There may be older Christian hymns, but the oldest hymn accompanied with musical notation is surely the fragmentary one found at Oxyrhynchus:

The Ancient Peoples blog provided a transcription and translation of the text:

[?πρ]υτανηω σιγατω μηδ᾽ αστρα φαεσφορα λ[ειπ]ε

[σ]θον[.].λει[…]ρ[…]ποταμων ροθιων πασαι υμνουν των δ᾽ ημων

[π]ατερα Χ᾽ υιον Χ᾽ αγιον πνευμα πασαι δυναμεις επιφοωνουντον αμην αμην κρατος αινος

[…]δ[ωτ]η[ρι] μονω παντων αγαθον αμην αμην

These lyrics have been translated as:

… Let it be silent

Let the Luminous stars not shine,

Let the winds (?) and all the noisy rivers die down;

And as we hymn the Father, the Son and the Holy Spirit,

Let all the powers add “Amen Amen”

Empire, praise always, and glory to God,

The sole giver of good things, Amen Amen.