Christmas Carols, generated by a neural network

Neural networks are a type of computer program that imitate the way that brains learn to solve problems. They’re used for face recognition, self-driving cars, language translation, financial decisions, and more. I mainly use them to write humor.

My process starts with a dataset - something that the neural network has to figure out how to imitate. Rather unfairly, I give it no instructions about whether it’s trying to write knock-knock jokes or invent Halloween costumes or begin a novel. It doesn’t even have any built-in knowledge of English. It definitely doesn’t have any clue what Christmas carols are, which is why this week’s experiment was so much fun.

The Times of London teamed up with reader/neural net hobbyist Erik Svensson to collect a mix of ancient and modern carols, about 240 carols in all, from “What Child is This?” to “Grandma Got Run Over by a Reindeer”.

When the neural network begins learning, it starts with a set of random rules about how to put one letter after another to make a Christmas carol. Since they are random, they are terrible rules, and when the neural network tries to apply them, it gets junk:

a a a na snn aca naa i s a i aa a a n a uugna nn na i a uaa a a i a a a a a sna aagt o aa

But it can check its rules against the real songs in its database and then make little tweaks to them that make them work slightly better. It learns to capitalize the first word of a line, it learns how often to do line breaks… and soon it begins to learn actual words - the most common ones first.

Hart fon the be the he br wong on the stor Christmas br he, or the wang

Christ, Christ, on bn a me the stord

Hont on thr st bong the wor

I he a s de poog the stow tome on be ser snur

After the neural network has spent many more rounds refining its rules, it begins to look a lot like Christmas.

Now, neural networks tend to pick up tone and vocabulary pretty easily, but struggle with making sense. So, the carol-trained neural network learned to produce a lot of lines that sound - well, joyful, at least.

The cattle around the Christmas will be

A very special Christmas with me

Hurry Christmas to you

Cup on the earth!

Still the loudly candlelight

Would praise His name.

The babe, the Son of Mary.

He sumbled their flowers and all.

The lord of the glory dawns

Give us the leave all away

A star is spent and red

Shake a cup a strend from the sky

Christmas is coming, the wind is come to you



Walkin’ him love, Dingle bells, jingle bells, jingle bells with bells are ringing



With a heart reindeer

But no more a stranger.

Santa baby, and Dancer, and Curry down

Happy Holiday

When the snowflakes will call the world wakes to bring

Glory bears and asses the air the angels sang

And Christmas tree



(Curry is not in the input data. Nor is Dingle. The neural net likes to invent words that it thinks sound sort of carol-y)

Other carols it generated sounded a bit more morose.

The fire is sleeping.

And crying,

Love love

What a King

Let’s take the little children of the grave!

For some reason, the Sandman figures very prominently in the neural net’s Christmas mythology, despite having been mentioned in the dataset only once. Sometimes the neural net latches on to particular words for no reason I can see. Maybe it’s a Neil Gaiman fan.

The sandman so be joyful now it was born today!

Gloria in excelsis Deo.



The sandman bright before Him.

The holly bears a berry bears

And star in the snow is born today!

The sandman so love to seek the world

The sandman so love so deep and sing and the sun

And this? In retrospect, I should have seen this coming:

The world and joy of the sleigh

Santa baby bore sweet Jesus Christ

The holly bears a berry,

And all the reindeer of the sky

The holly bears a berry and reindeer

He was born today!

And Santa baby bore sweet Jesus Christ,

And the chimney the angels sing.



When the snowman brings the snow

Christmas tree

Let’s take that road before

And Santa Claus comes tonight

He will bring us goodness and light



Santa baby, a blitzen,

And he was the sun and reindeer and earth.

The Saviour of the chimney tonight

The story of the chimney see

Santa baby, and blood and joyous so world and joy and good will to see

Santa baby bore sweet Jesus Christ

Fa la la la la la la, la la la la la la la la.

King of toys and hippopotamuses [sic] full of the light of that stood at the dear Son of Santa Claus

He was born in a wonderful christmas tree

Run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolph, run, run Rudolf the new born King.



The neural network also produced a few lines that, um, strained the bounds of good taste. If you want to read them, sign up here and I’ll send them to you.

