Spoilers ahead for the most recent episode of Game of Thrones and A Song of Ice and Fire.

In one of Bran’s three visions on Sunday, he discovers the origin story of the White Walkers: The not-quite-so-innocent Children of the Forest created them. His lesson is a bit rushed, and he doesn’t get all the answers he’d like. For instance, did they know what they were doing when they shoved what appears to be an obsidian blade (a.k.a. dragon-glass) into the chest of their first test subject, the man who would become the Night’s King? Did they ever have the means to control their Frankenstein creatures? And why did the White Walkers turn against their makers?

We don’t have all the answers, either, but we can fill in the blanks for some based on what we know from the books. When Leaf tells Bran, “We were being slaughtered. We needed to defend ourselves,” she’s referring to the coming of the First Men across the Narrow Sea to Westeros. In what would be considered the prehistoric period of Westeros — the Dawn Age — one of the original inhabitants of the land were the Children of the Forest. (“Before your Old Tongue was ever spoken, we had sung our songs 10,000 years,” one says.) Some compare the Children to elves, fairy folk, or land spirits, since their spirits and memories become part of the trees after death. The Children worshiped the old gods (the gods of the forest, stream, and stone) and were peaceful, at least until they were threatened by the invasion of men some 12,000 years ago.

The First Men, a new people to Westeros, arrived by crossing a land bridge across the Narrow Sea into the southernmost part of Dorne, where Westeros and Essos were joined. Thousands came, bringing new animals (horses, cattle) and carrying bronze swords. They started settling the land, creating roads, farms, and villages, as well as migrating north. At first, the Children welcomed the newcomers, “in the belief that there was land enough for all.” But then, the First Men started chopping down the weirwood groves, either because they didn’t realize the significance of the weirwood network, or they feared the faces in the heart trees, or they suspected the Children could spy on them. Whatever the reason, this meant war. Theon’s sister recalls a legend she heard about those days, “When the greenseers turned the trees to warriors.”

The Children were limited in their tools of warfare: bone, wood, stone, and obsidian. According to legend, they were able to call upon beasts to fight on their behalf, but all the direwolves, snow-bears, cave-lions, eagles, mammoths, and serpents were not enough. Some say the Children tried to slow the advance of the First Men, cutting off their routes, flooding the Arm of Dorne and the Neck around Moat Cailin, to try to break Westeros in two. (This is attributed to the Children using dark magic to call upon their gods “to send the hammer of the waters.”) The maesters of the Citadel, however, say these were natural events, not caused by the Children’s magic. Either way, the great floods broke the land bridge (now the Broken Arm) and made the Neck a swamp. But this was not enough.

The war lasted for generations, until the Children realized they could not win. Although the references are few and far between, there are mentions in the histories that the Children were “driven to a desperate act” — that they turned to sorcery “to stem the tide of these invaders.” The legends are vague on the specifics, and usually it’s interpreted that the “desperate act” was the flooding, since the magic fueling that required blood sacrifice. But this “desperate act” could have been when the Children created the White Walkers as a weapon to defend themselves. Where the legends pick up is when a peace accord was struck between the First Men and the Children, called the Pact. They divided up the land, saving the deep forests for the Children. The First Men took the Children’s old gods as their gods, and agreed to no longer cut down the weirwoods, which became sacred to both parties. This ended the Dawn Age, and ushered in the Age of Heroes.

Or so we’re told. But what if the Dawn Age ended in part because the peace accords were struck in haste, so the Children and the First Men could fight the White Walkers together? If their weapon got out of control when the White Walkers realized they could essentially warg the dead and raise an army of wights, the First Men and the Children might have realized that they needed each other to survive. Building the Wall together, gifting the Night’s Watch with obsidian — were these things the Children did out of guilt? It might explain why the remaining Children were willing to sacrifice their lives to save Bran: They feel responsible for their own unruly children.