The dark, secret and painful truth at the heart of every shaman is that they cannot possibly understand what they are doing. At least, this is according to many of us who worship at the altar of today’s technology. We’re probably less sure of one thing though — even the most skeptical among us might concede that somehow, in some situations, sometimes, shamanism might actually work.

If you’re concerned that this article is taking a misguided turn towards the mystical, let’s get straight to the point. There’s this growing notion of a shaman programmer. To be clear for what comes, what I refer to as shaman programming is the characteristic that systems are built without a thorough understanding of how their component parts work. They are built ritualistically. Real shamans [1] [2] are reputedly multi-skilled practitioners, especially well-versed in trance experiences and suchlike — these are not within the scope of this article, and are left to the anthropologists and others.

Professor AL. Couch called the phenomenon out in his article, Programming with Technological Ritual and Alchemy [3]: “I teach a generation of students unfettered by my compulsion to understand and, as a substitute, fettered by their own compulsion to do and experience”. Powerful stuff. He makes a clear distinction between programming patterns, where “concepts have been proven to work through widespread application”, and rituals, such as when an engineer locates and copies, say, a block of code from Stack Overflow and somehow makes it work in a specific situation. One problem with rituals, he suggests, is that modifications — however minor — are not guaranteed to work. Nor are such rituals well-documented, maintained or even properly tested.

The roots of this unease go further back. Just as soon as object orientation gained commercial traction in the early 1990s, pundits complained that this was turning developers into the mere manipulators of lumpy, block-level constructs, rather than designers and creators. It was de-skilling them, they claimed. Paradoxically, this ability to regard software artefacts as plug-and-play, reusable components used to be (and still is) touted as one of object orientation’s great benefits. This trend mirrors what happened when a generation of hardware technicians found themselves swapping out printed circuit boards after having spent their early careers performing component-level diagnosis and repair.

If we can use existing components to build robust, high-quality systems — successfully and repeatedly — are we at fault if we’re ignorant of their internals? This practice of chunking function into coarse-grained, often hierarchical, reusable black-box components has been a very successful strategy for mitigating complexity. As technology progresses however, the chunks themselves are undoubtedly becoming more complex, and more opaque. Try to explain to an educated layperson precisely why and how a deep learning system makes its decisions. And once you have explained, down to the level of weights, biases, activation functions and backpropagation, you will still feel the keen disappointment that you were unable to evoke a purer essence of exactly what was going on. Actually, it’s much, much worse than that. Deep learning is entirely opaque. Even if you can convince yourself that the result is right for one particular combination of weights, retraining the network using a slightly different initialization or training data will flip your intuition in a moment.

Danny Hillis, founder of Thinking Machines (where he worked with Richard Feynman [4]), in his book The Pattern on the Stone [5] contends that that no one is smart enough to understand all of the layers. Programming languages, algorithms and heuristics, memory and encryption, and so on are ‘mere’ abstractions building upon each other. He might argue that today, things are no different, that we’re still building and then climbing that eternal tower of abstractions.

If by using high-level abstractions, solving certain classes of problem is made simpler, then by definition, such problems become tractable to a wider population of developers and inevitably require lesser knowledge and skill to approach. Like a bad penny, the specter of de-skilling returns. To us at sourcerer, what seems likely is that we’ll see some more specializations emerge — some, the shamans, will be naturally drawn towards exploring what they can do by integrating many off-the-shelf or even custom components in new, interesting and exciting ways. Others will see themselves as component engineers. There will still be plenty of opportunity to exercise skill, ingenuity and expertise, regardless of which camp you fall into. We do believe, however, that good engineers will strive to understand as much as possible of those black boxes they rely on.

But there’s a chance we’ll also see the emergence of a third specialization, another category of developer, not a shaman, not an engineer, but an automation — an AI.

Github CEO Chris Wanstrath, believes that “the future of coding is no coding at all” [6]. It’s too bleak an opinion for us at sourcerer, but we think he’s onto something when he claims “the fastest road to making coding easier is to just automate the whole process”. Military experts are already predicting the last manned fighter aircraft. If Chris Wanstrath is right, we might be approaching the time when it’s credible talk about the last programmer. In our opinion, it’s more likely that new generations of developers will hone their craft as directed by an AI, using its growing capabilities to augment new hybrid development processes — imagine if an AI could write and run your unit tests, for example. In the future, shaman programmers will learn to converse with their AI advisors, teaching them until they can pick up the reins and perform the required tasks automatically.

In mechanics, there’s this notion called mechanical advantage. It expresses the ratio of the input force applied to a machine or mechanical device to the output force achieved. Think of a lever. Or think of the destructive power unleashed by a modern weapon with just the press of a button. Likewise, each new level of abstraction magnifies what’s possible with software, and we need not mourn the passing of the days as our new abilities are shaped to cope. No doubt, with great power comes great responsibility, but just think of the uncountable opportunities made possible by today’s shamanic incantations, such as:

pip install tensorflow

To close with some words from the preface of Danny Hillis’ book: “I etch a pattern of geometric shapes onto a stone. To the uninitiated, the shapes look mysterious and complex, but I know that when arranged correctly they will give the stone a special power, enabling it to respond to incantations in a language no human being has ever spoken. I will ask the stone questions in this language, and it will answer by showing me a vision: a world created by my spell, a world imagined within the pattern on the stone.”

The rise of the shaman programmer? Bring it on!