The evolution of the Temple Mount crisis has exposed Israeli officials’ shortsightedness. Despite all their experience, Prime Minister Benjamin Netanyahu and Public Security Minister Gilad Erdan adopted the police’s recommendation to install metal detectors after the killing of two policemen at the Mount. They didn’t consider that this move might be perceived by the Palestinians as altering the status quo or lead to mass protests in East Jerusalem, smaller protests around the West Bank and a surge in attempted terror attacks.

Most of the relevant security organizations – the Shin Bet security service, Military Intelligence and the Coordinator of Government Activities in the Territories – warned against the implications relatively early (and were accused of painting apocalyptic scenarios). The debate underscored yet again how hard it can be for intelligence agencies to predict the reactions of large groups, even when the agencies are often able to foil an attack based on a single social media post, out of tens of thousands, that a lone terrorist writes just before setting out.

Col. Michael Milstein, the COGAT adviser for Palestinian affairs, published an illuminating article last week describing the intelligence challenge in trying to analyze the people’s behavior in the territories and Arab countries. Milstein, a former head of the Palestinian research department at Military Intelligence, says this type of research is in dire straits “on both sides of the ocean.” He says that, in the past, intelligence research focused on analyzing the intentions of leaders and the enemy’s military capabilities.

Nowadays this research has to identify complex processes motivating the people or non-state organizations. But still, an intelligence analyst who’s an expert on another culture or mentality is considered less valuable than before. The focus is skills centering on thought processes and strategic analysis, combined with an overreliance on technology, which is expected to provide quick and precise answers to complex questions.

In the article, written before the current crisis and published by the Israel Intelligence Heritage & Commemoration Center, Milstein says intelligence may end up being relegated to operations only. He recommends the integration of modernity and tradition: the reinforcement of intelligence people’s familiarity with the language and culture, together with a mastery of modern skills like Big Data analysis. This combination doesn’t guarantee success in predicting the future, but at least it can improve our ability to accurately read the present.

Milstein describes the big problem intelligence officials don’t talk about often: the declining priority of a deep understanding of the enemy’s culture, language and history. He says this crisis only got worse after 9/11 and during the Arab upheaval now six and a half years old. Moreover, the number of intelligence experts capable of analyzing deep social and cultural phenomena in the Arab world has shrunk to “the verge of extinction,” and “discussion of the problem is practically nonexistent. Even worse, a counter-approach has emerged that says such skills are anachronistic and unnecessary in modern intelligence research.”

When Milstein writes about intelligence being confined to the operative dimension, he echoes processes happening in recent years at IDF intelligence. At the start of the decade, then-MI chief Aviv Kochavi spearheaded a revolution in which operational intelligence-gathering capabilities were dramatically improved, along with the way this intelligence was relayed to the commanders in the field. But this success had a price. Under Kochavi and his successor, current MI chief Herzl Halevi, operational intelligence, especially that for special operations, gained greater prestige, at the expense of the intelligence research department.

Military Intelligence, which greatly tightened its cooperation with the air force and later with the regional commands, is also a huge enterprise now, a conglomerate for producing targets. The advantages of Kochavi and Halevi’s moves will be proved in any military confrontation with Hezbollah and Hamas. But meanwhile, Arabic is being forgotten. Former officers from the research department say this outfit has no more than 10 fluent Arabic speakers. This may be one reason Military Intelligence had trouble predicting the changes in Hamas’ stance regarding the failed cease-fires during the last Gaza war.

These processes were in the background of last year’s resignation of Brig. Gen. Eli Ben-Meir as head of the research department. People in the department cited an improper balancing of resources, saying the balance between the two sides was radically off. Eisenkot was furious over the resignation and the revelation of the clash between the research-department chief and the Military Intelligence chief, but he backed Halevi. To this day, Ben-Meir hasn’t commented directly on the affair, with the exception of one vague Facebook post on the day of his discharge. One can expect more such posts at some point.