Why have Middle East’s oil-rich economies failed to diversify despite their tall promises and grandiose plans? The answer lies not in the absence of good technical plans or weak implementation, but in political incentives. If many other countries have been successfully able to diversify their economies it was not merely a result of good policies but the right political incentives of those who were in the driving seat.

An enabling political framework has been a common denominator in all successful diversification experiments. Botswana’s experience underscores the role of stable political coalitions and favourable initial and external conditions. At its independence, Botswana inherited multiple constituencies representing divergent economic interests. This was complemented with the presence of political competition and stable coalitions. A third important factor was a favourable external environment. Botswana’s membership in the South African Customs Union served as a positive inducement for sensible macroeconomic reform. Together, all these factors played a part in protecting the interests of non-resource sectors.

The Malaysian experience reinforces the same message. At Malaysia’s independence, the Chinese community represented a powerful de facto economic force by virtue of their control of the Malaysian private sector. Their continued presence counter-balanced any tendencies for the natural resource sectors to grow at the expense of the private sector. In the political domain, the consociational agreement between the ethnic Malay and Chinese communities fostered a system of power sharing that protected the economic interests of Chinese businessmen. Bad macroeconomic policy – especially an overvalued exchange rate – was politically unacceptable to Chinese interests. It was both bad policy and bad politics. If domestic political economy was helpful, so was the country’s insertion into the regional trade circuit, which created positive regional spillovers that supported private sector development.

Clearly, each case is different and must be analysed on its own merit. But politics provides a common thread across these accounts. And, this is where Arab economies are especially challenged. Saving a few cases, most countries in the region did not inherit strong and diverse economic constituencies that could have gained political voice after independence, and counterbalanced the dominance of the oil economy. An unfavourable external environment, resulting in negative spillovers from regional conflict and instability, served as another impediment to diversification. The Middle East thus lacked all three factors that facilitated economic diversification in other countries: strong political coalitions, diverse economic constituencies and positive neighbourhood effects. With this adverse legacy, is there any real hope for diversification? In this regard, I have the following three points to make:

Successful diversification requires a new political settlement that allows elites to concede greater space to the private sector;

Diversification is unlikely to succeed without a regional vision that fosters complementarities among Arab economies and creates a shared economic space to deal with emergent economic challenges common to all states;

Sustained economic change in the Middle East requires a wider set of concessions that go beyond domestic and regional political elites. It also requires a candid and constructive geopolitical discourse that reconsiders the trade-off between a narrow, short-term, vision of geostrategic stability and long-run development.

Let me briefly explain these in turn.

Given the primacy of the political, the debate on diversification must begin with a discussion of elite incentives and political concessions. If a closure of the economy benefits rent-seeking elites, what will persuade them to concede greater economic space? What concessions are needed from the ruling elite and what will persuade them to surrender their control of the economy and the associated rents? Perhaps, they need to be compensated for the loss of rents from a levelling of the economic field. After all, new growth strategies in emerging markets are built on a happy (even if fragile) coexistence of economics and politics.

The Chinese example serves to illustrate how economic reform can be aligned with the interests of political elites. The Chinese political experience is decidedly based on centralised authoritarian control. But the system allows a balancing of competing interests. It co-exists with considerable regional decentralisation where local leaders derive strength from patronage – just as in any other developing country – but are equally strongly incentivised to ensure economic growth in their localities. Economic growth yields clear political dividends for local elites. And bureaucrats face strong performance incentives. As a result, growth of the economy has become an integral component of the political objective function.

Beyond the oft-cited example of China, Africa’s recent success stories confirm the importance of elite incentives. Consider Ethiopia‘s recent economic transformation, which has placed it in the list of the 10 fastest growing economies of the world. Central to this growth experience has been the role of public investment in infrastructure and public enterprises, and the changing political orientation of state elites. The ruling political party managed to set up its own enterprises supported by specialised endowments geared towards promoting investment in underdeveloped regions. Although this model of party capitalism poses serious questions about market competition, it goes to shows that elites can favour an expansion of the economic pie when they are among its lead beneficiaries. This is, after all, a key point of North, Wallis and Weingast’s treatise on Violence and Social Orders. Change often begins with small outcomes and processes that are compatible with elite incentives. But, what begin as privileges for insiders can ultimately become universal rights for everyone else.

In short, the idea is not to search for the ideal growth experience that will uniquely fit all Arab contexts. Rather, it is to emphasize that whichever growth strategy the Middle East embarks upon should consider and accommodate political incentives. And, elites have rarely surrendered their economic control unless it became essential for their survival. The so-called “Arab Spring” was a recent tapping on the doors of power. Unfortunately, rather than resulting in any genuine economic concession, what we have instead seen, is business as usual. The only concessions that came through were financial concessions in the guise of cheap loans, salary hikes and free bonuses. But such temporary appeasement without changing the underlying rules of the game is unlikely to work for too long. And, the rules remain rigged in the favour businessmen in and around the royal circle. In North Africa, crony capitalism is rearing its head again, and insider deals continue to thrive across much of the region. In this backdrop, economic diversification will be difficult, if not impossible, to realise without a new political settlement that caters for a future beyond oil and conflict. At the very minimum, the region needs a new discourse on economic reform that mobilises public support for two or three fundamental concessions that elites must surrender for long-term economic revival.

Let me turn to the second idea. The argument, in brief, is that national diversification plans that disregard regional linkages in development are doomed to fail. It is important to recognize that, in the Middle Eastern case, the questions of national and regional development are closely interwoven. While national initiatives can kick-start economic revival, it will be difficult to sustain without regional market access. Few countries have effectively diversified without expanded markets and deeper trade reforms that regional trade liberalisation affords. Turkey’s recent economic success is built on a strategic cultivation of regional trade linkages. In Asia and Latin America, regional market connections offer an additional avenue for industrialisation through entry into global supply chains, which tend to conglomerate spatially. Arab countries are clearly disadvantaged in this regard. A larger coordinated effort is needed at the regional level to foster trade complementarities, establish regional public infrastructure, and relax trade barriers. Given the history of repeated failures at regional economic cooperation and the adverse security climate, this seems like a pipe dream. No matter how impractical, it will be difficult to sidestep the regional question in any new vision for Arab development. In political economy terms, the rationale for this is even stronger, since it is only through a regionally integrated merchant class that a stable constituency for economic and political reform will emerge. If the broader economic challenges faced by the Arab states are common, they also deserve a common response. Even if a cooperative solution does not serve the narrow factional interests of political elites, the Arab civil society must lend its weight behind the regional project.

This brings me to the role of geopolitics, the final element of my argument. In a region that has historically remained a hotbed of conflict and violence, it is difficult to conceive economic diversification in isolation from geopolitics. The powerful negative externalities emanating from regional instability have scaled back even the modest gains achieved on the economic front. Prior to the recent upsurge in violence, countries in the Levant had begun to witness falling trade costs and growing regional trade. These limited gains have been washed off by regional violence. Foreign military interventions in the guise of regime change have eroded state capacity, demolished public infrastructure and ripped apart the very social fabric of Arab societies. The region has been set back by decades.

If conflict retards development a genuine economic renaissance in the Arab world will also have geopolitical repercussions. Foreign powers have a deep economic, political and military footprint in the region. An economically independent Middle East can challenge the established patterns of external hegemony and undermine the prolonged legacy of divide and rule. In this milieu, structural economic change also requires a geopolitical concession from regional and global powers that have high stakes and influence in the Middle East. As the recent refugee crisis has shown, the spillovers from regional conflict are difficult to contain within Arab borders. This is an opportune moment to talk about concessions. A peaceful and prosperous social order is now of direct interest for the global community, especially Europe.

Foreign powers face a deep trade-off between narrow short-term strategic interests and long-term development. The human and economic cost of this policy trade-off is rising by the day. Yet an effective global response has been noticeably lacking. Since the start of the Arab Spring, economic development has been conspicuous by its absence in western policy discourse. There has been no grand vision for regional development on the part of multilateral institutions or Western governments. Initiatives such as Deauville Partnership and the Arab Partnership Fund were minuscule efforts both in size and significance, and simply substituted talk for action. On the other hand, we have seen a major escalation in the sales of military hardware to Arab states. Rather than using their “convening authority” to organize regional funding for a major development initiative, Western powers have instead sold billions of dollars in worth of arms to the GCC states since 2011.

In closing, economic diversification in the Middle East – far from being purely a technocratic affair – carries deep power implications, involving all three inter-locking spheres in the domestic, regional and geopolitical domains. By producing a greater number and variety of products, diversification not only increases the complexity of economic exchange but also risks generating independent constituencies whose political economy effects are neither neutral for domestic power structure nor for the prevailing geopolitical order. This calls for a more holistic understanding of the challenge of diversification.

The views expressed in this article are the author’s own and do not necessarily reflect Al Jazeera’s editorial stance.