United States President Barack Obama recently introduced DJ Patil as the new Chief Data Scientist of the United States Government. See video: Data Science: Where are We Going?

Why does the government need a Chief Data Scientist? We all know the government is dysfunctional, inefficient, overly expensive and not working well - especially the legal and healthcare systems. Yet we need quality and cost-effective government services for many things the private sector cannot or will not do.

Put simply, the current dysfunctional, wasteful, incompetent and inefficient government that we get today is unsustainable in the long term - it is hurting the economy, bankrupting the country and damaging faith in our political institutions. We need better government services cheaper and faster.

The hope is to build an ecosystem of data services and products in conjunction with data science techniques that add value to government thus making it work better and cheaper. In 2013 President Obama signed Executive Order 13642 making open and machine-readable government data the new default and created numerous Open Data Initiatives making huge amounts of data easily accessible to anyone.

So far so good. Yet where is the evidence that all this big data is making government perform better and cheaper? Do we even have the right measurement tools and techniques to quantify and measure government performance? Remember, if you cannot measure it you cannot fix it!

Moreover, many government dysfunctionalities and inefficiencies are a feature and not a bug to be quantified, understood and improved. Political considerations with powerful interests groups and constituent beneficiaries usually trump service efficiency, quality and cost-effectiveness when government provides services or transfers wealth.

In addition, government and political incentives are very different from market incentives. If we have any hope of improving government our society will need to change political incentives and goals. Does our polity have a reasonable consensus on what basic government services are required for a healthy and good society?

That said, in theory, a government Chief Data Scientist is a great idea if serious about improving government and creating a better future. In practice, the Chief Data Scientist will likely be impeded, hammered and rendered ineffective (if not destroyed) by special interests and numerous political forces when the data science evidence suggests taking away or reducing their power, money and free services.

As a result, the only realistic strategy to use data science to improve government is to reach some approximate societal consensus on base level government goals and services that we can afford and replace much of the bureaucratic Leviathan with a new government service system based on data science. This includes government service innovations in: high performance computing (HPC), smart data collection, machine learning and algorithms. Furthermore, it means experimenting with new machine / human partnerships delivering different service processes while defanging the bureaucracy that has strong vested interest in the status quo.

Machine learning algorithms can use old and new data to make optimal decisions as well as innovate and constantly improve services thus achieving better and cheaper services. Well designed social, legal and healthcare algorithms in partnership with human creativity will provide more effective and cheaper services. Additionally, algorithmic decision-making removes most human bias for greater fairness improving citizens faith in institutions. Creative humans can work with these new smart machines to design, improve and maintain algorithms and service processes.

Much of the government run legal and healthcare systems can be significantly improved using HPC and algorithms. The legal and healthcare systems are dysfunctional, unnecessarily expensive and provide outrageously bad service for citizens. Moreover, they are unsustainable in the long term as we simply cannot afford them even if they performed with better results.

Achieving the above will take time, willpower and much experimentation to perfect. Yet, we have no choice - our current government is simply bad, unsustainable and must begin to change. Yes, we will need to collect better (not necessarily bigger) quality and more relevant data, innovate new computing system architectures as well as develop new data science techniques.

Today's Chief Data Scientist needs to be a system creator. The focus today is on creating and building new data engineering systems and infrastructure, collecting better smart data and innovating data science techniques. In short, he or she needs to create new high-tech systems and processes based on a new machine / human partnership and experiment to find what really works.

The future Chief Data Scientist will oversee the government partnership between efficient machines and creative humans to constantly improve and innovate new processes to provide better and cheaper government services. Thus, he or she will be a system maintainer.

Therefore, I predict in thirty (30) years or so the government Chief Data Scientist will be the second most powerful person in government save the President. Yet the role, skills and duties will be dramatically different than those of today's Chief Data Scientist.

The question is, does new government Chief Data Scientist DJ Patil have the chops to create this new system? Is he a system creator or system maintainer? Will President Obama and future presidents provide the leadership, political support and funding required for data science to succeed? Can the political system reform to change how it provides services, re-align public / private relational incentives and allow data science to save our great nation from slow decay and potential bankruptcy?

We live in interesting times!

Note: President Obama quoted me (Michael Walker) by stating "data science is a team sport" (see video at 1:15). Faithful readers know I coined the phrase in my blog post entitled "'Data Science is a Team Sport" dated January 28, 2013. Imitation is the sincerest form of flattery so thank you Mr. President - I owe you one.

See: http://bit.ly/1kXsipc