Why New York City?

“The [2008 financial crisis] created a perfect storm of talent exiting afflicted institutions and investment money fleeing the public sector markets.”

First, some history

Contrary to popular belief, New York has actually always been a major data science hub. The reason New York is often overlooked as a cornerstone of the data science community is because almost all the city’s data scientists were previously locked within enormous institutions like banks, ad agencies, and major media companies. They held vague and cryptic titles like quant, statistician, or business intelligence analyst.

Instead of thinking of themselves as a community that could leverage their expertise to transform how data is used, they maintained focus on building specific analytical tools and products for their own particular domains of expertise. This went on for years.

But then something really tragic happened: the financial crisis of 2008, which delivered a huge blow to the city of New York; to both its denizens and its businesses. However, out of this calamity came an opportunity. The crisis created a perfect storm of talent exiting afflicted institutions and investment money fleeing the public sector markets.

Most significantly, New York’s data scientists started coming together to talk about their work and ideas. This gathering of diverse backgrounds and experiences spawned a unique, cohesive data science community that, quite frankly, doesn’t exist anywhere else in the world.

Diversity of talent and of thought

“As a diverse group comprised of skeptical academics, social sector employees, and public and municipal organizations, data scientists in NYC deal with the question of how data is really serving people on a daily basis.”

One of the reasons New York’s data community stands out from the rest is that it is the best at recognizing the need for both natural sciences and social sciences to come together to do truly great and innovative data work. Part of our job as data scientists is to be good at math and modeling complex systems, while also incorporating a deep understanding of human decision making, the most complex system there is. New York’s data science community is a diverse collection of talented data scientists who are uniquely able to balance and implement both of these components into their work.

New York’s data scientists have not only helped to build many innovative and successful businesses (AppNexus, Bitly, Tumblr, Kickstarter, Jet.com, Vimeo, Oscar Health, Enigma, Greenhouse, MongoDB, ZocDoc, OnDeck, Etsy, Venmo, Blue Apron, Fast Forward Labs, Clarifai, etc. etc.) but the Data for Good movement started here too. There are numerous New York-based organizations that have put using data for social good at the core of what they do, such as DataKind, Crisis Text Line, Murmuration, Mt. Sinai’s Arnhold Institute for Global Health, and Teachers Pay Teachers, as well as the NYC Mayor’s Office for Data & Analytics, founded in 2013 under Mayor Bloomberg.

As a diverse group comprised of skeptical academics, social sector employees, and public and municipal organizations, data scientists in NYC deal with the question of how data is really serving people on a daily basis. They also take the time to step back and ask “Okay, there’s lots of good work happening, but what are the limits?” New York City’s density and diversity support this kind of questioning, by keeping all of us in direct contact with the users and consumers we’re hoping to serve.

Density as a forcing function

“If you work at a software firm where you sit by yourself and imagine what your customer needs or wants are, you’ll never be as successful as you could if you’re able to walk down the street from your office and talk directly to your customers.”

Bustling sidewalks and crowded subways aside, New Yorkers living in close proximity to one another has benefits for folks in enterprise businesses as well. A common challenge for many startups is the constant need to better understand their users. Who is using our product? How are they using it? What issues are they having? If you work at a software firm where you sit by yourself and imagine what your customer needs or wants are, you’ll never be as successful as you could if you’re able to walk down the street from your office and talk directly to your customers. New York City has more Fortune 500 companies than any other city in the US. It’s HQ central for so many industries: media, advertising, financial services, banking, fashion, large-scale retail… the list goes on and on.

In New York, we almost take it for granted that on any given day, whether it’s at Data & Society, at Insight Data Science, or at Civic Hall, there are conversations going on about what it means to be a professional data scientist. Instead of staying buried in a text editor all day, many of us are grappling with the ethical and social challenges that come with data science. And in New York City, there is an appetite to discuss and share the latest trends and topics in data science.

The community’s diversity and willingness to ask the “hard questions” about what we’re capable of are reflected in the ecosystem of companies that have emerged in New York. One of the biggest challenges currently facing all data scientists is a lack of a clear career trajectory. What does it mean to be a “Head of Data Science” or “Chief Data Officer”? At the moment there are now a lot of data scientists but not a lot of data science managers. What does it mean to develop leadership inside this community? What are the things that we need to do to create the next generation of data science leaders from all the people sitting in our big tent?

The good news for the world’s future data scientists and managers is that building an innovative data team in New York is not only possible, but their best bet for opportunity and growth. The community’s diversity will allow them to combine methodologies that work well from fields with more established pathways like software engineering, scientific research careers, or product management, and mesh them with less obvious systems modeled after those in finance, healthcare, design, and media. Our ability to foster conversation and collaboration across these areas is what will eventually allow those of us in NYC to shape the trajectory of data science careers well into the future.