The formation of the boroughs also predated the opening of the New York City subway system by 6 years. The ambition to connect the boroughs of NYC via subway was made clear in the naming of the first subway company to operate in the city: The Interborough Rapid Transit Company. From its first day of operation (when it transported over 150,000 passengers) to it’s current status as the busiest rapid transit rail system in the western world, the New York City Subway has had a radical impact on the life of New York City and the mobility of its denizens.

City Hall Station: one of the first stations opened by The Interborough Rapid Transit Subway (IRT). Photo by Fred Guenther

Disenfranchisement of the Boroughs as a Unit

In 1989 the Supreme Court unanimously declared the NYC Board of Estimate to be unconstitutional. This decision was made on the grounds that Brooklyn (pop: 2,504,706 according to the 2010 census) had no greater representation than Staten Island (pop: 468,730), thereby violating the Fourteenth Amendment’s Equal Protection Clause. In the wake of this decision, the NYC Board of Estimate was abolished, and most of its governing responsibilities were transferred to the New York City Council, which consists of members drawn from a much more granular partitioning of the city with 51 council districts distributed (unevenly) across the 5 boroughs. Thus, while the constituent council districts of a borough have substantial political power in the New York City government (via their representative council members), the boroughs as a unit have far less political significance.

Using Data and Artificial Intelligence to Understand New York City

We formed Topos earlier this year to advance the understanding of cities through the interconnected lenses of data and artificial intelligence. While there are well-known tools such as the United States Census that use manual techniques to collect information about different locations, using data and AI enables a dynamic, highly granular, and globally scalable understanding of place — an understanding we think is valuable given the rapidly evolving nature of cities and neighborhoods around the world (the US Census, for example, takes place every 10 years, divides the country into 9 regions, and only covers the US).

Furthermore, we were interested in going beyond more familiar demographic viewpoints to capture the personality of a place, and what it feels like to actually be there. In a sense, we’ve developed a ‘psychographics’ of neighborhoods. As part of this endeavor, we have constructed a suite of features and indices about neighborhoods and cities that encompasses everything from topological analysis of urban form, to ambient light levels, to the prevalence of craft cocktails within a neighborhood.

From this starting point, we decided to re-envision what a five borough partitioning of NYC might look like in 2017 using techniques from Artificial Intelligence: vector construction, dimensionality reduction, and clustering.

Neighborhoods represented on a map (left) and as vectors (right)

From a 2D map to a 65D hyperspace

Mathematically, we can understand this suite of neighborhood features as a high dimensional vector space, where each feature is represented by a unique dimension; creating, in the case of this article, a 65-dimensional space. Each neighborhood[*] becomes a vector in this space, which can now be transformed and analyzed using a wide range of mathematical, statistical and computational techniques.

[*]: Here we make the problematic assumption that neighborhood = zipcode. We promise to tackle this assumption in an upcoming blog post.

From 65D to 16D

One of the challenges in constructing a collection of features is understanding the interrelationship between features. 4 dimensions that are tightly correlated reveal much less than 4 completely independent dimensions. This becomes especially important in understanding the ways that entities described by features relate to one another — an understanding that forms the basis of several machine learning applications. For this reason, high dimensional spaces are often transformed through the use of various dimensionality reduction techniques.

Starting with a 65 dimensional space, we applied Principal Component Analysis (PCA), resulting in a 16 dimensional, linearly independent space that captures 86% of the variance of the original 65 dimensional space.