Big city network performance: the New York City urban mobile experience



10-6-15 | Patrick Linder

Introduction

Mobile life in the big city



The beauty of mobile is that it promises not just greater productivity and connection but also greater freedom. Ideally, you can use your mobile phone wherever you go, whenever you want. In our daily life, we take advantage of that freedom by using our phones in a wide variety of locations, across incredibly diverse conditions.

Our recent State of the Mobile Union report provided you with a broad overview of mobile network performance across the US, within all 50 states, and across the 125 most populous metros. But we know that in addition to being part of a nation and having an area code tied to your phone, your mobile life is made up of a zip code, a street number, and your own unique combination of family and friend connections.

In short, your mobile experience is incredibly varied. Your daily mobile life in a big city is a tapestry built of different places, special moments, and everyday movements, each of which puts a different level of stress on mobile networks.

For this report, we put the New York City area through the most comprehensive study of mobile performance available anywhere. Our report starts with a never-before-seen look at how the carriers fared in the dense Manhattan Urban Zone. We’ve long tested the broader New York City metropolitan area as a whole, but this first section shows performance within the heart of Manhattan. We’ve then complemented that look at the Manhattan Urban Zone with performance reviews of greater New York City area sporting venues, transit stations, campuses, and airports. We know your mobile life is varied; you deserve a report that shows performance across all the areas you use your mobile in the big city.



london-eye

carousel

Results overview

To give you information that best fits your unique mobile life, we’ve continuously expanded our scientific testing to include even more areas where you use your mobile. We’ve long tested performance in airports, metro areas, within all 50 states, and across the nation itself. In the first half of 2015, we expanded yet again: our in-depth, non-scored testing of key urban zones within big cities works in conjunction with our new series of Transit Stations, Venue, and Campus RootScore Reports to offer a detailed look at the performance challenges faced within a big city.

Our existing line of Metro Area RootScore Reports are based upon government-established Census Urbanized Area (CUA) boundaries. These Metro Area RootScore Reports show performance across the breadth of a market and include results from areas that are within CUA boundaries but often beyond the central population core. For instance, the Census Urbanized Area for New York also includes a large swath of the Tri-State area. Our new Urban Zone study offers a more focused look at performance within the densely-populated areas that are central to your life in the city.



For this special report, we’ve gone one more step and complemented the deeper dive into performance within the densely-populated Manhattan Urban Zone with results from New York City area venues, transit stations, select campuses, and airports. It’s a comprehensive look at performance across your mobile life in New York City.

performance-tally

Detailed insights for each section can be found below. Overall, the big takeaway is that competition between the networks is heating up. Verizon has often led our scoring in other reports, and Verizon once again offered excellent service in our testing of mobile life in the New York City area. Sprint and T-Mobile, however, also caught our attention with strong performances in the densely-populated area of New York City. Continuing a trend we have seen in our other recent testing, Sprint’s reliability and call performance were particularly strong, while T-Mobile’s speed and data performance were especially noteworthy.

The dense urban environment is where millions of people spend the majority of their mobile lives. The performances from Sprint and T-Mobile in our look at the New York City urban experience means not just improved competition in the city itself but could augur a new mobile performance race elsewhere as upgrades continue.

The other surprising change was the relatively modest performance from AT&T. In many of our previous reports, AT&T has closely trailed Verizon. Our most recent New York and the Tri-State Area report, for instance, showed Verizon only slightly ahead of AT&T in our Overall Performance rankings. However, within the densely-populated Manhattan urban zone and the variety of other dense locations we tested in the New York City area, we saw increased competition from Sprint and T-Mobile. If results from these locations are any indication, investments from Sprint and T-Mobile are having a significant impact and helping to reshape the mobile performance race.

icons

Urban zone



Reporting unique coverage and congestion challenges

Big city centers can present big challenges for mobile coverage. As hubs for tourists, commuters, and residents, city centers are asked to handle a wide variety of mobile demands from a constantly fluctuating number of people. Architectural challenges and limits on tower placement and other network equipment make handling the traffic demands even more complicated.

To provide an in-depth look at mobile performance in this dense urban environment, we test extensively indoors, while walking outdoors, and even during driving along the urban streets. And because a big city center is not homogenous, we always test a variety of different and representative districts within the urban zone. The end result is a city study like no other, providing a look at how coverage challenges unique to dense city centers impact your mobile life.

The urban zone is the heart of your mobile life in a big city. Whether walking along the busy sidewalks, stopping for a meal at a crowded restaurant, or riding in a taxi to your next location, you expect your mobile to work without interruption or delay. You want reliable and fast performance that lets you effortlessly make calls, send texts, or connect for email and other data activity.



To give you a full picture of the Manhattan urban zone, we tested call, data, and text performance under the same conditions that you experience in your daily mobile life. The boundaries of the urban zone reflect where population is concentrated and typically follow the city limits defined by local government. An urban zone often includes several well-known city districts and we extensively test these important areas. Within the Manhattan urban zone, for instance, we tested seven districts: the Financial District, the Garment District, Midtown, Soho, Tribeca, the Upper East Side, and the Upper West Side.

We also know that spending time in the city center often means walking between locations, so we tested while walking throughout the urban zone. And because performance inside buildings is so critical to both business and leisure time in the big city, we spend approximately 50% of our time testing at indoor locations throughout the urban zone. For a full view, we then supplement our testing with a drive study along urban streets. All told, we visited 277 indoor locations and collected more than 72,000 total tests. The end result is the most comprehensive look at Manhattan mobile performance available anywhere.

With multiple unique districts, an urban zone is a diverse space. Because performance can vary from location to location, we’ve included not just metrics for the urban zone as a whole but also highlights from various districts within the urban zone. Looking at the districts in which each carrier was fastest and slowest points to how performance can vary within the concentrated urban zone.

While Urban Zone results do not receive RootScores, we report on reliability and speed metrics to give you insights into how each network’s performance might impact your mobile life.

We found a hotly contested performance battle between the networks within Manhattan.

Speed insights



Bragging rights as Manhattan’s fastest network for download speed belong to T-Mobile. Verizon, however, wasn’t far behind. In fact, we found a sharp, two-tiered difference between the networks when comparing download speeds in Manhattan, with T-Mobile and Verizon clearly faster than AT&T and Sprint. In fact, T-Mobile and Verizon more than doubled the median download speeds recorded by AT&T and Sprint across all of our urban zone testing.

While Sprint was slower than T-Mobile and Verizon, its performance is a strong sign of continued network improvement. Indeed, Sprint outpaced AT&T for both median download and upload speeds. That said, all carriers offered both median download and upload speeds near or above 5 Mbps. Speeds above 5 Mbps, as our chart shows , provide a smooth experience for many consumer activities. In fact, data speed follows a law of diminishing returns: you need enough to do your daily mobile activities with ease but beyond that the extra speed does not necessarily have a dramatic impact on your experience.

For a more direct comparison of the speeds we found in Manhattan, we’ve included how long it would take to download a song to listen to while walking within the city. While downloading a song on T-Mobile’s network would be marginally faster than on Verizon’s network, all networks would take less than eight seconds to deliver a song to you.

time-needed-song

Reliability recap



reliability-testing

All networks performed excellently in our reliability testing. That’s great news for customers in Manhattan and suggests that your mobile life should move forward with minimal interruptions. Call failure rates, for instance, were low for all networks. Call performance in the urban zone would most likely meet the demands of nearly all consumers.

urban-zone-performance

We also had little to no trouble getting connected and then staying connected to each carrier’s data network. Data failures were incredibly rare during our web/app testing: we connected successfully nearly 100% of the time to each network and were then able to stay connected until our task was complete nearly 100% of the time as well.

In sum, with both strong data and call reliability from all networks, consumers should expect a smooth experience within Manhattan.

urban-reliability

Indoor versus outdoor differences

We know that time in the dense urban core often means time spent inside, whether in an office _globaling, a restaurant, or in an apartment. To give you a clear picture of mobile performance in the urban zone, we extensively test at indoor locations. Performance inside a _globaling can be tricky, where dense _globaling materials and architectural differences present challenges for coverage. Not surprisingly, whether looking at median download speed or blocked call rates across all of the urban zone, the networks all performed better in tests performed outside of buiuldings rather than inside.

indoor-outdoor

Variation within the urban zone

Finally, keep in mind that performance can vary widely within the urban zone itself. Differences in architecture, congestion, and provisioning between districts mean that speed and reliability aren’t uniform across the urban zone. To show this type of variation, we’ve identified where each carrier was fastest and slowest, along with where each carrier experienced the most and least blocked calls in our testing. As you can see, we found wide variation for each carrier at different locations within the urban zone itself. While performance within the urban zone was in general good for all networks, your mobile life can experience fluctuations as you move through the city.

urban-median-download

Takeaway

The tight race between the networks and strong performances from Sprint and T-Mobile in Manhattan are good to see but shouldn’t necessarily be surprising. While not typically a strong performer in our state or nation testing, T-Mobile has often performed well in our testing within major metro areas. Performance within big cities is often a top priority for networks, making the dense urban zone the frontline of the mobile performance battleground. With so many potential customers in such a condensed space, network investments are often directed to urban zone upgrades first.

In this sense, the close race we saw within Manhattan could be a predictor for greater competition across other areas moving forward. Verizon continues to offer excellent performance, but results from our look at Manhattan suggest that the other networks are ready to challenge Verizon for the top spot in our testing.

Performance within the dense urban zone is only one component of your mobile life in the big city. For a full view of your urban mobile experience, also see how the networks compared during our testing of transit stations, venues, campuses, and airports. Read on to see how your network fared.

Venue

Fan-plus-phone demands are changing the game

Our mobile phones have changed what it means when we go to a game, concert, or other big event. We want reliable mobile performance so that we can upload pictures to social media, share our experiences online, and perhaps even stay in touch with the office.

Stadiums are also taking advantage of mobile connectivity. At more and more venues, you can order food or watch replays of the action from your seat. In short, mobile networks have helped change the game at venues and events. Our unique “science meets spectator” approach shows you exactly how the networks are handling these new fan-plus-phone demands.

Your mobile life is built upon not just everyday usage but also the ability to stay connected with others through sharing special moments. Think about attending a big game (or a concert or anything else at a big venue). Chances are your phone is going to be an inextricable part of the event. After all, we don’t just watch the game while we’re at the stadium: we take pictures and want to upload them to social media, email them to friends, and maybe even send a quick video of the action to those who aren’t there. Strong mobile performance during a game, concert, or other event helps keep your mobile life connected with others.



Venues, however, can be incredibly complex areas for networks to cover. Congestion from crowds, dense construction materials, and even location can impact your mobile experience. Read on to see how the networks compared and how performance at New York basketball and baseball venues could impact your mobile life.

Quick hit: Verizon earns top honors at each venue

venue-winners

Speed insights



Verizon stood out when comparing speeds. At each venue, Verizon offered the fastest median download and upload speeds. That said, speeds were in general slower at the venues than what we found in the metro area itself. With a median download speed of 2.6 Mbps and a median upload speed of 4.1 Mbps, Citi Field was the most difficult venue for Verizon. That said, Citi Field proved a challenging environment for all networks, and Verizon still won our RootScore Award in this venue.

T-Mobile might have offered fast speeds in our testing of the Manhattan urban zone, but performance dropped considerably within each of the venues we tested. Barclays Center, in particular, gave T-Mobile trouble: the often speedy T-Mobile recorded a slow 1.0 Mbps for median download speed and a markedly slow 0.1 Mbps for median upload speed.

Sprint also recorded consistently slow speeds in our tests, with median download and upload speeds below 1.6 Mbps at each venue. With the exception of performance at Citi Field, AT&T was generally faster than Sprint and T-Mobile. AT&T’s speeds, though, always trailed those recorded by Verizon.

The slower speeds we saw in our venue testing can have a significant impact on your mobile life and make sharing a picture or video of your experience difficult. Consider, for example, the difference in time needed to upload a picture at the fastest and slowest speeds we recorded. Sending a 3 MB picture at the 9.2 Mbps median upload speed Verizon recorded at Yankee Stadium would take less than 4 seconds; sending that same picture at the slow 0.1 Mbps speed T-Mobile recorded at Barclays Center would take more than 4 minutes.

time-needed-picture

Reliability recap

reliability-testing

venue-report-card

Verizon’s reliability stood out in our venue testing. Verizon was, in fact, the only carrier to achieve a mark of Excellent at getting and staying connected at each venue. With strong reliability marks, Verizon customers likely could share their experience by uploading a picture at the big game with little trouble.

Sprint trailed the other networks when considering reliability at venues. Yankee Stadium was a particularly challenging venue for Sprint: we were able to get connected to Sprint’s network during only 40% of our tests.

A venue-to-venue comparison shows Madison Square Garden standing above the other venues when looking at reliability. Rates for both getting connected and staying connected were high for all networks at Madison Square Garden. With the exception of Sprint’s difficulties, reliability at Yankee Stadium was also strong.

Takeaway

Venues are among the most challenging areas of your urban mobile life. Slower speeds and lagging reliability often make sharing your mobile experience more difficult than in other urban locations. While Verizon won our RootScore Award at each venue, the other networks often trailed by a large margin. With reliability difficulties and slow speeds, Sprint’s RootScores were particularly low at Barclays Center, Citi Field, and Yankee Stadium.

Venues are only one component of your mobile life in an urban environment. For a full view of your urban mobile experience, also see how the networks compared during our testing of dense Manhattan, transit stations, campuses, and airports. Read on to see how your network fared.

Transit

Making connections at stations and terminals

Millions of people use the subway or rail to commute into and out of city centers each day for work. A mobile phone is often a commuter’s best friend, providing a connecting link to work or entertainment. While waiting in line at stations, commuters use their phones to catch up on email, read online news, look in on social media, and even check to see the updated arrival time for their route.

With so many commuters flowing in and out of stations, network congestion can create mobile performance issues. We know you want not just your commute but also your mobile experience to move smoothly without interruption or delay. Our testing of network within large terminals and stations shows what to expect.

Getting around in the city often means using rail service. Both rail and your mobile phone, in this sense, act as connecting links for your urban life. Congestion and dense _globaling materials in rail stations, however, can make finding good data service a challenge. Read on to see how the networks compared and how performance differences could impact your everyday mobile life.



Quick hit: Verizon and T-Mobile battle it out for bragging rights in our testing of transit stations



transit-winners

Speed insights

In the stations we tested, Verizon was the clear speed king. Verizon outpaced the other networks to record both the fastest median download and upload speeds at Penn Station. Verizon’s 26.0 Mbps median download speed at Penn Station was especially fast and more than doubled the 12.0 Mbps speed T-Mobile recorded. In addition, Verizon had the fastest median upload speed at Grand Central Station and tied with T-Mobile for the fastest median download speed.

At the other end of the spectrum, Sprint’s speeds were markedly slow and never surpassed 1.0 Mbps for either median download or upload speed. Perhaps not surprisingly, Sprint’s LTE footprint at each station was small. Less than 20% of our download tests connected to Sprint’s LTE network at Penn Station, while less than 40% of download tests accessed its LTE network at Grand Central Station. As Sprint expands its LTE footprint in these transit stations, we expect speeds to increase. For now, however, speeds were slow and, as we show below, could hamper your daily mobile activities.

The other interesting storyline is the difference we found between the stations themselves. Speeds were, in general, slower at Grand Central Station than at Penn Station. Consider the drop for speed leaders Verizon and T-Mobile. Verizon recorded a fast 26.0 Mbps median download speed at Penn Station but fell to 7.2 Mbps at Grand Central. T-Mobile, meanwhile, fell from 12.0 Mbps median download at Penn Station to 4.2 Mbps at Grand Central. Differences in architecture, provisioning, and congestion can all impact performance, and as our data shows, your experience can vary widely between stations.

If good mobile service is available, waiting at a station for your train to arrive doesn’t have to be wasted time. Good speeds, for instance, let you download songs to listen to during the commute home. To show how the variations in network speed can impact your mobile life, consider how long it would take to download a song for each carrier at each station. At the top end of the spectrum, Verizon’s 26.0 Mbps median download speed at Penn Station would let you download a song in just over 2 seconds. It would take you more than a minute to download the same song at the 0.6 Mbps speed we recorded for Sprint at Grand Central Station.

time-needed-song-transit

Reliability recap

reliability-testing

The big story is that no network performed well enough to earn a mark of excellence in our reliability testing at Penn Station or Grand Central Station. That said, reliability was much better for all networks at Penn Station than at Grand Central Station. Indeed, at Penn Station, all of the networks narrowly missed our threshold for excellent performance; at Grand Central Station, in contrast, the networks often fell significantly short in our reliability testing.

Normally a leader in our reliability testing, Verizon’s results were uncharacteristically poor at Grand Central Station. We had trouble both getting connected and staying connected to Verizon’s data network at Grand Central Station. Although Verizon was the fastest network at Grand Central Station, its reliability issues weighed down its RootScore and allowed T-Mobile to win the Overall RootScore Award for Grand Central Station.

transit-report-card

Takeaway

Mobile connectivity is an important part of traveling through the city. Strong mobile performance in rail and subway stations keeps your mobile life on track. In general, performance and scoring was lower in the stations we tested than in the broader New York City area.

For a full view of your urban mobile experience, also see how the networks compared during our testing of the Manhattan urban zone, venues, campuses, and airports. Read on to see how your network fared.

Campus

Learning which networks make the grade

Good mobile performance is a key element in today’s college experience. Mobile is an integral part of both learning time and time away from the classroom. Whether connecting to the Internet to do research for an assignment or connecting with friends on social media, mobile network performance is vital to activities on campus.

Large campuses, in fact, often rival small cities not just in size but also in mobile traffic. Our network performance testing shows which networks make the grade and which have more work to do.

Fast speeds and strong reliability from mobile networks are helping change the way students learn. We compared performance to see which networks provided the best mobile experience at select campuses within New York City.



Quick hit: Sprint, T-Mobile, and Verizon bring home the most first place finishes

campus-winners

Speed insights

T-Mobile recorded the fastest speeds, registering over 20 Mbps for both median download and upload at Columbia University. Verizon wasn’t far behind, however, recording a 15.4 Mbps median download speed at Columbia. Sprint and AT&T both recorded solid median download speed but Sprint’s 0.9 Mbps median upload speed at Columbia was markedly slow.

The speed landscape looked different at Fordham University, where AT&T and T-Mobile were both much slower than at Columbia. Verizon again offered a strong performance and recorded the fastest median download speed (12.4 Mbps) seen at Fordham. Sprint's 9.7 Mbps median upload speed, meanwhile, was much faster than what we saw at Columbia.

In sum, while T-Mobile offered the fastest overall speed, Sprint deserves a nod for performance at Fordham and Verizon was the most consistent network, recording fast speeds at both universities.

These differences in speed impact your mobile experience. Consider how long it would take to upload a photo (3 MB) to share. At the 20.6 Mbps median upload speed T-Mobile recorded at Fordham University, your photo would take less than 2 seconds to send. If you were to send the same picture at the speed Sprint recorded at Columbia, however, it would take more than 27 seconds for your picture to upload.

time-needed-picture-campus

Reliability recap

Sprint, T-Mobile, and Verizon shared first-place honors in our Network Reliability and Call Performance testing at Columbia University. At Fordham, Sprint and Verizon once more topped the Network Reliability and Call Performance rankings, while T-Mobile fell into a tie for third with AT&T in each category.

In general, both call and data reliability were good at both universities. Data reliability met our threshold for excellence in every instance except one: we had trouble getting connected to T-Mobile's network in our testing at Fordham University.

Sprint and Verizon deserve special mention for consistently excellent call performance in our testing. Sprint, in fact, recorded zero drops or blocks in our call testing at Fordham University. At Columbia, Sprint was again perfect when looking at blocked calls and backed that up with a very low 0.6% failure rate for dropped calls.

T-Mobile’s call performance was similarly excellent at Columbia but its blocked call rate was the highest of all networks at Fordham. Blocked call rates were higher for AT&T at both universities.

Keep in mind, however, that even the relatively “high” call failure rates seen by AT&T and T-Mobile still only represent 3 out of every 100 calls being impacted. In short, call performance is most likely good enough for nearly all customers, but there are slight distinctions between the networks for those who demand the highest possible service.

campus-report-card-call

reliability-testing-call

Reliability during our web/app testing was fantastic. We were able to get connected and stay connected with ease in the vast majority of tests for each network. The only slight bump was getting connected to T-Mobile’s network at Fordham University, where T-Mobile missed a mark of Excellent by a narrow margin.

campus-report-card

Takeaway

With college attendance figures that can at times rival smaller towns, networks face challenges provisioning for these key areas. All carriers provided generally acceptable service at Columbia University and Fordham University, offering performance to keep your mobile life on campus moving smoothly.

Campus performance is only one component of your mobile life in an urban environment. For a full view of your urban mobile experience, also see how the networks compared during our testing of dense Manhattan, transit stations, venues, and airports. Read on to see how your network fared.

Airport

Taking flight with first-class mobile performance

Airline delays aren’t the only slowdowns that can hamper your travel experience. Take a look around any airport and you’ll see people focused on their mobile phones. We use them for everything from downloading boarding passes to online entertainment to fight travel boredom.

With thousands of people potentially trying to access network resources at the same time, dense construction materials, and even some areas at times located underground, airports present unique challenges for mobile coverage. Our in-depth testing of data performance at airports shows you which networks and airports offer first-class service.

With mobile usage restricted during flights, time spent in the terminal often means a rush to catch up on email, download a show for watching on the flight, or sending a last second status update to social media. Given the importance of mobile at airports, flight delays aren’t the only slowdown that hampers your travel experience; lagging mobile internet performance can also act as a significant pain point. Read on to see how the networks compared and what you can expect at Newark International Airport (EWR), John F. Kennedy International Airport (JFK), and LaGuardia Airport (LGA).



Quick hit: T-Mobile wins two, AT&T wins one, while Sprint and Verizon are shut out

airport-winners

Speed insights

Newark International Airport (EWR) was home to the fastest median download speed for any area airport, with Verizon recording 25.9 Mbps. Newark also earned distinction as the airport with the fastest median upload speeds (T-Mobile, 16.7 Mbps and Verizon, 16.2 Mbps). T-Mobile’s performance at Newark was representative of a broader trend we saw for the network. T-Mobile showed strength across all New York area airports for median upload speeds and never dropped below 7.7 Mbps.

AT&T’s speeds were fast at each airport, but the network particularly shined at LaGuardia (LGA), recording an airport best 15.1 Mbps for median download speed. LaGuardia was also Sprint’s fastest airport: Sprint recorded a fast 9.3 Mbps for median download speed. It was a different story for typically speedy T-Mobile and Verizon at LaGuardia. While T-Mobile recorded a quick 7.7 Mbps for median upload, its 3.4 Mbps for median download was much slower. Verizon, meanwhile, was slower at LaGuardia than at any other area airport and recorded an uncharacteristically slow 0.7 Mbps median download speed. With such slow speeds, Verizon finished in last place at LaGuardia.

These differences in speeds can have a significant, real-world impact on your travel experience. Consider it in terms of a common traveler activity like downloading the latest 45-minute episode of your favorite show in HD (600MB) to watch during your flight. At the speeds Verizon recorded at Newark (EWR) and JFK, your download would be complete in plenty of time before boarding. At the other end of the spectrum, your download could take nearly 2 hours on Verizon’s network at LaGuardia (LGA).



time-needed-episode

Reliability recap

reliability-testing-call

Reliability was excellent at all New York City area airports. AT&T deserves special mention. With zero data failures, AT&T easily achieved a mark of excellence for both getting connected and staying connected at all three airports. With only a small handful of failures, Verizon also earned a mark of excellence at all three airports. When Sprint and T-Mobile did not achieve Excellent status, they missed the threshold by a narrow margin. In short, your airport experience should be a smooth overall.

airport-report-card

Takeaway

T-Mobile won more Airport RootScore Awards in the New York area than any other network, but each carrier performed well overall and had its own highlights. AT&T took the RootScore Award at LaGuardia, while Sprint’s speeds at LaGuardia show promise moving forward. Reliability was strong for each network, with AT&T and Verizon especially standing out.

Airports are only one component of your mobile life in an urban environment. For a full view of your urban mobile experience, also see how the networks compared during our testing of dense Manhattan, transit stations, venues, and campuses. Read on to see how your network fared.