Glasnost: Results from tests for BitTorrent traffic shaping

Over the last three years, a large number of users from locations around the world have used our tool, Glasnost, to test whether their BitTorrent traffic is being shaped by ISPs. On this page, we present results from the tests conducted between January 1st, 2011 and November 5th, 2011.

The source code of our tool has been publicly accessible for more than 3 years now. You are welcome to download and inspect the code. Please contact us if you find any bugs or have questions, comments, or suggestions.

We published a technical paper describing our data collection and analysis methodology in the 7th Usenix Symposium on Networked Systems Design and Implementation (NSDI 2010). The paper explains in detail the potential sources of measurement errors and the checks we apply to minimize their impact on our results. You can download the paper in pdf format here.

Links to older releases of Glasnost results can be found here: February 1st, 2009, November 9th, 2008, and July 25th, 2008.

1. How accurately can users detect BitTorrent traffic shaping by ISPs?

We designed Glasnost to enable individual users to detect if their BitTorrent traffic is being shaped by ISPs. We can control the accuracy with which users can detect traffic shaping. But, in the process, we have to make some hard trade-offs between false positives, where the test indicates traffic shaping by ISPs even when they are not, and false negatives, where the test fails to detect traffic shaping even when ISPs are shaping traffic.

For details on how we can choose the trade-offs, please follow this link.

In brief, we can set the controls to detect any traffic shaping that reduces BitTorrent bandwidths by more than 20% with a 4% chance of false positives. Alternately, we could configure the controls to detect any traffic shaping that reduces BitTorrent bandwidths by more than 50% (higher false negatives) with less than 1% chance of false positives (lower false positives).

2. What can we infer about traffic shaping policies of ISPs?

Glasnost has been designed to enable individual users to detect if their traffic is being shaped by ISPs. Glasnost data from individual user tests can be used to infer if an ISP is deploying traffic shaping for at least some of its customers — by checking if the percentage of Glasnost tests for an ISP that indicate traffic shaping far exceeds the chance of false positives. However, Glasnost data cannot be used to accurately infer how widely traffic shaping is deployed within an ISP, i.e., the percentage of the ISP's users who are affected by traffic shaping.

For more details on how we can infer shaping policies of ISPs, please follow this link.

3. Are ISPs shaping BitTorrent traffic?

We present our analysis results for BitTorrent traffic shaping by ISPs world-wide in the table below. The table shows, for each ISP, the number of Glasnost tests ran by the ISP's customers as well the percentage of tests for which we detected traffic shaping of (a) BitTorrent uploads on any random port, (b) any TCP uploads on BitTorrent ports, (c) BitTorrent downloads on any random port, and (d) any TCP downloads on BitTorrent ports.

Note 1: When trying to infer whether an ISP is shaping traffic, it is important to compare the percentage of tests that detected traffic shaping with the chance of false positives. The greater the difference between the percentage of shaping detected tests for an ISP and the false positive rate, the more likely it is that the ISP is deploying shaping for at least some of its customers.

Note 2: To obtain the results, we configured our analysis such that there are few false positives (estimated to be less than 1%), but we fail to detect any traffic shaping that affects BitTorrent bandwidths by less than 50%.

Note 3: Our estimate of false positives, derived in our NSDI 2010 paper, is based on measurements conducted over wired (cable / DSL) access networks. The false positive rate might be higher for wireless access networks like WiMax or Cellular broadband networks. For such networks, it is better to allow for a higher false postive rate when inferring the presence of traffic shaping.

Note 4: We aggregated the results for different autonomous systems (ASs) owned by an ISP. For example, Tiscali UK and OPALTELECOM are two different ASs owned by TalkTalk. So we merged the results for user tests from both ASs under TalkTalk.

Country ISP # tests Percentage of traffic shaped (chance of false positives: 1%) BitTorrent uploads Uploads on BitTorrent port BitTorrent downloads Downloads on BitTorrent port Argentina Cablevision 125 0 0 0 0 Argentina Telecentro 102 1 50 5 3 Argentina Telecom Argentina 156 0 1 1 4 Argentina Telefonica de Argentina 138 0 1 0 0 Australia Dodo 132 0 0 1 2 Australia Exetel 104 0 3 1 0 Australia iiNet 272 0 0 2 8 Australia Internode 230 1 1 1 4 Australia Optus 748 1 1 22 24 Australia Telstra 747 1 4 1 3 Australia TPG Internet 381 0 0 1 3 Australia Westnet 120 0 0 5 7 Belgium Belgacom 136 0 0 0 0 Belgium Telenet 647 16 17 1 1 Brazil Canbras Net 223 15 15 2 6 Brazil Embratel 115 4 13 5 8 Brazil Global Village Telecom 801 0 1 1 4 Brazil Hi 1510 1 1 2 26 Brazil NET Servicos de Comunicao 1142 16 15 2 4 Brazil NTT America 31938 3 6 5 8 Brazil Telecomunicacoes do Brasil 145 0 0 0 0 Brazil Telesc 1035 0 0 2 9 Brazil Telesp 866 0 1 2 3 Canada Bell Aliant Regional Communications 261 0 1 1 2 Canada Bell Canada 1031 41 39 20 21 Canada Canaca-com 106 24 25 12 13 Canada Cogeco 264 1 0 1 0 Canada Distributel 143 6 6 4 6 Canada EastLink 267 4 3 2 1 Canada MTS Allstream 108 0 0 0 0 Canada Rogers 2713 2 28 2 3 Canada Shaw 1740 12 18 0 1 Canada TekSavvy 333 24 26 15 15 Canada Telus 462 1 0 1 1 Canada Videotron Telecom 206 0 0 0 0 Chile Moviestar 111 0 0 1 3 Chile VTR Banda Ancha 401 0 17 1 3 Croatia T-Com Croatia 121 0 0 1 0 Czech Republic Telefonica o2 133 0 0 1 1 Denmark TDC Data Networks 155 0 0 0 0 Egypt TEData 293 1 8 1 0 Estonia Elion Enterprises 103 0 0 0 0 Estonia Elisa Eesti 178 4 85 1 0 Finland Elisa 175 0 1 0 0 Finland Telia Sonera Finland 123 1 0 0 0 France France Telecom – Orange 306 0 0 0 1 France Free 386 0 0 0 0 France Numericable 106 0 0 0 0 France Societe Francaise du Radiotelephone 221 6 9 1 1 Germany Deutsche Telekom 473 0 0 0 0 Germany Kabel Deutschland 393 1 18 0 27 Germany Telefonica o2 447 0 0 1 0 Germany Vodafone 167 1 2 0 1 Greece Forthnet 144 0 0 0 0 Greece Hellas OnLine 286 0 0 0 1 Greece Ote 423 0 0 1 0 Hong Kong PCCW 167 10 11 16 22 Hungary Digicable 104 2 4 0 2 Hungary Magyar Telekom 271 0 0 0 0 Hungary UPC Broadband 1617 1 0 1 3 India Bharti Airtel 1311 0 1 1 6 India Mahanagar Telephone Nigam 317 0 0 0 0 India National Internet Backbone 689 0 0 1 1 India Reliance Communications 103 0 0 1 1 India Tata Communications 119 0 0 1 5 Indonesia Telekomunikasi Indonesia 191 0 2 2 1 Ireland Eircom 103 1 1 0 0 Israel 013 NetVision 160 19 20 18 22 Israel Bezeqint 474 18 22 17 29 Israel Smile Communications 185 0 9 6 6 Italy Fastweb 483 0 2 1 0 Italy Opitel 326 1 10 4 4 Italy Telecom Italia 2786 0 0 0 0 Italy Tiscali Italia 745 0 0 9 10 Italy WIND 1264 0 0 1 1 Japan NTT 471 2 26 1 20 Japan Softbank 136 1 4 1 4 Japan Technology Networks (@NetHome) 172 4 79 1 8 Japan Vectant 140 0 3 0 0 Lithuania TEO LT 135 0 1 0 1 Malaysia TM Net 579 1 23 3 7 Mexico Uninet 226 0 0 0 1 Netherlands KPN 217 0 0 0 0 Netherlands T-Mobile 108 0 0 0 0 Netherlands Telfort 140 1 0 0 0 Netherlands Ziggo 377 0 0 1 1 New Zealand CallPlus 335 0 8 0 1 New Zealand Telecom New Zealand 162 1 2 1 1 New Zealand TelstraClear 109 0 0 1 0 New Zealand Vodafone NZ 129 0 0 4 1 Norway Telenor 324 0 1 0 0 Philippines Globe Telecoms 136 0 1 2 1 Philippines Smart Broadband 431 3 2 2 3 Poland Netia 177 0 1 0 1 Poland Telekomunikacja Polska 447 0 1 0 1 Poland Toya 131 3 73 0 0 Poland UPC Polska 315 18 18 0 1 Portugal Cabovisao 102 20 15 15 21 Portugal Optimus 120 0 0 0 0 Portugal Telepac 303 1 2 2 2 Portugal ZON TV 1482 0 15 1 1 Romania RCS & RDS 321 0 0 0 0 Romania Romtelecom 115 0 0 0 0 Romania UPC Romania 143 0 0 0 1 Russian Federation Beeline 156 0 1 0 2 Singapore SingNet 416 3 3 22 59 Singapore StarHub 515 8 13 6 10 South Africa Internet Solutions 289 1 9 3 7 Spain Jazz Telecom 198 0 0 0 0 Spain ONO 456 0 0 1 2 Spain Orange Espana 328 0 0 1 0 Spain Telefonica 683 0 0 0 1 Sweden Com Hem Sweden 337 0 0 1 20 Sweden Tele2 126 0 2 3 0 Sweden Telia Sonera 177 0 0 0 0 Switzerland UPC Cablecom 137 0 0 0 1 Taiwan Chunghwa Telecom 1529 0 2 2 5 Taiwan Digital United 682 8 24 1 2 Taiwan Hoshin Multimedia Center 170 3 2 2 6 Taiwan Sony Network Taiwan 148 0 2 0 2 Taiwan Taiwan Fixed Network 387 3 43 5 7 Thailand TOT 225 2 4 6 25 Thailand True Internet 220 5 10 17 45 Trinidad and Tobago Columbus Communications 138 2 3 6 12 Turkey Turk Telekomunikasyon 277 0 0 1 2 United Arab Emirates Du 196 2 69 1 49 United Arab Emirates Etisalat 170 0 1 7 2 United Kingdom BE 1105 4 7 1 2 United Kingdom British Telecom (BT) 2852 16 17 56 61 United Kingdom Easynet 752 0 0 0 0 United Kingdom Eclipse Internet 111 0 0 0 1 United Kingdom PlusNet 181 0 0 1 0 United Kingdom TalkTalk 2861 9 17 13 22 United Kingdom Virgin Broadband 2938 14 22 15 32 United States AT&T 3841 0 0 0 1 United States Cable One 132 0 0 0 0 United States Cablevision 769 0 0 1 0 United States CenturyLink 1071 0 0 0 1 United States CenturyTel Internet 111 0 0 2 0 United States Charter Internet 1740 0 0 0 0 United States Cincinnati Bell 103 0 0 2 0 United States Clearwire 192 4 4 3 1 United States Comcast 9477 0 0 1 1 United States Cox 2116 0 0 0 0 United States Frontier Communications of America 123 0 0 1 0 United States Insight Communications 189 0 0 1 0 United States Midcontinent Media 102 0 0 0 0 United States RCN Corporation 246 0 0 1 0 United States RoadRunner 5549 0 0 0 0 United States Suddenlink Communications 296 0 0 1 0 United States Verizon 2055 0 1 1 0 United States Windstream Communications 182 0 0 1 1 United States WOW 197 0 0 0 0 Vietnam VNPT 124 0 0 3 1

4. Where can we download the raw data and analysis scripts?

You can download the raw data collected during Glasnost tests from Measurement Lab. Please follow the instructions on the page.

The tools that we used to analyze the data can be downloaded from here.

We encourage you to analyze the data and inspect our scripts. If you have any comments, questions, or suggestions, we welcome you to contact us.

5. Inferring shaping policies using data cited in New York Times article on Glasnost

Glasnost project was featured in a New York Times article on Novermber 14th, 2011. The article cited the percentage of Glasnost tests for which we detected traffic shaping for ISPs world-wide. When inferring whether an ISP is deploying traffic shaping, it is important to view those percentages in the context of the potential measurement errors, all of which were not emphasized sufficiently in the article. In particular, we wish to emphasize the following:

Note 1: The results cited in the article were obtained using an analysis configuration that detects any traffic shaping that reduces bandwidth by more than 20% (compared to 50% in the table above). With this more aggressive configuration, the chance of false positives are higher, around 4% (compared to 1% in the table above).

Note 2: More importantly, the article cited aggregated results from 4 different BitTorrent tests (BitTorrent uploads, BitTorrent downloads, TCP uploads on BitTorrent ports, and TCP downloads on BitTorrent ports), each of which has an measurement error of around 4%. That is, a user is reported to have observed traffic shaping, even if any one of the four BitTorrent tests she runs detects shaping. So the resulting measurement error in the aggregated results could be as high as 4 x 4% = 16%.

Note 3: When mapping AS names to ISPs, the analysis used an older (2008) version of the mapping table from IANA.

You can download an xls file containing the results cited by the New York Times article here.

As the aggregated results (column H in the .xls file) have a high chance (16%) of potential false positives, it is hard to infer with high confidence that any ISP for which the percentage of users that detected shaping is smaller than 32-48% (a factor of 2 to 3 larger than false positive rate) is actually shaping traffic. A more accurate inference can be drawn from the results of individual BitTorrent tests (columns C through G in the .xls file) which have a significantly lower chance of measurement error at 4%. To further improve the accuracy of inference, we recommend using results presented in the table above, where the measurement error is estimated to be lower than 1%.

For example, for US ISPs like AT&T and Verizon the percentages of tests that were reported to have detected traffic shaping fall within the range of false positives. Results presented in the table above (with tighter bounds on false positives) shows more clearly that there is no evidence of traffic shaping by these US ISPs.

On the other hand, the article correctly reports that BitTorrent traffic shaping appears to be more common in certain European countries like Britain. The percentage of tests that detected traffic shaping are so high (70% or higher) for some European ISPs that they exceed the chance of being accounted for by false positives (16%) alone. In fact, results presented in the table above (with tighter bounds on false positives) also show that some European ISPs are likely deploying BitTorrent traffic shaping.

This page is part of the research on residential broadband networks at the Max Planck Institute for Software Systems.

The results presented on this page were compiled by Marcel Dischinger and Krishna P. Gummadi on 28. November 2011 as part of the Glasnost project.

If you have questions or feedback, you can contact us via e-mail: