Yesterday, I was testing out some privacy settings in Windows and ran across a bunch of stuff that concerned me. So I ranted about it on Twitter (see thread):

Now those who know me know that I rant about this stuff all the time. Except this time it got quite a few retweets than usual and a lot of others ran with it. As a number of people pointed out, there were some problems with how I applied a couple of the group policy settings. And looking back, I can’t even say I’m 100% sure if I rebooted after applying the settings (although I did do a gpupdate). So no, this was by no means a clean test. It wasn’t meant to be a published finding, it was a Twitter rant.

Not all the criticism towards how I set my settings was valid but I’m not going to bother addressing that. Instead, I ran more formal tests in a controlled environment to get more accurate results.

But first let me explain that I have been using Windows exclusively on my desktop for more than twenty-five years. In the early 90’s I did Windows tech support for a major computer company. In the late 90’s I worked for a software company as Director of Microsoft-Based Development. I wrote a column for SecurityFocus.com on Windows security. I have written for Windows IT Pro Magazine, Redmond Magazine, Windows Web Solutions, Windows Secrets and others. I also wrote a book on ASP.NET security. Microsoft awarded me with the Most Valuable Professional (MVP) award seven times. Windows is kinda my thing.

But that thing changed with Windows 10. A shift in Microsoft’s philosophy has lead to a massive collection of data from Windows computers. For me, it’s not only a privacy issue but a security issue. — it’s hard to control what is happening on your computer when you aren’t in control.

But back to my tests. As I mentioned before there were too many variables I had in my more casual tests and I was a bit sloppy with some settings so I started with a clean build. This is what I did:

Installed the OS (Windows 10 Enterprise Build 15063) in a VirtualBox virtual machine (CentOS host) with no network adapter. Installed virtualbox client extensions. Applied the Windows Restricted Traffic Limited Functionality Baseline that Microsoft publishes (more info). Manually uninstalled Solitaire and Feedback Hub, the only apps left Windows would let me uninstall. Shut down the virtual machine. Added NIC tracing in VirtualBox using this command:

vboxmanage modifyvm “Win10ETest” — nictrace1 on — nictracefile1 windows.pcap Enabled the NIC. Started the virtual machine. Logged in. Pinged 8.8.8.8 to verify network connectivity. Let it sit untouched overnight.

To save you all the suspense, yes this test resulted in much less activity than my initial test (put away the pitchforks). Less, but still too much (get out the pitchforks).

What was the difference? The main difference is that the baseline sets many more settings than I did in my test. Another part of it surely was the fact that I did not set all of the settings I thought I had set. For example, I only set two settings for disabling SmartScreen, instead of considering all of these:

For the record, I don’t recommend disabling SmartScreen.

Of course, you don’t need to set all of those, there is some overlap, I’m pretty sure you only need to set 2–5.

And several people noted that I had set the Allow Telemetry policy incorrectly. Now this was just sloppiness on my part and totally my mistake, but you can see how others might find it easy to get confused with the incorrect way to disable telemetry (enable the policy and then disable below, if you scroll down it in the dialog box it explains this):

The wrong way to do it

And yet compare that with the correct way to disable SmartScreen (this time set the policy to disabled, ignore the box below):