Windows 10 Feature Updates – The Challenge of Servicing in the Enterprise

I hope I never have to build another Task Sequence again. Seriously. Wouldn’t that be nice? I know that’s not truly feasible, but can’t a guy dream? It’s not that I dislike Task Sequences, it’s just that I strongly believe that Modern Desktop Management (AutoPilot/Device Reset/Feature Update servicing/etc.) is going to continue to improve and we should continue to attempt to adopt these tools with as much effort as we spend on customizing Task Sequences.

I continue to have conversations with people who tried Feature Update servicing and had a bad experience so they went back to Task Sequences because they are easier to manage. I totally understand the feeling, we even did the same thing in our organization. I also understand that there are plenty of organizations that simply cannot use Feature Update servicing due to many factors including bandwidth, infrastructure and 3rd party software (which are very valid reasons). I’m hoping to challenge you to reconsider your opinion on Feature Update servicing without Task Sequences and give them another go. I’m still learning and Microsoft is still improving, so I can only hope that this becomes easier to adopt as time goes by.

The Enterprise Challenge

My organization, along with many others, are just coming to the Feature Update table after dealing with a slow and agonizing migration off of Windows 7, while others have been managing Windows 10 for several releases now and dealing with the challenges involved with ensuring their clients are on a supported Windows 10 version. I’ve been watching many others deal with the many missteps and challenges that were prevalent in early Windows 10 releases and feel like this is one of those times where being late to the game may actually save have saved us some headaches.

SCCM/MDT Task Sequences continue to be the recommended approach for Feature Updates for several reasons, including flexibility, user visibility, reporting and more. However, my hope is that there are organizations with simple requirements that can benefit from Windows 10 Servicing without Task Sequences. There are several obvious areas that make servicing of Feature Updates more difficult to adopt than Task Sequences but I if you stick with me, I’ll share what we are doing and what’s working.

Before continuing, I’ll mention that I’ve recently made some contacts at Microsoft who are eager to hear feedback on the process and help improve things. I’m hopeful that we will see this space continue to get better!

Some of the obvious challenges I’ve seen with Feature Update servicing include:

Limited documentation from Microsoft – There a just a few pages in the docs for Windows 10 that cover options for managing Feature Updates via servicing. The examples are very high level and basic and since most people are using Task Sequences, it has been hard to find (m)any sources to help navigate enterprise management of Windows 10 Features Update servicing.

Reporting capability is lacking when compare to Task Sequences

Poor reliability track record – previous releases have stumbled and many people who tried Feature Update servicing previously have given up and gone with Task Sequences instead. As of 1803, Custom Actions were added, which many people haven’t looked into.

Dynamic Updates – We have been fighting to stay on top of Dynamic Updates for the past year or so and now they have been moved out of WSUS and are pulled directly from the internet – this just doesn’t work for many orgs with bandwidth limitations.

Features On Demand and Language Packs – Some people have seen where they don’t get preserved during Feature Updates and are somewhat difficult to manage.

For the above reasons alone, the safe bet is Task Sequences for Feature Updates but I also know that the time I’ve spent on building and managing Task Sequences compared to the work required for Feature Updates is enough motivation to give up on one of those things and move forward.

Real World Implementation

We just spent the past 6 months upgrading from Windows 7 to Windows 10 1803 (1803 because 1809 kept having issues initially and 1803 was a huge improvement over 1709). We knew that we’d be up against the 18-month support window for 1803, so as soon as we could, we began working on a plan for 1809. As we began testing Feature Updates, we ran into 4 main challenges that I built workable solutions around:

Non-Blocking Driver & App Compatibility Blocks

802.1x/NAC whitelisting

Error Handling

Logs/Reporting

The solution and scripts provided in this series are copies of what we deployed to our enterprise. We deployed the update as available for about 2 weeks with a required deployment set for a Thursday night, just before a holiday weekend (maybe could have had a bit better timing). Within a week of the update deadlining over 90% of our 1803 devices have been upgraded to 1809. As we began looking at the remaining devices, we found a few devices that had upgraded, but SCCM/WSUS weren’t updating the status and kept triggering a reboot every 1.5hrs. These were easily identified and remediated by resetting SCCM client policies. We didn’t encounter a single instance of a user’s machine failing outright or even rolling back due to an error.

Note: My organization doesn’t support additional language packs and we haven’t had to deal with major driver issues, so I haven’t spent any time working with them at this point, though I likely will do some testing in the future.

A Bit About Application Compatibility

One of the major areas that causes organizations issues is application compatibility. We have been leveraging Upgrade Readiness and now Desktop Analytics to manage our upgrades since we began our Windows 7 to Windows 10 migration. It gave us the insight into our application compatibility which greatly reduced our application testing efforts and allowed us to rapidly move to 1809 with no application issues.

I don’t mean to imply that we don’t have any application compatibility issues. We do. However, we were able to use these amazing tools to help drive our testing and remediation efforts and most importantly, identify devices which SHOULD NOT be upgraded yet. This is why we have been able to user Feature Update servicing with confidence. In fact, we built up our confidence in Upgrade Readiness while we were building our 7 to 10 Task Sequences and were able to In-Place upgrade over 50% of our environment (the remainder were hardware lifecycle replacements) without having to perform any device-based pre-assessments.

By shining a light on Application Compatibility, we have been able to build up confidence in the toolset and worry less about applications causing issues with deployments.

A Note About Drivers

Since I started this post, I have done some basic driver analysis and found that existing drivers from 1803 were migrated to 1809 without issue. If you know that you will have driver issues for specific models, you should either pre-upgrade the drivers if possible OR pre-stage the new drivers and add a line to setupconfig.ini to reference the staged drivers. You just need to get the drivers onto a known location on the device before the feature update AND point setupconfig.ini to that location.

Marius Wyss, a Microsoft PFE, built a solution for managing his setupconfig.ini using a silent Task Sequence. It has great examples for how to manage Drivers and Language Packs. I’m not saying that you need to use a Task Sequence to manage it, just want to share ONE possible option, which is pretty clever. You can find more info here: https://github.com/MrWyss-MSFT/PrepareIPUTaskSequence

The Enterprise Solution

We identified 4 main hurdles to overcome when deploying Feature Updates so I built a set of scripts and other components to address each of them. Combining them, we now have a relatively robust solution that has streamlined our Feature Update servicing. I built each piece to be as modular as possible, so only use what’s required for your organization. The list below matches up the challenge with a detailed post about that topic and associated scripts.

I will also included a guide in the repo (soon-ish) for how to manage the included scripts if they aren’t intuitive. At a minimum, to get started with Feature Update servicing, you should deploy Setupconfig.ini to your clients with the Compat option set to Ignore Dismissable Warnings. This setting alone will reduce your Feature Update issues greatly. Everything else just adds nice-to-have features to be more like the Task Sequence experience.

[Setupconfig] Compat=IgnoreWarning

Looking Back

It has been over a month since I built the solutions above (and wrote the post honestly). In that time, I’ve had time to fine tune some things in our environment and assess the effectiveness/necessity of the toolset. (I have some updates to the scripts that I need to publish to the repo – nothing major, just a few minor bug fixes/tweaks. You can find everything here – if you Watch the repo, you will get alerted when I publish updated scripts.).

Currently, we run the copy logs routine for success and failure. I think I may update it to only copy logs on failure since I don’t think I even looked at any logs that were copied.

I debated whether running setupdiag, whether or not there was a failure, and I am glad that I did. I’ve actually added a baseline to SCCM now that will run setupdiag on any machine that hasn’t run it (it doesn’t report anything for new builds, just upgrades). This gives me a more complete picture of deployment when we run reports.

I’m not entirely happy with how each solution needs to be managed. I need to combine them a bit more to make changes easier to handle. I know, I should just use a Task Sequence right???

Drivers – I still need to figure out how I want to deal with these. I plan to add some things for managing them for the next feature update (I hope).

Summary

I hope that you are inspired to give Feature Updates another go. I know I’ve covered a lot of ground in these posts, but there is even more that needs to be said and plenty of room for Microsoft to close the gaps needed to make Feature Updates more accessible to everyone.