The following was shared as a twitter thread. Here’s the medium version.

Screenshot from twitter (by author)

1/ Apple has a software problem. Here’s how it plans to fix it. via @markgurman // Let’s take a step back and talk about the broader context and product development at scale. Lots follows… via Mark Gurman Bloomberg

2/ Several important points are conflated in the broad discussion about Apple and software:

• Quality

• Pace of change

• Features “versus” Quality

• Innovation

3/ Scanning the landscape, it is important to recognize that in total the work Apple has been doing across hardware, software, services, and even AI/ML — in total — is breathtaking and unprecedented in scope, scale, and quality. Not saying that lightly or trolling. It just is.

4/ Few companies have done so much for so long with such a high level of consistency. This all goes back to the bet on the NeXT code base and move to Intel for Mac OS plus the iPod, which began the journey to where we are today.

5/ The pace of change has been remarkable. In the 10 years from when Apple acquired NeXT OS X was reinvented in a completely modern architecture. And in the next 10 years the iPhone went from that code to where we are today.

6/ One must consider in those 20yrs there were releases every 12–18 months *for the entire time*. While some were larger/smaller, there wasn’t much comparable anywhere and certainly nothing without some big gaps. There were some massive architectural bets playing out over years.

7/ Microsoft Office released ever 18–30 months from about 1990–2010 but had a shaky start so you could say that from 1995 it kept that cadence but today puts out mostly modest visible changes to be SaaS like.

8/ The pace, scope, and quality of change is unprecedented in the industry. That’s a debt to the whole team, especially to the leadership like Jobs and Forstall (and the people in place now who were there). Apple was disrupting the PC market with a unique POV but no one knew.

9/ The only comparable project would be IBM System/360 — the creation of IBM 360 hardware and software. The PC of course, but the scale (even at the time) was much less. Windows NT clearly had the scope/scale of software but had been done before (VMS) and built on PC h/w.

10/ A fantastic read on history of the IBM 360 can be found here — a must read for anyone thinking they are creating a lot of new stuff at scale. IBM invented so much. Outstanding book! smile.amazon.com/IBMs-Early-Sys…

11/ What is lost in all of this recent discussion is the nuance between features, schedule, and quality. It is like having a discussion with a financial advisor over income, risk, and growth. You don’t just show up and say you want all three and get a “sure”.

12/ On the other hand, this is precisely what Apple did so reliably over 20 years. But behind the scenes there is a constant discussion over balancing these three legs of the tripod. You have to have all of them but you “can’t” but you have to. This is why they get paid big $.

13/ In practice when building Office (and later Windows) whenever someone on the team would panic and ask “are we date driven, feature driven, or quality driven” we would just roll our eyes and pull up a chair…This was so common we just called it conversation #37 and move on.

14/ A massive project like an OS (+h/w +cloud) is like a large investment portfolio and some things will work (in market) and others won’t, some things are designed to return right away, some are safe bets, some are long term investments. And some mistakes…

15/ Customers don’t care about any of that and that’s ok. They just look for what they care about. Each evaluates through their own lens. Apple’s brilliance is in focusing mostly on two audiences — end-users and developers — tending to de-emphasize the whole “techie” crowd, even IT.

16/ When you look at a feature like FaceID and trace it backwards all the way to keychain — see how much long term thought can go into a feature and how much good work can go unnoticed (or even “fail”) for years before surfacing as a big advantage. That’s a long term POV AND focus.

17/ This approach is rather unique compared to other tech companies that tend to develop new things almost independent of everything else. So new things show up and look bolted on the side of what already exists. (Sure Apple can do that to, but not usually).

18/ All the while while things are being built the team is just a dev team and trying to come up with a reliable schedule and fix bug. This is just software development.

19/ There’s nothing magic about this. It goes back to a balancing act. Mature orgs just manage this the whole time. There are processes and approaches that you use so you never face the absurd notion that this is a zero sum trade off between quality, schedule, features.

20/ What happens to a growing project over time is that processes and approaches need to re-thought. It just means that how things once scaled — tools like deciding features, priorities, est. schedules, integration test, etc — are no longer scaling as well. That happens. ¯\_(ツ)_/¯

21/ What I think it happening at Apple now is not more dramatic than that. What they had been doing got to a point where it needs an adjustment. Reality is that for many at Apple it feels dramatic b/c it might be first time they have gone through a substantial “systems” change.

22/ From outside it looks more dramatic. Ppl look for cause & effect. Like w/Windows ppl said “need a big change ‘because’ of Vista” when reality the system just needed to mature anyway. Office had a big system change in ’98 but no one talked about it outside absent a ‘moment’.

23/ In my view the ‘moment’ is being manufactured a bit right now because of the perception that the Apple products have become less stable or…”buggy”. This is where the “signals” about the state of the world can get confusing.

24/ In any absolute sense the quality of Mac/iOS + h/w are at quality levels our industry has just not seen before. Think of the scale of iPhone X release. From zero to 30M in months. That’s just insane. And it works better/more reliable than just about anything else I can buy.

25/ How does that explain general “buggy” feeling w/ so many super smart/skilled people saying products are suffering? It’s because of the depth and scale of usage that comes w/success. A responsibility.

26/ Look, there are bugs. You (and Apple) can make a list of them. But mostly this is about change. I know people say that isn’t the case but it is. On any absolute scale number of bugs — non-working, data losing, hanging mistakes — in iOS/Mac is far far less today than ever before.

27/ I can’t prove this but I’ve also worked on some really big projects where people said the same thing and we had tons of data. Apple has the same data. What is different is that at scale a bug that happens to 0.01% of people is a lot of people. A stadium full or more.

28/ No one ever anywhere has delivered a general purpose piece of S/W+H/W at scale of 1B delivering such a broad, robust, consistent experience. We don’t have a measure for what it means to be “high quality”. I can say that in any absolute sense, Apple has exceeded everyone else.

29/ The more a product is used the more hyper-sensitive people get to how it works. The human brain is extraordinary in how it recognizes even the slightest changes in responsiveness, performance, and sequencing of operations.

30/ It’s incredible how even smallest changes can become disorienting. We used to say how this is rooted in primordial instincts for survival — the ability to spot a tiger in the grass that might jump out to eat us. Same ability to detect small change drives us nuts on computers.

31/ But what happens to a team as complexity evolves is simply the challenge of coordination and more importantly consistency or leveling of decisions across a complex system. This is particularly acute if the bulk of the team has only known the previous few years of success.

32/ So Apple will just renew the engineering process. It means thinking about how risk is analyzed, how schedules are constructed, how priorities are set. This is literally what it means to run a project and what we are all paying them to do.

33/ They have more data and understanding to make adjustments than anyone. The only thing I think is fair to say from the outside is that this is not nearly as dramatic as it is getting made out to be…

34/ Rest assured that this has been brewing for a couple of cycles. Especially at the edge of the organization where it probably felt like “gee it is tougher to get things done” or at the top of the org where it probably felt “hmmm this should be further along”.

35/ The idea though that this is some massive shift to focus on one dimension of the overall product process: quality OR features OR date is just **nonsense**. Nothing of scale is thought of or executed that way.

36/ You start a project with aspirations. You start with a known set of resources. You have a date. Building a plan is iterating across these constraints. Different people on the team/org with different inputs. It is an iterative process.

37/ Big projects run poorly are “date driven” or “we’re getting this whole thing done (famously “second system syndrome”). Lame projects are “we’re fixing bugs” (used to call this “re-indenting all the source code”.

38/ Big projects, well-run are going to look across all systems and a long-term POV and develop a portfolio of what to do where/when. That’s how you get something like a new file system years in making to show. Or killer FaceID. Or all of a sudden have tons of cloud features.

39/ Ultimately at MS used to have Conversation 37:

• Eng wants to do nothing but fix code // BUG BUG

• Sales wants new product every year w/new quotas

• Press would like a new thing each month

• Techies — revisit core UI, add options on demand

• IT — no change, ever :-)

40/ But a product at scale serves all those needs and a strong leadership team has a long-term point of view PLUS it knows why the company exists and what it wants to do. That’s how a plan is developed. That’s what it means to build at scale.

41/ Some people say “oh consumer products need yearly releases” or “enterprise need constant value for SaaS”. The only thing you need is to do good products when you have them. In the scheme of things market timing except for seasonal-only products isn’t how to scale for 1B.

42/ The best example of being caught in your own timing myth was how Detroit decided 5–7 years for cars until Japan did it in 3. But now redesigns are like 7+ because the need to do so is much less than cost.

43/ Growth hacking or “move fast break things” sounded great until it wasn’t. This especially doesn’t/never worked in enterprise. Again, adopting a methodology absent building a great product *always* fails. “Internet time” was kind of a bust the first time around.

END/ So to me on Apple, even as an outsider, I feel confident saying that this isn’t reactionary/crisis or a response to externalities. Importantly it isn’t a massive pivot/“student body left”. It’s a methodical and predictable evolution of an extremely robust and proven system.