New year, new Linux – or, in the case of Ubuntu, two. As in years past, Canonical's distro gets two updates in 2017 – the spring and autumn releases numbered and named respectively 17.04, Zesty Zapus, and 17.10 – name TBD, actually.

As ever there will be UI and experience fiddling – Zesty Zapus sees changes in windows management, the organisation of applications and there a new Mir abstraction is planned called Miral.

What about some really big changes, like the Unity 8 shell and Mir itself?

Ubuntu 16.10, October 2016, marked the seventh stab at a pre-release version of Unity 8 but no actual final product.

And what an incomplete release it was: not working with proprietary Nvidia and AMD Radeon GPUs or virtualisation software and suffering poor 3D.

Speaking after 16.10, Canonical product manager Richard Collins was reported to have claimed the next "big target" was "to evolve the Unity 8 experience so that it runs as both a touch and a desktop experience".

All eyes on you then, Zesty Zapus, in a few months' time.

But what is Unity 8 and why's it a big deal?

Unity as a name and project began life in 2010 as a new UI for desktops and laptops and it arrived swiftly – in the following year. However, the idea morphed to offer the same screen and user experience on all devices regardless of mouse or touch. Put Ubuntu running Unity 8 on a phone and it'll render as a phone, put it on a PC and it'll render as a PC, put it on a tablet and it'll render as a tablet. That's the idea anyway, and it was analogous to ideas floating around Redmond for a single version of Windows running on PCs, phones and tablets – the same UI and same "experience". One brand, development and runtime.

That was part of the idea of Windows 8 anyway, and the Metro UI.

Coming with Unity 8 is Mir, the planned display server replacement to the predominant X Window System, which Canonical announced in March 2013.

X Window System is an industry standard for bitmap displays in Unix-like systems such as Ubuntu and is the product of Stanford University, MIT and IBM. Canonical wants to build its own display server, however.

Four years on, though, the dream remains unrealised with Mir like Unity 8 available only as a preview.

Unity 8 is, by the reckoning of Ubuntu daddy Mark Shuttleworth, a year late.

And it's Canonical's founder who has given the engineering team the necessary latitude they need to complete their work. "Unity 8 has taken longer than I anticipated," he told The Reg during a recent interview. "I thought a year go we would be there."

What's the big hold up then? High expectations and the very low bar on failure, he reckons.

The ability to span different hardware isn't, as you'd suspect, the problem, Shuttleworth says. It's building something that works as it should everywhere and doesn't piss people off.

"The challenge to that [Canonical engineering] team is to deliver a desktop as good as this desktop," Shuttleworth said. "If we are going to deliver code to 20 million people, it's not a science project, it's not for fun – you can't screw up people's desktops. It's a very high bar and it's taken longer to get over.

"But I've given them that room. We [as people] have 30 years of desktop habits. If I put something out there that's not good quality, the comments on Reddit are going to be really ugly. We can't move that stuff in front of 20 million people until there are no rough edges."

Isn't it time to face reality and move on? If weeks spent in software development are like dog years on delivery, what are years? Surely, if Shuttleworth giveth the time Shuttleworth can taketh away.

"It may be my white whale," he admits. "I feel the team has earned a fair shot and I may take my carrots."

A pause.

Carrots?

A reference from the younger Shuttleworth's school days, I later learn: a metaphor for something you didn't enjoy at the time but that's probably good for you in the long run.

"But what is the nature of risk," he presses on.

Suddenly you get a flavour of why Canonical's founder has given the Unity 8 team so much of that lattitude.

"Unity 8 delivers a unified set of experiences across all the kinds of personal computers," he says. "I care about developers – I need to give them a Linux environment wherever they want to do their developing – if that's on a phone it needs to be on their phone. If it's on their goggles it needs to be on their goggles.

"Ironically, most seem to be on a MacBook!"

Unity 8's saving grace then may be the fact so many Ubuntu Penguins run on Apple laptops, obviating the need to stretch to replicating the experience on a long list of devices that recedes to the horizon.

Microsoft may have moved on, but he feels validated by the fact Microsoft was even there. "When we said we were going to do one piece of code for phone and tablet, people thought we were loopy! Now, I may be loopy but at least I have company. They gave up on the form factor but not the experience."

Canonical has a history of tackling the plumbing of its Linux to build a better distro for this core developer market.

Ubuntu exploded on to the scene during the early 2000s when the year of the Linux desktop was permanently mañana and Microsoft had lost a generation of developers to Apple thanks to Windows Vista and the rise of MacBooks.

Ubuntu 7.04 blew Linux heads away in 2007. Among its "innovations" the ever so humble and now taken-for-granted addition of Wi-Fi that worked out of the box. The Ubuntu installer was superior to those of Red Hat and Debian, with one early Ubuntu evangelist joking that the word "Ubuntu" was African for "Can't install Debian".

Back then, Red Hat was the best-known and most successful of the commercial Linux distros, but Red Hat prioritised corporate users, not individuals, and its focus was the server, not the desktop or laptop.

Ubuntu established a loyal fan base through Canonical's work building community via forums and blogs and through tools such as its bug tracker.

"I genuinely love enabling other people to do amazing things," Shuttleworth says. "It's the most rewarding thing. Developers are a really interesting crowd. And they see the future before everybody else – they are a bit like artists. And if you can enable them to create things more efficiently, we help the future show up faster.

"How do we do that? By looking really closely at what developers are playing with and where they are struggling. I was one of those developers in 1997, I was looking at the internet and I figured I could use Linux to do things much faster but it was really hard.

"I said Linux should be easy and the established guys aren't interested, so I decided to make Linux easier, and that's what took us away with Ubuntu for the desktops, and it was very specifically Linux for developers."

The work to build that Linux has seen Canonical rework some of Ubuntu's fundamentals. Some have accused Canonical of suffering from a "not invented here" syndrome.

Canonical has tackled system initialisation and management (init), combined desktop and web search (Dash and Lens), application development and deployment (JuJu and Charms), container hypervisors (LXD) and at volume manager and file systems (OpenZFS).

Not everything has gone down well.

Dash in 12.10 was an über search engine that paired your innocent desktop/laptop queries with returns designed "suitable" from Jeff Bezos' mega warehouse Amazon.

Not surprisingly, people hit the privacy klaxon and Dash got turned off.

In 15.04 Canonical forged ahead with controversial system management tool systemd despite protests from a similar move across the pond in the United States of Debian.

Init would have been preferred but in Ubuntuland but then there was no init; Ubuntu's author Canonical employed its own (natural) systemd – called upstart.

With 16.04 Canonical implanted the open-source ZFS (OpenZFS) into the heart of its Linux. ZFS is a revolutionary storage system that combines volume manage and file system; it gives you not just the physical discs but also the volumes.

ZFS was built by Sun for Solaris, now owned by Oracle, with Sun releasing the code as OpenZFS under an open-source licence it created called CDDL.

Revered ZFS and OpenZFS maybe, but the Software Conservancy Council and open-source firebrand Richard Stallman have claimed OpenZFS binaries cannot be legally merged and distributed with any GNU Linux binaries, which are under a GPLv2 licence, because OpenZFS uses CDDL and the two licences are incompatible in what you are allowed to do with the code.

And yet, so far, legals have yet to fly.

Shuttleworth insists there's no legal issue, having taking advice from former Free Software Foundation general counsel Eben Moglen.

"Without that, I simply wouldn't have done it. Oracle has no issue. It was one of those issues where it was waiting for somebody to take that step."

Now containers and micro services are here and Canonical's taken its own approach rather than simply use somebody else's. Called LXD, it's a pure container hypervisor for running Linux operating systems and apps unmodified in a VM. LXD is an extension of widely used LXC for virtualisation software at the operating system level within the Linux kernel and takes its place in the field with Docker and CoreOS.

Can we expect more Jujus, more LXDs, more – ahem – Unity 8s?

"I think that is conceivable," Shuttleworth said. "But I'm not sure that I'm saying yet." ®