Key GNOME component updates

Did you know...? LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

GNOME is a massive project with a lot of moving pieces that are directed by separate individuals so, at times, it can be hard to up to date with where things are moving. At GUADEC 2016 in Karlsruhe, Germany, developers provided updates on the Flatpak packaging format, the GTK+ Scenegraph Kit (GSK) API, and a new logging system for GNOME applications.

Flatpak

Alexander Larsson presented an update on the progress within the Flatpak project over the past year, followed by a look at what comes next. In addition to "the great renaming," he said, the past year saw the project move from Freedesktop.org to GitHub, add several tools, and add lots of polish.

The new features include flatpak-builder, a program that builds Flatpak packages on the command line. Previously, he said, users had to write their own scripts using low-level APIs, which certainly hindered adoption. Support was also added for AppStream metadata, so Flatpak packages now provide user-level information (such as descriptions of the program). This allowed the GNOME Software installer tool to add support for managing Flatpak packages.

Package installations are now system-wide by default (although per-user packages are still supported). Permissions are now handled by PolicyKit, which makes system-wide installs more reliable. The first iteration of portals was added, enabling users to mediate an application's requests for access to potentially risky services. The project also released a stripped-down version of the basic container used in Flatpak packages, called bubblewrap, that can be used to isolate any program.

The Flatpak package format was also extended to support portable "single-file bundles" that can be sent via email or exchanged on USB sticks, with no system infrastructure required. The first Flatpak runtimes (separate packages provide the system APIs a Flatpak requires to run) were also released; a GNOME 3.20 runtime and a generic Freedesktop.org runtime have been produced so far. New architectures are now supported as well, including 32-bit and 64-bit ARM. Finally, several external adopters have started using Flatpak bundles, notably LibreOffice and Endless Mobile; KDE has also done some initial work toward building a compatible runtime.

Moving forward, Larsson said, portals will be an area of major work. Several new portals are needed, including one for access to DConf, one for sharing media (such as over social networks), one for accessing contacts, and several new device classes (including optical drives, USB sticks, and game controllers). More infrastructure work is needed to support some of the portals, however. Wayland is still missing pieces like cross-window processing (which makes clipboard sharing problematic), PulseAudio "is completely unsafe" at present, and the Pinos camera-access API lacks a viable permissions model.

Larsson added that he hopes to see the Flatpak work dispersed more throughout the GNOME project in the future, since, at present, it is almost a solo effort. The GNOME Release Team could maintain runtimes, he suggested, and individual application teams could build their own Flatpak packages and maintain the necessary build information and metadata. Further out, Larsson speculated that the project might need to consider "long-term support" runtimes that provide a minimal set of services but would be suitable for applications that are rarely updated (such as proprietary programs). It would also be nice, he said, to offer a personal package archive (PPA)-like site for Flatpak distribution and to support paid programs.

GSK

Emannuele Bassi began his session by noting that "the GSK talk" had become a GUADEC staple, much like "the Clutter talk" was before it. GSK is Bassi's Clutter replacement, a drawing API that will allow application developers to create relatively free-form user interface elements by defining geometric primitives in hierarchical layers and transforming them using standard positioning, visibility, and animation operations.

But GSK is still in development, so, rather than cover the same ground as last year by discussing incremental progress, he chose instead to spend most of the allotted time recapping the history of how GTK+ has drawn pixels to the screen over its lifetime. The project is close to 20 years old, he said: the first release was in 1997, roughly when MP3s first hit the internet, Windows 95 dominated the OS marketplace, and the browser wars had not yet started.

At that time, "the apex of GUI toolkits was Motif," and GTK (with no "+") looked exactly like it. Each widget was backed by an X11 window, each had its own color map on the X server, and all drawing commands were sent over the wire. The 1.2 release in 1999 added the first theming support, but it was not until 2.0 in 2002 that GTK+ was "all grown up" and the project decided to "bring the light to other platforms" like Windows and Mac OS X. That meant reimplementing the X11 drawing API for those platforms, which is what led to GDK.

By the time GTK+ 2.6 came along, several new projects had been started, most notably Cairo and the XRender extension. GTK+ added support for them, replacing X11 calls with Cairo calls and moving to client-side window rendering—but support for the deprecated APIs was not removed. In 2011, GTK+ 3.0 was released, and OpenGL drawing was added—although it was not yet taking advantage of GPUs. GTK+ also added support for the CSS drawing model, which replaced the Cairo drawing model.

But the Cairo API has remained part of the 2D rendering stack, even though it is only optimal for a few specific pipelines, Bassi said. That is because it turns out that "OpenGL is bad at drawing GUI elements unless your elements look like Quake." Thus, GTK+ today continues to use Cairo "for what it's good at" while using OpenGL for what it was good at.And OpenGL has been getting better, he said; in the long run Cairo may eventually be dropped, but for the time being it will remain supported.

This approach is "the way out," he said; retaining both the Cairo and OpenGL APIs lets GTK+ make more efficient use of multiple cores, without forcing application developers to rewrite everything for a new drawing API. Moreover, maintaining support for an old API while adding a new one has always been GTK+'s approach: the same decision was made when adding GDK to X11, when adding Cairo to GDK, and when adding OpenGL to Cairo.

This history lesson, Bassi said, was meant to reassure developers that the current work on GSK would not disrupt their applications. He is still working hard on GSK itself, optimizing rendering on the GPU not the CPU. "So are we in 'the future' yet? We're pretty close," he concluded. GSK can do high-quality animations, 3D transformations, compositing and rendering CSS primitive, and more. But it is important to remember that there will be always be something new coming down the pipeline, he said.

GSK will not be available in the upcoming GTK+ 3.22 release, but it is likely to be merged for the subsequent version.

Structured logs

Philip Withnall discussed his recent work adding a structured logging facility to GLib. The library's existing logging interface is GLog, "which everyone has been using forever," he said. But GLog is rather limited, so he has been borrowing ideas about structured logging from journald and other projects in order to develop a superior replacement. The work is being done in upstream Glib, even though Withnall has been doing it on behalf of Collabora.

The upshot is that structured logging uses a set of "key: value" fields, rather than a simple string, for each log message. There are numerous benefits. In particular, structured logging allows each log message to have a unique message ID, enabling searches within the log and in the source code. There is a new writer function that replaces the old GLog log handlers, and the new Glib log API hooks into journald, so that it can tell if journald is running and whether or not log messages are being directed to the journald socket.

The new API supports context pointers in log messages, which allows logging complex state information without the use of global variables. And the API also supports color console output, he added, "which I think is probably the most important change." In light of the new features, he said, "we're encouraging people to log everything," then to do whatever filtering is necessary in the log viewer, rather than trying to fine tune what messages are logged in advance or requiring users to set an "esoteric" log level via a command-line switch.

Withnall then recapped the shortcomings of GLog. In that system, each library would install its own log handlers to do "something different" with log messages from applications. How various log domains and log levels interacted was, essentially, undefined, and there was little that a developer could do to customize behavior. Some features could be tweaked in a log handler, but others (like how fatal errors were handled or when to abort on an error) were hard coded in GLib.

There was also some global state that various libraries on a system would fight over (such as which errors were regarded as fatal), leading to unpredictability. And there was no way for unit-testing code to test for complex sequences of messages; the GTest facility could watch for a specific sequence of messages, but it cold not cope with optional messages or variation in the delivery order.

Next, he presented the new structured log format. The key-value pairs in a log message do not need to be formatted into a string immediately, so that task can be put off as long as is convenient. Keys can be namespaced to implement application-specific message features. Performance is potentially better than the old log API, he said, although it depends on how it is used. Applications that do string formatting on all of their log messages will, of course, pay the price for that. Without string formatting, there is a zero-copy path all the way to the socket output.

Porting existing programs to the new API is straightforward, he said. Developers merely need to include:

#define G_LOG_USE_STRUCTURED #include <glib.h>

and add a g_message("foo") call in their code. This will pass through the message ("foo") to the structured logging facility, plus the file name, line number, and function name. Full control over the log message can be achieved with the g_log_structured() function, which lets the developer supply whatever key-value pairs are of interest.

There is also a g_log_structured_array() function ("the turbo option") that can be used to pass in an entire array of structured log fields. Among the other changes is a new G_DEBUG_HERE macro that is expanded into the standard GLib debug-message call.

Each application can also now define its own log-writer function; this is a change from the old API, in which log handlers were implemented in libraries and application code had little to no flexibility over log message content or logging policy. There are default log-writer functions provided (one that simply logs to journald and one that sends output to stdout ) that application developers can extend as required.

There were a few questions from the audience at the end of the session. One person asked how message IDs are generated; Withnall replied that they can be any string that the developer wants—which prompted Christian Hergert to suggest that development tools like Builder could be configured to generate them automatically. Lennart Poettering asked if the facility could be extended to handle error codes; Withnall responded that it sounded like a feasible idea and promised to explore it.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2016.]

