The big picture

Drupal 8 had very high goals for Blocks and Layouts. We were planning on user interfaces for landing pages, "node type overrides," and something akin to the Context module, and Spark had laid out plans for a responsive layout builder. It's now clear most of these interfaces will not make it into D8 core - they are each complex interfaces requiring considerable effort, experimentation, and gestation, and as Dries explained in his post-18th blog post, the time for such things is now past. However, this may not be such a bad thing.

While having such user interfaces in core is a good eventual goal, the real challenge in this problem space - the one inhabited by core blocks, Context, Panels & CTools Page Manager, Display Suite, and others - has never really been the UI. Rather, the difficulty has been in the lack of fundamental API concepts for defining what a "page" is. Without such APIs, contrib has been forced to create its own notions that are a) mutually incompatible and b) inherently limited by virtue of being in contrib. A unified underlying API is the only problem that we truly need to solve in core - and we are very close to that right now.

But we are not giving up on user interfaces completely. At the very least, kicking everything to contrib would be unwise because an API without implementations is doomed to inadequacy, at best. Also, it would be a significant regression: whatever its flaws, the existing blocks administration system (at admin/structure/block ) serves an important basic function, and we need to replicate it. And so, that is probably our final, major goal for SCOTCH: make a new block administration page that more or less matches existing functionality, but is built atop the new first-class page model. By using these new tools that are designed for extension, the new administrative user interface should cease to be a dead-end that contrib must struggle with or hack around, and will instead be a natural foundation from which a healthy, more interoperable contrib ecosystem can arise.

How?

It's easiest to explain the new system by in the context of changes to the admin/structure/block page. In previous versions of Drupal, blocks were configured per theme; there was one tab atop admin/structure/block for each enabled theme. To support this, themes had to provide a list of regions in their .info file or a hook, and a corresponding page.tpl.php that would display those regions. Those regions would be used everywhere the theme was in use, giving rise to the notion that "Drupal has one layout that bends a lot." (courtesy merlinofchaos)

In Drupal 8, the the page template ( page.tpl.php ) is changing drastically: its job is being taken over by a layout* - inspired by, but an advancement over, the Panels plugin of the same name. Each layout plugin has its own regions and corresponding Twig template. Core will provide at least a few layouts out of the box; modules or themes will be able to provide more, and themes will (of course) be able to swap in their own Twig templates to replace the core ones if desired. So, no more "bendy" layouts - we can build them for specific purposes, keeping them terse, tight, and beautiful. The layouts system and API have been in core since last September/October, and while it still needs a few tweaks, all we really lack now are layout implementations.

The new block administration user interface will be the one place core allows users to place blocks into these layouts. The basic purpose of the interface will be the same: to allow block placements that apply sitewide. But block placement per theme no longer makes any sense - layouts will provide the region list and page template. Nor does it makes sense to have a tab per layout - the goal of layouts is to provide choice, not to force more configuration. Plus, when would each of the different layouts be used?

Enter displays*. Displays are configuration entities that record the arrangement of blocks within a particular layout. This information is analogous to the block configuration we used to store per-theme - block visibility, ordering within regions, etc. - displays just make it portable. What’s different about them is that they are a complete picture of what is shown on the page. Whereas in Drupal 7 we had mechanisms like hook_page_alter() that any module could use to inject markup onto the page (and consequently, no way of predicting what would be shown on any given page) in Drupal 8, markup only makes it in if the display has a record of it. Making the display a canonical record gives us a predictable page model; a predictable page model enables reliable "metaprogramming" or middleware-like patterns, including (but hardly limited to) things like caching. And displays are what we save to at the replacement for admin/structure/block .

...almost. There are actually a few different flavors of displays, and what the new blocks administration will save to is called a "master display." Master displays are rather like master slides in presentation software: they define shared elements (blocks), then normal displays inherit from the masters, but can make their own local additions and changes. When the master is updated, every normal display inheriting from that master is immediately updated, as well.

Our plan is to ship core with two master displays: "frontend" and "admin", alongside two identically named normal displays that inherit from their respective master. These displays will be bound to “routes”(which are roughly like hook_menu() items of old) such that they coincide exactly with the default and admin themes (which we intend to retain): the admin master display will be used for all admin paths, and the frontend master for all other paths. This simple and intuitive default configuration should be a good starting point for users, helping ease them into a complex system.

Master display inheritance is a major reason that we believe this system will foster a healthy, interoperable contrib ecosystem. It bridges the gap between the two rival philosophies of page building: the site-wide or "global" orientation of Context module and blocks in the past, vs. the more page, entity, or "local" orientation of Panels and, to an extent, Display Suite. Master displays provide the global outlook, but flow seamlessly into the localized control of an individual display. Our hope is that, by providing the base API and an interface for managing master displays, contrib will be able to focus on building somewhat more use-case-targeted interfaces and applications:

Landing pages: conceptually, these need not be any more than a custom entity type with a display coupled tightly to it.

Customizable dashboards & profiles: a display per user, placed at a special private URL. In most use scenarios, the site builder would probably want to preselect a master display to be used by all dashboards. This system could easily accommodate the dashboard on Drupal.org, for example.

Per-node-type visuals: the primary use case of Display Suite and a major use case in Panels, this would entail a display that's bound to a single node type.

There are plenty of other cases. But every one of these systems could save everything they need to as a display, and follow standardized patterns for binding those displays to routes. And all would benefit from having a master display to contain shared elements, like a "site logo" block, the primary links menu, or the toolbar, so that they need not be manually re-added locally on each and every display.

These use cases are complex enough that it merits a contrib cycle for us to experiment on top of the new API, figuring out how to make them play nicely together and how to get the interfaces really, really right for the different use cases.

Note: we intend to rejigger names: "layout" is planned to become "template", and “display" to become "layout". Probably.

What else?

The preceding ideas represent a consistent and attainable overall goal for visual site-building tools in Drupal 8. It's most of the picture for SCOTCH, but not quite all. This system also needs to be able to accommodate sophisticated caching mechanisms like ESI and hInclude; significantly improving cacheability was a major goal justifying the creation of WSCCI and SCOTCH in the first place.

There's also a fair bit of replumbing that needs to happen for all these pieces to work together: we need to move html preprocess logic elsewhere; we need to change the way CSS and Javascript assets are declared and managed.

Additionally, we still have follow-up issues from the original blocks-as-plugins patch, committed back in early January. Dozens of them, in fact. Some are resolved by the architectural switch to displays, but most still need addressing.

How to help

As we are now in feature freeze, the committers have suggested that we shift work over to a sandbox repository; the goal is to keep core “releaseable at any time,” which means less incremental, more big-chunk work. We’ll be using the SCOTCH repository. We hope this will be a good thing for ongoing SCOTCH work, as many of our patches have suffered from the incremental approach as reviewers were unable to see the benefits of functionality in small patches in isolation from the rest of the (in-progress) systems that complement them.

Here's our current high-level hit-list. Jump in on the following issues, or see the Blocks-Layouts tag for an up-to-date list:

Come visit us in #drupal-scotch, and ping sdboyer or EclipseGc. We're there almost all the time.