Browsing joomla.org, I recently noticed the following in the Joomla roadmap:

The Joomla! CMS [content management system] seeks to create a variety of distributions of the CMS to address a variety of common niche markets.

Good idea!

As a Drupal developer with a longtime focus on distributions ("distros"), I'd like to support this Joomla initiative. As a first step, as followup to posting in the Joomla development forum, I thought I'd try to write up some observations on Drupal, Joomla, and distros.

I've written this with an intended audience of Joomla developers. But if you're a Drupal contributor, please wade in as well!

What is a distro?

Technically, any time you package up and distribute a version of Joomla that's different from what's on joomla.org, you've got a new distribution. Great! But that's not really what I'm talking about here.

I'm talking about: how can you produce a bundled version of Joomla that provides free and out of the box pretty much everything that's needed for a particular type of website?

Say, for example, I'm going to build an ecommerce site. Rather than having to select, download, install, and configure and integrate a bunch of plugins, plus maybe write some custom code, I can instead just download and install a version of Joomla that has everything I need. Presto, there's my ecommerce site. Yes, from there I might hire someone to train me or extend the site, but out of the box I get all the basics covered.

And not just on some hosted online service. For example, the software as a service Nonprofit Soapbox Joomla-based product is sometimes, mistakenly, referred to as a distribution. Nope. For starters, it's not distributed. It's not even open source.

Some distros may focus on bundling software or solving a technical need rather than creating a preconfigured site for a particular sector. In the Drupal world, this type of distro includes Acquia Drupal, which bundles Drupal core with a bunch of contributed modules, and Pressflow, which provides performance enhancements. This type of distro serves a need. But, again, it's not my primary focus here.

Instead, I'm talking about the niche-focused distributions hosted on drupal.org. Like the Open Atrium groupware, Commerce Kickstart for ecommerce, and OpenChurch for, yes, churches. And if you're wondering who I am and why I'm interested, have a look at the Open Outreach distro, of which I'm a lead developer.

Why distros?

Given the millions of Joomla (or Drupal, or Wordpress) sites out there, there's a ton of duplication. Yes, it's great to have a CMS that's flexible, but endless repetition of variations on the same custom site development kinda defeats the purpose of open source development. Not to mention it gets tedious.

A niche-focused distro captures the configuration you'd otherwise repeat and presents it as a starting point. Further development can benefit not just one site but all adopters of the distro. Like other Joomla or Drupal development.

Some Drupalisms and their Joomla equivalents

For lack of an established vocabulary, each CMS develops its own jargon, often - just to make things extra confusing - taking the same terms and giving them different meanings. See Plugins, Modules, and Themes, Oh My! for a handy glossary.

Exportables and bundles of configuration

Before digging into specifics, some key concepts by way of background.

A niche-focused distribution is more than a bunch of general-use extensions. It's all the configuration that's the stuff of site building. And it's not just about a one-time initial setup. Sites on a distribution also need to get updates--to configuration as much as to code.

The obvious shortcut to a distribution is a variation on a database dump. That's basically the approach outlined a couple of years back on how to use a commercial backup extension to roll your own Joomla distribution. It's also the approach that was taken for the first Drupal distros. Just configure a site the way you like it, export it at that point, and you're set to clone it as many times as you like. Easy! Quick!

But not so flexible or updatable. What if some of your users don't need or want all the stuff you enabled? What if, next week, you extend your configuration or fix a problem--how do existing users get your fix?

In Drupal, the answer is so called "exportables"--configuration items that are exported into code. See an overview of exportable configuration in Drupal. While it's possible to export a whole site's configuration into a single bundle, for purposes of distro development what's preferable is to export into distinct bundles of configuration, each meeting a specific use case. That way, users of a distro can pick and choose, customizing their site with just the functionality they need. For example, a project management distro might include several optional bundles of configuration: project, group, issue, event, and so on.

In Drupal 7, the Features module is the standard tools used to package up bundles of exported configuration. As well as a UI for creating and managing "features" (configuration bundles), it provides bridging for types of configuration that don't natively support exportables.

For Drupal 8, the Configuration Management Iniative (CMI) has extended exportables to cover all types of coniguration. A recent blog post outlines how exportable features are shaping up in Drupal 8. The CMI is one of the elements slated for inclusion in Backdrop, a recent Drupal fork.

Distro enablers and what Drupal does--and doesn't do

So, in no particular order, here are some notes on what Joomla might be able to learn from Drupal. I've arranged them around a number of "distribution enablers"--that is, factors that - at least for Drupal - have been important in enabling or supporting distribution development and adoption. I've also added some notes about Joomla--sometimes just questions, since I'm a Joomla newbie. If you can answer the questions, please do and I'll update the list.

Support in core for distributions

Why it's important

As a distro developer, I want the core software to work with me--not against me.

How it's done in Drupal

Drupal supports natively multiple "install profiles", which are similar to modules (extensions). Install profiles can alter the site installation process, adding their own steps. They can also include plugins (modules, themes, libraries). Core includes two install profiles, "standard" and "minimal". To create a distribution, you simply add a new directory, e.g., profiles/myprofile, and put the profile for your distribution there, along with any contributed plugins it requires. See documentation on distribution development.

Room for improvement

The install profiles that ship with Drupal 7 core aren't particularly useful as models, as they rely on direct database or API calls rather than exported configuration. This should change in Drupal 8, thanks to the Configuration Management Iniative (CMI).

Joomla notes

Is there an API for modifying the Joomla install steps? Where is relevant documentation?

Infrastructure to support distributions

Why it's important

Packaging up a distribution is a lot of work. As a distro developer, I want some tools that make the job easier.

How it's done in Drupal

While the first Drupal distros were hosted externally, the drupal.org site now includes highly specialized infrastructure support for developing and packaging distributions. Developers focused on drupal.org infrastructure development added this in two rounds of community-supported development. First, the core infrastructure was put in place. Then in a followup round of development, infrastructure devs funded by companies with an interest in distros put in place a major set of enhancements. The Drupal Association played and continues to play a key and direct role in funding, coordinating, and providing this and other Drupal project infrastructure.

Room for improvement

The solutions require a lot of custom code, something that's added to the upgrade burden for drupal.org and is part of the reason the site is still on Drupal 6, years after Drupal 7's release.

Joomla notes

Joomla legacy code has the JoomlaCode repository, while core has moved to Github, supported by the jIssues application. What would it take to extend jIssues to support distro packaging?

Source code and updates available through publicly accessible repositories and download packages

Why it's important

Building a distro typically involves packaging up a lot of extensions along with core and supporting libraries. To use the code, it's not enough that it has an open source license. It also needs to be available.

How it's done in Drupal

In Drupal, developers and shops typically sell services--not code. The drupal.org site offers more than a listing of plugins. Rather, drupal.org provides a (Drupal-based) project management solution for Drupal code projects, including Git source code repositories, release management, a ticketing system, and security oversight.

Room for improvement

A custom project management solution is costly in terms of development and can easily fall behind other available options. Some Drupal developers are drifting elsewhere, mostly to Github.

Joomla notes

Much of the Joomla scene centres on selling software. Even presumably "non-commercial" projects often include blocks to code access, like mandatory registration on a host site.

If there's one key barrier to distros in Joomla, I'd say this is it. Commercialized code sets up huge barriers to distro development, since as a distro developer what I need above all is the ability to package up and use the best my open source software has to offer. If that code is stuck in some private repository, it's a drain rather than a gain.

The JoomlaCode site featured some relevant infrastructure and the newer jIssues application is promising.

Probably the single most important step to enable distro development in Joomla would be to strengthen and promote publicly accessible code. This might require changes to the extensions directory to, at least, provide profile to extensions with publicly accessible source code--ideally, code managed through a Joomla community infrastructure including jIssues.

Strong GPL protections

Why it's important

As a distro developer, I don't want to have to be concerned with licensing issues and the integrity of the code I'm including.

How it's done in Drupal

Rather than relying on soft, voluntary compliance, Drupal includes clear developer documentation and direct protections for GPL integrity, including enforcement and removal from drupal.org code hosting. Longtime Drupal contributors including Gerhard Killesreiter (killes), Larry Garfield (Crell), and Derek Wright (dww) have provided focused and sustained leadership in this area. GPL protection is written into the distribution packaging system on drupal.org via a whitelist approach. Non-drupal.org code must be explicitly whitelisted before it can be packaged into a drupal.org hosted distro.

Joomla notes

For someone coming from Drupal, Joomla looks in comparison (no offense intended!) like a GPL wild west. One of the first "non-commercial" extensions that I browsed from a link on the Joomla extensions listing offered a "personal use only" download and then wanted to charge for a "commercial use" version. Without a central repository with clearly enforced rules, I'd be at a loss as to how to ensure code integrity, short of a painstaking process of individually evaluating each package I wanted to include.

Update notifications

Why it's important

As a distro developer, I include a lot of code in my distro and I need an easy way to make sure it's all up to date.

As a distro user, I want to make sure the code I'm running is up to date, particularly for any security releases.

How it's done in Drupal

The Drupal update manager allows sites to poll a central server that provides update data in an XML format. Since almost all Drupal code is hosted on drupal.org, projects get update notifications out of the box.

Joomla notes

Joomla core includes update notifications. For extensions, update notifications rely on the individual extension authors implementing an update server. In practice, only a subset of extensions support updates.

Strong institutional backing for distro development

Why it's important

It's a lot of work to build out and support the enabling toolset for distributions and the distros themselves.

How it's done in Drupal

The Drupal distro toolset was built out alongside the first "modern" Drupal distro, OpenAtrium. Development Seed, a (then) Drupal shop, built OpenAtrium as a generic tool for use by large clients including the World Bank, and at the same time built out the Features toolset. When Development Seed changed its focus, OpenAtrium and the enabling toolsets moved to Phase2, a company already focused on building out Features-based distros including OpenPublish. Most of the well maintained distros are produced by Drupal shops that generate business from building sites with their distros. In many cases, distro development includes a range of other independent contractors or shops that also build sites with the distros and contribute plugins and other enhancements.

Standard site-building tools

Why it's important

As a distro developer, I need tools to build out configuration, and the more standardized those are the better.

How it's done in Drupal

In recent major releases, there's been an increasing move to bring key configuration management tools into Drupal core, including custom content types (added in Drupal 6), fields (added in Drupal 7), and Views (added in Drupal 8). Even in contrib, there is a tendency for usage to converge on a best solution. With Views, for example, there wasn't a question of which query builder would be a candidate for core inclusion--Views was already the clear choice. Factors that facilitate this convergence include (a) non-commercial code, so that the best solutions are universally accessible and (b) the usage statistics ranking of projects on drupal.org, so that site builders easily see which solutions are gaining adopters.

Joomla notes

Joomla looks to be relatively challenged in both standard tools and the bringing of those tools into core. Consequently, it may be a bit harder to build integrated solutions.

Standardizing on site building solutions could be a key step.

Extensions that facilitate distribution development

Why it's important

As a distro developer, I want to have handy tools I can use both to author and to deliiver my distro.

How it's done in Drupal

Drupal contributors have produced a standard toolset for producing Drupal distributions. At the core is the Features module, which includes an advanced UI for capturing and exporting bundles of configuration that meet a particular use case.

Joomla notes

Are there equivalent extensions in Joomla?

Command line tool and packaging format

Why it's important

Distro development involves a bunch of repeated tasks, like exporting bits of configuration. Please, save me endless clicks! It also involves gathering up code from disparate places.

How it's done in Drupal

Alongside Features, the Drush command line tool includes key functionality supporting distro development. The "drush make" format makes it possible to specify, in a simple text file, a custom set of code that will be packaged up to produce a full distro.

Joomla Notes

Is the Joomla CLI an equivalent base here?

Flexible core

Why it's important

Key to being able to produce radically different distros on the same code base is ensuring that very little in the core CMS is fixed or required.

How it's done in Drupal

Drupal 7 has a minimum of required core modules. Modules providing more specialized functionality have steadily been moved out of core into the contributions repository.

Joomla notes

There have been efforts to build a "light" version of Joomla to facilitate distributions, including Square One and JLite. These are useful examples, but they've hit the same barriers that early Drupal distros did--it's way too much work to have to maintain a largely forked version. Instead, it will be important to focus effort on refactoring the core CMS so it has a minimum of required components.

Tangentially related: there was public musing about whether Molajo was a Joomla distro or a fork, but according to lead developer Amy Stephens it's neither--see comment below.

Distribution-friendly hosting solutions

Why it's important

As a distribution user, I want hosting options that make it easy to install and update my preferred distro.

How it's done in Drupal

Many Drupal hosts offer special support for distros. The Drupal Aegir hosting platform includes built in support for distributions. The Pantheon system supports distributions.

Joomla notes

Are there Joomla hosts interested in supporting distros?

Further reading on distros in Drupal

A 2006 blog post by Drupal founder Dries Buytaert charted the early development of Drupal distributions.

The 2012 Linux Journal Drupal Special Edition (free download) included in depth articles on distributions in Drupal.

Key Joomla strengths

Looked at from outside, Joomla has some tremendous strengths that should provide a solid base for building out distro support.

Huge install base

The sheer number of Joomla sites out there means there's a lot of potential market for distro-based sites. As site owners face the inevitable need to upgrade to new versions, getting off a customized one-off site build and onto a distro can be a big attraction. Upgrading an existing site to a distribution can save a ton of time and expense in upgrade costs and future maintenance.

Strong production and community leadership

Having an inclusive and participatory structure should help a lot in marshalling resources to achieve the (significant, but doable) tasks of enabling and supporting distro development.

Strong developer and adopter communities

There's no shortage of skilled Joomla talent to draw on.

Institutional backing

It takes a lot of backing to successfully make changes and fund infrastructural development. Open Source Matters and other key Joomla institutions would be important players.

Next steps

So that's it for an initial brainstorm. To emphasize again: my aim here is to start a conversation about what Joomla devs might learn from the Drupal experience re dos and don'ts of facilitating distro development. I'm not at all suggesting that Joomla should follow Drupal's "model"--just that a close study of how Drupal has done distros is probably a valuable input into making it happen in Joomla.

What did I get wrong? What did I miss? Where can we take the conversation from here?