- Drupal core has become larger and more difficult to maintain in recent releases. How can we make it easier for increasingly overloaded core developers to maintain the software?
- The rapid growth of Drupal has led to a much more diverse pool of users. How can we account for the different -- and sometimes incompatible -- needs of that very diverse userbase?
- The same complexity and tight coupling that overloads core developers also makes building major products and sites on top of Drupal difficult; how can we simplify that work?
Those Sound SeriousIn many ways, these concerns overlap with each other. The traditional solution to serving more diverse needs is to add more functionality to Drupal core, but that exacerbates the problem of developer burn-out and increases the complexity of building focused products on top of Drupal. On the other hand, pruning lots of code from Drupal core would reduce the maintenance overhead and the complexity of customizing it, but would leave site builders and newcomers with a less useful tool.
A Way ForwardCore developers Nathaniel Catchpole (catch) and Daniel Kudwien (sun) have been two of the most active participants in the conversation, and my "Product, Framework, or Platform?" session from DrupalCon London summarizes my take on some of those issues. Last week we had a chance to talk about some of these issues, and while there's still a lot to iron out, quite a bit is becoming clear. Rather than simply hunting for modules that experienced developers often ignore -- and evicting them from Drupal core -- our community needs to assess what Drupal core is and what it should be. There's broad agreement that we need to establish a set of heuristics that our community can use to weigh new functionality against its shared vision of what Drupal should be, and examine existing functionality to see if it's still appropriate for core. These are hard decisions, because our community has rarely invested in building consensus around Drupal's purpose, target audience, and internal architectural philosophy. The current explosion of interest in these topics, and the demand for viable solutions, tells us that the time is right to tackle them.
What is Drupal?One of the key problems we face in talking about Drupal is the relatively clunky terms we use to describe different parts of our relatively complex software stack. Often, discussions pit Drupal's potential as a "Development Framework" against its possible future as a "Product for end users", but that binary split gives a false picture of how our system really works. In his 8/31 blog post, Nathaniel Catchpole addressed this issue by naming five distinct layers of the Drupal software stack, and I think they're a great way to break things down.
The Web FrameworkThe foundation of that stack is the essential "web framework" code for Drupal. It's the portion of Drupal that provides essential services any web application would require. Rendering data into HTML and other formats, identifying and loading plugins, handling HTTP requests and responses, caching, database and storage access, and so on all live at this layer; today, most of this code is found inside of Drupal's
The Building BlocksMoving up in the stack, we find the "building blocks" layer that Catch referred to as the "Platform" in his blog post. This is where we begin to encounter Drupal-specific approaches to CMS and web building problems. The Entity system, users, nodes, taxonomy terms and vocabularies, actions, roles and permissions, FieldAPI, text formats, and our concept of "Blocks and regions" live here, and can be used to assemble features and functionality. Reusable user interface tools and UX components are also here: things like drag-and-drop tools, Drupal 7's Overlay mechanism, and so on are all reusable tools used to craft usable interfaces. Most of the functionality at this layer is invisible or irrelevant to the people who visit a Drupal-powered web site: they don't see Views and flags and nodes, just galleries and favorites and articles. That type of visitor-oriented functionality still has to be assembled out of the building blocks. Most of the "Big ideas" in Drupal's history made their splash at this layer: Node Types, CCK, Views, Context, and so on. As Effulgentsia points out in his comment on Catch's post, the Web Framework and Building Block layers are what most people want to keep when they talk about "streamlining" Drupal or making it into a simple framework. Although a Drupal site could run using nothing but the components that exist at this layer, it would be hard to build it without the piece that comes next...
The Site Building ApplicationIf the raw materials of a Drupal site live in the Building Blocks layer, the tools for assembling those materials into a working web site live here, in the "Site Building Application." It contains the tools that are used when building a new site from scratch via Drupal's own web interface. Examples include administration forms for creating content types and modifying their fields, as well as modules like Views UI that exist solely to create and configure new site elements. (This layer doesn't include day to day user management and content management, however -- just the tools used to actually create and configure a site.)
User-Facing featuresWe've talked about the framework, the building blocks, and the site-building tools that we're all familiar with. Now comes the layer that excites everyone: user-facing functionality.
End-User ProductsThe capstone on this Drupal stack is one we're not as used to talking about in the community: products for end users. When all the tinkering and programming and theming is done, a "product" is the end goal. A single-user blog for a book lover, a portfolio site for an artist or photographer, a local newspaper's web site, and so on are all concrete "products" formed by the careful use of all of Drupal's other layers. What really lives at this layer? Often, they're one-off web sites built for a particular person or organization's needs. Increasingly, the world of Installation Profiles and Drupal Distributions has started capturing common types of sites in the form of reusable products like Open Atrium, Drupal Commons, and OpenPublish. Hosted services like Drupal Gardens, Buzzr, and SubHub are also products, because they package a particular administrative interface with a particular set of features, and target users with a set of needs more specific than simply "building a Drupal Site." At the annual Do It With Drupal conference, one of our tracks is a parade of Fantasy Sites -- clones of popular sites like Stack Overflow, BaseCamp, Flickr, and others built in Drupal. They're built on a short timeline using existing Drupal building blocks, and deconstructed for the audience by the team that created them. It's consistently one of the most popular events at the conference, because people love seeing how compelling sites were created. One of the greatest challenges faced by new users of Drupal is the spartan quality of the standard installation profile that it ships with. Although it pre-configures a handful of content types, it doesn't target a particular use case or type of web site, and leaves many users shaking their head as they evaluate the whole platform. Real products living at this layer of the Drupal stack are all about making clear and conscious choices, then implementing them with the tools Drupal provides.
The Drupal Platform
What does it mean?So after all those words, what have we gained? Does this way of discussing Drupal help us understand some of the burning questions our community faces? Looking at our ecosystem and our codebase through the five-layer-lens can help us, and I think there are several critical conclusions we can draw.
- The long tail of Drupal users build features and products with the middle layer -- the "Site Building Application." They may know PHP and write custom modules to fill gaps, but the "Framework" and "Building Block" layers are a means to an end for them. Even code-centric developers need something at that level, whether it's Drush or custom-written tools.
- Calls to split the "framework" and "product" portions of Drupal core into separate projects would allow developers to obtain a slimmed down version that only includes the first two layers. However, the question of what should be bundled into the primary "Welcome to Drupal" introductory download would still remain. We can't avoid strategic choices simply by changing our infrastructure.
- One of the biggest pain points articulated by developers is the huge number of interconnections and cross-dependencies between different parts of Drupal Core's code. While it doesn't solve all of our problems, segmenting our code effectively along the boundaries discussed above can help. This would take discipline, though: loosey-goosey interaction between different subsystems and different layers is a big part of Drupal coding today.
- Dissatisfaction with canned features like Blog, Forum, and Tracker spiked when the community shifted to a "building blocks" approach. Those modules are useful, but no longer reflect our preferred way of solving problems. They're easier to understand, but less flexible, and have stopped evolving.
- Our preferred way of solving problems -- combining small building blocks into reusable features, and features into finished products -- is baffling for newcomers who have not yet mastered the building blocks themselves.