Warning! Super-geeky post ahead!

Drupal's node system is extremely powerful, and has made it one of the most flexible OSS CMS systems around. however, when it comes time to actually build and render nodes for display, our APIs are showing their age. *Many* modules often alter the contents of a node before it's output, themers often need much finer control over the data when it comes time to output it, and often HTML is insufficient. Output as JSON data, PDF, XML, and so on is essential for many complex sites. What problems do we currently face, and what are some of the possible solutions?

This post is a stab at capturing some of the diverse discussions going on around Drupal-space and describing some of the solutions that are being proposed.
Feedback is welcome. I'd love to see some of these things make it into 6 before the freeze, but I'd love even more for the community to iron our a really solid path forward.

The problems

• Namespace collisions
  The existing $node object is a dumping ground for properties: every module gets a chance to add properties to it and very few use defined namespaces like $node->modulename. Common nouns like 'state' or 'vote', and ambiguous ids like 'pid', 'vid', and 'sid' can easily collide. There is no way to attach metadata to these properties (what module they belong to, for example) without excessively verbose naming schemes. See http://drupal.org/node/148420
• Rigid theming and rendering
  After the node is loaded as an object, its display content is built as a structured array. However, only some chunks of a node's content are included: Taxonomy terms, links, submission information, etc are handled separately and cannot benefit from rendering system features.
  After this content is built, we collapse that array to HTML and use it to replace the existing $node->body variable before passing the node to the theming layer. Themers are unable to control this rendered content, but $node->body and $node->teaser have already been overwritten.
  Themers who want to make simple changes -- altering the order of fields, showing the first field in a list but linking to additional fields, etc. -- have no recourse other than writing a module from scratch to alter the node.
  See http://www.drupal.org/node/134478
• Only two display modes
  Nodes are displayed in a wide variety of contexts and for a variety of purposes, but there's only one way to control what contents are assembled: the TRUE or FALSE $teaser variable. $page and $links also exist, but these are tacked-on special case booleans. Piling on additional boolean params for every special case is obviously unacceptable (witness the refactoring of the l() function).
  See http://www.drupal.org/node/144608
• Single output format
  Nodes are built and rendered for display in HTML. While this is often sufficient, other cases (pdf rendering, XML and JSON output, separation of attachments for RSS and mail, search indexing, etc.) all have additional requirements that can't be captured using the normal node rendering system.
  See http://www.drupal.org/node/145551
• Custom hacks become necessary
  Due to the limitations listed above, there are numerous special case functions in core that wrap node_view() in hard coded workflows, or re-implement it with minor variations. For example:
  node_show(), which forces comment listings to display after node bodies and marks the node as being read by the current user.
  Book module implements a custom version of node_view() when creating print-friendly pages, simply to give modules a chance to modify nodes for print.
  RSS feed generation and search indexing also re-implement node_view(), in order to extract field information before the node is collapsed to HTML.
  Contrib modules are in the same position: many bypass node_view(), to avoid its hard-coded limitations.

Proposed solutions
These proposed solutions are not necessarily dependent on each other. They are definitely complimentary, as they collectively solve quite a few problems. But they can be implemented one by one as we're able to bring development resources to bear. Some -- like the ability to build nodes in a structured form, are certainly central to the plan.

• Build the node as a structured array
  Rather than dumping properties onto the node object, we should build the node as a structured array from the start. We already have a proven format for storing structured data -- the FAPI-style #prefixed array. This allows us to append descriptive metadata onto each piece of data as necessary (source module, access control information). In the future, it's also possible to map schema data directly to the node using metadata in our SchemaAPI.
• Pass structured data to the rendering layer
  Because we're building structured FAPI style arrays on node_load, only a few additional properties are necessary (#type and/or #theme) to make the entire node object renderable to HTML by default. While some modules will obviously need to add data at render time, or modify the default node structures for display, this is less work than rebuilding everything from scratch.
  Passing this structure to the display/theming layer -- rather than a node object with rendered HTML embedded in it -- means more flexibility at render time and less doubling of work when themers need to override the default node layout.
• Support 'viewing styles' for nodes
  The true/false $teaser variable and the true/false $page variable are insufficient for representing the range of settings nodes are displayed in. Instead, we should support a more flexible $style parameter, with 'teaser,' 'full,' and 'feed' being the core-provided defaults.
  Rather than tagging additional parameters like $page and $links onto the build params, a second $options parameter should contain a keyed array of various optional flags to control rendering. Other modules could respond to the contents of this options array as overrides to their normal rendering behavior.
  Contrib modules should be able to expose their own styles beyond teaser, full, and feed. Modules that currently provide configurations options for the explicit teaser/full modes should instead provide configuration options for each option in the list of currently available render modes, in the same way that per-content-type options are often provided.
• Support output formats for nodes (and other data)
  Rendering to HTML is the default output format for Drupal, naturally. We special-case certain output paths to generate RSS formatted XML, but this solution is less than ideal. The 'options' array mentioned above (to control node rendering) should support the ability to control the desired structure (HTML, Javascript, XML, PDF, PHP serialized data, etc.) and format (ATOM, RSS, JSON, PHP Array, etc.)

So. Yeah.

Feedback is welcome. I've got two fairly heavy-duty patches in the queue for Drupal 6 that address the rendering side of things, but the architectural questions are bigger than just rendering -- it's an opportunity for us to carve out a new, more robust system that will help Drupal grow into a more connected, more flexible system.

Thoughts? Talk, follow Drupal hackers!

Comments on this post will automatically be closed three months from the original post date.