Posted on September 16, 2011 // Short URL

Drupal Drama: Burnout, D7 Retrospective & Product vs. Framework debate

In this podcast, Kent Bye talks with Jeff Eaton and a couple of the top Drupal core contributors from Drupal 7, Daniel "sun" Kudwien and Nathaniel "catch" Catchpole. They discuss the maintainability of Drupal core, a retrospective of the ups and downs of the Drupal 7 development cycle, and alternate roadmaps for the future of Drupal core. Developer burnout is also becoming more of an issue, and they talk about the importance of recruiting more "code gardeners" to help review and evaluate core patches.

Although the online comments about these topics have been contentious at times, we feel that they need to be discussed, with the goal of improving Drupal for everyone. Listen in for a wide-ranging and engaging discussion about the Drupal project and the concerns some of the core contributors are wrestling with today.

Show notes

At the beginning of DrupalCon London, one of the top Drupal core developers named Daniel Kudwien (aka ""sun") posted a blog called "The Drupal Crisis" about the woes of the Drupal 7 development cycle. He claimed that "Drupal core is not maintainable anymore. There's too much cruft. Too many half-baked features that no one actually maintains." and that "We need to be a solid framework, and also a basic but extensible CMS, like we've always been."

After lots of discussion on this post, sun wrote a follow-up blog post with a number of conclusions, including, "We need to discuss, propose, agree, and define what we are able and want to maintain in Drupal core," "which core features are actually product features," and "whether we can or want to move product features into a separate product project in the long-term."

Channelling this interest into action, sun created a number of different issues within the drupal.org issue queue including this list of Discussion items and this "Make core maintainable" issue, which generated nearly 300 comments.

The "Make core maintainable" issue quickly turned into a discussion of which modules to remove from Drupal 8 core, and the question of what should stay and what should go led some core developers like Crell, eaton and catch to call for a more defined set of heuristics that the Drupal core maintainers could use in determining what should be included within the Drupal core product.

This discussion eventually led Jeff Eaton to create a spin-off issue called "Establish heuristics for core feature evaluation" that tries to lay out a list of criteria for evaluating which features should stay and what modules should go.

Asking what modules belong in core and which ones should be removed can be answered differently depending on whether you see Drupal as a primarily as a generalized CMS product or whether you see it as a development framework for a specific site. It's actually both a framework and a CMS product, but this tension has brought the Drupal project to a crossroads when trying to make these types of decisions of what to include in the main project and what core developers are responsible for maintaining.

Eaton talked more about this tension between product and framework at DrupalCon London in his presentation called "Product, Framework, or Platform?" Instead of creating a false dichotomy between the Product vs. Framework, eaton is using the term of "Platform" in order to incorporate both the CMS product and anything else that can be created with the module system and development framework.

Another top core developer Nathaniel Catchpole (aka catch) wrote up a couple of related blog posts including "Why all the drama?", where he does a bit of a retrospective as to what went wrong with the D7 development cycle from his perspective.

Catch also took a stab at expanding on the Drupal Framework vs. Product debate in another blog post called "Framework, Platform, Application, Features, Product, Workload" where he reiterates that Drupal is a hybrid of the two. He lays out a more nuanced and complicated 5-layered system for how he thinks of it, which is mentioned in this podcast a couple of times.

At the bottom of his post, catch points to a number of follow-up and related initiatives. Catch and sun created the "Unofficial Drupal 8 Framework initiative" that includes a lot of the low-level API clean-up.

In this podcast, catch and sun talk about the need to simplify and de-couple the system.module's responsibilities in order to do unit testing and make core debugging more manageable. Here's a list of issues tagged as part of this framework initiative.

There's also a number of issues tagged with the Platform Initiative, which is centering around the heuristics for which features should be in Drupal core, and which ones should be removed.

Another related project is eaton's Snowman initiative in order to create a number of installation profiles that give users a more options for what flavor of Drupal product that they'd like to use. There's also a meta issue from eaton "[meta] More flexibility for core installation profiles."

A related topic throughout all these discussions has been burnout of core developers. Randy Fay held Core Conversation about Burnout at DrupalCon London that is well worth watching. Fay also looked into a lot of research about burnout, and wrote an excellent series of blog posts:

In his Drupal Crisis post, sun says that "there are many more Drupal contributors who kept silent about their personal feelings for too long. In the end, we have several critical communication problems about fundamental topics in our community." This has been contributing to the feeling of burnout.

One solution to core burnout is to recruit more help for the core development workload. Sun and catch talk about the need in the Drupal core development community to recruit and mentor more people who "code gardeners" can test and evaluate patches.

Bringing up issues and having productive conversations are the first steps to finding solutions. We hope this discussion provides a valuable contribution to the conversation.

Other related links

Brought to you by the Do It with Drupal Conference October 12-14th, 2011 in New York City.

Comments

Steven Jackson

Developer Burnout

I recently left a pretty large Drupal consulting firm due to burnout. I've been on the Drupal track for several years and think it's a very useful framework but much of the Drupal "development" or consulting community consists of non-computer science or programming background individuals. So much of Drupal can be configured with clicking around on some contributed modules (which is great from a user/client standpoint) that we have had this surge of people who understand Drupal but not development and as soon as it comes down to really getting to use Drupal at it's base level and creating new and exciting modules or functionality - or simply altering some contrib code - the consultant workplace grows chaotic. In the firm I was at, of all the 30-50 consultants, I was the only one with Computer Science degrees and, importantly, the only one with even a Science based degree. I know that degrees do not necessarily make one a good programmer and there are plenty of people without degrees who could program circles around me but these are not the developers attracted to Drupal or at the least the developers who stick around for very long. The community is now saturated with non-developers trying to traverse a robust and complex tool and in the end putting up shoddy websites that look pretty. These non-programmers, however, are really good at selling their 'expertise' and are edging out developers, causing a mass exodus of core developers from the project.

Reply

Steven Jackson

To my point about developers

To my point about developers vs. non-developers, with computer science or strong programming backgrounds comes a fundamental understanding of data structures, process and data modeling, and the general development process. What we have more and more in the community are what I call "if" scripters -- people who say they know development but when it comes down to it, they mean they can write an "if" statement to test some predefined variable to make that so-sweet cornflower blue appear instead of #fff; but throw some basic object or array creation/traversal or sql CRUD and their facade crumbles. I heard an interview one time where the "developer" said they had extensive MySQL knowledge but when the interviewer as the difference between left-join and right-join they choked - this interviewer also asked about basic class inheritance. It was a very smart tactic to weed out the wrong people. I bet they are a very strong Drupal firm.

Reply

earthday47

Respectfully disagree

I don't think we have too many non-Computer Science types. I think programming knowledge is essential to do anything productively with Drupal, even theming, which would is the usual design-centric part of a CMS - and that's the problem.

I have a Computer Science degree, I know plenty about MySQL, procedural and OOP, CRUD, and all the things you mentioned, and it still took me a year or more to be comfortable with Drupal's APIs - and I still regularly struggle with it. I think it's supremely unfair to all the smart so-called "middle of the road" developers to call them the "wrong people".

The whole point of a CMS is to empower more people, of all skill levels, the ability to be cool things.

Reply

Mark

CS Degrees aren't the answer

Before I start I want to point out that I am one of the people in the community without a CS degree and I tend to get frustrated when this gets brought up.

I am sure that over the past 6 years while I was learning development and working on Drupal sites at the same time I made a lot of terribly stupid errors, I'm too terrified to even look at some of my early code to know for sure. :)

What exposure to Drupal has allowed me to do though is learn a lot of the discipline of software development without going into debt to learn Java or .NET development from a middle of the road university's computer science department. This is a very important part of what Drupal is to me and I would hate to see that change.

In my experience a CS degree on someone's resume is not a sure indication of performance. Love of your craft and a desire to continually get better are a far better indication, but that is incredibly hard to measure. I would say that a CS degree is a symptom of the former, but it is definitely possible to slog your way through a CS program and still be a terrible developer.

I agree we need a way to cut through the facade and call out bad code from mediocre developers, but how do you do that without scaring off the next crop of contributors?

Reply

earthday47

Dries' responses

The group mentioned Dries' response to this product vs. framework debate. I agree that it was a bit dodgy. I remember thinking that starting D8 development at Chicago seemed rushed, but I can understand why. Everyone was so frustrated with the delay. But, we needed to sit back and analyze the process, refine, and then jump into Drupal 8. This is kind of the fallout of that omission.

Reply

Chris Calip

I like Jeff Eaton's

I like Jeff Eaton's Postmortem suggestion and it's true it cannot be a day process. Might I suggest core dev team do a week or month long retreat.
Just floating the idea.

Reply

wernerglinka

Core + Contributed Modules + Install Profiles = Turn key product

I am a themer and site builder, not a developer but I have managed large development efforts and have been active in various volunteer organizations. I can second what Eaton pointed out: The problems Drupal development is encountering are nothing new. And like in any volunteer driven organization you always have a buzzillion opinions on what to do, so here is mine:

I seems like the basic structure to deliver a turn key product is already in place with installation profiles. With that mechanism in place doesn't it make sense to have the core effort concentrate on core functionality, performance and a forward looking api, e.g. take out ALL product features. Match available dev resources with the project scope. That will ease the time demand of core devs.

Reading some comments on Drupal.org it seems like some people want all high-use modules in core. I believe that is exactly the wrong direction given the limited dev resources. As Drupal grows in complexity it will get more and more difficult to get devs who understand the whole system. Putting emphasis on install profiles might get new contributors who are not core dev class but serve the idea of being able to do something out of the box with Drupal well. Isn't the call for a build-in back-end wysiwyg editor easily solvable with an install profile. Maybe there needs to be some thinking around how to make it easier to create install profiles.

So Core + Contributed Modules + Install Profiles = Turn key products would satisfy users who demand Word Press ease of installation and also provide a lean, fast, easy to maintain framework/engine.

Reply

Sean Boran

interesting discussion, great team

This Podcast has been a valuable adding to the posts I'd already seen from sun and chx. Listening to catch, sun and eaton together was motivating indeed . I like the ideas, open discussion and vision they have going forward. Thx to Lullabot for this poscast!

Reply

John Jacobs

Levels

Hi Jeff (and Jeff and all the Lullabot crew)
Hey what a great episode of the LB podcast, thanks so much for tackling head on some thorny issues, I love it when you really dig into some Drupal stuff for us, this truly is a great example of what makes the open source community conversations so great!
I have been listening to your podcasts for many years now and really have gotten so much out of them.
Okay down to the niggle. I am prompted to comment now because this great show could be so much improved by a simple audio mastering technique. My request? Please normalise your sound files before publishing them, there is at least 6db (double the volume available to you at the push of a button. Very significant on the bike at the gym and on public transport, in other words all the places that your nerds fans listen to you.
Keep up the great work and thank you very much from a Drupal fan in Australia
cheers John

Reply