by February 19, 2007

Best practices in open source development

However, gaining the full advantages of open source requires a fundamental shift in development practice. And people who are not "in the know" on this point can often get into trouble when they continue to work in a traditional way, and a negative attitude about open source software (it's not me, it's them!) can result.

ed to just accept whatever functionality comes with it; you can modify the software to do whatever your heart desires, and you're never "locked in" with a particular vendor when you need additional functionality. There is a community of people developing, using, and testing the software, which tends to lead to higher quality and faster growth. Plus, in terms of initial costs, it's generally cheaper than proprietary software (often times, free).

However, gaining the full advantages of open source requires a fundamental shift in development practice. And people who are not "in the know" on this point can often get into trouble when they continue to work in a traditional way, and a negative attitude about open source software (it's not me, it's them!) can result.

This article provides some best-practice tips and advice that we at Lullabot employ while working on on development projects. While this article talks primarily around our experiences with Drupal, its lessons should still apply for anyone working with open source software.

Best Practice #1: Learn how to Extend The Software

Drupal has a little motto that goes something like:

If you have to "hack core" (change core files) to get Drupal to do what you want, you're probably doing something wrong.

This advice holds true for most other open source projects; always perform investigation into the tools and techniques the software makes available for customization. These may include a "skinning" or "themeing" system to customize the look and feel, the ability to extend functionality through modules, extensions, and add-ons, or configuration options that can be altered to fit your needs.

Your first goal when working with open source software should be to try and "get in the heads" of the developers of the software, and determine how they intend for the project to be extended. While it can appear easier on the surface to just bolt on a chunk of functionality you're missing, doing so means inadvertently undoing all the collective work and thinking the community around your project has poured into the problems you're facing.

Learn what tools and techniques are available to you, and you'll have a much easier time building your project.

Best Practice #2: Do. Not. Fork.

The second you change any of the default core files of an open source project in any way, even for something as small as changing the text here or there, you have created a "fork." This hurts you in a number of different ways:

  • Difficulty in getting support. Your changes may cause subtle bugs to appear, and you're going to have a very hard time finding other people to help you track them down, if they can't reproduce the problems themselves.
  • Difficulty in upgrading. Each change you make, no matter how minor, needs to be carried over each time you upgrade the site's code base. If you forget a change, your site behaves differently. If a security patch changes a line where you've made a customization, you then have to hunker down and do a bunch of analysis and testing as you attempt to merge the changes and ensure that you're still getting the proper fix.
  • Maintenance headaches. Because this is your code that you've written, you're on the hook for maintaining it, testing it, and improving it. More on this and why it's a bad thing in a bit.

Resist the urge to fork, and instead...

Best Practice #3: Participate within the community

There are many advantages to using open source software, but probably the biggest advantage is the virtually limitless resources involved in the open source ecosystem. Millions of people around the world are testing the software, fixing problems, and adding new features. It's simply impossible to reproduce this kind of momentum with a small team (or even a very large team).

Yet, many people only see open source software as a cheap (or free) alternative to building things from scratch, and completely ignore the community aspect of the software. This is detrimental, both to you and your clients, and to the software project itself.

Let's imagine a scenario where you find a bug in your open source project of choice. Your natural inclination might be to just troubleshoot and fix the bug in your local copy, and then move on with your day. After all, we're all busy people, and client deadlines aren't getting any shorter.

An alternate, and more effective, approach is to do the following:

  1. Search for the bug. When you very first encounter a bug, before you even attempt to fix it, your first instinct should always be to search for the bug in the issue queue or bug tracking software used by the project. It's possible someone has already found the bug, and reported a fix for it. If so, you just saved yourself some work.
  2. Report the bug. If the bug isn't there, the second thing you should do, again before you even attempt to fix it, is submit a detailed bug report. Often, developers are very active on issue queues, and they are also more familiar with their code base, so they can often find and get to the root of a bug before you can.
  3. Submit your fix. Assuming someone in the community didn't get a fix out to you before you figured it out, always submit your fix back. Not only is this good "karma," as you've just saved the next person from having to do this (and karma in an open source community is your very best asset), but often you can get good feedback from other developers as to whether or not your fix is the best solution, and either way, you help ensure the fix gets put into the "official" product, which means you no longer have to maintain it.

Let's use another example. A client needs some crazy new feature for a site you're building.

  1. Search for an existing module. Even if there's not a "100% fit," a 90% fit (or even 50% fit) is a better starting place than nothing.
  2. Submit a feature request. If there's a module that will work as a starting point, just as with bug reports, before you start coding anything submit your intent to work on the additional functionality. You may get other people interested who will offer co-development, testing, or even just some cold, hard cash toward its development. Or, the module author might chime in with how they already tried it, how it didn't work, and how to use X instead.
  3. Contribute new code (a new module for example) to the community. If there's not already something out there that fits your needs and you need to build it, submit it to the community in whatever they're using as a "forge" system (CVS, Subversion, etc.) If possible, the best thing to do is place your code there as early during the development as possible. This allows others to try out your changes and provide testing, bug reports, and usability feedback long before it reaches your client's site. Procrastination can create more custom code for you to maintain: lots of people have really good intentions of committing stuff back when all's said and done, but often it's easy to get side-tracked.
  4. Work out of the issue queue. When you add a new feature to your module, make a feature request for it, attach a patch, and commit it. When you fix a bug, make a bug report for it, attach a patch, and commit it. The discipline you show here will help pay off ten-fold by getting more eyes on the code, and by making changes small and easy to isolate should something go wrong.

By pushing fixes and features back "upstream," you help ensure that future sites you build will have the functionality you're looking for, rather than having to constantly re-invent the wheel. You also turn into a collaborator on the project, rather than a user.

Advantages of following best practices

Working in this manner has the following advantages:

  • It helps save you time (and saving time saves you money). You might find answers before you even begin your quest. Another developer can come along and code something before you even have the chance to do so.
  • It earns you karma. In addition to the "warm fuzzy feeling" that contributing gives you which is a reward in its own right, open source communities tend to be "meritocracies." Opinions of individuals are generally weighted by the community, either consciously or subconsciously, by the level of contribution they've given to a project. Gaining a reputation as a contributor often means your questions get answered sooner, and it also means more business for you as new clients see you being committed to the platform, and other contributors in the community refer additional work to you.
  • It increases the quality of your code. More eyes mean more testing, more feedback on the approaches you've taken to problem solving, and in the end more flexible, usable, and easy-to-understand code and functionality.
  • It lets the community help maintain your stuff. Putting stuff out in the community means you increase your chances of someone other than just you using your code. And that's a very good thing! These other people will contribute their own bug fixes and features to your module, and help test it for bugs. They can also help with upgrades between major versions. With custom code, you are the only person who is able to do this.

Disadvantages of following best practices

There are not really any, other than it takes a bit more time to do up front. You can help offset that by building that time into your contracts, and by client education. Explain to your clients about how open source is different, and that in order to get the very best "bang for their buck," it's important to leverage the community development model. This will help ensure that they enjoy all of the benefits of open source, including not being "locked in" to a proprietary fork, having the highest-quality code, and being "future proof" for upgrades.

"Real world" implications

Lullabot has built a number of sites using these best practices, which has resulted in thousands of contributions back to the Drupal community in the form of themes, modules, and core patches. That's a win for us, a win for our clients, and a win for Drupal. You can't get much better than that. :)