Lullabot Ideas

We know stuff. We empower you to know stuff too.

Tales from the D7 Front

Article by Karen Stevenson

Tips and tricks from upgrading several sites from D6 to D7

I wanted to experiment with the D6->D7 upgrade path to see how well the CCK data updates are working, so I was looking around for some real sites to upgrade. Like many of you, I maintain a bunch of 'friends and family' sites in addition to my 'day' job. Usually those sites are the last ones upgraded, but it occurred to me that they would be good fodder for doing some testing of the site upgrade process. I've been upgrading them, rolling back, and re-upgrading them over and over and I found a number of tips and tricks I thought I would share.

Clean House First

You know how when you move you always are amazed at the amount of junk you forgot you had lying around? Your site probably has some junk in it, especially if it is a few years old. These sites date back to Drupal 4.7 and include the results of a lot of early experimentation where I was installing and then removing lots of modules just to see what they would do. All that cruft is still in my databases. Time to clean it out.

First thing to check is whether there are any modules I never used, or previously disabled but never uninstalled. Disabling unused modules gets them off the list of things I need to worry about. Taking the extra step of uninstalling them will often clean a bunch of cruft out -- database tables and variables that they used that I no longer need.

But that won't necessarily find all the extra tables, especially if I installed a module and then removed it without uninstalling it, or used some modules that didn't properly clean up after themselves. I downloaded and installed the Schema module to help me find database tables that I no longer use. It provides a handy report that compares the schema of my installed modules with the ones actually in my database, to find things like those Aggregator tables that I don't need any more.

Any place I'm using Features in D6 that define content types, I need to do one more thing before I upgrade. I need to go into the node_types table and find any content types that have the module name of 'features' and change it back to 'node'. Without this step, when I disable the Features module all those content types will be dropped and will no longer be available in D7. Once everything is ported to D7 I will have to re-create the D7 feature from scratch, there currently is no upgrade path for the Feature itself, but my data will be fine.

Fields are in core in D7, so I won't need CCK's Text and Number modules, or modules like File Field and Image Field. I don't want to actually uninstall any of them though. If I uninstall them my fields will be marked inactive and my data deleted, and I need that data for the field upgrade.

Identify Changed Module Needs

Next I need to think about the modules I used in D6 that I won't need any more in D7, and new ones I will need that I didn't use before. One example there is the Admin Menu module. D7 has a nice built-in Toolbar module that I'm going to use. So I want to disable and totally uninstall the Admin Menu in D6 before I do my upgrade, so that it will clean all the Admin Menu items out of the menu tables. I used Modal Framework in D6, but won't need it in D7 with the built-in Overlay module, so I'll uninstall it, too. I won't need jQuery UI or jQuery Update now that jQuery UI is in core, so I can get rid of them.

I have numerous modules enabled in D6 that define CCK fields that I won't need in D7. I will still need the D7 version of CCK for the Content Migrate module which will upgrade my field data from the CCK format to the format now used by core. In D7 I will also need:

In D7 I could rely on the core file and image handling and not add any contrib modules, but I plan to use the new Media module and its relatives. That whole collection of modules can provide image galleries, image plugins for WYSIWYG editors, and a lot of other nice features. So I have a list of modules that I used in D6 that I won't need any more and another list of modules I will use in D7:

Out With the D6 Media Modules:

In With the D7 Media Modules:

I will also need a couple other modules that weren't required in D6:

  • Entity (required by Field Collection and Media modules)
  • CTools (required by the D7 version of Views)

Do the Upgrade

Now I'm ready to perform the upgrade. I use the Backup and Migrate module to easily create a backup of my D6 database before I start. I have to create an empty database for the D7 version of the site and a folder on my directory where I can put the new site. I won't re-use the old location and old database, I want to leave them pristine so I can go back and re-use them if necessary. And I'm doing this NOT on production but locally so I can be sure everything works before I do anything to my live site.

There are several ways to actually do the upgrade. I can do it manually, or I can use Drush to make it much easier. I'll describe both approaches.

Upgrade Manually

For the manual upgrade I have to download and unzip a copy of Drupal 7 into my new site folder. Then I need to copy the D6 version of the settings.php file into the new directory. The only thing I need to change is the name of the database, if I'm using a different one for the D7 version. Other than that I don't make any other changes, the upgrade process will alter it as necessary for D7.

The easiest, best way to to the upgrade is to start with just the core code, no contrib modules. So navigate to the D7 folder and run update.php on only core. The official upgrade instructions say to disable all contrib modules before you upgrade, which can take a while, especially if you have lots of module dependencies that prevent you from disabling a module until all its dependencies are disabled. Running update.php without any contrib modules in the modules folder will basically accomplish the same goal -- which is to keep any contrib module updates from running until I'm ready for them.

Then I have to find, download, and unzip D7 versions of all my contrib modules into my sites/all/modules folder. That means looking each one up to find the right D7 version of the code. This step is really time-consuming. Part of the reason for cleaning house first is to avoid doing this for modules I don't even need.

Once I have added all the contrib modules I have to go back to my modules page to make sure they are enabled, then return to update.php to allow the contrib modules to do any updates that are needed.

Upgrade with Drush Site-Update

There are actually two ways to use Drush for the upgrade. I'll want to be sure to be using Drush version 4 or higher for best results. I prepare an empty database for the D7 version of my site, as above, and decide where I want to place the new files in my directory. I don't need to create the directory, Drush will do that.

I need one more thing to give Drush enough information to know what to do, I need to create a Drush alias file for the new site. If I already am using Drush aliases, I can add an alias to my current aliases. If I wasn't already using aliases, I would create a new file called 'aliases.drushrc.php', add it to the folder where my Drush code lives, and put something like the following text into it (I can give the sites whatever aliases I want, they don't have to be 'old' and 'new'):

<?php
$aliases
['old'] = array(  // The alias I want to use for the old site.
 
'uri' => 'www.oldsite.com', // The url of the old site.
 
'root' => '/var/www/drupal6', // The physical file location of the Drupal root for the old site.
 
'db-url' => 'mysqli://username:password@localhost/oldsite', // The db_url string for the old site's database.
);
$aliases['new'] = array(  // The alias I want to use for the new site.
 
'uri' => 'www.newsite.com', // The url of the new site.
 
'root' => '/var/www/drupal7', // The physical file location of the Drupal root for the new site.
 
'db-url' => 'mysqli://username:password@localhost/newsite', // The db_url string for the new site's database.
);
?>

Then I just type the following into a command line and watch it go:

drush @old site-upgrade @new

If I want more information about what it is doing I can append '--verbose --debug' to the command.

This will create the new site directory, copy the old database to the new database, add the core files to the new site location, run update.php on the core files, add the contrib modules, and run update.php on them.

Upgrade With a Drush Make File

Using Drush Site-Upgrade is pretty slick, but it does not do everything. It does not find a specific version of module code for my contrib modules if I need something that is not standard or not yet marked as supported. And it does not add the new modules I will need in D7 that did not exist in D6. And in some cases it adds modules to D7 that I won't actually want to use.

A more fine-grained approach is to create a Drush Make file for my upgrade that contains the exact modules and versions that I want in D7 and let me grab them all at once. It will also do things like add external files needed by some of the new modules, like the mediaelement library required by the Media Element module.

The easy way to do this is to go to the old site and type the following:

drush make-generate mymakefile.txt

This will create a Drush Make file for my old site. Then I can edit it to change the core version to D7 and fix the versions of each of the contrib modules to the version I want to use in my D7 site. I can also remove modules I no longer need and add new ones that are required.

Then I create an empty directory for my new site, place the make file in it and type:

drush make mymakefile.txt

That will populate my site with all the new code. I copy the settings.php file from the D6 site to the D7 site, copy the D6 database to the D7 database, navigate to update.php, and run update.php to update the database. As noted earlier, I may want to add an extra step of moving the contrib modules out of the sites/all/modules folder while I run update.php on only core, then move them back and run it again with the contrib modules in there.

Final Steps

I can then go to admin/modules and enable any new modules I need plus several modules that won't be turned on by default in an upgrade:

  • Toolbar
  • Overlay
  • Image
  • Contextual Links
  • Dashboard

Many things were updated automatically but if I was using CCK in D6 I won't see any fields in D7. I have to update my field data as a separate step. I install the Content Migrate module (in the D7 version of CCK), then navigate to admin/structure/content_migrate.

There I will see three sections: a list of all the fields that are available to migrate, fields that cannot yet be migrated (because I don't have the right modules installed and available yet), and fields that have already been migrated.

After I migrate the fields to move the D6 field data to D7, I can go to the content types pages to confirm that the fields are there and structured correctly. I can roll the migration back if it doesn't work right and try it again later until I have the results I need.

Conclusion

I hope these tips are helpful. The upgrade process is never totally painless, but hopefully these ideas will make it a little smoother.

Comments

Great tips

Hi!

Great tips on how to upgrade from D6 to D7. This comes at the right time for me. I also have a question.

I'm updating a website from D5.x and I intend to take it all the way up to D7. However, my theme framework (zen breaks quite considerably because they've changes the way sub-themes work ) and a whole bunch of modules have changed quite drastically... Drush does not find the appropriate versions because the current versions are no longer supported. What would you suggest in this situation?

You'll probably want to use

You'll probably want to use the Drush make file method. Create a make file from your current site as a starting point. Figure out which modules are no longer needed and remove them from the list. Figure out which news ones are needed and add them. Then go to each project page to see what is the right version and put that version into the make file.

Excellent tutorial, Karen

Thanks for the excellent tutorial for the upgrade process from D6 to D7. It covers lots of things I was fuzzy about in clear, easy-to-understand language. I'll give a shout-out to the Docs team.

Media Browser

Awesome info, many thanks for writing this!
One thing, the link to Media Browser is coming up with the abandoned Drupal 6 module. Did you mean Media Browser plus?

http://drupal.org/project/media_browser_plus

Looked for just plain "media browser" for Drupal 7 but couldn't find it.

Thanks for a great article.

K.Webb

Yep, good catch. I made that

Yep, good catch. I made that change in the article. Thanks!

Edit settings.php?

Thanks for a nice write-up. I have just one question: You say that you copy the settings.php file from the D6 site to the D7 site. Don't you have to edit the database settings too - so the new D7 database is used? Or did you consider this so obvious that it wasn't worth mentioning ;-)

No, you do not make any

No, you do not make any changes to the D6 settings.php file. When Drupal sees the old-style settings.php information it knows you are doing an upgrade and will fix it for you. If you change it manually Drupal won't realize you haven't yet run the upgrade process.

[edit] Actually, the one change you *would* make is to change the database name if you are using a different database for the D7 version of the site. But don't try to create a D7-style database info array, leave it in the D6 style.

I was in fact thinking about

I was in fact thinking about the database name. I didn't understand how Drupal 7 could guess the new database name ;-)

PS! Looking at your Drush aliases it seems you too changed the database name and hence for clarity you should mention that you also edited settings.php.

I see your point and I made a

I see your point and I made a change to try to clarify that :)

empty database?

Nice tutorial, makes me less scared...

But I'm missing a point somewhere. What do you mean create an empty database? That would just install D7, not upgrade? Surely I need to upgrade the D6 database with all its content?! (and yes, keep a backup somewhere)

I don't make changes on the

I don't make changes on the original database. I create a new one and copy the D6 data to it. That way I always have the original as a fallback. If you prefer to write over the original you can, once you are completely sure you have a reliable backup.

awesome tutorial

Yep-- yet another amazing tutorial from one of the bots.

However, I can't help but wonder how we could ever expect a 'normal' human being to handle the upgrade process.

I fully agree with the 'drop is always moving' philosophy (so I'm not one of those that take every opportunity to lobby for longer support periods or more supported versions), but do we seriously expect non technical users to take this process on?

I totally agree it is

I totally agree it is challenging. Lots of effort has gone into trying to make it as seamless as possible, and it gets better with every Drupal version, but it is definitely not painless. If you are a developer you can jump in and try to make this better in D8 :)

I keep trying to dive into

I keep trying to dive into core dev but I always end up feeling way over my head and drowning, lol. But you're right-- time for me to try again!

Great Upgrade article

Thanks, Karen. I wish I had read this before I tried an upgrade earlier in the week. The drush steps would have helped me. Unfortunately, I had to roll back to D6 since there are some issues with Views, Calendar and Date fields.

cleaning house

In addition to using schema to clean out unused tables, you'll also need Variable Cleanup module to clear out unused variables:

http://drupal.org/project/variable_clean

Good start

Nice to see some decent documentation/tutorial for a tricky process. Funnily it stops at the exact moment it becomes interesting.
Tried a sandbox upgrade myself a few weeks back; the custom fields seemed to go okay. Things started to go wrong in the Imagecache section (presets not being recognized and images disappearing). But the main problem was that I rely heavily on Nodereference and all these links were gone. Not a clue if anyone is working on an upgrade path for that module...
Also curious about the state of Thickbox. Colorbox seems to be its successor but that does not feel very safe yet.

Normal Human Beings Can't Follow the Upgrade Path

I'm a normal human being and I can't even follow the upgrade process. I realized drupal is free software, but for 95% of drupal users version 6.x is the end of the road. If I knew it would be this confusing, I would have waited for drupal 7 to launch and not started on drupal 6. Silly me for thinking the upgrade process would be doable my a mere mortal.

It is easy

Dear Anonymous:

I'm a normal human being too. The upgrade process for Drupal seemed daunting at first, but then after reading and re-reading the documentation, and trying it out myself, I find that it's actually surprisingly easy to follow.

Upgrade With a Drush Make File?

Hi Karen and thanks for the nice article.

I've run into one problem though as I'm running multi-sites off one code-base. I'd like to update one site at a time using Drush. I tried to create the makefile but I get an error using the following command:

drush -l http://mydomain.com make-generate mymakefile.txt

The error is: The drush command 'make-generate mymakefile.txt' could not be found. [error]
I'm using Drush ver 4.2
What am I missing?

Sounds like you don't have

Sounds like you don't have Drush make installed. It is a separate module, not included with the rest of Drush automatically.

** Blush ** Yup, you were

** Blush **
Yup, you were right. I did not even know about the Drush Make module.
Thanks.

Schema Module

I would so love for someone to do a schema module how to! Any chances you might consider? It seems both really important and powerful, but sorely under-documented.

Thanks for this doc!

An intro video

Hey Ryan, I have a short intro video for schema (http://www.lullabot.com/videos/database-fun-schema-module). It is a bit old and is for Drupal 6, but the basics should be there. I agree that a more in-depth look would be nice though. (I'll add that to my list of topics to tackle. ;-))

Super useful!

Thank you! Thank you! This is such a useful resource. A great primer as we just start to dive in to D6-7 upgrades! I really like your "clean house" related tips, and the info on CCK!

Great Post

Fantastic post. I'm looking to build some D7 sites soon and make heavy use of Media, so your list of modules was brilliant and just what I needed. I've been aware of the Media module for D7 for awhile but not most of the others.

Also looking to upgrade some older D6 sites to D7 this summer, so once again, thank you for the well written post. Rock on!

Couple things

First, I have tried both Toolbar and Admin_menu. No comparison. Admin_menu is my goto for running the site - forget Toolbar.

Second, I haven't yet done an upgrade from D6 to D7, but I know from D5->D6, you couldn't just throw all your contribs in at once. They had to be done in a certain order, which I had to find myself. For example, Date had to be installed early in the process so that date fields got updated before Views looked at them (particularly Calendar). I ended up having to do the contribs in about five waves to get them done right. Hopefully, with Fields in core, this will be less of an issue.