Mike and Matt discuss various aspects of Lullabot's Support & Maintenance Department, and how it differs from Client Services, with Director, David Burns, and Project Manager, Cathy Theys.

This Episode's Guests

David Burns

Thumbnail
That special place where people, code, and business requirements meet is the place that I want to be.
Transcript

Transcript

Matt Kleve
For April 1, 2021, it's the Lullabot podcast. Hey everybody, it's the Lullabot podcast, episode 252. I'm Matt Kleve, a senior developer at Lullabot. With me as always, cohost of the show, senior front end dev, Mike Herchel. Hey Mike.
Mike Herchel
Hey Matt, how are you doing?
Matt Kleve
I'm pretty good. We're going to do this thing again, it's been a while.
Mike Herchel
It's been too long.
Matt Kleve
Let's get the band back together.
Mike Herchel
Good idea.
Matt Kleve
Google tells me there are millions of Drupal websites on the Internet, and we at Lullabot know about a few of them.
Mike Herchel
We do.
Matt Kleve
Yep. And Google also tells me that Drupal 9 was released on June 3, 2020.
Mike Herchel
Sounds about right.
Matt Kleve
I'm going to guess that all of those millions of websites are running Drupal 9 and the latest greatest hotness of Drupal leading edge code.
Mike Herchel
I would maybe step back from that one, Matt.
Matt Kleve
We're in a state where there are Drupal 7 sites that exist, there are Drupal 8 sites that exist, there are Drupal 9 sites that exist, and they're all different in their own special way, and they're all actually technically supported and up to date at this time, right?
Mike Herchel
Yeah. So how do we handle this with so many different versions?
Matt Kleve
We have some friends in Lullabot that do that for us now. We've got people for that, right?
Mike Herchel
Yes.
Matt Kleve
We do. On the podcast today, we have our Lullabot's director of support and maintenance, coming to us from Philadelphia, David Burns. Hey Dave.
David Burns
Hey Matt. Hey Mike.
Mike Herchel
Hey, how are you doing?
David Burns
I'm doing well. Very glad to be on the podcast with y'all today.
Mike Herchel
Thanks for being on. Also with us is our support and maintenance project manager, Cathy Theys, coming from somewhere Virginia, Washington D.C.-ish, right?
Cathy Theys
Yeah, I like to keep it vague.
Mike Herchel
Yeah, you're a little bit of everywhere. How are you doing today?
Cathy Theys
I am doing super. I'm really happy to be talking about maintaining Drupal sites.
Mike Herchel
It's an important topic.
Matt Kleve
It is an important topic. A lot of times everybody talks about people implementing the latest, greatest, the new APIs, the best way of building new things with all of the coolest new tools, but at some point all of that new stuff gets put into use and not everybody is undergoing a huge website build all the time, and that's why the Lullabot support and maintenance team exists, right? How long have you been doing that?
David Burns
So officially, Lullabot support and maintenance, I looked at the existing SOWs, began January 1, 2019, so really only two years ago.
Matt Kleve
And what kinds of things do you do?
David Burns
Well, before we get into that, I'll just give a little more background than just the start date. So the reason that we got into support and maintenance is because we had a few people that were working on projects and Lullabot tries to have one developer on a project for full-time, like that's the only thing you work on and it can be for three months, six months, however long it takes to build that project.
Matt Kleve
For example, Mike is a front end developer and Mike you're on one project, right? You're full-time committed to the build out of this one project.
Mike Herchel
Absolutely.
Matt Kleve
I'm on a project, I'm full-time committed to this build out of this one project. But that's harder when you were trying to do support, right? That's what you're trying to say?
David Burns
Yeah, it's much harder and that's kind of why the department came into existence. So we started getting inquiries that like hey, let's have you work on this site that you've already built, stick around, hang out with us and fix bugs and implement new features. And we started taking on some of that work, but it wasn't officially under a specific department. And I was brought in to assist the people that were doing that type of work at the time and it was working well. So Seth and Matt and people from the leadership team reached out, it was like hey, there's a need for this in Drupal and there's other Drupal shops doing this, so let's commit to doing it. And that officially gave the Lullabot support and maintenance brand.
Matt Kleve
So can you talk a little bit about the mindset that might be a little bit different to somebody's who's doing maintenance on a Drupal site?
Cathy Theys
I think one of the things that we think about is the scope of the request from the customers. And so we're like is this a maintenance type task, is this a security update, is this a regular kind of update, is this a bug fix, is this a feature request, how big is the feature request, what exactly are they asking for? And so I think the overall mindset is always wanting to do the little maintenance tasks to keep things running smoothly and nicely, but keeping your eye out for requests that are more than they seem to be.
Mike Herchel
And that never happens with web development, does it?
Cathy Theys
Never.
Matt Kleve
A lot of times people talk about the metaphor of building a Drupal site with Lego, right? You have a bunch of Lego blocks and you're building this huge house or something. And I think a lot of times when you're maintaining a site or working with an active site, long term, swapping out that one blue block might be a little bit difficult too, because of all the other things that it's resting on it would seem, right?
Cathy Theys
Yeah, you need to be careful. We have a lot of tools that we use to catch unintended changes. And helping our customers utilize those tools and bring those into the project is something that we do that might be like a big project, but its goal is to increase automation to make maintenance easier. So we do some automation to help catch unintended consequences.
Mike Herchel
Let's jump into ... This is one of the questions that I had for you right here is you have updates, you have dependencies, you have imports. What types of things can you automate, when does it go wrong? What type of things are still manual?
David Burns
So one of the things that is challenging about support is the context switching, the amount of context switching we do, right?
Matt Kleve
Which is kind of why developers who are on full-time projects, like Mike and I, don't end up changing projects. That's something that Lullabot has carved out in saying, "Context switching is hard, it's best if our people can stick to one thing," but that's not what you get to do, is it?
David Burns
Not exactly.
Cathy Theys
No, we get to specialize in context switching. That's like-
Matt Kleve
Well, that's fair. That's fair and that's an important skill.
David Burns
I like that. So one of the things that we do to help reduce or streamline context switching is we standardize on stuff, right? So there's DDev that allows you to have a config file and you run a command and it spins up in local site.
Matt Kleve
It sits in front of Docker, right, DDev?
David Burns
Yeah, DDev's built on top of Docker. So that's one system, but we started using Lando way early on, like one of the first-
Matt Kleve
Which is the same idea, right? I've used both in consecutive projects and I couldn't tell you the difference other than I used a different command to do stuff.
David Burns
Under the covers, under the code that's running, it's all Docker based. It's all the different layers that get added on top of it and the command you can use and the pre hooks that exist that let you do more advanced things. So as a department, we decided we're going to use Lando, and what we've done, we started out with Lullabot.com, which is our own site where we get to explore and test and figure out scripts-
Matt Kleve
Which Lullabot hires you as a maintainer for Lullabot.com, don't we?
Cathy Theys
Well-
David Burns
I'm not sure if hire's the right word. It's like you do maintenance and we're sending you a paycheck, so take care of this thing that our whole company cares about.
Matt Kleve
Well, we do, yeah.
David Burns
And I'm totally on board with that.
Matt Kleve
Okay. Okay, that's good. Sorry, Dave.
Cathy Theys
Yeah, but it's a great guinea pig though and we're really empowered to try out things and it's a very safe environment for us, because we know that if we impact the customer, we're impacting us and we'll fix it right away. So it works out really well to let us test run things that we might want to maybe future standardize in support, we can try it out on Lullabot.com first.
Matt Kleve
So you were talking about putting everything into Lando. So if you were starting with a new client, do you end up then putting all of their stuff into a container, into Lando and going from there?
David Burns
Yeah-
Matt Kleve
Is that step one for you because you need to specialize your process?
David Burns
So some of the projects come to us already having these type of tools implemented. Some of them may be using VirtualBox, the VM system, and we'll take a look at that and be like, is there enough reason to stick with doing that and what is the time it takes to just swap in all the fancy good stuff that we know works well with Lando? And I would say 9 times out 10, we wind up integrating Lando.
David Burns
We do some really neat stuff that when you do the start, it'll go ahead and fetch the latest database and not import it, but give you a command in the output and says, "Just paste this and you'll get a fresh copy of the site running locally." We do fun stuff for some clients where we actually compile themes when we run Lando start. We were noticing that some of the clients were coming to us and be like, "Hey, this page looks different than the live site, why is that?" And we would get up and say, "Hey, remember to build the theme."
David Burns
When you start repeating things over and over again, you're just like let's script this, let's have somebody fresh to the project be able to come in, run one command, and basically have the whole site running, and that's what Lando's been able to provide for us.
Cathy Theys
Yeah, it's a really good investment that we will look at, because sometimes like Dave said, we'll keep what automation the customer already has. But if it's compelling or if they don't have one and we set up Lando, that investment really pays off later, because we have people switching projects. Even if they stay on that project, they're still going to be switching in and out of it, because they're switching between three projects.
David Burns
Yeah, it's not uncommon for me to have a day where I'm running Lando Start on four different sites.
Matt Kleve
I'm going to guess you need a decent sized hard drive for all that.
David Burns
Yes. We have some massive projects, Georgia.gov, GovHub is one of them.
Matt Kleve
Which was a Lullabot project that we then passed off into the support world, right?
David Burns
Yeah, and that's how we get a fair amount of the work within support, I think over 70% is passed over from client services. But we were noticing that when we were onboarding new people to the GovHub platform, we'd have to bump the default Docker setting from 80 gigabytes of space up to 120 gig, just because how massive the platform is, how many databases it's pulling in.
Matt Kleve
And that's not a Docker issue, that's just the website is huge, right? It's a big multi-site website and yeah, there's just a lot going on there.
David Burns
Exactly.
Mike Herchel
So outside of local environments, is there any other automation and, or standardization that you can do across the different projects?
David Burns
That's actually been a big push as of the last quarter of last year and a major focus of this year. We have a lot of clients that are on Pantheon, but we also work with some that are on Acquia, some that are self-hosted. And the thing that allows us to context switch quicker, reduce the amount of errors is to after you're committing code locally and pushing up to be able to build environments and see the work that a developer's doing, and a Lullabot built tool called Tugboat QA is perfect for that. That integrates with GitHub, GitLab, Bitbucket, even privately hosted repos.
Matt Kleve
Tugboat.qa is the website, and we'd done a previous podcast with the Tugboat folks, Mike.
Mike Herchel
Yeah.
David Burns
And what's great about Tugboat, it has integrated the Lighthouse integration for Google that gives you SEO scores, performance scores, accessibility scores, at a very general level, you still want experts to go in and do the fine detail stuff. But we're able to look at that, we're able to look at the visual diffs that come with Tugboat. We provide one of each content type, the major landing pages. We even have a workaround that allows it to screenshot and do visual diffs on node add, node edit, and editing pages, stuff like that.
Mike Herchel
That's pretty cool. Can you just explain for our listeners what a visual diff is really quick?
David Burns
Sure. So we have this concept of base previews, basically what does your live site look like right now. And then you make some code changes, you commit it to Git, Tugboat spins up an environment. And then Tugboat goes hey, let me take a screenshot of this new PR that you've been working on and I'm going to overlay it with a screenshot of that base preview and it's going to do a pixel by pixel check looking for any variations.
David Burns
And Tugboat's able to say this page is like 98% perfect, and sometimes these slight variations are a different image showed up or something shifted. Anything above 90, I wouldn't be too, too concerned about. But once we start seeing numbers in the 80s and 70s, we're like hold on, we definitely don't want to merge this in, in its current state, let's take a closer look at that specific page and figure out what's going on.
Cathy Theys
Yeah, it's really useful.
Matt Kleve
As you know Mike, and I'm sure Dave and Cathy, CSS cascades and a small change here might affect elsewhere, right?
Mike Herchel
CSS-
Matt Kleve
And so a visual diff would super be handy to do that, and that's what I was thinking about. I know that a lot of times when I come on to a new project, even if it's not a support one or they have an existing website, I get a ticket to change something and then I start working on it and I'm like I hope I'm not breaking the rest of the website with this change, but I think I'm doing the right thing. There's always that hesitation I guess, and so having these tools set up I'm sure help out a bunch, and not just you on the support and maintenance team, but the client as well who can leverage that too, right?
David Burns
Yeah. We could send a Tugboat link for that PR environment, they could click around as if it was the live environment, they could add content, remove content, anything they want to do and give the thumbs up, hey, this is ready to go. And then what's really neat is once they click the hey, this is ready go, usually a lot of tickets would stack up in the course of a sprint or a couple weeks and they'd tag a release and deploy it.
David Burns
We're starting to look at the GitHub Actions tool that basically says once you merge this into whatever you're main branch is, we're going to see GitHub or whatever send off a web hook and be like this is okay and approved and we're just going to push that out to the dev environment. So we always have an environment that's the latest and greatest code.
Cathy Theys
Yeah, and usually that dev environment is usually then on the customers hosting infrastructure, not on Tugboat. So that goes through Tugboat and then moves into something that's closer to what will actually be deployed with the live site.
Matt Kleve
Does hosting matter? Are you hosting agnostic, or do you want to support these two things?
David Burns
We have preferences, which reduce context switching and it's good when we have a lot of clients using the same hosting, but right now we are okay and we accept any enterprise level hosting or anybody that's doing custom hosting.
Matt Kleve
Sure.
Mike Herchel
So if I'm using, FTP you won't support me?
David Burns
Nope. We need version control, that's where we're at these days.
Matt Kleve
FTP on a GoDaddy, right?
David Burns
Mm-hmm (affirmative).
Matt Kleve
GoDaddy-
David Burns
And I just want to be real clear, our primary purpose within support and maintenance is not to be system architects. We take care of the Drupal side, the application side. We will happily reach out to the support team of those hosting providers if there's performance issues that are not caused by the application and coordinate a fix with them if the support client doesn't feel comfortable doing those things.
Cathy Theys
Yeah. So we work with all these different hosting companies, a lot of the common Drupal ones and custom things that customers have, but that gives a wide experience set. One of our customers can say, "Hey, I'm seeing this weird issue," we go through the catalog of all the other customers that we've had and the ones on all these things and we can quite often be familiar with that host and maybe that specific issue, because we've worked with the host on a different customer earlier. And so we can really help out smooth some of that over and make those resolutions come quicker for our customers.
Mike Herchel
Makes sense. So Dave, you mentioned automating things with GitHub Actions and tools like that. On the Lullabot.com repo, I see all these notifications come in for Dependabot, to the point where I actually set up a little filter, so I don't have to look at those. Can you talk about maybe your dependency automation and maybe other things that you might be doing?
David Burns
Yeah. So a lot of our support clients have limited hours that are available to us within a month, and early on we noticed that somebody that's with 30 hours a month, a majority of our time would be spent writing a Composer update for a specific module, going in, testing that change and things like that. And then I heard about Violinist IO through Twitter and I was like this is the greatest thing ever, one pull request per module update and you get to layer on-
Matt Kleve
I don't know anything about it, Dave, can you explain that to me, Violinist?
David Burns
I'm sorry, what's that?
Matt Kleve
Violinist.io, I don't know anything about that.
David Burns
Yeah. So all it really does is look at your composer JSON file and it runs a Composer update dry run, looks for anything that is out of date or there's an update available. It also has some options around do you only want to create a pull request for modules that have security updates as opposed to minor point releases. And when it sees one of those, it'll go ahead and create a pull request for that. And included in that pull request is some of the change logs if the module maintainer has been good about that, and it'll print it right in a comment or the description of the pull request and be like, here's all the commits.
David Burns
And with all things that we have layered on top in Tugboat, we automatically get the visual diffs, the Lighthouse scores and everything, and then we're also experimenting with automated tests. So if everything passes after Dependabot or Violinist IO creates that pull request, we're feeling pretty confident this is safe to merge in. We'll go in and look at the config sync page to make sure there's no overwritten config. We'll click around on the places we know that that module interacts with the site to just give us a lot more confidence.
David Burns
But in setting up tools like Violinist and Dependabot, we found that what used to take maybe 30 hours a month for our team to work on, we're really only spending five. That frees up 25 hours to get to know the client, their project, the things that they're struggling with their editorial system, or let's start talking about bugs or small feature enhancements. So we're providing a lot more service by layering in these automated tools.
Matt Kleve
Those sound like really great tools for you, but they sound like really good tools for anybody who runs a Drupal site.
David Burns
Definitely recommend going and checking them out, see if they're a fit for your organization or your own personal sites.
Mike Herchel
We're adding the link for Violinist.io to the show notes.
Matt Kleve
We're talking to David Burns and Cathy Theys of the Lullabot support and maintenance team. After this, we'll talk a little bit more about things that they find when they dig into Drupal, coming up right after this.
Mike Herchel
Welcome back. We're talking to the crew from Lullabot support and maintenance department. So as all Drupal developers know, there's a million ways to do anything in Drupal. Is there anything that really, really stands out where you looked at something and you're just like oh no, or you're like, I should charge these guys double, or something like that?
Cathy Theys
What have they done?
Mike Herchel
I know.
Cathy Theys
Why?
Mike Herchel
Any SQL queries in the template files or anything like that, that you've seen, that you can think of? You don't have to name names.
Matt Kleve
Can you still do that in Drupal 8?
Mike Herchel
Not in Drupal 8, but you can in Drupal 7.
Matt Kleve
I'm doing all kinds of awesome Java script right now on a Twig file.
Mike Herchel
You shouldn't admit to that.
Matt Kleve
I only say that to make the front-ender's blood pressure go up. But Dave and Cathy, I can see by your 1000 yard stares that everybody's website is a little bit different.
Cathy Theys
Yeah, I'm trying to think. Usually, things are a little different customer to customer, but it's more interesting than shocking, and so it might be ... I think we approach it as an opportunity to figure out how does it work and why did they pick it this way, because we need to know that in order for us to have confidence that when we change it, we're changing it in the way we intend.
Cathy Theys
And so, quite often we'll see things that we don't understand and we'll just be like what is this? But then we talk to our coworkers and we talk to people in support and we talk to people in client services, and sometimes I talk to Drupal people in Drupal.org Slack about things and I'm just like, hey, I'm working on this thing and I don't know anything about it, can you help me please figure it out?
Matt Kleve
The hive mind is pretty great working within Lullabot, and of course being a part of the Drupal community, there's always that place you could reach out and that's cool.
Mike Herchel
Yeah, totally.
David Burns
I would think that one of the things I say is we live in a world of riches when it comes to support and maintenance. We are given these projects that client services team has worked on, built from the ground up, and we see things like in a Lando file or the way they're compiling things and we're just like, this is totally useful, we're going to mark this down and try to reuse it as much as possible, again, working towards standardization to reduce context switching.
David Burns
So that's been really, really helpful and we'll even wind up in situations in support where they need somebody that had built the original site to do a specific larger task that's outside of the scope of support and maintenance. So working directly with Lullabot, we're able to say, "Hey, come in on this same support SOW and address this feature." It doesn't happen as often as I'd like, but when it does happen, it's so great to be able to see the cross sharing of people, between both teams.
Mike Herchel
I remember I came in and I did a small, what was it? Like a one week project where the goal was just to make this navigation menu for an EDU accessible. It was one of those situations where the drop downs were not keyboard accessible, and so I'm like, all right, I could do this. And it was fun because I was doing stuff like that and I got it working within two days and I was like well, I could do these other things, too and they were like yeah, and that was a lot of fun.
Matt Kleve
These-
David Burns
Yeah, that's ... Sorry, Matt.
Matt Kleve
These quick contracts aren't necessarily normal, but it's fun when it works out, Mike, right? You've got some availability-
Mike Herchel
Yeah. I was in between client projects.
David Burns
Yeah, we've been reaching out frequently for accessibility audits, which has been really insightful, definitely something I'm wanting to learn more about, and menus keep coming up all the time. Menus and accessibility are like the number one thing that we're having to reach out to get some super-duper experts outside of support maintenance to help out with.
Matt Kleve
Side bar, Mike. We've got a bunch of new accessibility experts. It seems to be there's been a really strong resurgence inside of Lullabot and I know we've done a ton of podcasts about it, but we should think about doing that again?
Mike Herchel
Yeah, totally. I actually know who I want to invite. I'm not going to announce this person right now.
Matt Kleve
It's a secret.
Mike Herchel
Yeah, but I am friends with them. Well, they wouldn't call me a friend, but I would call them a friend.
David Burns
Mike, you've got so many friends.
Mike Herchel
They would be like, stay away.
Cathy Theys
I'm trying to think what unusual things do we see in support. And the other thing that I thought of was I see a lot of interesting things because some of the sites that we work on have been around for a very long time. So it could be like 8 years, 10 years. I don't know how long has Drupal 7 been around, right? It's just been ages, and so I can see the timeline of the way the popular things have changed in Drupal, because on one site, it might be like compiling SASS and be using Compass. And on another site it's like pure CSS and I can see different stages of things.
Cathy Theys
And so we do see a lot of wild differences, but they made sense for that project at the time under the constraints that they were on. They were just doing the best they could whenever they had the development budget and they were working on the site and they were like, well, we need to do this quick, or we need to do this rigorous. Whatever their constraint was, they did the best they could. And then their site's been round for ages and ages and it needs maintenance and they come to us, and we're just like, we understand.
Matt Kleve
I like your attitude. I've come around to that in the last few years too. It's the not what the hell was this person thinking, but what caused this person to think that way, because there was something going on at that time and this was the right way. You give them a little more grace, right? It's like the person before me, they were in a situation and that might not make sense now, but yeah.
David Burns
It all boils down to time and cost, right? We'll see projects and we'll be talking with the customer and they're like, this thing we built two, three, four years ago is just so troublesome, and just talk-
Matt Kleve
And it was a temporary solution two or three years ago.
David Burns
Yeah. So one of the things we'll come do initially is an audit and make sure the site's using best practices, making sure modules are in the right places, make sure there's a patch Makefile for D7, make sure you're using Composer for D8, and just get us to a baseline that we know when we bring anybody else into the project, they're seeing less and less of the surprises.
David Burns
So that is part of the standardization and cleanup thing we're focusing on, and sometimes the challenge there is we want to do these things that provide everybody a better quality of life, but the client is wanting these many features to be focused on. So it's a constant balancing act of we're going to deliver the thing that obviously you're paying for our services, but we also need to take care of these things to make everybody's life easier, not just the Lullabot team.
Matt Kleve
You were talking about how long it's been since Drupal 7 came out, but Dave, I know you and I both have built Drupal 6 websites for Lullabot, so yeah.
David Burns
There are no Drupal 6 sites right now that I can think of in support and maintenance and thank goodness for that, because-
Mike Herchel
Would you take a Drupal 6 site?
Cathy Theys
Oh my gosh, I don't know.
Matt Kleve
6 to 7 is a whole lot quicker migration than 7 to 8, but I know that a lot of people are wanting to go 6 to 8, or 9 I guess at this point.
David Burns
I think it would be hard for us to flat out say no, we wouldn't accept some at some point. We're always going to try to do an audit before we say yes, and if we're feeling good and we're on the same page with the direction the site owners want to go, like hey, it's on Drupal 6 now, but within a year we plan to migrate Drupal 8, Drupal 9, those things are taken into consideration.
Mike Herchel
Yeah, that's makes sense.
Matt Kleve
After that conversation, if it's something that you don't feel comfortable undertaking, do you have friends or enemies you refer them to?
Cathy Theys
There are companies that were approved or advertised Drupal 6 long term support vendors, they're called something slightly different. Yeah, we might refer to a friendly competitor, because there's some people, they did that, they're good at that.
David Burns
And they still do that.
Cathy Theys
They should keep doing that, they're great at it, but it's different.
Matt Kleve
I would think that newer versions of PHP would start to become the issue with running those older versions of Drupal, because you find a new host right now, they're going to run PHP 7 point something and Drupal 6 might laugh at that.
David Burns
And that's one of the things we would look for, like where is this site hosted? Chances are Drupal 6 might be using SVN instead of Git. So I don't think we would want to add the overhead of having to have every person on our team remember all the SVN commands.
Matt Kleve
That wasn't that long ago, Dave. You and I have been on a Lullabot project together that used SVN.
David Burns
Absolutely.
Matt Kleve
It's something that is in recent history if we want to call it that, but shoot, that's been like nine years.
Mike Herchel
I wouldn't call that recent, Matt.
David Burns
Not [crosstalk 00:33:05].
Matt Kleve
I don't know, is Drupal in your recent history, Mike? It depends on I guess your perspective. But Cathy, go ahead.
Cathy Theys
Oh no, I was just going to go back to the Drupal 6 question.
Matt Kleve
Oh go ahead, yeah.
Cathy Theys
I think we would raise our eyebrows at that, but we do have Drupal 7 customers and we are an official, approved Drupal 7 extended support vendor.
Matt Kleve
More to come on that later, right?
Cathy Theys
Yes.
Matt Kleve
Well, the end of life for Drupal 7 is in November '22, so we've got a little bit left that you can still use Drupal 7 and feel mostly okay with it. Drupal 8's end of life is 2021 though, right, this year in November?
David Burns
Yeah. So the reason for that is the version of Symfony that is shipped with Drupal 8 is going to go end of life, and Symfony is not in Drupal 7, so we have to follow where Symfony is going. I do think it's weird that a newer of version of Drupal is not being supported as long as an older one, but I understand how they got to that reasoning.
Matt Kleve
Sure. Well, so they'll upgrade.
Cathy Theys
Yeah. It's a lot easier to migrate from Drupal 8 to Drupal 9.
Matt Kleve
Kind of like I was talking about 6 to 7, I think 8 to 9 is probably about the same as 6 to 7 was.
Mike Herchel
I would say easier than that. I mean, a lot-
Matt Kleve
It depends on what your custom code is, right? If you were doing bad things in your custom code in Drupal 8, then it's going to be harder, right?
David Burns
I think 6 to 7 there was a bit more schema changes because we were starting to explore the concept-
Matt Kleve
And I remember, didn't renderables became a bigger deal at that point, or am I wrong? I thought there was something in the theme layer that became trickier between 6 and 7, but it's been a while.
David Burns
It's so long ago.
Mike Herchel
Yeah. We should do a podcast on that, Matt.
Matt Kleve
Okay.
Mike Herchel
The 6 to 7 upgrade.
Cathy Theys
What?
Matt Kleve
I'm sure there's actually probably one out there, right?
Mike Herchel
I know, right?
Matt Kleve
Like the old Jeff Robbins, Matt Westgate podcasts of 10 years ago.
Cathy Theys
I know we're laughing, but I'm trying to think in my head, would that ever being a thing somebody would want to do, migrate from Drupal 6 to Drupal 7. And the only reason I can think that somebody would want to do that is maybe if they wanted to migrate to Backdrop, because Backdrop is kind of like Drupal 7, and-
Matt Kleve
A fork, right? It's a fork, it's a friendly fork. It's not a fork like the open source community's angry with them. They're providing-
Cathy Theys
Right. It was forked from Drupal while Drupal 8 was under very intense but prolonged development, and so it's kind of a snapshot of Drupal 7. It's got the best things of Drupal 7 in it, but it's got the features of Drupal 8, but the code underlying it is different. And so if it was too expensive to migrate from Drupal 6 to Drupal again, like to Drupal 9 would be where you would migrate now, and that's too expensive, somebody might want to migrate to quote "Drupal 7" except that Drupal 7 that they migrate to is really Backdrop.
Matt Kleve
That's interesting. I Googled something real quick, and so there was actually a Lullabot podcast, Drupal 7, Are You Ready? Episode 90.
Mike Herchel
Nice.
Matt Kleve
Mentioning that in an article even more interesting. There was an upgrading from Drupal 5 to Drupal 6 that came out over 10 years, that's a really great article that's written by Mr. David Burns.
David Burns
I have Lullabot.com articles? I need to start writing some more of those. But while we're talking about the upgrades, recently we have started focusing on the Drupal 8 to Drupal 9 upgrades within support. That's one thing that we do because the upgrade process is so much easier than it has been in the past. One of the things we do is drop in the Drupal Rector module and the upgrade status module, and we run an audit report to give us an overview of the level of effort. It's basically upgrade your modules to versions that support D9, that's fairly obvious.
David Burns
Drupal Rector will scan your custom code and be like this is deprecated, go ahead and replace this with this. And it even ships with a command you could run that's like, we'll try to fix as much of that as we can automatically, and I found it works for about 80%, if not more, of the code that needs to be changed. Getting the D9's a much, much more enjoyable experience than going from Drupal 7 to Drupal 8.
Cathy Theys
Yeah, and for some sites it might be a maintenance task, and for other sites it might be oh wait, we need a dedicated person for this. This is a little bit more customized and needs some more attention, in which case we would talk to the customer about that and maybe hand the project through our sales team back to client services.
David Burns
Even right now with a project like GovHub, which is by far the biggest Drupal site I've worked on in terms of amount of code and sites that it's running, if the SOW has enough months and the client's willing to work with the support team in a long enough timeframe that we could do sprint models, where we do five modules here, until we get everything done, that can still land in support, it's just how quickly does the client need to be on Drupal 9. And if it exceeds our quote-unquote "support bucket" we would send that back over to client services.
Cathy Theys
Yeah, that's a good point.
Matt Kleve
What does your typical contract look like?
David Burns
So it breaks down into three tiers. The entry level, which is like 30 hours a month, and that really focuses on the initial security audits and recommendations, performance audit, module and security updates, integrating Dependabot, and then just critical bug fix, it's like this is affecting the user experience. And we find that 30 hours a month, 30 to 50 is a good place for those type of projects.
David Burns
And then the next tier would basically be like a consultation and mentorship. It includes everything that was mentioned in the first tier, and then on top of that, we do implementation and architectural guidance. So they'll reach out to us and be like we're looking at these three modules, which one would you recommend if we're trying to do X, Y, Z and we'll give the pros and cons of each. We'll also be able to do pull request reviews, a little bit of project management and road mapping, and that's for clients that are like 50 to maybe 80, 90 hours a month.
David Burns
And our third tier is basically dev augmentation. A lot of the clients we work for have small departments, like one or two people and they are hiring us just to do continued development to help augment their existing team. It includes everything from the first two tiers that I mentioned, but also includes maybe one developer part time, or two to three developers part time depending on how many hours are available. We'll also do ... We'll set up more robust automated tools, we'll probably do some Cypress IO testing or Nightwatch. Is that what it's called, Nightwatch JS that's in core?
Matt Kleve
Sounds good to me.
Mike Herchel
Yeah.
David Burns
So those are the three tiers and our rates are similar to the hourly rates that any Lullabot project would have, just because the type of work we're doing is support, we still operate at the same cost for Lullabot.
Cathy Theys
Yeah, I like it when our contracts are long and retainer-based and are big enough to have multiple people on it. Those are my favorite ones.
David Burns
Those are the ones that allow us to really get to know the platform, really get to know the client and their team, and then really talk to them about what their experience is with the site that we've built, or the site that we're maintaining for them. And those feel much more like partnerships where we're bouncing ideas off each other trying to reach a common goal, where the other ones, while they're great, are just like we've got the feature set or this site's not going to change much, we just want to make sure there's no security flaws that people are going to take our site down.
Cathy Theys
Yah, they're different. But they're also like it's whatever the customer needs at that point, right? When they need and want that collaboration and investment and exchange of learning and improvement of their infrastructure, we're going to be excited about that. But when they don't need it, and what they really need is just bare minimum, maybe because of the project lifespan or maybe because of the budget fluctuations, we're going just help them do whatever they need to do with whatever they got.
Matt Kleve
And as a-
Cathy Theys
And it changes too. We might have customers that will keep renewing their SOWs with us, but their needs change over time and so maybe they become a smaller contract. Or maybe they've got a big initiative and so they're growing the contract and we can adjust with that. That changes too.
Matt Kleve
As a support team you're growing too, right?
David Burns
We are. We are doing interviews right now to bring on a front end developer. We have a very strong support team right now, but nobody that has ownership of that front end expertise. We're a lot of full stack. We're really, really strong site builders, really strong back end developers and we can hobble our way through a lot of these front end requests. We get it done, we run tests to make sure everything's passing accessibility and delivering what the client needs, but we don't have somebody that we can just say, "This client's asking us about accessibility, this client wants to refactor the whole menu system on their site, go do it." So this new position we're hiring for, we're hoping to have that person onboarded maybe by April.
Matt Kleve
Very nice.
Mike Herchel
That's exciting.
Matt Kleve
That is exciting. What makes a support developer special? Are they just gluttons for punishment?
David Burns
Yes.
Matt Kleve
Okay.
Cathy Theys
I think it's like what we've been talking about, a little bit of it is attitude, like compassion and empathy, and understanding that decisions were made in the past. We're not building things from scratch, we're not making new decisions and we can consult and we can give advice if that's what's needed, but that's not the usual thing. And so I think you have to be okay with that. You have to be like I can't redesign this, I can't make this exactly follow my vision I have in my head. I'm working within the constraints of the system that it is. And it makes it easier, I don't know, mentally when you approach that from a place of wow, they're in a tough spot, they must have been in a tough spot before. It makes it easier to deal with that.
David Burns
I would say there's also a really strong communication because of the context switching, knowing what project's somebody's talking about and being very clear as to the type of thing that you're working on, because just saying the name of a project, may not be enough to have the person you're talking to fully understand and grasp, because they've been working on this other site for a bit, documenting things a bit more.
David Burns
And what Cathy said around the compassion. Clients are coming to us when something's broken and it'll sometimes feel like putting fires out all the time, but to understand that we're here to help them and we don't want to add any complexities to what they're already experiencing. So to take that, give solid estimates of how soon we could expect this thing to be fixed and always keeping the client up to date if our estimate is changing. I think client services does a good job of this, but they get to work in larger teams, with longer timelines, with bigger number of hours in a month. So being very cautious and aware of how we use the time and communicating that back to the client's like a skill set that our support team does very well.
Cathy Theys
Yeah, and I think being patient too, because we do see a lot of opportunities for improvements, but because we don't want to add any stress onto our customers, we have to be able to make a note of that and come back to it later, when the time is right for that. So you want to be looking for improvements, but also be very patient in terms of waiting for the right time for them, because we don't want to ignore those opportunities either.
Matt Kleve
Very good. David, anything else you'd like to add?
David Burns
I'm sure there'll be a link to the support and maintenance page on Lullabot.com, that has a bit more info-
Matt Kleve
Maybe.
David Burns
What's that?
Matt Kleve
Maybe.
David Burns
Maybe. And that will give a breakdown of some of the services we do and how to contact us if you need help. I should probably mention how our SOWs are structured. We like to work with a client for at least a quarter and then as we build trust in that relationship, we'll consider expanding an SOW to six months or even a year if they know they're just going to be sticking with this version of Drupal.
Matt Kleve
So it sounds like you wouldn't generally want to be like hey, I want you to fix my photo gallery this week?
David Burns
Yeah, this isn't like a jump in, jump out, because there's so much overhead with setting up the Lando, getting familiar with code base after like hey, we've got 30 hours this whole engagement that 15 hours of that is going to just be involved in conversations, gathering requirements. We really need three months to build our understanding and that relationship.
Matt Kleve
You probably feel like you're more useful to them at that point too, so being able to commit some time and not just fix the Drupal. Cathy, anything more you'd like to add as you point towards wrapping this up here?
Cathy Theys
No, I think support is great. We get to see so much variety of things and, I don't know, I feel like we really help our customers, and so it's interesting and rewarding.
Matt Kleve
So Mike-
Mike Herchel
Yes, Matt.
Matt Kleve
... go support.
Mike Herchel
Yeah, go support.
Matt Kleve
And quite frankly, this may have come off as an infomercial, but I think there's a lot that could be gleaned by somebody who is a one person shop, say at a non-profit or something-
Mike Herchel
Oh wait, there's more.
Matt Kleve
... maintaining a website, right?
David Burns
One more thing.
Mike Herchel
Yeah. For the low, low price of ...
Matt Kleve
But I think somebody who's-
Mike Herchel
No, I get it.
Matt Kleve
... running their own website, might be able to glean a few things and think maybe if I change my processes this becomes a little easier, and automated testing or visual diffs, those tools could be very useful to them in keeping things going.
Mike Herchel
Absolutely. Yeah, absolutely. They're useful for everybody, not just folks in the support and maintenance department, but Drupal as a whole.
Matt Kleve
Three easy payments. 877-Lullabot. Call 877-Lullabot. That's our real phone number, by the way.
Mike Herchel
Yeah, I'm not surprised.
David Burns
It's funny, I have to always go to the website to actually see what the numbers are when somebody's like hold on, what number should I use, because I always go about it just typing the word Lullabot and have them figure it out on their own.
Matt Kleve
Well, thank you both.
David Burns
Thanks for having us. See y'all.
Matt Kleve
See you.
Mike Herchel
All right, thanks, bye.
Cathy Theys
Thanks.

If you enjoyed this Episode, you may also enjoy...

About host Matt Kleve

Portrait of Matt Kleve
Matt Kleve has been a Drupal developer since 2007. His previous work in the media sparks a desire to create lean, easy to use workflow processes.

About host Mike Herchel

Thumbnail
A senior front-end developer, Mike is also a lead of the Drupal 9 core "Olivero" theme initiative, organizer for Florida DrupalCamp, maintainer for the Drupal Quicklink module, and an expert hammocker