A Legacy Leap with Drupal Retrofit

Matt Kleve: For October 19th, 2023. It's the Lullabot podcast.

[Intro music]

Matt Kleve: Hey everybody, it's the Lullabot podcast episode 265. I'm Matt Kleve, a senior developer at Lullabot. And today we're talking about where we've been, where we're going upgrading from Drupal seven. But you know what? That's something that you hear a lot from a lot of different people about these days. With the end of life of Drupal seven, everybody's thinking how are we going to get off of that? We're going to migrate to all the latest versions of modern Drupal, and it's a lot of work. We got something a little bit different today. With us today we have a principal software engineer from Acquia. He's the maintainer of PHPStan, a Drupal extension, and the companion Drupal check runner. Drupal Test Runner, a PHP storm plugin running run it for running Drupal's test suite. All of the really great automated tools that you use when working on Drupal. He's the author of the Drupal eight Development Cookbook, one of the first published books on Drupal eight. From Wisconsin, matt Glaman joining us. Hey, Matt.

Matt Glaman: Hey, thanks for having me.

Matt Kleve: Also with us, we have a senior front end developer from Lullabot. She's CPACC certified in Digital Accessibility, an organizer of A11yTalks. She's contributed on accessibility for the Olivero theme, the Claro admin theme, and Drupal Admin UI. From Perry, Kansas, joining us, Kat Shaw. Hey, Kat.

Kat Shaw: Hey, thanks for having me.

Matt Kleve: Awesome. And I know Kat, you did a bunch of work prior to your time at Lullabot, you know, in the D7 world and supporting some D7 sites and that kind of thing. And.

Kat Shaw: Yeah, yes, I did.

Matt Kleve: So you've got all that deep knowledge of that along with you know, your awesome front end knowledge, too. So, Matt, we were at the Twin Cities Drupal camp, and I listened to you talk your talk about Drupal Retrofit? And I thought, wow, this is great. Of course, I think I saw you talking about it on the the platform formerly known as Twitter. Some time ago, you were kind of pontificating on. Wouldn't it be great if we had this, this, this middleware, right? Explain to me what you're doing with Retrofit these days and how it's kind of a novel idea.

Matt Glaman: So Retrofit I first talked about it at mid camp during a lightning talk and explained it as my best, worst idea that I've had in four years. So this is something I've been thinking about for a while, where we focused a lot in Drupal on doing data migration from Drupal seven to Drupal eight, nine, ten and so forth, but we never really thought about the code part of it. It was just kind of we said, that's life. You have to rewrite your code. But what if we didn't have to rewrite our code? What if we could plug in, like a shim or some kind of library that let your Drupal seven code just work in Drupal 9, 10, 11, 12? Because there's a huge effort in rewriting your code that has your business logic, those legacy applications that have been running fine for eight years and had all this money invested, it's really hard to just go and rewrite that when you have probably developers who didn't originally write it. So Retrofit is a tool that wants to solve that gap and make it easier to lift your code from Drupal seven into a modern Drupal application.

Matt Kleve: So that's different than the way Drupal has always been in the past, though, right? Because Drupal is all about, you know, the drop is always moving, old code be damned. Your content will be golden. We'll make your content good to the latest version. But you got some work to do, huh? So the idea here is that we're going to try and take some of that work off?

Matt Glaman: Make it a lot simpler. And this just focuses on Drupal seven code. Now, there could be ways in the forward in the future to help ease major version upgrades. So one of my other tools is the composer lenient plugin, which lets you install any module with any version of Drupal and then patch it. So that way let's say it doesn't work with Drupal ten yet. This allows composer to download it. Well, Retrofit could also help bridge that gap if you for some reason you're managing 50 sites, 50 different code bases, and you just didn't have time to get to them all. Because even like a year, that's a lot of code bases to touch. And this could allow you to run some of that deprecated code in the newest major version of Drupal without it crashing.

Matt Kleve: You mentioned the lenient, right? That sounded like a wild and crazy idea that didn't make any sense. And why would you ever use that? But if you're trying to upgrade to a modern version or a different version of Drupal, and the module doesn't yet support it, but it does support it with the patch you need composer lenient and then the patch, and then you can run the old version of the module patched on the new version of Drupal.

Matt Glaman: Yeah, essentially composer would say, sorry, this isn't compatible, but it gives you a chance to say, hold on. I'm okay with that because I will make it work. And, you know, lenient and Retrofit because even think of Retrofit beyond your own code. If you did have like some Drupal seven contrib that didn't upgrade. This is open source. This is part of the risk of using open source code is things may not work in three years, or there may be a maintainer that doesn't release things fast enough. And this is going to help with your own code. But maybe you're the Drupal seven site that hasn't been able to upgrade because that one obscure contrib. You made a decision about ten years ago that shouldn't hold you back. And if you don't want to rewrite the code, Retrofit should help you at least get a lift so that we don't have to worry about rewriting it.

Matt Kleve: So explain to me how it works. If if I were to go install Retrofit today, what is it? What does it cover? I guess just kind of take me through the, the what it's going to do for me today?

Matt Glaman: Yeah. So Retrofit gets installed in a new Drupal ten site. So you would create a new Drupal ten code base, and you do composer require Retrofit-drupal/Retrofit, and that will plug into Drupal and its service container and kind of hook into all of its APIs and say, oh, you're discovering blocks. Well, I'm going to call hook block info because that was the API in Drupal seven, and I'll go find all the blocks, and then it provides a class. And then that class calls all the old API methods. So that way if you define blocks in your Drupal seven module, it will call all the same hooks. And luckily the render system is mostly the same. Render arrays for a lot of things could still just work. So your block can start outputting content without deleting the hook lines and creating a new class. You know it's not hard work, but it's still work. It's tedious work to have to go change things around. Same with your right now it can do discovery of theme hooks such as theme functions. Drupal seven allowed you to write theme functions, and in Drupal eight they said, no, we're not doing that. It's Twig templates all the way. This actually renders your theme hooks inside a Twig template. Same with PHP template so you don't have to rewrite everything. Hopefully, let's say instead of it being 100% of your code, it's maybe 30 to 40%. And that's that is that's a huge deal if you're not touching every single line of code.

Matt Kleve: So this is all kind of based on the idea that the business logic doesn't change, but the scaffolding around it does. Right?

Matt Glaman: Yes. Yeah. Let's think about the other way that what I was thinking of is I've been a senior developer for a while now, like team lead reviewing code. And I know at the end of the day, my head hurts sometimes when I'm just reviewing code all the time, and I can't. I've never been on a Drupal seven to Drupal ten upgrade. A few that are were on were e-commerce sites, so we just re-imported the data, right? It didn't matter. We didn't migrate data. I can't imagine. Actually, those we did have to rewrite business logic. Hold on. Sorry. So I was on a few Drupal seven and Drupal ten rewrites, and we had to reinvent the business logic. And that's a lot to review. That means you have to go check the code. You have to check with the developers. You have to check with the business, like, hey, we have to adjust this. Does this still make sense? Like, is this business process still correct? And if there's a way that you could just lift that and plug it into a new scaffolding without having to dust off any old knowledge, or, you know, somebody wrote this and they're gone and nobody really knows how it works. But it works if we can avoid those conversations. That's the goal.

Matt Kleve: 7 to 10 with commerce too would mean rules is gone. So...

Matt Glaman: Yeah.

Matt Kleve: That would have been a pretty good change.

Matt Glaman: It's not going to fit for everybody. You know because then rules and like even like chaos tools provided a bunch of plugins or like its plugin system. So there will be things that don't map correctly, such as I was on a lot of sites with analyzer. I'm not sure how that would really work, but the idea would be a lot of your code that interfaces with like entities and nodes should just work.

Matt Kleve: So what does work at this point is there is there a list that's documented somewhere that we could take a look at if we wanted to?

Matt Glaman: I need to see if I've made a properly documented list. I don't think I have.

Matt Kleve: I heard you mention blocks and nodes.

Matt Glaman: Yes. So blocks.

Matt Kleve: Entity, entity hooks? That kind of thing?

Matt Glaman: No, not. I started to work on entity hooks, and then I paused because I realized everybody brought up themes. So...

Matt Kleve: Yeah.

Matt Glaman: Let's go down this. I had to pull up the code quick. So hook menu. That was the first thing I did. So your hook menu.

Matt Kleve: Maps to routes pretty pretty straightforward right?

Matt Glaman: Maps routes to links to actions to tasks. Because before in Drupal seven hook menu defined everything. And then Drupal eight became individual API's. So that is mapped forms. So if you have a hook menu that defines a form the form works. We it gets put into a form class. And luckily that API hasn't changed a lot. So it really does just work. Blocks, theme hooks, if you have a theme and you've overridden templates using TPL templates, those get picked up and respected. That's a little bit more wonky because some of the underlying theme theming things have changed. But I took Drupal seven, Bartik put it in a Drupal ten site, and I could see my nodes themed via Bartik from the Drupal seven land.

Matt Kleve: Did you just tell me that you were you have functional Drupal seven theme running on Drupal ten?

Matt Glaman: Yes. Now there was there had to be some finagling around style sheets, because I have not found a way to shove those in, if you will. Where it could take the style sheets definition and create a library dynamically for you. And that's mostly because Drupal is still attached to hooks and not event subscribers. Just some of some of the magic to make this work is very locked away inside parts of the Drupal API. That was just too hard to swap out, if you will.

Matt Kleve: Well, I just one thing I was, I think we were talking about this or I was hearing you consider trying to do that, and I was thinking about how the theme system used to be pluggable back in the day, and I think now it's less so. But man, it would be nice if it were again, because that would actually work, right? Without having to to do a bunch of workarounds, I guess.

Matt Glaman: Yeah. And one thing I'm considering to provide, and I don't know if anybody's done this before, but is a drush command that's bundled with Retrofit that allows you to convert your theme to that says, oh, I need to Retrofit this theme. So take the .info file and convert it to a modern info.yml. And maybe that builds your libraries.yml for you. So kind of like a code migrator. I know there is like the module upgrader, module package, whatever it is.

Matt Kleve: Yeah.

Matt Glaman: But I don't think I don't know if it works for themes. I haven't actually dove into that because the idea is I don't want people to have to rewrite code. I want it to just work. But I think for that, like info files, that's a you got to touch those no matter what.

Matt Kleve: Yeah. So that's the one thing you're saying you have to change from your Drupal seven to a Drupal ten is your info file turns into a YAML file, right? Yeah.

Matt Glaman: Yep. And it maps, it's pretty much

Matt Kleve: And that that allows Drupal to recognize it. Right? So then your hooks can be called and. Yeah.

Matt Glaman: Yeah. Yep. That's the one part of Drupal that's really hard to just to get in and modify. And I just fear the level of effort on my end is so high that if that's the one thing you have to do is, you know, replace an info file to info YAML, it's not if that's if that's the hardest part of this, that's a win in my opinion.

Matt Kleve: Kat, I'd love to hear what you think about the front end.

Kat Shaw: I was wondering did you mention forms?

Matt Glaman: Yes.

Kat Shaw: The forms API?

Matt Glaman: So I how I've been working on this is I took the examples module from D7, and I kind of just copied it over and I was like, all right I'm going to walk through the for like the menu example. And all of those paths work. Like let's do form where forms were built into it. I went to the form and you can do tutorials one through seven that iterative, iterative that build up the form over time and it submits even Ajax. So form API plus Ajax from Drupal seven code works in Drupal ten, which kind of blew my mind.

Matt Kleve: [Laughter] But I wouldn't have expected that like form API is is is not that different I suppose?

Kat Shaw: Yeah.

Matt Kleve: I mean, it's, it's gone from being the procedural functions to stuff it in a class because classes are better, I guess, but.

Matt Glaman: The API hasn't changed that much. The internals have. That is one thing that Retrofit does is form state was always an array, and form state handles all the information of a form that's being built and submitted. So it decorates it with a version of form state that implements the array access interface, which basically says you can touch this object as an array. So that way, you should be able to fetch like form state values. It knows to map that, to form state get value and the value you would like.

Kat Shaw: That's really cool. Have you have you looked into the like how it affects the accessibility features that because I know Drupal nine, Drupal ten, even Drupal eight had a lot of accessibility features that were introduced that weren't really in Drupal seven. So, you know, like the RL alerts and the just even the, the link on the, on the top for going to the main content and things like that.

Matt Glaman: That is a good question. I know if somebody were to take their theme over verbatim, then they would not receive those updates. But that's kind of in the nature of the project is one of the reasons behind this also was not just so that people can be lazy. So that way or like miss out on features, but let's say that they don't have time. Like they're running short on time. They don't have time to finish this project. It gives them a chance to get on Drupal ten, and then use the next year or two to keep refactoring away their code. Or I think a lot of us are used to each Drupal major version being like a replatform because it is a replatform. And when you replatform, you usually do a theme redesign. So the idea is that you don't need to bundle upgrading from Drupal 7 to 10 with a theme redesign. It could be we will do this update, then we will redesign the site, and then we'll do a maintenance package to refactor away the code and get off Retrofit. So it's kind of a way just to help people. You know developers, that these teams are going to have to maintain the legacy platform and a Drupal ten site at the same time, and this is a way that they can maybe just say, all right, we're not going to do feature dev for three months for one quarter, because we're going to do this upgrade and then we're back on track versus saying, oh, we can't do this for a year. We can't. We'll have to maintain two platforms. So there would be a loss at any of the accessibility items that were added to Drupal ten or to modern versions of Drupal and their themes, if they ported their theme over verbatim.

Kat Shaw: What I really like about this is it really creates a bridge that allows the time. That's why I when you were you're talking about that. And I, you know, I hope you don't mind. I actually worked at a Douglas County, Kansas for, like, over 13 years as a webmaster, was my title. And one of the things I did in my last years working there was actually create some Drupal seven sites, and I actually felt bad when I left because then shortly after that, Drupal eight was created and I was like, oh my gosh, I cannot believe I did that. I should have chosen Drupal eight. And they actually struggled with this exact issue. And they had to migrate to Drupal nine actually. So this topic is really, really interesting to me. So I'm really glad that you have created this, because I think that they could have used it because they actually ended up going the route of having to spend a lot of time and money.

Matt Kleve: I don't think this gets away from having to know about your Drupal site and know how it runs, but it's kind of provides some shortcuts to at least get the major version of Drupal running faster. Right?

Matt Glaman: Yeah. Yep. And I think it fits this. It fits this market of people who invested in Drupal as their platform and application, not, you know, the one example that I had is there's like a family snowmobile club, and I built it on Drupal 7 13 years ago. I'm not going to use Retrofit to port that theme over. I'm telling them to go to Squarespace because that's that. That's easier for them now. They don't need these features, but people have actually said this is a great platform because Drupal is a framework, yet a content management system, and we've embedded it in our business. This allows them to keep that investment without, you know, it's not throwaway to do the upgrade, but the idea is that they can just get more bang for their buck out of their previous investment.

Matt Kleve: 13 years ago, Squarespace wasn't was what it is today like as far as like these site builder things Weren't as numerous or as good as might, you know, somebody might be able to use today. So, I mean, it makes sense that, you know, the technology has moved on and and maybe there's a better way to do it. And maybe that isn't the best way to, to. I mean, Drupal isn't the best thing to run a small site. So yeah, I mean, that's that's good to hear, but giving them that bridge is is important.

Kat Shaw: Yeah. I just kind of going off of what I was talking about that I wonder how many sites like that, like local governments especially that have used the are on Drupal seven have changed to another platform just because of the stress of converting. I know that with the example I gave with Douglas County, Kansas, they actually considered going to other platforms because of that. And I really fought hard with them because I kept in contact. Tried to give them pointers on how to to keep with it, and luckily they did. But I wonder how many have gone off of the platform because of that, and that's too bad. I think.

Matt Glaman: I mean, it's bad, but I'm less of the oh, no. I think it's just because of my history in like the e-commerce realm when I was that commerce guy slash and replatforms happen all the time. You get new management in, you get some new version in. Somebody was like, oh, I know we can do this better and more efficient. We've, we've taken sites off Magento onto Commerce, onto then big. Then they were like, oh, it's going on big commerce. Change is inevitable and replatforms happen a lot. And it usually happens when there's somebody who wants thinks they, they know somebody and can save more money by the other platform or they are facing a big you know, upgrade cost. Okay, well, if we're spending money, let's reevaluate how we're going to spend the money. So we might lose some sites out of it. And I think that's okay because that's just the natural way things go. We will. I think Drupal's innovating in a way that will win back or more like sites with the capabilities. But I want to help provide a tool that lets those that know it's the system they want, but they know it's going to be too hard to upgrade, and they don't have a fleet of like, security engineers to be running their Drupal seven sites behind the scenes.

Matt Kleve: Well, and that timeline is now set, right? At least.

Matt Glaman: Yeah.

Matt Kleve: Again. Set again, because it seems like that Drupal seven end of life has been pushed back a couple of times. But now I kind of believe this one, right? And what are we, a little over a year out?

Matt Glaman: You've got four quarters. If you didn't, you basically need to start planning now. So that way you can execute in Q1 to hopefully launch. So like so that way you can start developing January 1st and then hopefully launch it at the end of Q3. So what September 1st launch it, so that way you can use September October to make sure everything is okay. And then I think the like it's January 5th I think is the end of life. Something like that. So off day. So you have three quarters nine months to do the development and launch if you haven't started yet, realistically. So that way everything can be tested and the bugs are worked out before end of life happens because you want that Drupal seven site shut down before the end of life. You got holidays right around then, so it's a pretty expedited timeline at the moment.

Matt Kleve: Now, if you just had a tiny snowmobile snowmobile club website like your example, I mean, that seems like a really long timeline to to get something upgraded. But folks who are maintaining, you know, a large Drupal seven install that has a ton of content and a ton of nuance because there's always some new feature that needs added along the way. And stuff gets bolted on to the sides. And, you know, as, as time progresses, it's not necessarily straightforward. And the person that did that might not be there anymore. And it's a lift, it's definitely a lift. And we hear about that all the time when folks are trying to to do these upgrades.

Kat Shaw: I was wondering if you had any hiccups during this process of creating this.

Matt Glaman: Oh, I've had so many hiccups. But the funny part is, the hiccups have come along over the past three years because I'm always finding ways like, how can I do Drupal, like use Drupal, but build an app like you would Symfony or Laravel, like trying to use Drupal, like I like Drupal's caching system and entity system, but I don't want to have normal Drupal. So I've always tried to do quirky things with it. Some of the biggest difficulties are just knowing where to hook into the system. So I'm basically living through Xdebug. So if anybody's like Matt, how do you know so much about Drupal? Like, well, I decide to do ridiculous things and then I use Xdebug and I end up stepping through the entire lifecycle of building a page. One of the big ones I think was around Drupal add JS and Drupal add CSS. So in Drupal seven, you used to say, I'm going to add random JavaScript to the page or random stylesheet to the page. And in Drupal eight that throws an exception or well, it. Yeah, if you would attach things in the wrong way, it throws an exception to do this optimized library system. So I found a way to polyfill or not polyfill, but to back port, those two functions and bubble JavaScript settings to the page. So if you do Drupal add JS but you're adding things to the JavaScript settings API, those will show up. But then it still collects all the files you wanted to add via like the JavaScript and the CSS.

Matt Glaman: And I'm finding a way, I'm finding a way to still throw that onto the page for you. So that way it still works the way it did, but it will happen at such a late runtime that it doesn't mess with the pages ag- like aggregates of files or cacheable metadata attached to any of that. So that's probably like one of the biggest hiccups is I always think like I'm right there and then I break the site somehow. Or I'm adding support for custom fields right now. Or rather, I do have it, and for custom field formatters to render it, but I'm adding in support for accessing field data as an array at the same time. So in Drupal, like in Drupal seven, you do field get items and then the entity, then the field name, and it returns it as a, as an array list as an array, each item in there and I have it working and luckily I have a contributor. Darren O has like stepped in and like made a PR instead of reviewing my PRS, which is great. I was like, hey, I'm testing this out and it broke my code because I forgot to do an underscore underscore call. So in Drupal ten, fields are classes or objects. So they can have custom methods. Well, my decorated field item class didn't support that, so it crashed. But it allowed the Drupal seven code to go field bracket RGB for the like the RGB property and work. So...

Matt Kleve: Which is that's straight out of the examples again, right? The RGB that's.

Matt Glaman: Yes, straight out of the examples.

Matt Kleve: I recognize that one.

Matt Glaman: So I think that's one of the biggest quirks is, how do I make the Drupal seven code work but not break Drupal ten code at the same time? And that will happen a lot as I add more entity support because that, Drupal seven didn't have entities. It it kind of did. And then there's the entity module, and then Drupal eight was like, bam. First class entity support. And object oriented support at the same time. So finding ways to, it wouldn't be hard for to have folks change like arrays to arrow access for things like from an array to an object, but what if they didn't have to? Again, it comes down to what if you didn't have to change that line of code? And I'd like to reduce the amount of lines of code people have to change to save everyone's sanity and reduce bugs.

Matt Kleve: Yeah, so it sounds like there could be challenges. Again, it's it's it's there's a lot of things that are functional right now in Retrofit, but it sounds like there's still some bleeding edge, especially if you decided to to push the extent of Drupal. Right?

Matt Glaman: Yes. Yeah. Especially if you if you did really go all in and have some robust custom code. And I don't think anything's impossible. The hardest part is just knowing what people have done. So that way the right areas are retrofit, if you will. You know, like what parts of the Drupal API actually should be have a backwards compatibility layer instead of everything?

Matt Kleve: What kinds of things would be more of a challenge right now? If we had Drupal seven site and we wanted to use Retrofit, what wouldn't work well?

Matt Glaman: I would say the theming layer. Because, as I said, I ran it with Bartik.

Matt Kleve: Sure.

Matt Glaman: When somebody like there's different there are different properties and global variables that are littered throughout pre-process and process functions that when it gets called, it breaks like as soon as I try to run Bartik, it was like, oh, you're accessing a block as an array or regions are different now. Or before breadcrumbs, the page title were properties on the page and not blocks. So I think once people start to use a theme with it, they're going to find all kinds of ways that the site doesn't run or that Retrofit isn't filling a gap. And then to me, that's a good problem because you open an issue and we can solve it. But I think that is going to be more challenging than the entity system, in all my honest opinion. Or just some of the like this obscure APIs that I don't even know about or think about, because I work with Drupal in a very specific way, and others might be on a whole other side of it that I didn't even consider.

Matt Kleve: Kat, what did you used to use as a starter theme?

Kat Shaw: Oh, goodness.

Matt Kleve: I just knew that there were lots of people that that felt like Bartik was a good starting place. I always hated it. So. Yeah.

Kat Shaw: Yeah. You had to. Yeah. Bartik. Classy.

Matt Kleve: Zen was kind of...

Kat Shaw: Zen.

Matt Kleve: ... that world for a while, right? Probably predates Drupal seven even.

Matt Kleve: I was on the Alpha Omega.

Matt Kleve: Oh!

Matt Glaman: And that's it. That's what I forgot. As we left Twin Cities, I made a note to take, the theme for that snowmobile club is an Alpha Omega theming setup. I meant to take that theme and try to run it on Drupal ten to see. All right, I know this was popular. If somebody took Adaptive theme was a big one, the very site builder-y theme, I have no idea. Like some things are just going to be can't do it. But you know what? If you're if that's the bulk of your site is the theme, hopefully you could retheme it easily. Hopefully.

Matt Kleve: What I hear is that it's not a magic wand, but it's a tool because it's going to try and execute it and probably throw an error. And then you go back and look at the error and figure out what the actual problem is. And probably you end up changing your old code or maybe massaging something to make things work. It's not a poof, it works. I've installed this code and now it's awesome. But it's, i've installed this code and now I have the potential of it working, and I might have to...

Matt Glaman: Yeah.

Matt Kleve: ... put in a little effort to, to to do some stuff on my own.

Matt Glaman: I think that's the best way to put it. And it also gives you an immediate feedback loop so you can take the code and run it. And if you get a fatal error, it's like Retrofit didn't work. Or you can see how you have to fix your code for Drupal ten without rewriting all of your APIs, because right now you have to say, oh, I have a hook menu. Okay, let's make a controller. Let's make the routing.yml file. Let's do all this. Or you just copy/paste it in and you go try to visit that page and you see if it crashes or not. If it doesn't crash, congratulations. Great! You test a little bit more if it crashes, like, okay, I need to adapt this part of the code. Like you're not having, you're chance to get immediate feedback on the level of effort even is much higher.

Kat Shaw: Mm-hmm.

Matt Kleve: You have to be the kind of person that's going to say yay, I got a different error this time. Right?

Matt Glaman: Yes.

Matt Kleve: There are, there are people that are like oh crap, I got an error. I'm done. But yay I got a different error. Is progress right? To folks that might be a little more comfortable doing that work. We're talking with Matt Glaman software engineer at Acquia, and his latest brainchild, Retrofit upgrading from Drupal seven.

[Intro music]

Matt Kleve: Welcome back to the Lullabot podcast. We're talking about Drupal Retrofit. Matt, when I was listening to your session at Twin Cities Drupal Camp, somebody else in the room described it as the latest, greatest way to murder puppies or something like that, right? Because this is this is I mean, it's kind of an interesting attack to to plug in old code to a new site, and it just kind of feels dirty, doesn't it?

Matt Glaman: I mean, haven't you gone to a Drupal seven site and then opened a template file and there's like business logic there? So much that it looks like a WordPress theme?

Matt Kleve: Queries in your tpl.php file? Yeah.

Matt Glaman: Yeah. So you know, there's there's I've called it like Frankindrupal's? That was one like there'd be like Frankindrupal's running around with Drupal seven code and Drupal ten. Let's face it, most agents are not sorry that that was a shot. Most builds are like that. Like when you go look at Drupal sites, they're not perfectly engineered systems. You know why? Because they're not products that are being built by product teams, necessarily, and they've got deadlines. So I've had the privilege of most of my life working on sites that are treated like a product. So we are kind of given that time where we find ways to say it has to be built this way. That's not the case. A lot of people like you've got three months to pull it off or you've got to do this one. It's like, we'll get back and fix that technical debt, don't you worry. And then two years later.

Matt Kleve: Of course we will. Right? Yeah.

Matt Glaman: Yeah. So that's where I was like, you know what? There's there's always going to be things that you look at and go, oh, how about that? But this just makes it so that system can actually be upgraded.

Matt Kleve: Yeah.

Kat Shaw: And it's flexible. You know it's flexible. It gives it you know, you have different deadlines. So one one place is small enough. It might have a longer deadline and another place larger and have a shorter deadline. So that's great I like that. I was wondering what what was your inspiration for the for Retrofit?

Matt Glaman: So one of the big inspirations behind this was just, I like to have wild experiments with Drupal code, you know, like I could just be doing PHP stuff, but I'm like, well, let's find a way to do it with Drupal. And I think it's just because it's a really good framework. You know, I have worked with Symfony and Laravel, but there's just things that Drupal has invented that are just great, like our caching system. I wish that was like a standard versus what exists in PHP, PSR land or the Fig, whatever. So I just like to do ridiculous things and then I want to talk about them because I look at like Laravel is a good one. It is like a hype factor over there. Like some of the things I'm like, you could just calm down a little bit, but people are doing innovative, cool things and they talk about it and they support each other and like, this is cool. I wish Drupal did more of this. And I think we do like we talk about sites that are built with Drupal, but maybe not like I built this thing, that lets Drupal act in a completely different way than just build a website or, you know, run in a different way. So that was one of the other motivational factors is like, all right, I need a new wild project to work on. And we had this deadline coming up and people are going to be talking about it. So why not build a tool that kind of just jumps into the middle of that? And it seems to also go in my wheelhouse. Oh, there's PHPStand Drupal, which is being used for all the major version checks composer lenient plugin to help people download stuff and patch it. So why not something to let people have a runtime tool to help with those upgrades?

Matt Kleve: Is this okay like Aquia in there role in in being Aquia and a part of the Drupal community like do they encourage this kind of work?

Matt Glaman: This is completely separate.

Matt Kleve: Okay.

Matt Glaman: So this was my own personal endeavor. I like especially like even PHPStan Drupal, I try to separate my open source work from work.

Matt Kleve: Okay, that's fair.

Matt Glaman: Like if Acquia I have to do open source work like the Aquia DAM module, that's something I built. That's open source, but then I try to keep my personal...

Matt Kleve: DAM being Digital Asset Management.

Matt Glaman: Digital Asset Management System.

Matt Kleve: Yeah. Okay.

Matt Glaman: Yeah. Like Acquia Search and Acquia Connectors it's on a team, and we maintained that.

Matt Kleve: It just seems like it's different from how normal Drupal should be. And I was wondering if you had gotten any flack from that, from your...

Matt Glaman: No.

Matt Kleve: ... Your job or anyone in the community or anything like that.

Matt Glaman: No. But I didn't also like, run around be Acquia's paying my time to do this because they're not.

Matt Kleve: Okay.

Matt Glaman: It wasn't like a it's not like it was like a sanctioned or unsanctioned project. It's just that is when I start, like, you're able to go have your chase your own personal interest. It doesn't have to like your open source work isn't, like, tied.

Matt Kleve: Yeah.

Matt Glaman: To Acquia. Like you're still able to do whatever makes you happy or not so.

Matt Kleve: Cool.

Kat Shaw: So I was wondering what are the future plans with Retrofit?

Matt Glaman: So right now my focus is just on the Drupal seven items, but then on how to help people get off of Drupal nine to Drupal eleven, because guess what's happening in June next year? We get Drupal 11. So Retrofit already provides a backwards compatible layer for module load install. The difference is you'd have to go find your function call and do backslash Retrofit Drupal backslash module load install. So you have to namespace the function. So that would be kind of the next step is like first like let's make it so Drupal seven sites can run or get upgraded. And then let's start adding different you know, backwards compatibility layers for other deprecated APIs. One such thing in a pull request that I need to merge is the module handler. They removed git hook implementations. Something of that matter. Well it's there, it's just not a public function anymore. Guess what? You decorate the service, which means you just kind of wrap it and then you can bring that method back. Is it a good thing or a bad thing? We'll let people decide. But does it make somebody upgrade their site a little bit faster? It sure does. So as the feature will be stabilizing backward compatibility layers for Drupal seven and then adding things for current versions of Drupal, so that way people can upgrade along the way and contrib. So it is under its own GitHub organization. So it's like github.com/retrofit/drupal. And then the Retrofit package is the main one. But there could be like a C tools package. And that helps bridge the gap for C tools in Drupal seven. Because Drupal seven C tools provide a lot of the inspiration for plugins in Drupal eight. So I think there might be a lot that there could be a lot of functionality that gets mapped in correctly. I don't know, I haven't tried it yet, but that would be one idea is make different packages for different ecosystems that are pretty big in Drupal, beyond Drupal core.

Matt Kleve: You know, at the root of this, the more I think about it, it's not all that different than the helper module. Are you familiar with that? They serve different purposes. It's I think the helper module is more like Dave Reid's personal pastebin of things that I do often and, you know, methods or ways that I do these common tasks a whole lot easier. In that it's just kind of like this middle, middle layer of, of translation code.

Matt Glaman: Yeah.

Matt Kleve: Anyway, just just something that I was, I was thinking of there. So based on that, one thing everybody's thinking about is that the Drupal seven upgrade, I guess happens like everybody's being forced to upgrade or should at least be very strongly encouraged to upgrade because security support is ending. And that's kind of the biggest reason to update, because you might be flapping in the wind with something you don't know is a problem until it's too late, because the Drupal security team is no longer on the job. When you start plugging old code into your site, you're still kind of taking your life into your own hands, right?

Matt Glaman: Yes. One of the big, like, every single security issue from Drupal seven was about unescaped output because we had PHP template. So you had to manually escape all user input. And then we got Twig.

Matt Kleve: Sure that was that was the whole, like you always put in your database the whatever was in the form, like whatever the user wanted goes into your database and then on output filter it as necessary, as necessary, as needed. I guess, you know, whatever type of data you're expecting, make sure that other types of data aren't there.

Matt Glaman: Yep. So you'd always filter on output and Twig does that automatically. But in Drupal, for our templating system, that was up to the developer to do.

Matt Kleve: So that's because you tell Twig what it is. Right.

Matt Glaman: No. Just because Twig just does that automatically, like it has sandboxing built into it. Or you're like, or kind of like a...

Matt Kleve: You can make you can tell Twig that this is a raw variable and if you want to put a PHP script or something...

Matt Glaman: Yes, so actually that's what you have to do. In Twig, if you were to try to print like HTML from a, from like a plain HTML string. It will sanitize it for you. You would have to explicitly say no, this is supposed to be printed raw, kind of like in React. So for anybody that does React, it's like dangerously set HTML equals. Right? Like it's some absurd way to say treat this as raw HTML to prevent unescaped output from being rendered. So modern templating systems have a way, it auto escapes but then you have to, like, do something special to say no render this HTML in the raw way it's supposed to be. Drupal seven, it was always raw and then you had to filter it yourself. So that is one big concern. Take field labels, for instance. In Drupal seven, when rendering a field, it would always do like filter the label, whatever the function was called, like filter xss on the label before...

Matt Kleve: Check plain, filter xss, there was a series of them depending on how seriously you needed to take things out. Yeah.

Matt Glaman: Yeah. So Drupal did a lot of those things behind the scenes. So Retrofit doesn't. It could in some ways. But the main thing is all those functions are in Retrofit Drupal. So you, the developers should have been doing that. Now it doesn't mean they did. But the way I think of it is, yes, Drupal took care of parts of it, but developers were still supposed to be doing it. So it might add a bit of a gap there in terms of user input that could be exposed. That was used to like have a pre-process function it ran through, but a lot of it was still on the developer. So I don't think it's any worse off, but it does add that extra, be mindful. And then there needs to be like the proper approach. Is it one where Retrofit Drupal integrates the same way that Drupal seven did and calls it on the same things? Or is it one of buyer beware, update your Twig templates or update your templates. Or this is like for instance, if your theme doesn't have a field.tpl.php file, it's going to be rendered by the Twig template. So it's not a problem. So there's there's all these gotchas. Yes, it's a problem, but I'm not sure how big of one it necessarily is. And I know some folks will say it's security. Any security problem is a big one. But we have to be realistic and think of consequences that this. If you're worried about your your filtered output, chances are you already took care of it and it will just work. Or you should just double over your code as you do this.

Matt Kleve: At the end of the day, you're responsible for your custom code, like one way or the other, right? Regardless of how it's written or what system it's designed to be natively executed in.

Matt Glaman: Mm-hmm.

Kat Shaw: I was wondering has have you touched on the the database abstraction abstraction layer at all in seven?

Matt Glaman: Oh! Yes. All of your classic DB underscore functions are back. That was one of the first things I did because luckily that was really straight forward. Like you just DB select is it's a lot of the same arguments as well. So those are back and in place and seem to work. I haven't done as much testing around that, but all the functions are there and they map in like DB or DB like because again, that API changed in the way it looks. But in terms of like arguments, it's the same thing. So the arguments to the old functions match to the new methods. It's just now all object oriented under the hood.

Kat Shaw: That's awesome.

Matt Glaman: And then some things for variable "get" and "set" are also adapted. Because in Drupal seven we didn't have config, we had variables. Now variables "set" and "get" will push things into the state system which is like the modern variable. Or if it's like a known item such as like menu rebuild, yeah that was one, like if you did variable set menu rebuild Drupal would trigger a menu rebuild. This calls the router and says rebuild the router or get site slogan. Load the site config and return the site slogan. So there's some mapping in there. And those are things that as people work with it, we can expand the compatibility.

Kat Shaw: You're bringing back some memories with some of that [laughing].

Matt Glaman: I know as I was digging in, I was like, oh, I forgot how we did these things.

Matt Kleve: [Laughing]

Kat Shaw: Yeah.

Matt Kleve: Well, I mean, Drupal seven will have existed for a long time now, right? What was the the the date. I.

Matt Glaman: I, I know I haven't worked on Drupal seven since 2016.

Matt Kleve: I was looking that up.

Matt Glaman: Was it like 2010?

Matt Kleve: I gotta pull it up because I, I, I know where to. Oh 2011, January 2011. It had a good run [laughing].

Kat Shaw: Yeah. It definitely did.

Matt Kleve: So what are your what are your future plans, continue to to do more work? Is there anything specifically you're you're itching to get into and finish the progress you've started or?

Matt Glaman: So I want to finish the field support. But I also don't want that to bog me down. I might merge it as is, and then we can address some changes along the way, like any bugs. But documentation, I want to get documentation up because you asked like, what does it cover? And I was like, I've got a list in my slide deck, but I don't have it on the website yet. And it's one of like, I built a site using Next.js. I was like, oh, I want documentation. Well, then I got to. Then I was like, how do I want to do documentation? Do I want it to be in this private site, or do I want it to be in the repo? And then I got stuck in decision paralysis. So one is just getting documentation up in somewhere form and making it so folks can contribute to it as well. And then finding ways to offer support, because I have a feeling organizations are going to want this and be able to come up with a way to ask somebody questions and finding a way to do, like I did a poll earlier and a lot of folks might be interested in a premium ticketing system. So that might be something where if you, your organization can subscribe to like get access to a ticket system to share more private code, because that is one problem. This is a compatibility layer that helps your custom code run, but you don't necessarily want to post your custom code on a GitHub issue. So that's one thing that I was considering as well.

Matt Kleve: And it's a GitHub issue. I mean, I think we mentioned that earlier. Just wanted to point out that if you wanted to go take a look at it, it's a it's on GitHub.com. The project is called Retrofit in a Retrofit-Drupal organization.

Matt Glaman: Yes. And I'm also you'll also find a demo. There's a repo called Demo Examples where I'm trying to have a Drupal ten site that you can clone down, run with DDEV and see Retrofit in action.

Matt Kleve: You also have a website.

Matt Glaman: Yes. So retrofit-drupal.com and that's where the documentation will live as well.

Matt Kleve: Is what is that Drupal. What is that site? Is that Drupal site?

Matt Glaman: That is a, that is Next.js.

Matt Kleve: Okay. I was hoping it was a Drupal ten site running Drupal seven code or something entirely meta like that.

Matt Glaman: Oh, that'd be great.

Matt Kleve: [Laughing] Not that it would make any difference, because that's kind of the point, right? Is like, you don't know what's running under the hood. It's just, you know, it just works. And that's the important part, getting it back to work. Kat, is there anything else you'd like to talk about or wonder about when you're thinking about Retrofit? As we kind of point toward wrapping up.

Kat Shaw: The only thing is, I noticed that you mentioned that it runs on previous versions of Drupal too. So if there's anybody out there that has a Drupal six site, which I guess is a possibility this should work with that too, so.

Matt Glaman: Yeah, I mean, I do know of some people that still have Drupal six Uber cart sites that are running because also like I think I know of at least one, it's behind the scenes because it works and it's now they're like order management system because it works. But, you know, eventually you do kind of want to upgrade it to something that runs on PHP eight. I don't know, I highly doubt their thing does. But you know, so it could be used for older and newer.

Kat Shaw: Yeah. That's great. I really, really love this. This is a really great project. So thank you so much.

Matt Glaman: Thank you. I like to hear that because it went from just being like, I don't know, to like, oh people like this. Great.

Matt Kleve: Matt, anything else you want to talk about when it comes to Retrofit?

Matt Glaman: No. That covers just about everything. It's just, you know, try it out. It there are the docs at least say how to get started or check out that Demo Examples and just plug in your code and see what happens. Like I would love to create like some online runner like PHPStan has like just try it, but that's a lot of engineering work and just give it a shot. And if something doesn't work, open an issue because I'd love to find ways to handle those edge cases.

Matt Kleve: And yeah, I think more people using it helps you, right? Because you understand what works and what doesn't work and things that might need changes, right?

Matt Glaman: Yep.

Matt Kleve: And you mentioned your slide deck. You've been taking this on the road talking about it. Where else are you going?

Matt Glaman: I don't know. I need to find my next stop. Twin cities was the first one, and I need to plan out. Maybe Florida Drupal Camp will be the next one that works out for me.

Matt Kleve: That usually happens in the spring. If I like early spring, I guess.

Matt Glaman: I think February like 22nd this upcoming year I think.

Matt Kleve: That that'll be great.

Kat Shaw: Perfect time to go to Florida.

Matt Glaman: Yes.

Matt Kleve: Thanks, Matt. We wish you luck.

Matt Glaman: Thanks.

[Intro music]