Lullabot Ideas

We know stuff. We empower you to know stuff too.

Drupal as a URL Shortener

Article by Jeff Robbins

Announcing Lullabot's short URL site: lb.cm

URL shorteners are the bane of the internet. However, for those of us who use Twitter on a regular basis, you know that 140 characters can be limiting. URLs can theoretically be 2,000 characters or more, but most are in the 40 to 80 character range. That's almost half your tweet! So we use URL shorteners to not only shorten the URLs, but (in the case of some services like Bit.ly) to track clicks and get statistics about who clicked on these short URLs.

The problem, however, with the popular URL services is that the more that they are used, the longer their URLs get. Both Bit.ly and Tiny.cc are up to 6 characters in their autogenerated URL paths. Add in the URL protocol (http://), the hostname (bit.ly) and the initial slash, and you've got minimum 20 character URLs.

So why use someone else's short URL service, when you can create your own, branded, shorter short URL service with URLs like lb.cm/a or jen.cm/j? A branded URL shortener can also be a sign of prestige or add a layer of trust. For instance, Flickr's URL shortener, flic.kr only links to Flickr photos. When you click on a flic.kr URL, you know what you're going to get. Google has a similar service with goo.gl and WordPress has recently launched wp.me. Even if you open it up for the public to use, as we have with lb.cm, there's just a certain geek prestige in having your own URL shortener.

Why Drupal?

So why use Drupal as a URL shortener? There are several single-purpose stand-alone Open Source URL shorteners out there already. And Drupal is sometimes criticized as being big and slow. URL shorteners need to be small and fast. On the other hand, we're not really expecting bit.ly-level traffic here, are we? Maybe we can consider Drupal.

So what do we get out of using Drupal as a URL shortener?

  • User management - Bang! You've got users with accounts and passwords and permissions.
  • Views support - Provide a nice listing for users of the URL's they've created
  • Security - We can build on top of Drupal's existing security around database requests, user permissions, and
  • Internationalization - A localized URL shortener? A multilingual URL shortener? It's all built-in to Drupal.
  • Hooks - Drupal's hook system is what allows you to build modules. By using hook_boot() for a URL shortener module, we can get in before Drupal bootstraps most of its code. This essentially trims some of Drupal's fat and allows us to do just a single database query to try to load a given short URL. If it exists, "bingo!" we can redirect the user without invoking a complete Drupal boostrap.

And what if our URL shortener does become super-popular? Well then we'd probably start doing some cool tricks with Memcache, maybe a little Varnish, and probably adjust our .htaccess file so that Drupal would never get touched when an existing short URL is requested. All of the Drupal goodness without any of the Drupal bloat!

Building our URL Shortener: lb.cm

I've been using bit.ly for several years and I really like it. My goal was to build something with a similar feature set including:

  • choice of custom (create-your-own) or auto-generated URLs
  • click tracking
  • user-owned short URLs
  • a web services API for shortening and expanding URLs
  • case-sensitive URLs (more characters in less space = shorter URLs)
  • click-to-copy-to-clipboard link on resulting short URLs
  • the ability to alias multiple domains to the same redirects (bit.ly and j.mp)
  • a javascript bookmarklet to autofill a page's URL
  • role-based rate limiting to prevent anonymous overuse of the API, conversely to allow special users unlimited use of the API

I also wanted

  • the ability to add special character Unicode (UTF8) URLs
  • Views support to allow admins to create custom listings

The funny thing about a dedicated URL shortening site is that it doesn't really use nodes! On our site, there's currently only one published node, the about page. The rest of it is just blocks and pages from the ShURLy module.

ShURLy Module

My initial research turned up the Shorten URLs / Short URL combo of modules. Shorten URLs is an abstracted module for using the web services APIs of sites like bit.ly, is.gd, tinyurl.com, etc., to create short URLs using the Drupal interface. Short URL is an API add-on to Shorten URLs which allows you to turn your own site into a URL shortener. I gave them a once over and found the user interface to be confusing with lots of options for things I didn't need and lacking in most of the features which I was looking for. Perhaps most importantly, upon inspecting the way that they were storing their information in the database, I couldn't figure out a way to provide the features I was looking for and also give administrators an upgrade path.

So I decided to start from scratch and create my own module. It's been a while since I've done a from-scratch module and I thought it would be fun. And it was! The result is ShURLy, a module which is now available on drupal.org. It implements all of the features I was looking for. It's really easy to set up and it's got what I hope is a pretty self-explanatory user interface. Download it and check it out for yourself. I'd like to know what people think.

The End Result

The real project here was to create a site, not a module. So that's what we did! The end result can be seen at http://lb.cm. It's open to the public for anyone to use. Anonymous users can create auto-generated short URLs. But if you want to really see the beauty of it, create an account for yourself and you'll be able to see your URL history, get click tracking, and create your own custom URLs. You can even create unicode URLs like http://lb.cm/♥, http://lb.cm/✌, or http://lb.cm/¡ǝɯosǝʍɐ-sı-sıɥʇ.

It's about as simple a Drupal site as you're going to see. It's mostly just ShURLy and a really nice theme design by Lullabot's superstar designer, Jared Ponchot. Once you're logged in, you'll get to see Views in action showing your URL history in a block at the bottom of the page. Views also provides the admin page listing history for all users.

The site also has web services, courtesy of ShURLy, and you can find documentation for them here. Jerad Bitner has already created (the beginnings of) a module called ShurlyBot to allow users to create short URLs in an IRC channel using Drupal's Bot module.

This was definitely one of those weekend-fun-turned-actually-useful projects. A URL shortener is a great example of something that seems simple until you really start digging into it. But now that it's done, we're happy to have lb.cm, and we're happy to contribute ShURLy back to the Drupal community.

Comments

j.mp

Please note that http://bit.ly also uses the http://j.mp alias, which is even shorter.

Redirection API?

Jeff I'd love to pick your brain on a common redirection API that could help solve a lot of duplication with redirection and short URL modules (think Path redirect, Global redirect, Short URL, FeedBurner, etc). For example, every module has to implement a hook_init() or hook_boot() to perform the database lookup to find the redirect for the current path. What if the API module performs a single lookup and then if the redirection loaded is a ShURLy redirect, pass it on to the module.

I'm already working on providing this API on http://drupal.org/project/redirect and I'd welcome feature requests to help modules of this type. We already provide an extensive API and storage, pre-redirect access control and even some cools stuff! For exmaple, allowing an empty page with the 'Location' header to be stored in the page cache. On the next request, we don't even have to do anything. Drupal automatically fetches the page from cache and sends the location header for us.

Interesting

I actually started ShURLy with a dependency on Path Redirect and I was using Path Redirect to do most of the heavy lifting. But I soon realized that if I wanted to do more advanced stuff, I would need to roll my own redirects. But, yes, as someone who's been involved in both the development of the Path Redirect module and now ShURLy, I would love to talk about how a common redirect API might work. My first inclination is to worry about efficiency, speed, and exactly how common the redirecting functionality of these different modules really are. But I guess that just makes me a better candidate for a healthy debate! :-)

Drupal it!

Hi

Here's a Drupal flavoured URL shortener i've been working on.

http://drpl.it

[Still in alpha version]

and a free wallpaper for you ;)

http://img820.imageshack.us/img820/8444/drpl.png

squidwrench?

Why is lb.cm going to a site called squidwrench about "promoting the interaction of creative and technical people in the Mid-Hudson Valley"?

Ha! Someone registered the empty string short URL!

http://SquidWrench.org for the win!

The rest of us can still get there via http://lb.cm/node !

Thanks for making this first, since this is a module i needed to make... but i need it in Drupal 7, maybe i'll be bouncing back and forth between the http://drupal.org/project/redirect and http://drupal.org/project/shurly issue queues...

Another simple solution

Automatic node titles (http://drupal.org/project/auto_nodetitle) + CCK redirection + path auto seems like it would get you most of the way there. Using something like time stamp or a mashup of uid and nid could make random stamps of a reasonably small size (especially for personal use)

Simple?

That sounds simple to you? Maybe this could be implemented as a Features module feature.

Just to Clarify

As the maintainer of the Shorten URLs module, I'd just like to clarify that the module is primarily an API for other modules -- a black box, so to speak -- where other modules can call shorten_url() and get back a shortened URL from "somewhere." It's not a native URL-shortening service like ShURLy; in fact the development build has support for ShURLy as one available shortening service.

great app

I use the modules on my site. Great to have. Use them for twitter, facebook, and email newsletters. You have complete control and the redirect is controlled by you

good module cut url

Great!
It's the module I was looking for, I'll try it on my drupal site!
Thank You for sharing Your knowledge!

Go.USA.Gov

I'm sure you just missed this, so, I just wanted to draw your attention Jeff...
http://go.usa.gov is a Drupal based URL shortener developed by the US government in connection with members of the DC Drupal Community (SeanR, more specifically). All the reasons you mentioned above were the reasons for even taking on the project, and to actually continue to have the data available on a Gov't site. There were so many shorteners last summer, that we all knew some would go away like Cligs and Trim, and then what would happen to the data?

There's even a short video about what we did on the Gov2.0Expo site. http://www.gov2expo.com/gov2expo2010/public/schedule/detail/12260 perhaps there are folks here interested in what took place and where they hope to go in the future.

Drupalove

Great module, I'm loving its features.
Thanks for sharing Jeff!

LxC URL Shortener

Hello,

Check out LxC URL Shortener
Let me know if you like it.

Thank you.