Google App Engine: First Thoughts

by Marty Alchin on April 8, 2008 about Django

Like much of the Web development world, I caught wind of Google’s new App Engine launch last night, during the NCAA championship game. I wasn’t following the Scobleizer video though, so I was relying on the TechCrunch article and people in #django and #django-dev to fill me in on what all the fuss was about.

Well, once I realized what was going on, my interest was piqued. I hadn’t delved into any of these new scalable systems yet, and with Google offering free access to the first 10,000 developers, I figured it was worth staying up an extra half-hour to see if I could get in. Turns out the game went into overtime, making me stay up that long anyway, so it all worked out. Like everyone else last night, I was waitlisted immediately, but that didn’t surprise me in the slightest. Nor was I at all surprised when I received my invitation this morning when I woke up.

So I promptly signed in, created an app and took a look around the dashboard. This was the first time I was surprised. That is, I was surprised by how little they offer in the way of feedback about a site’s operation. Maybe more will show up once I actually get the app configured and running, but so far I’m not impressed. I thought they’d at least have built Google Analytics into the dashboard, even if I did have to include the JavaScript manually. Oh well, I guess it’s two dashboards to see what’s going on: one for the site itself, and one for its visitors. Not a deal breaker, but unfortunate.

Reading over the documentation, I’m surprised how similar many of the features are to Django’s own. Not identical, of course, but similar. Python is the language of choice, at least to start, which makes me incredibly happy. Anything this high-profile that funnels people to Python is better for everyone. It also supports WSGI, the standard interface for Python Web interactions, which means that Django’s HTTP handling and middleware should work just fine, unmodified.

Among the differences, one of the first documentation pages I was linked to talks about running Django on App Engine, and it raised my eyebrows a bit. At the top, the details are very encouraging, stating boldly that “it is possible to use nearly the entire Django stack on Google App Engine, including middleware.” I cheered, but was curious what “nearly” meant. What isn’t possible? Reading down a bit, I found my answer: “the only necessary adjustment is modifying your Django data models to make use of the Google App Engine Datastore API….” Uh oh.

Reading down further still, my fears were confirmed.

Since App Engine does not support Django models, leave all DATABASE_* settings set to an empty string. The authentication and admin middleware and apps should be disabled since they require Django models. The functionality these provide is covered by the App Engine Users API and Admin Console respectively. Sessions also depend on Django models and must be disabled as well.

You read that right. No Django models. For any application. At all. While Django’s documentation touts the ability to swap out any part of the stack for another, many proponents of SQLAlchemy have pointed out - and rightly so - that swapping out Django’s ORM for something else automatically removes a large number of apps from your library. This includes not only some of the apps included with Django itself - including admin, auth and comments - but much of the vast array of third-party apps that are available. It’s always been a given that Django’s ORM would always be available to a Django app, so most people use it. Now that Google has taken it away, it leaves Django severely crippled.

Of course, if you’re building an app from scratch for App Engine (which most people will be), you can just write your models to use the Datastore API and be done with it. After all, as the docs state, App Engine provides admin and auth features for you, so you’re not losing much of the Django distribution itself. However, that also means either rewriting third-party apps you might otherwise use, or at the very least, modifying their code as well, so that they’ll work on the platform. If you’ve ever tried to merge local changes with those from a third-party developer, you know that it’s possible, but it’s not very fun.

To be fair, App Engine does allow you to provide your own copy of Django, should you so choose, which allows you to run trunk. In particular, this removes the claim that you have to disable Django’s own session handling, since in trunk, session engines can be configured for a specific app or operating environment. All we need is a session engine that uses the Datastore API and django.contrib.sessions is back in business. Once queryset-refactor lands in trunk, it might also be possible to write a database backend for App Engine that would allow any app to run properly. Also, my own filestorage patch will make it possible to be store and retrieve files directly within the Datastore or possibly even use to write files to the file system.

One other thing I noticed missing was any mention of caching. To be fair, the limitations on the execution environment mean that most apps won’t have the resources to cache anything anyway, but the whole thing had better be pretty fast to be attractive in the long run. Maybe the Datstore transparently caches things in memory for a while, who knows? Otherwise, some form of caching will need to be implemented, and there’s no great way to do so at this point. The closest that seems possible would be to store completed output back in the Datastore, so that it at least doesn’t need to be queried or calculated each time. This still requires a trip to the Datastore on every request, but depending on what an app does, that could still be a big win. I’d much rather see something in memory, though, like memcached. Of course, it’d probably be some massively distributed system called TemporaryDataStore or something, but I’d be fine with that.

All in all, I’m pretty much in wait-and-see mode right now. I don’t have the time to devote myself to writing an app for it, but I do have one in mind that I think would play quite nicely within its resource constraints, and wouldn’t rely on any third-party apps, so I could get by without Django models just fine. As I have time, I’ll play around with the local development environment and try to put some things online and kick the tires a bit. For the most part, however, I’ll be reading what other people have to say, offering my own thoughts along the way. I’ll probably also post my thoughts on more of the bundled features, including the Users API and the URL Fetch API.