Empty Thoughts

I had a need to colorize the nicks for the new DjangoBot Logger. Instead of managing a collection of names and corresponding colors we decided it would be more simple to just “hash” the nickname using a colorize filter. This causes the same nickname to always appear in the color.

Based on our discussion on the latest episode of This Week in Django we have decided to go ahead and set up a #django-dev IRC channel. This channel will only be for questions related to developing Django itself, not general user questions.

This week we are pleased to have Eric Florenzano on the show. We discuss PyCon 2008, the world-wide sprint, all of Eric’s projects and a few notable blog posts, a few source commits, a new Django book, and recent changes to the Django Bot. This is definitely one of our best shows yet!

So it appears that while we have all been whimpering about missing inheritance functionality in Django models, Malcolm Tredinnick has been extremely busy getting it implemented. Changeset 7126 adds two very cool pieces to Django’s model options: abstract base classes and model inheritance.

You can definitely feel the momentum in Django these days. So, it is no surprise that Sams Publishing chose to offer Django in its Teach Yourself series. This one is written by Brad Dayley and is titled Sams Teach Yourself Django in 24 Hours.

This Week in Django is a weekly podcast about all things Django.

This week we talk about a few source commits, a discussion about unicode usernames, Some cool projects from the community, the Tip of the Week, and a couple of questions from the IRC.

In this screencast I cover converting an existing application to the NewForms-Admin Branch of Django. We will cover hooking the NewForms-Admin into our urls.py module, extracting out the inner Admin class information into a NewForms-Admin class, handling the prepopulate_from, and organizing our admin class information into a separate module.

Neil Roberts implemented the Django template language in JavaScript. Pretty cool stuff. Find out more at Ajaxian

Django Screencasts are now available in flash format at ShowMeDo. It took me a while to get everything working on their end, but Ian and team have been wonderful and quite helpful. Speaking of ShowMeDo, they have a bunch of Python and Django videos as well as others. Great stuff. I encourage you to check it out.

This Week in Django is a weekly podcast about all things Django.

This week we talk about a few source commits, the new IRC bot, a few development level proposals, Some cool projects from the community, the Tip of the Week, and a couple of questions from the IRC.

In case you missed it, DPaste is now officially Pony Enabled. Pbx rocks!

By the way, for more pony goodness learn all about Paul Bissex on the This Week in Django 6 podcast.

In this screencast I cover how to implement application based settings in a way that keeps the application pluggable. Watch this screencast and learn more.

If you plan to attend PyCon 2008 in Chicago (and you should plan on it) you can now proudly display PyCon 2008 Badges on your Web site.

In this screencast I cover how to map specific URLs to the Flatpages contrib. There are times when you want the advantages and flexibility of flatpages, but also do not want URL mapping to be open ended and are uncomfortable with the fallback middleware approach. Watch this screencast and learn more.

Downloads

MP4 Edition (23.7 MB, 15:14, H.264, MP4)

OGG Edition (25.2 MB, 15:14, Theora, OGG)

IPod (11.9 MB, 15:14, H.264, MP4)

Requires the latest version of QuickTime, VLC Media Player, or comparable software capable of playing MP4 container format and the H.264 codec.

Feeds Available

iTunes Feeds are available, So please choose the iTunes feed if you want automatic downloads in iTunes. Please show me some iTunes love.

iTunes Feeds

Django Screencasts – MP4 Edition

Django Screencasts – iPod Edition – Still waiting for it to show up

Regular RSS Feeds

Django Screencasts – MP4 Edition

Django Screencasts – iPod Edition

Show Notes

• Django Documentation on Flatpages

This Week in Django is a weekly podcast about all things Django.

This week we have our special guest Matthew Wensing on the show to discuss his work with the Django GIS Branch (GeoDjango). We also cover EveryBlock, Some cool projects from the community, the Tip of the Week, and a couple of questions from the IRC.

Shabda Raaj has created a site called 7Days7Apps where he is attempting to create seven Django applications in seven days. At this point he’s on day three. It’s an interesting idea and sure to generate some marketing for his site (already #53 on programming.reddit.com), but it is a bit of an exaggeration to call these applications. So far he has created a ToDo List, a DPaste “clone”, and a time tracking application.

It is really good to see people that are passionate about Django and want to be able to demonstrate that passion. Although, as expressed in some of the reddit comments, I can’t help but wonder if it would have been more impressive to create one well thought out app with a nice design in seven days.

As promised, Brian Rosner, co-host of This Week in Django, has just released a screencast on Using Git with Django. Although this is Brian’s first screencast, he has put together a very professional tutorial. It’s enough to wet your appetite for using Git with Django.

I expect, and hope, that we will see more great screencasts from Brian in the future.

Simon Willison just released a very cool Web site called Django People. It’s a great mashup that allows Django developers all over the world to be able to get connected and find out more information on those in the community and where they are located.

David Cramer just posted some interesting information on a caching layer for Django’s ORM that he has been developing. David is a lead developer for Curse, so he has some experience with scaling applications. It is early work, but looks very promising. The code is available for review on Google Code.

This Week in Django is a weekly podcast about all things Django.

This week we discuss the opening of PyCon 2008 registration, an important changeset, Some cool projects from the community, the Tip of the Week, and a couple of questions from the IRC.

Carl Karsten (CarlFK) just hoped onto IRC and announced that the PyCon 2008 Registration for Chicago on Friday, March 14, through Sunday, March 16, 2008 is now open.

This Week in Django is a weekly podcast about all things Django.

This week we have a very special guest this week. Paul Bissex, creator of dpaste.com among other things, joined us for an in-depth interview. During the interview he reveals a very special surprise, a new Django book he is writing with two co-authors, Jeff Forcier (bitprophet) and Wes Chun (author of Core Python Programming).

You don’t want to miss this one!

I am pleased to announce that Brian Rosner has agreed to join me on This Week in Django to co-host the show.

Peter Sheats mentioned today in the #django IRC channel that he wrote a script to autoload his installed application models automatically in the shell. This is something that I’ve had on my todo list for some time. All the importing when I’m trying to quickly test out something small in the shell can get quite cumbersome.

Peter’s approach is a short and nifty piece of code. Check it out and save yourself a lot of typing. Now I owe Peter a beer.

miracle2k (do not know his real name) posted great entry titled FuzzyDates, or: One Django Model Field, multiple database columns. In this article he presents an in-depth approach to implementing a model field in Django. It is especially insightful given that he is actually using a single field to store into two database fields. His particular use case is to implement a fuzzy date field, which in itself is also quite interesting.

I plan to implement a complex model field in the coming days, so this is just perfect to help me get into it.

PyCon 2008 in Chicago, IL will be offering tutorials on Thursday, March 13. The official PyCon blog just announced the tutorials that will be available. There looks to be over 29 tutorial sessions available, including a Django tutorial by Jacob Kaplan-Moss and a Django Code Lab with Jacob, Adrian Holovaty, and James Bennett. There’s also looks to be beginning and advanced tutorials on SQLAlchemy which could be quite interesting. Ah, decisions.

This Week in Django is a weekly podcast about all things Django.

This week Brian Rosner joins me once again. We discuss the Akita interview of Adrian Holovaty, a few changesets, the Queryset Refactor branch, a Tip of the Week, and a few questions from the IRC Channel.

Fabio Akita (not the Fabio) has an excellent interview with Adrian Holovaty, creator of the Django Framework.

I’ve been following Akita’s work for some time and I’ve enjoyed it immensely. Recently he reproduced the original Rails screencast, to update it. I also enjoyed his two-part interview with Avi Bryant.

Even if you’re not doing Rails stuff, Akita’s feed is a Must Subscribe.

This Week in Django is a weekly podcast about all things Django.

This week we have a very special treat. Brian Rosner will be joining me to give an update on the NewForms-Admin branch as well as provide input during the rest of the show.

You do not want to miss this one!

People ask about blogging apps for the Django platform all the time. I have not investigated each and every one of these, but so far I still feel pretty confident that not one of these is fully baked in any sort of manner comparable to something like WordPress or Typo. That said, there’s a lot of great code here, and if you’re not interested in creating your own, use one of the following.

Well I finally got my feeds to show up in iTunes for the This Week in Django netcast. Here’s the relevant bits

This Week in Django in a weekly podcast about all things Django. This Week in Django 3: Django Book Online, dumpdata changes, On disk sqlite for tests, Django Snippets, a Documentation app, a Pownce Widget, MEDIA_URL in context, and a couple questions from IRC.

Today there was a question on the #django IRC channel from a user name “Joe” about how to go about retrieving selective fields from a table. The issue is that Joe wants to be able to get all Links that are tied to a specific Category but only retrieve select fields. You might want to do something like this if you needed to retrieve a subset of information and you wanted to keep from pulling particularly large fields like BLOBs or Text fields.

In This Week in Django 2 – 2007-12-16 I mentioned in closing that I often use Unipath to keep things portable. This post shows you how.

Stefan Imhoff just created a set of Django folder icons for Mac OS X. To install just select a folder, hit command+i, and drop the folder icon over the image icon in the upper left-hand corner.

Thanks so much Stefan.

Lately I have been thinking a lot about how well Django adapts to new users. I noticed that there seems to be an influx of first-timers. Although these programmers come from a variety of backgrounds, there are some areas within the framework that consistently cause stumbling blocks. In a lot of ways it is part of the reason that I’ve devoted a segment of This Week in Django to the subject. So what can be done to make Django more new user-friendly?

This Week in Django in a weekly podcast about all things Django. This week highlights PyCon 2008, ModelForms changes, Practical Django Projects book, Sphene Community Tools, Two-Faced Web, authentication decorator on generic views, and a couple questions from IRC.

It looks like all the updates to the Django Book have been made available to the online version. Check it out now: http://www.djangobook.com

This Week in Django in a weekly podcast about all things Django. This episode discusses the Django Sprint, a few important Changesets, some notable things going on in the Django community, the Tip of the Week, and some fodder from the IRC channel. Please see the Show Notes below for all the pertinent information and links.

In this screencast I cover NewForms Personalization. Often you have a drop down or some other field in your NewForm that you need to have personalized, or changed based on the logged in user or other instance criteria. This screencast will show you how to accomplish that in your applications.

Very exciting news!! Those of you who follow James Bennett’s blog, The B-List, closely already know the news. For those of you who don’t (yes both of you) there is great news on the Django resource front. James Bennett just announced that he’s writing a Django book titled Practical Django Projects. It will likely not hit the streets until June 2008 time frame but it sure puts a spring in my step tonight.

Mattias Winther just wrote an excellent post, Growl and Django, describing how to use Growl and Django together to get notifications from your Django application to your Mac OS X system. It’s a real clever combination and really opens up the possibilities for other interesting forms of integration.

Well it looks like it’s finally here. The Definitive Guide to Django: Web Development Done Right is finally shipping. If you haven’t ordered your copy yet, head on over to Amazon and get it today.

On the heels of James Bennett’s post titled “Where is Django’s blog application”, the folks at Blogcosm just released Blogmaker. Blogmaker is a full-featured, production-quality blogging application for Django. It supports trackbacks, ping and comments with moderation and honeypot spam prevention.

As many of you know, Django recently introduced an enhancement that automatically auto-escapes all template variables. It’s a very elegant solution and something that I’m really excited about. Bob Follek recently wrote an interesting post asking the question Why The h Can’t Rails Escape HTML Automatically?”...

It looks like Adrian Holovaty just committed an update that modifies the functionality of the Send to DPaste enhancement which I wrote about previously. There are two modifications…

I think by far one of the coolest features to come out of the Django Sprint is the new “Send to DPaste” button on the Exception information page. The Exception page was already very slick and informative, but this feature makes it quite delightful.

Well the Django Sprint is over and I thought I would take this opportunity to comment on my overall feelings about the process.

Final live blogging entry on the Django Sprint…

The official Django Spring ended yesterday (I’m in the USA by the way), but I know that there will be still quite a few people plugging away on issues so I thought I would continue to do so as well…

Continued live blogging on the Django Sprint. I’m about done for the day.

Continued live blogging the Django Sprint…

Continued live blogging on the Django Sprint.

12/1 – 11AM

Well theju and I just finished up testing the patch for ticket #3632. We’ve decided to let the triagers decide if we need to add tests for it. The console testing we did proved the patch worked fine. It’s such a small change. I think if I have some time later today I’ll go ahead and add tests to it.

Things are really getting into full swing at this point since the Lawrence folks are online. It’s great to see everyone working together on all of this. A lot of open source projects could really benefit from this type of approach. Not only does it get a lot of things accomplished, but it also allows a bunch of people to get involved in the process and realize that contributing is not as scary as it might sometimes seem. This whole experience is a great community builder.

Time to go get a Christmas tree…

Continued live blogging on the Django Sprint.

Today (depending on which timezone you’re in) the Django Project is doing a worldwide sprint. It’s a time where developers all of the world come together in person and through IRC to work on squashing bugs and making enhancements. You can find more information about the sprint at the Django Project Wiki. I’m participating in the sprint and plan to do some live blogging on it as we go.

Looks like some of the PyCon 2007 talks are available for download. I really looking forward to these since I was not able to attend this year.

In my last post on Customizing NewForms Nick pointed out that I had forgot my save() method in the form class definition. The problem was a simple copy and paste error but as I began to copy in the save() method it occurred to me that there might be a simpler way to handle saving in NewForms. This post offers an alternative solution.

In my last post I covered a very basic introduction to getting NewForms working in your application. If you’ve followed along then at this point you should have a functioning add/edit form that posts back your information and saves it in the database. This works great as long as you want every field in your model to appear on the form. Generally that’s not the case. In this post we’re going to look at five ways to modify which fields that appear in the NewForms output.

NewForms are a wonderful thing, and once you “get it” there is an “a ha” moment and you begin to appreciate the architecture behind the new forms implementation. Until then it is easy to find yourself struggling with getting basic functionality working. This post will be a simple walkthrough on creating newforms.

If you have global custom management command extensions or if you have suggestions for some that would be useful to you in your daily work with Django, we want you. Yesterday I posted a new project on Google Code, called Django Command Extensions, that will hopefully grow into a nice repository of custom management command extensions.

I had a need to generate a new secret key for a Django project that I’m working on. Often when I start a new project I’ll just copy an existing project template that I have which has all the bits and pieces in the right places. This helps me to get going quickly without a lot of fuss. Although when doing so I always need a new SECRET_KEY for the settings file. Instead of doing it manually this time, I decided to create a new custom management command and make it part of my global management command extensions project.

While creating the most recent Django Screencast on Custom Management Commands, I had it in my show notes to mention how you can call Django’s management commands from an external script. There’s probably several different ways to accomplish this, but the simplest and “baked in” way to do this is through the use of the call_command function.

I mentioned in the Django Screencast that I would be putting together a cheatsheet on Django’s custom management commands, so here it is. I hope you find it useful.

In this screencast I cover adding custom management commands into your Django applications. Custom management commands are a method of extending Django’s built-in commands that are made available through the manage.py or django-admin.py scripts.

In this screencast I cover implementing model based feeds using the syndication framework.

We’ll start where we left off last time and enhance our simple blogging application by adding feeds for individual sections of the website. We will walk through modifications to the urls.py, implementation of a model based Feed class, and changes to our base template to expose the feeds. Finally I will briefly cover how you can use Atom feeds in place of RSS.

In this screencast I cover implementing Django’s syndication framework into an existing application.

Björn Kempén has written a nice post titled Simple middleware for database driven IP ban using django where he describes a method of using middleware to handle IP based banning in your Django application. This post really illustrates how easy it is to tie into the request / response cycle to add in customized functionality.

Just the other day I heard about two interesting Django projects. Ross Poulton, creator of the Django Sites website, is at it again. This time with a community site providing a centralized repository for Django Apps. Things are still baking on this one, so I don’t have a url to point you to at the moment, but keep an eye on Ross’ blog.

I’m especially excited about this project because this has been something on my personal ToDo list for some time. Ross did such a great job with Django Sites, I’m eager to see what he comes up with. For a while now I’ve been offering some Google Foo that I got off of the #django IRC Channel as a way to find needed apps. It is a relief because now I will have somewhere to point people when they are looking for an app, like Django-Tagging.

The next project is a Django App which not only will fill a nice void but also provide some best practices examples for creating your own apps. Collin Grady, James Bennett and others have teamed up to create Counterpoint, a Django Forums app. The need for a forum app comes up quite often, so I’m glad to see that these folks are cranking away on it. I got a chance to play around with a “live” version the other day and I was quite impressed.

It is great to see the community continue to build on such a great platform.

One other notable commit this past week is Changeset 5877. This particular changeset helps dry up generic views a bit. The modification sets the default slug_field key to 'slug'. Why is this important?

I think someone is reading my mail, or should I say mind. For the past several days I had been drafting a preliminary patch and rationale for proposing that the manage.py system be opened up to allow extensibility of custom commands. My initial motivation for this functionality was due to my Newforms Generator. I wanted to be able to inject this behavior into manage.py without having to submit a patch and get buy-in. Well thankfully it looks like someone else had the same exact idea. With Changeset 5925 we get this functionality.

Coming from Ruby / Rails world I have really missed Rake and the simple extensibility of that tool to automate common tasks. Now with extensibility in manage.py I can do a lot of the same things I was accustomed to previously.

The docs online are not updated yet to reflect this change but the Changeset patch has all the information you need to get started.

Very cool stuff.

UPDATE

As noted in the comments, it looks like this one was rolled backed for a bit. Something about it not working. :) I’m sure the required fixes will be put in place shortly.

In the August 6th Django Weekly Roundup written by Clint Ecker a comment was posted by “Mac” that expresses concern over the strength and dynamics of Django development. Now “Mac” is likely just a troll, but let us say for a moment that “Mac” has a legitimate concern. “Mac” suggests that because the Django Weekly Roundups tend to be more about what’s going on in the Django Community (outside) than the core Django development itself (inside), that somehow that is indicative of a lack of core development interest. The opposite is actually true. Let me explain.

The Django ORM is quite flexible and provides a lot of ways to get at the data you need without having to write raw SQL, although that option is always available. Every once in a while someone on the #django IRC channel will ask about LEFT JOIN functionality. Although, the ORM doesn’t support LEFT JOINs “out of the box” (as best as I can tell), there are creative approaches to the problem.

Did you ever need to display the SQL that Django is generating, within your shell? I just might have an answer.

I’m still enjoying the nuggets of goodness in the Django Master Class slides. I love this little tidbit.

Yesterday there was a thread of discussion on the Django Users Google Group about the best way to implement an age in years calculation. The outcome turned out to be quite a creative implementation.

Today there were two questions in a row on the #django IRC channel about getting Django’s Syndication Framework working. The syndication framework is used to add syndication feeds to your Django project. Unfortunately for the two individuals that I was attempting to help we got nowhere with resolving their issues. It struck me as odd because when I implemented syndication for the first time “it just worked.” I decided to dig into the source of the syndication framework contribution in an attempt to better understand exactly what is going on under the covers.

Setting Up Our URLConf

The first problem that we dealt with was an invalid URLConf specification for the feed. Although the documentation doesn’t state it explicitly, you are required to specify the url parameter in your URLConf definition. So in this case your best bet is to just follow the documentation exactly. A proper URLConf ends up looking like this:

(r'^feeds/(?P<url>.*)/$', 'django.contrib.syndication.views.feed', {'feed_dict': feeds}),

You can futz with the prefix if you want, for instance ’^/rss/feeds/’, but the rest of it should pretty much remain the same. The above URLConf specification will grab all url text after /feeds/ and pass it into the feed() method in the syndication view. The url parameter must at least contain one item after the /feeds/ part of the url. Without this you will get an error that states:

slug isn't registered. 

This error is a bit confusing, especially if you have a slug field on the model that your feed is based upon. The terminology is correct, but sometimes it can throw you off. The url parameter is made up of two parts: slug and params. The slug in this case is the first bits after the /feeds/ url. The params is everything else after that. So for instance if we have a url that looks like this:

/feeds/latest/

The slug will be ‘latest’ and the params will be empty. Likewise if you have a url that looks like this:

/feeds/category/django/

The slug is ‘category’ and the params is ‘django’.

The slug is the lookup value into your feed_dict that was specified in the URLConf. So in the two cases above you need to be sure to have a ‘latest’ and a ‘category’ key in your feed_dict. Continuing with our example, I should have a feed_dict that looks like the following:

feeds = {
    'latest':   LatestEntries,
    'category': LatestEntriesByCategory,
}

If you do not have these specified properly then you will receive an error message that says something like:

Slug 'category' isn't registered.

So if you get that message you know you have a mismatch between what’s in the url and what’s specified in your feed_dict. Ensuring that those two items are synced up will solve most of the problems.

Creating Our Feed Class

That next part of the implementation is creating your Feed class. Continuing with the example above I might implement my Feed classes like this:

from django.contrib.syndication.feeds import Feed
from models import Entry, Category

class LatestEntries(Feed):
    title_template = 'feeds/title.html'
    description_template = 'feeds/description.html'

    link = '/blog/'
    title = "My Blog" 
    description = "Latest entries in my blog." 

    def items(self):
        return Entry.objects.order_by('-created_at')[:5]

class LatestEntriesByCategory(Feed):
    title_template = 'feeds/title.html'
    description_template = 'feeds/description.html'

    def get_object(self, bits):
        if len(bits) < 1:
            raise ObjectDoesNotExist
        return Category.objects.get(slug=bits[-1])

    def title(self, obj):
        return "My blog for category '%s'" % obj.name

    def link(self, obj):
        return obj.get_absolute_url()

    def description(self, obj):
        return "Blog entries recently posted in category %s" % obj.name

    def items(self, obj):
        return Entry.objects.filter(category__slug=obj.slug).order_by('-created_at')[:10]

The first example is simplified by the fact that we only have a slug and no params. There are a couple of interesting things to note in the above code. First I’m specifying my title_template and description_template explicitly. If not specified the names for each of the above classes would be ‘latest_title.html’, ‘latest_description.html’ and ‘category_title.html’, ‘category_description.html’ respectively. Since the templates end up being fairly generic, I specify them in my Feed class so the same templates can be used for all of my feed classes.

Also notice that you can define the properties of your feed using either simple variables or defining methods. For instance, in the LatestEntries class I define description as a simple variable that has the description of my feed. In the LatestEntriesByCategory, instead I define description() as a method on my class, because I want to utilize the item object instance (obj) to provide a category relevant feed.

Another item of note is the get_object() method defined in the LatestEntriesByCategory Feed class. This method is responsible for returning the object that is being looked up. In other words if our url is /feeds/category/django/, the object returned should be an instance of the Category class that is for the django Category. We’re doing some basic checking to be sure that a category was specified, since a /feeds/category/ url doesn’t make sense without specifying the category we want. We then look up the category and return the Category instance object. If the object doesn’t exist or isn’t returned, you will receive a FeedDoesNotExist exception.

There are a lot of methods you can override in your Feed class definition to get the behavior you desire. For instance, if you want to specify an Atom feed instead of an RSS feed, that’s possible. All of the available additional methods are documented quite nicely in the Django Syndication Framework documentation. One you get a basic feed implementation working I encourage you to look into all the different possibilities.

You also may run across another error:

Give your Entry class a get_absolute_url() method, or define an item_link() method in your Feed class.

This message is pretty clear. If your model class, in this case Entry, doesn’t have a get_absolute_ur() method I suggest you add one. It’s pretty integral to a lot of things in Django so it’s best to have one defined. You may also define your own item_link() method in your Feed class if you want to do something funky with the links that are provide for each Feed item. So keep that in mind.

Sort of along these same lines, the Sites framework is required if you’re using a version of Django before changeset 5654. That changeset removes the requirement for the Sites framework. If the Sites framework is not installed the domain information is garnered from RequestSite, the current HttpRequest SERVERNAME information, instead.

Defining the Templates

In addition of the above we need one more piece to complete the puzzle, our templates. The templates are responsible for rendering the feed data, just like templates in your application are responsible for rendering the view data. As a result, the templates end up being very simple. In our case we will just use the defaults:

{{ obj.title }}

{{ obj.description_html }}

After doing all of the above test it out. Hopefully everything is in place and things will come up roses. If you’re using the Safari web browser and testing it against localhost, you might see the following error:

Safari can't open the page 'localhost:8000/blog/feeds/category/django/' because it cannot redirect to locations starting with 'localhost:'.    

As far as I can tell this is an issue with Safari when trying view a feed against localhost. On live sites it works fine. Also if you test in another browser like Firefox you should not have a problem.

Although the syndication framework is pretty simple to understand and implement there are pitfalls you need to watch out for. I hope that this information helps explain a few of the problems you might run into when implementing feeds on your site. I also hope that next time there’s a question on the #django IRC channel I can be a bit more helpful.

Even though you’re seeing all the ugly details in this post, I continue to be amazed at the simplicity and elegance of this implementation. As often heard in the Django community, “They make it stupidly simple.”

The past few days I’ve been making my way through the Django Master Class slides that were part of a presentation put on by Jacob Kaplan-Moss, Simon Willison, and Jeremy Dunck at OSCON 2007. I’ve been enjoying them immensely. There’s a lot of great stuff in there, like information on Signals, Ajax Javascript validation, and Template Tags. I’m looking forward to digging deeper into each of these items.

Highly recommended.

Recently I needed to implement some Ajax functionality into a Django app I’m building. Although Django has full support for working with Ajax and the Ajax library of your choice, out of the box it doesn’t steer you in any particular direction in this regard. Because of this I had been a bit hesitant about attacking this particular problem, fearing I would end up losing days of productivity trying to get something going. Luckily my fear was unfounded. It turns out that it’s fairly easy to implement, and quite simple to get your head around it.

One of the things I really needed was an easy way to serialize objects of data as an Ajax response back to my client. I searched around and through a combination of various forum postings and IRC, I was able to come up with the following generic json serializer. Like Generic Views, it allows me to easily make requests without a bunch of extra view code for standard cases. The implementation that I ended up with looks like this:

from django.http import HttpResponse 

def serialize(queryset, root_name=None): 
    if not root_name: 
        root_name = queryset.model._meta.verbose_name_plural 
    return '{"total": %s, "%s": %s}' % (queryset.count(), root_name, json_encode(queryset))  

def json_generic_query_serialize(request, queryset, criteria=None, parameter='query', root_name=None): 
    if criteria:
        qs = queryset.filter(**{'%s' % criteria: request[parameter]})
    else:
        qs = queryset
    return HttpResponse(serialize(qs, root_name=root_name), mimetype='text/javascript')

The only item you’re not likely to have is the method json_encode by Wolfram Kriesing. Believe it or not I found this tidbit on a dpaste post and it was exactly what I needed. Later I found out that Wolfram blogged about this item on his website. I encourage you to read his reasons for writing this nice tidbit.

The reason the default json serializer (django.core.serialize) doesn’t work well for my purposes is because the default implementation is built around the idea that the objects that are serializable need to be deserializable as full objects as well. I’m using the Yahoo! User Interface Library as my Ajax library and YUI will not work well with the default json structure provided by Django.

Just in case the json_encode method gets removed from dpaste I’m reprinting it here. I want to stress again that this is the work of Wolfram Kriesing, not mine. I included this method in the same file as my serializer methods posted above.

import types
from django.db import models
from django.utils import simplejson as json
from django.core.serializers.json import DateTimeAwareJSONEncoder
from decimal import *

def json_encode(data):
    """ 
    The main issues with django's default json serializer is that properties that
    had been added to a object dynamically are being ignored (and it also has 
    problems with some models).
    """ 

    def _any(data):
        ret = None
        if type(data) is types.ListType:
            ret = _list(data)
        elif type(data) is types.DictType:
            ret = _dict(data)
        elif isinstance(data, Decimal):
            # json.dumps() cant handle Decimal
            ret = str(data)
        elif isinstance(data, models.query.QuerySet):
            # Actually its the same as a list ...
            ret = _list(data)
        elif isinstance(data, models.Model):
            ret = _model(data)
        else:
            ret = data
        return ret

    def _model(data):
        ret = {}
        # If we only have a model, we only want to encode the fields.
        for f in data._meta.fields:
            ret[f.attname] = _any(getattr(data, f.attname))
        # And additionally encode arbitrary properties that had been added.
        fields = dir(data.__class__) + ret.keys()
        add_ons = [k for k in dir(data) if k not in fields]
        for k in add_ons:
            ret[k] = _any(getattr(data, k))
        return ret

    def _list(data):
        ret = []
        for v in data:
            ret.append(_any(v))
        return ret

    def _dict(data):
        ret = {}
        for k,v in data.items():
            ret[k] = _any(v)
        return ret

    ret = _any(data)

    return json.dumps(ret, cls=DateTimeAwareJSONEncoder)

Once the generic serializer is in place, it becomes easy to use. We need an entry in our urls.py file that routes the Ajax request to our generic serializer. A typical implementation will look like this:

categories_dict = { 
    'queryset': Category.objects.all(), 
    'root_name': 'categories',
    'criteria': 'name__icontains',  
} 

urlpatterns += patterns('', 
    (r'json/categories/', json_generic_query_serialize, categories_dict), 
)

Notice how flexible this implementation is. We can pass any criteria we want for finding our items. If we wanted to simply return all items, we would only need the ‘queryset’ item.

So what does this look like in the Form definition in our Forms.py file? It depends on what we want to accomplish. As an example, I’m implementing an AutoComplete field from the Yahoo! User Interface Library. My implementation looks like this:

category = forms.CharField(max_length=255, 
                           label=u'Category', 
                           widget=AutoCompleteField(source='/blog/json/categories/', 
                           schema=["categories", "name", "id"], 
                           options={ 'animSpeed': 0.1, 'autoHighlight': True, 'forceSelection': True, 'typeAhead': True }))

In the above sample I’m using a custom AutoCompleteField widget that I created that gives me all of the auto complete goodness that I need. I’ll write more about the implementation of that custom widget field in another blog post.

As you can see, solving the problem of easily getting serialized data in json format makes the whole Ajax process much simpler. Although Ruby on Rails bakes this functionality into the framework at times it becomes a two edged sword. It sure makes it easy for new users to start using Ajax immediately with little or no knowledge, but it also makes it quite painful to use any Ajax library that has not been “anointed.” I have no person preference of one implementation over the other, it’s just important to recognize the advantages and disadvantages of differing approaches.

Django’s newforms has a couple of very useful helper methods for generating a Form class based on a model (form_for_model) and a model instance (form_for_instance). Both of these are quite useful, but quickly I find that I want to override the default widget and/or hide certain fields that I will be handling in the POST operation, etc… In no time I reach a point to where I often realize that it’s just easier to define my own Form class for my model and make all my specification in there. You can read all about creating your own Form classes in the Django Documentation on newforms.

The problem is I’m lazy. If I have a model with a fairly large number of fields, this can be a tedious task. To counteract this, I decided to create a simple generator that I use to create the Form class for me based on the model I pass in. It can also accept just certain fields if you know that you want to limit the list of fields in your Form class. The generator is not perfect and will sometimes include defaults, leading to more verbosity than necessary, but it’s a great way to get going on custom forms.

def describe_form(model, fields=None):
    """ 
    Returns a string describing a form based on the model
    """ 
    opts = model._meta
    field_list = []
    for f in opts.fields + opts.many_to_many:
        if not f.editable:
            continue
        if fields and not f.name in fields:
            continue
        formfield = f.formfield()
        if not '__dict__' in dir(formfield):
            continue
        attrs = {}
        valid_fields = ['required', 'initial', 'max_length', 'min_length', 
                        'max_value', 'min_value', 'max_digits', 'decimal_places', 
                        'choices', 'help_text', 'label']
        for k,v in formfield.__dict__.items():
            if k in valid_fields and v != None:
                # ignore defaults, to minimize verbosity
                if k == 'required' and v:
                    continue
                if k == 'help_text' and not v:
                    continue
                if k == 'widget':
                    attrs[k] = v.__class__
                else:
                    attrs[k] = v

        params = ', '.join(['%s=%r' % (k, v) for k, v in attrs.items()])
        field_list.append('    %(field_name)s = forms.%(field_type)s(%(params)s)' % { 'field_name': f.name, 
                          'field_type': formfield.__class__.__name__, 
                          'params': params })

    return '''
from django import newforms as forms
from models import %(object_name)s

class %(object_name)sForm(forms.Form):
%(field_list)s
''' % { 'object_name': opts.object_name,  'field_list': '\n'.join(field_list) }

The code is pretty straightforward. I generally just import it and call the describe_form method from a shell. Obviously there’s a a lot more I can do with this, such as more intelligent ignoring of defaults. In addition, I would like to wire it into django-admin.py so I can do something like:

./manange.py describe_form blog.Post

I could then have it output to the shell, or redirect it to a file.

The template system within Django is, in my opinion, one of the crown jewels of the web framework. There are a lot of opinions about what is the “correct” implementation of a template system—and most will never be convinced differently. For me, the template system in Django provides the right balance of flexibility and constraints.

On a recent project I needed to be able to provide Next and Previous functionality in a website (similar to the implementation of Polls at LJWorld). Out of the box I couldn’t find this functionality, but luckily the developers of Django made it stupidly easy to add your own tag and filter libraries.

If you’re interested in adding your own tags or filters, I highly recommend downloading the source to any of the following projects:

• django-template-utils
• django-comment-utils
• typogrify

Each of these provide excellent examples. In addition, you should definitely look at the Django Documentation on Django Templates for Python Programmers.

For my project, I started with the some code from django-template-utils and then modified to fit my needs. I began with the parser token part of the the tag and ended up with the following.

def do_next_object(parser, token):
    """ 
    Retrieves the next object from a given model, in that model's
    default ordering, and stores it in a context variable.

    Syntax::

        {% get_next_object [object] using [field] as [varname] %}

    Example::

        {% get_next_object object using pub_date as next_object %}

    """ 
    bits = token.contents.split()
    if len(bits) != 6:
        raise template.TemplateSyntaxError("'%s' tag takes five arguments" % bits[0])
    if bits[2] != 'using':
        raise template.TemplateSyntaxError("second argument to '%s' tag must be 'using'" % bits[0])
    if bits[4] != 'as':
        raise template.TemplateSyntaxError("fourth argument to '%s' tag must be 'as'" % bits[0])
    return NextObjectNode(bits[1], bits[3], bits[5]) 

There’s nothing too complex going on here. Mainly this part of the code is just validating that the user has sent in all the right bits and pieces. How you structure the language of your template tag is up to you. For me I thought the above read quite nicely. Remember, you’re creating these for designers not developers, so try to refrain from using developer based terms.

The real work occurs in the overridden render method in the template node. Mine ended up looking like the following:

class NextObjectNode(template.Node):
    def __init__(self, obj, field, varname):
        self.obj, self.field, self.varname = obj, field, varname

    def render(self, context):
        obj = resolve_variable(self.obj, context)        
        try:
            context[self.varname] = getattr(obj, 'get_next_by_%s' % self.field)()
        except:
            pass
        return ''

There’s really only two interesting lines of code in here. Tokens received by the tag are received as strings. Well obviously I needed the actual instance object of model that I was working with, so I could use the get_next_by_FOO convenience method that Django automatically adds to each DateField or DateTimeField in my model. Luckily there’s a helper method called resolve_variable that does this heavy lifting for us. It takes the string and the current context and converts it back into its object form. Once I had the actual object all I needed to do was call the get_next_by_FOO method for the specified DateField or DateTimeField and stuff the results into the context.

Using my new tag ends up looking like this in my page template:

{% load taglib %}
{% get_next_object object using pub_date as next_object %}
{% if next_object %}
  <a href="{{ next_object.get_absolute_url }}">Next Question</a>
{% endif %}

The full source for these tags is available here. Enjoy!

Django on Gentoo (DoG) Linux image. This is built on the base Gentoo image that I posted about previously . I added a whole bunch of goodness to run Django. Included are a whole set of startup scripts that build up the system and initialize the application.

AMI ID: ami-29947140
AMI Manifest: eminent-ami/django/10/image.manifest.xml

About this AMI

This image contains a Gentoo Linux 2007.0 installation with Django application support. The following applications / libraries are installed and/or configured:

• Gentoo Linux
  • dhcpcd,
  • logrotate,
  • all packages are recent and updated,
  • AMI tools have been installed and are fully working even for volume imaging,
  • EC2 meta-data is fetched from 169.254.169.254 into /var/spool/ec2/meta-data,
  • EC2 meta-data and user-data include files in /var/spool/ec2/meta-data and /var/spool/ec2/user-data respectively,
  • Disabled password authentication in /etc/ssh/sshd_config
  • Modifications to startup according to Amazon documentation
  • the portage tree has been bind-mounted to /mnt/usr/portage to preserve space on the root file system,
  • the /tmp directory has been bind-mounted to /mnt/tmp to preserve space,
  • the /var/lib/postgresql directory has been bind-mounted to /mnt/var/lib/postgresql,
  • the root home directory contains a dev-copy directory that is needed to build the volume images,
  • all packages have been recompiled to correctly respect the Xen environment.

• Django
  • Apache 2.2.4 installed (not in startup) and configured with Python support,
  • Python 2.5.1,
  • mod_python,
  • Postgresql 8.2.4 (not in startup),
  • psycopg2
  • Subversion 1.4.3 (no repositories are setup)
  • Django trunk version checked out into /home/django/django and installed as normal into the Python site-packages directory (django-admin.py copied to /usr/local/bin/),
  • Support packages: libjpgeg, zlib, freetype2, zip, and unzip,
  • Python packages: PIL 1.1.6, FeedParser 4.1, Markdown 1.6, Unipath 0.1,

• This image contains the following daemons / services:
  • local
  • net.eth0 / net.eth1
  • netmount
  • dhcpcd
  • sshd
  • syslog-ng
  • vixie-cron
  • apache 2 (disabled by default)
  • postgresql (disabled by default)

Sample Application

The instance can accept a set of user data parameters for setting up an application. I’ve provided a default application to indicate the process for creating your own application setup scripts. If you run the instance with the following parameters it will startup a complete running default application based on the Django Vote tutorial.

ami-29947140 -k gsg-keypair -d "app=mysite-setup=http://s3.amazonaws.com/eminent-ami/mysite-setup.tgz"[/code]Once the server is up and running you should be able to to enter the public hostname (starts with ec2) into your browser and view the application. The Admin section of the application is in the location /admin. Credentials are:

Username: admin
Password: password

The default application will be installed into the /home/django/mysite directory. The setup scripts for the application will get installed into /opt/mysite-setup. The Admin media directory and the mysite media directory are both symbolically linked to the /var/www/mysite/htdocs directory.

Postgresql Configuration

Postgresql is installed but there is no database setup and the service is not started by default. The database directory /var/lib/postgresql is bind mounted to /mnt/var/lib/postgresql. The configuration files have been configured according to the Gentoo Postgresql How-To (http://gentoo-wiki.com/HOWTO_Configure_Postgresql) document with the included logging changes. Configuration files are symbolically linked in from /etc/postgresql for those that like that sort of thing. A sample of creating and restoring the database can be found in the mysite-setup scripts for the Sample Application. Write Ahead Logging (WAL) is not implemented, but would be necessary implementation detail for most sites.

Creating an Application Setup Package

The mysite-setup package (http://s3.amazonaws.com/eminent-ami/mysite-setup.tgz) is the base requirements for your own application setup package. At a minimum you are required to have a bin directory with a file in it called setup.sh. Within the setup.sh script you are free to put any setup instructions you need or pass those tasks off to other scripts written in the language of your choice.

Currently the configuration of this image requires that the application setup package is publicly available and downloadable via curl (see the application-setup.sh script). I plan to add security implementation details in the future.

Thanks

Grateful thanks to