Method combinations in Python


Django is a popular web development framework for the Python programming language. Each request to the server is handled by a view function, and in the typical case where the response is going to be a web page, the view function makes queries to the database, assembles a context dictionary based on the results, generates an HTML page using a template, and returns the result to the client. (The context dictionary is just a collection of values which the template can refer to.)

When you have lots of view functions there tends to be a lot of common functionality: for example, every view that requires the user to be authenticated has to check the session credentials; and every view that accepts a form submission from the client has to decode and validate the form contents. In Django 1.3 they introduced a system of class-based views to allow each piece of common functionality to be factored into a base class or a mixin, so that you can put the user authentication code into an AuthView class and the form validation code into a FormView class, and then combine these bits and pieces using class inheritance.

Where this becomes a bit of a pain is in the assembly of the context dictionary. Django’s built-in class-based views expect there to be a get_context_data method returning a dictionary. So each class needs to put its own piece of data into the context while co-operating with all the other classes in the hierarchy. So you end up with code like this:

class AuthView(View):
    # ...

    def get_context_data(self, **kwargs):
        c = super(AuthView, self).get_context_data(**kwargs)
        c.update(user = self.get_authenticated_user())
        return c

class FormView(View):
    # ...

    def get_context_data(self, **kwargs):
        c = super(FormView, self).get_context_data(**kwargs)
        c.update(form = self.get_form())
        return c

class AuthFormView(AuthView, FormView):
    # ...

Which involves a rather high boilerplate-to-content ratio, but worse than that: it is also highly error-prone. It’s natural to create a new get_context_data method by copying and pasting an old one and changing the c.update line. But wait! You also have to remember to change the class name in the super() call too.1 If you forget, then if you are lucky you’ll get

TypeError: super(type, obj): obj must be an instance or subtype of type

and if you are unlucky the object in question will happen to be an instance (via inheritance) of the class you forgot to change, and so the super() call succeeds but skips parts of the inheritance hierarchy, leaving you wondering why your HTML pages have mysterious gaps in them.

Common Lisp programmers will recognize this as a problem of method combination. Thinking abstractly, when you call a method on an object you get some kind of combination of the methods of all the classes to which the object belongs. In many object-oriented languages, there is only one way of combining methods, and that is to call the most specific method (the first method found when looking at the classes in method resolution order). This is such a common implementation strategy that it is easy to forget that you might want to have other kinds of combination. And in the particular case of assembling the context dictionary in a Django view, we want the union of the results of calling the methods in all the classes to which the object belongs.

In the Common Lisp object system, method combinations are first-class objects. There are ten built-in method combinations, and you can define your own using define-method-combination. Were we developing our views in Common Lisp, we might find the append method combination useful.

In Python, things are not quite so nice, but classes have a mro method that returns their superclasses, in Python’s standard method resolution order.

>>> AuthFormView.mro()
[<class 'AuthFormView'>, <class 'AuthView'>,
 <class 'FormView'>, <class 'View'>, <type 'object'>]

Using type(obj).mro() we can make a method combination that calls the method in each class to which the object belongs, and takes the union of the results:

class BaseView(View):
    def get_context_data(self, **kwargs):
        c = super(BaseView, self).get_context_data(**kwargs)
        for cls in reversed(type(self).mro()):
            if hasattr(cls, 'extra_context'):
                c.update(getattr(cls, 'extra_context')(self))
        return c

class AuthView(BaseView):
    # ...

    def extra_context(self):
        return dict(user = self.get_authenticated_user())

class FormView(BaseView):
    # ...

    def extra_context(self):
        return dict(form = self.get_form())

We loop over the classes in reverse method resolution order so that subclasses can override context values provided by superclasses, if they need to.

  1.  In Python 3 you can just write super()—with no arguments—and it does the right thing, avoiding the risk of error due to forgetting to change the class name. But at the time of writing, Django only runs on Python 2.