Dynamic Functions

by Marty Alchin on November 22, 2007

Python supports a neat little trick that I had never heard of before I delved into the Django source. I doubt its usage is really all that rare, but I started Python because of Django, so that was naturally my first exposure to it. In a nutshell, Python allows you to define and call functions using any number of arguments, and it does in a very clear, concise way. Unfortunately, if you’re not familiar, it only looks concise. It isn’t immediately clear just by looking at the code without documentation.

This may seem old hat to most of my readers, but this came up just tonight in IRC with someone new to Python, so I’d like to try my hand at explaining it.

Positional argumenets

In Python-speak, positional arguments are any arguments that aren’t named when you call a function. That is, which variable they go to in the function is defined by their position — first, second, third — instead of their names. This is how most languages work, and is intuitive to most programmers, regardless of their backgrounds.

Some functions, however, may find it useful — or even necessary — to accept any number of arguments. These wouldn’t normally be just default arguments, Python deals with those a different way. Instead, this would be used when a function can deal with any number of objects, which are typically all of the same class. For example, you might write a function that adds numbers together, but you want it to be called as sum(1, 2, 3) or sum(1, 2, 3, 5, 8, 13, 21).

The key here is to define a single argument to accept all the numbers at once, and to tell Python what you’re trying to do. You can instruct Python what’s going on by adding a single asterisk (*) to the beginning of the argument’s name. Consider the following example, which implements the sum function described above.

def sum(*numbers):
    total = 0
    for number in numbers:
        total += number
    return total

There are better ways to do this, of course, but it illistrates the point. Essentially, the function receives a single argument, which is actually a tuple of all the positional arguments that were sent to it. That way, the function can just iterate over it and deal with them accordingly.

Django uses this approach in many places, including when adding objects on the reverse side of ForeignKey relationships.

>>> author.article_set.add(article1, article2, article3)

For reference, most Python code uses the name args for this variable argument, though this is just a convention. You can use any name you like, as demonstrated in the example above, which uses the name numbers.

Keyword arguments

Python also supports keyword arguments. Function definitions are the same, but you can call a function and identify arguments by name instead of by position. This allows them to be out of order, or to skip over some default arguments entirely, supplying just the arguments that are actually important. In order to support variable arguments for these, a different modifier is needed, and it works slightly differently.

The ability to accept variable keyword arguments is denoted by two asterisks before the argument name. Other than that, the definition looks pretty much like the one for positional arguments. The biggest difference inside the function itself is the fact that the variable will be populated with a dictionary instead of a tuple. The keys will be the names provided for the arguments, and the values will be the values given to those names.

def print_(**values):
    for name, value in values.items():
        print "%s: %s" % (name, value)

This is used very often in Django, from populating models to most of the database API. Again, for reference, the name typically assigned to this variable is kwargs, but could be anything you like.

Using them together

These options, of course, can be used together, even alongside traditional arguments, both required and those with defaults. The only real concern here is the order. Required arguments still come first, following by those with defaults. After that, the single-asterisk (positional) argument can be used, if needed, followed by the double-asterisk (keyword) argument, if necessary.

def func(self, verbose=False, *args, **kwargs):

But the good news doesn’t sop there!

Calling functions dynamically

Functions can be called using these same modifiers to arguments, even if the function wasn’t defined to take arguments this way. Just like Python can take the arguments that were given and turn them into tuples and dictionaries, it can take tuples and dictionaries and turn them into standard arguments.

>>> def add(a, b, c):
...     return a + b + c
... 
>>> positionals = (1, 2)
>>> keywords = {'c': 3}
>>> add(*positionals, **keywords)
6