profile

Rodrigo Girão Serrão

🐍📝 partial function application

published2 months ago
4 min read

Hey there, how are you doing?

In this issue of the Mathspp Insider we will talk about functools.partial and the advantages of being able to apply a function partially.


The motivating example

Last week we tackled the problem of finding the longest word in a list of words.
We covered many different solutions, and I used those examples and the code you sent me to give a talk at PyCon Portugal 2022:

video preview

The final solution we arrived at, that I emailed you last week, was this one:

def get_longest(words):
    return max(words, key=len)

I convinced you this was a great solution because it solves our problem with an idiomatic use of the appropriate built-ins in Python,
max and len, via max's keyword argument, key.

Today, I want to show you an alternative formulation of this solution, which I think is even more elegant.

Partial function application

Looking at the implementation of get_longest above, you will see that the function has a single line, the call to max.
What is more, the call to max has a part that is fixed: max(..., key=len).
The only thing that changes is the ... that is replaced by the list of words that is passed in as an argument, words.

In a sense, the function get_longest provides a blueprint: max(..., key=len).
Then, we just fill in the remaining parts of the blueprint with the list of words.

Another way to talk about this blueprint is by saying that the keyword argument key is set to len and that that part of the function never changes.
The only thing that changes is the first argument of max.
Now, maybe you won't believe me, but this is a very common pattern!

It is common for us to be interested in changing one argument of a given function and, at the same time, leave the arguments of the other functions fixed.
Let me show you some examples.

Examples of partial function application

Let me show you a couple of meaningful functions that are implemented all in the same way:

  1. we pick a Python function (in these examples, I will stick with built-in functions);
  2. we define a wrapper function around it;
  3. the wrapper function sets one or more arguments to fixed values; and
  4. the wrapper function takes another argument that is placed in a specific position of the built-in.

Get the longest word

The example we have already seen takes an iterable of words (like a list of words) and returns the longest word:

def get_longest(words):
    return max(words, key=len)

In this example:

  1. we picked the built-in max;
  2. we defined the wrapper get_longest around it;
  3. the wrapper set key=len; and
  4. the wrapper function takes the argument words and puts it in the first argument of max.

In this example, we took max and derived a new function by setting key to the built-in function len.

Convert from binary

This example takes a string representation of a binary number and converts it to a decimal number:

def from_binary(bin_string):
    return int(bin_string, base=2)

print(from_binary("11011011"))  # 219

In this example:

  1. we picked the built-in int;
  2. we defined the wrapper from_binary around it;
  3. the wrapper set base=2; and
  4. the wrapper function takes the argument bin_string and puts it in the first argument of int.

In this example, we took int and derived a new function by setting base to the integer 2.

Here is just one more example:

String representation of long lists

This example takes a list and builds a string representation, with one element per line (useful for long lists):

def newline_join(items):
    return "\n".join(items)

print(newline_join(["Rodrigo", "@mathsppblog", "rodrigo@mathspp.com"]))
"""
Rodrigo
@mathsppblog
rodrigo@mathspp.com
"""

In this example:

  1. we picked the built-in join;
  2. we defined the wrapper newline_join around it;
  3. the wrapper set the first argument to "\n"; and
  4. the wrapper function takes the argument items and puts it in the first argument of "\n".join.

Now, this might look structurally different from the two other examples, but we can actually rewrite this example to make it much more similar to the other two:

def newline_join(items):
    return str.join("\n", items)

This alternate implementation is equivalent, and it plays with how methods are resolved in Python.
After all, "\n".join(...) is essentially the same as str.join("\n", ...).

So, in this example, we took str.join and derived a new function by setting its first argument to a newline "\n".

Having fixed arguments is useful

As you can see, having fixed arguments to functions is useful, and lets us interpret those functions in a completely new light:

  • the built-in max became a function that finds the longest word;
  • the built-in int became a function that converts binary strings into numbers; and
  • the built-in str.join became a function that creates newline-separated representations of lists.

If you come up with any other examples of how fixing arguments to functions is useful, feel free to reply to this email and let me know of those examples!
I'll be waiting!

functools.partial

functools.partial is a built-in tool that was created to help you exploit these patterns.

By using functools.partial, you can derive new functions by doing partial function application of other functions you already have.
“Partial application” refers to the act of setting one or more parameters of a function to some fixed values.

functools.partial can be used like so: you start by giving it the function you want to do partial application in, then you provide positional arguments you want to set, and then you give it the keyword arguments you want to set.

For example, in the case of the function get_longest, it goes from

def get_longest(words):
    return max(words, key=len)

to this:

from functools import partial
get_longest = partial(max, key=len)

Notice how we got rid of the keyword def that usually defines new functions.
With this approach, we define the function get_longest in a more functional way, which is helpful because it highlights the pattern we are exploiting: the built-in max is the star of this implementation, together with the fact that the parameter key=len is fixed.

Now I have a challenge for you!
With the help of the documentation for functools.partial, reimplement from_binary and newline_join with functools.partial.
If you need any help, you can also reply to this email with your questions!


Thanks for reading, and I'll see you next time!

Rodrigo.