In addition to accepting functions as parameters, higher-order functions can also return functions as result values.
Let’s implement a function to calculate the sum of variable parameters. Normally, such a summation function is defined like this:
def calc_sum(*args):
ax = 0
for n in args:
ax = ax + n
return ax
But what if we don’t need to calculate the sum immediately, but instead want to compute it later in the code when needed? We can return not the sum result, but the function that performs the summation:
def lazy_sum(*args):
def sum():
ax = 0
for n in args:
ax = ax + n
return ax
return sum
When we call lazy_sum(), it does not return the sum result, but the summation function itself:
>>> f = lazy_sum(1, 3, 5, 7, 9)
>>> f
<function lazy_sum.<locals>.sum at 0x101c6ed90>
The actual summation is performed only when we call the function f:
>>> f()
25
In this example, we defined a function sum inside lazy_sum. The inner function sum can reference the parameters and local variables of the outer function lazy_sum. When lazy_sum returns the function sum, the relevant parameters and variables are preserved within the returned function. This kind of program structure is called a Closure and possesses immense power.
Please note another point: each time we call lazy_sum(), it returns a new function object, even if the same arguments are passed:
>>> f1 = lazy_sum(1, 3, 5, 7, 9)
>>> f2 = lazy_sum(1, 3, 5, 7, 9)
>>> f1 == f2
False
The calls to f1() and f2() do not interfere with each other.
Notice that the returned function references the local variable args within its definition. Therefore, when a function returns another function, its internal local variables remain referenced by the new function. This makes closures easy to use but not so easy to implement correctly.
Another important point is that the returned function is not executed immediately; it is only executed when f() is called. Let’s look at an example:
def count():
fs = []
for i in range(1, 4):
def f():
return i * i
fs.append(f)
return fs
f1, f2, f3 = count()
In the example above, a new function is created in each loop iteration, and then these three functions are returned.
You might expect calling f1(), f2(), and f3() to produce 1, 4, 9 respectively, but the actual results are:
>>> f1()
9
>>> f2()
9
>>> f3()
9
All return 9! The reason is that the returned functions reference the variable i, but they do not execute immediately. By the time all three functions are returned, the variable i they reference has already become 3. Hence, the final result is 9 for all.
Note:
A crucial rule to remember when returning a closure is: the returned function should not reference any variables that might change in the future.
To fix this, we can create another function and use its parameters to bind the current value of the loop variable. This way, regardless of how the loop variable changes later, the value bound to the function parameter remains unchanged:
def count():
def f(j):
def g():
return j * j
return g
fs = []
for i in range(1, 4):
fs.append(f(i)) # f(i) is executed immediately, binding the current value of i
return fs
Let’s check the results again:
>>> f1, f2, f3 = count()
>>> f1()
1
>>> f2()
4
>>> f3()
9
A disadvantage is the longer code length, which can be shortened using a lambda function.
nonlocalUsing a closure means the inner function references local variables of the outer function. If we only read the value of the outer variable, we find that calling the returned closure function works perfectly:
def inc():
x = 0
def fn():
# Only reads the value of x:
return x + 1
return fn
f = inc()
print(f()) # 1
print(f()) # 1
However, if we try to assign a value to the outer variable, the Python interpreter will treat x as a local variable of the function fn(), and it will throw an error:
def inc():
x = 0
def fn():
nonlocal x # Declare x as nonlocal
x = x + 1
return x
return fn
f = inc()
print(f()) # 1
print(f()) # 2
The error occurs because x, treated as a local variable, hasn’t been initialized before x + 1 is calculated. What we actually want is to reference the x variable inside the inc() function. Therefore, we need to add a nonlocal x declaration inside the fn() function. With this declaration, the interpreter recognizes x in fn() as the local variable from the outer function, which has already been initialized, allowing x + 1 to be computed correctly.
Tip:
When using closures, before assigning a value to an outer variable, you need to declare it as non-local using the nonlocal keyword.
Use a closure to return a counter function that returns an incrementing integer each time it is called:
def createCounter():
x = 0
def counter():
nonlocal x
x = x + 1
return x
return counter
# Test:
counterA = createCounter()
print(counterA(), counterA(), counterA(), counterA(), counterA()) # 1 2 3 4 5
counterB = createCounter()
if [counterB(), counterB(), counterB(), counterB()] == [1, 2, 3, 4]:
print('Test passed!')
else:
print('Test failed!')
return_func.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
def lazy_sum(*args):
def sum():
ax = 0
for n in args:
ax = ax + n
return ax
return sum
f = lazy_sum(1, 2, 4, 5, 7, 8, 9)
print(f)
print(f())
# why f1(), f2(), f3() returns 9, 9, 9 rather than 1, 4, 9?
def count():
fs = []
for i in range(1, 4):
def f():
return i * i
fs.append(f)
return fs
f1, f2, f3 = count()
print(f1())
print(f2())
print(f3())
# fix:
def count():
fs = []
def f(n):
def j():
return n * n
return j
for i in range(1, 4):
fs.append(f(i))
return fs
f1, f2, f3 = count()
print(f1())
print(f2())
print(f3())
A function can return a computed result or it can return another function.
When returning a function, always remember that the function has not been executed yet. Avoid referencing any variables in the returned function that might change in the future.