Closures and Higher-Order Functions

Why Closures Matter for Scientific Code

In scientific computing, you frequently need families of functions that differ only in their parameters:

  • A noise generator for different SNR levels
  • A regularizer with varying strength Ξ»\lambda
  • A kernel function parameterized by bandwidth Οƒ\sigma

You could use classes with __call__, but closures are often simpler, more Pythonic, and produce cleaner APIs. Understanding closures is also the key to understanding decorators (Section 2.3).

Definition:

Higher-Order Function

A higher-order function is a function that does at least one of:

  1. Takes a function as an argument (e.g., map, filter, sorted)
  2. Returns a function as its result (e.g., function factories)
# Takes a function as argument
sorted(data, key=lambda x: x.snr_db)

# Returns a function
def make_adder(n):
    def adder(x):
        return x + n
    return adder

add5 = make_adder(5)
add5(10)  # 15

Higher-order functions enable powerful abstractions: callbacks, strategies, and function composition.

Definition:

Closure

A closure is a function that captures and retains access to variables from its enclosing scope, even after that scope has finished executing. The captured variables are called free variables.

def make_scaler(factor):
    # `factor` is a free variable captured by the inner function
    def scale(x):
        return x * factor
    return scale

double = make_scaler(2.0)
triple = make_scaler(3.0)

print(double(5))   # 10.0
print(triple(5))   # 15.0
print(double.__closure__[0].cell_contents)  # 2.0

The inner function scale is a closure because it "closes over" the variable factor from the enclosing make_scaler scope.

Definition:

Function Factory

A function factory is a higher-order function that returns a new function configured by its arguments. This is one of the most common uses of closures in scientific Python:

def make_gaussian_kernel(sigma: float):
    """Create a Gaussian kernel function with fixed bandwidth."""
    coeff = 1 / (sigma * np.sqrt(2 * np.pi))
    denom = 2 * sigma ** 2
    def kernel(x: np.ndarray, center: float = 0.0) -> np.ndarray:
        return coeff * np.exp(-((x - center) ** 2) / denom)
    return kernel

narrow = make_gaussian_kernel(sigma=0.5)
wide = make_gaussian_kernel(sigma=2.0)

The factory pattern avoids re-computing constants (coeff, denom) on every call β€” a form of precomputation via closures.

Theorem: Closure Variable Binding Rule (Late Binding)

Closures in Python capture references to variables, not their values at the time of closure creation. If the captured variable is later modified, the closure sees the updated value. This is called late binding.

Think of the closure as holding a pointer to the variable's memory cell, not a snapshot of its value. This is why loop-variable closures are a classic Python trap.

Theorem: Closure-Class Equivalence

Any closure can be rewritten as a callable class (using __call__), and vice versa. The two representations are semantically equivalent: closures capture free variables in __closure__ cells, while callable classes store them as instance attributes.

A closure is essentially a lightweight class with one method and some state. Use closures when you need a simple function factory; use classes when you need multiple methods or complex state.

Theorem: Map/Filter vs Comprehension Equivalence

For any map(f, iterable) or filter(pred, iterable), there exists an equivalent list comprehension (or generator expression) that produces the same result. The comprehension form is generally preferred in Python for readability.

map and filter are inherited from functional programming traditions. Python's comprehensions achieve the same result with less syntactic overhead.

Common Mistake: The Loop-Variable Closure Trap

Mistake:

Creating closures inside a loop that all capture the same variable:

callbacks = []
for snr in [0, 5, 10, 15, 20]:
    callbacks.append(lambda: run_simulation(snr_db=snr))

# All callbacks use snr=20 (the final loop value)!
for cb in callbacks:
    cb()  # All run with snr_db=20

Correction:

Use a default argument to capture the current value:

callbacks = []
for snr in [0, 5, 10, 15, 20]:
    callbacks.append(lambda snr=snr: run_simulation(snr_db=snr))

# Or use functools.partial:
from functools import partial
callbacks = [partial(run_simulation, snr_db=snr) for snr in [0, 5, 10, 15, 20]]

Example: Closure Factory for Noise Generators

Create a function factory that produces noise generator functions for different noise types and power levels. Each generator should accept a signal shape and return noise samples.

Example: Function Factory for Regularization

Build a function factory that creates regularization penalty functions parameterized by type (L1, L2, ElasticNet) and strength Ξ»\lambda. Use it to sweep over regularization strengths in an experiment.

Example: functools.partial for Scientific APIs

Use functools.partial to create specialized versions of a general simulation function without writing full wrapper functions.

Closure Factory: Parameterized Functions

Explore how closures create families of functions. Adjust the parameters to see how the captured values affect the output function's behavior (e.g., Gaussian kernels with different bandwidths, regularizers with different strengths).

Parameters
1

Closures vs Callable Classes vs functools.partial

FeatureClosureCallable Classfunctools.partial
Syntax overheadLow (nested def)Medium (init + call)Minimal (one line)
State accessVia closure cellsVia self.attrVia .func, .args, .keywords
Multiple methodsNot supportedFull supportNot supported
Serialization (pickle)Usually failsWorks if defined at module levelWorks
IntrospectionLimitedFull (attrs, repr)Good (.func, .keywords)
Use caseSimple factoriesComplex stateful callablesParameter pre-filling
PerformanceFastestSlightly slowerSame as direct call

Closure Factories for Scientific Computing

python
Complete implementations of function factories for noise generators, regularizers, kernel functions, and activation functions.
# Code from: ch02/python/closure_factories.py
# Load from backend supplements endpoint

functools Patterns: partial, reduce, lru_cache

python
Practical examples of functools utilities in scientific workflows: partial for API specialization, reduce for aggregation, lru_cache for memoization.
# Code from: ch02/python/functools_patterns.py
# Load from backend supplements endpoint

Why This Matters: Closures in Research: Hyperparameter Sweeps

In machine learning and signal processing research, function factories are the natural way to parameterize experiments. Instead of passing a dozen parameters through every function call, create specialized functions via closures:

# Create a family of loss functions for hyperparameter sweep
losses = {
    f"l2_alpha={a:.1e}": make_regularizer("l2", alpha=a)
    for a in np.logspace(-4, 2, 20)
}

# Run experiments in parallel
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor() as executor:
    results = {name: executor.submit(train, reg=loss)
               for name, loss in losses.items()}

This pattern appears in PyTorch's learning rate schedulers, scikit-learn's custom scorers, and Optuna's objective functions.

See full treatment in Chapter 6

closure

A function that retains access to variables from its enclosing scope after that scope has finished executing. The captured variables are stored in the function's __closure__ attribute.

Related: free variable

free variable

A variable used inside a function that is not defined in that function's local scope. In closures, free variables are captured from the enclosing scope and stored in closure cells.

Related: closure

Historical Note: Closures: From Scheme to Python

1975-2006

Closures were first implemented in Scheme (1975), a dialect of Lisp designed by Guy Steele and Gerald Sussman. The concept was formalized in their famous "Lambda Papers" (1975-1980). Python gained proper closure support gradually: nested scopes were added in Python 2.1 (PEP 227, 2001), and the nonlocal keyword for mutable closures arrived in Python 3.0 (PEP 3104, 2006). Before nonlocal, Python programmers used the ugly "mutable container" hack (count = [0]) to work around the limitation.

Key Takeaway

Closures create parameterized function families. Use function factories to generate specialized functions (noise generators, regularizers, kernels) that capture their configuration at creation time. Remember that Python closures use late binding β€” use default arguments or functools.partial when creating closures in loops.