> ## Documentation Index
> Fetch the complete documentation index at: https://python4ai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Advanced Iteration & Decoration

> Combine closures, decorators, iterators, and generators to build powerful, memory-efficient, and monitorable data pipelines.

Closures, decorators, iterators, and generators are among Python's most powerful advanced features. While they are highly effective on their own, **combining them** allows you to build elegant, memory-efficient, and easily monitorable applications.

For example, you can use decorators to profile or rate-limit generator streams, or use closures to maintain state inside custom iteration logic.

***

## Learning Objectives

After completing this lesson, you will be able to:

* Apply decorators to generator functions to monitor, time, or log value generation.
* Understand the execution flow differences when decorating normal functions vs. generator functions.
* Combine multiple generators into data pipelines and manage them using decorators.
* Use closures to configure custom iterators dynamically.

***

## Pattern 1: Decorating Generators

Decorators are commonly used to log, time, or rate-limit functions. However, when you apply a decorator to a generator function, you must be careful about **when** code executes.

### The Generator Decorator Nuance

Recall that calling a generator function does not execute its body; it immediately returns a generator object. If a standard decorator wraps a generator, the wrapper function will run when the generator is created, but **not** when values are yielded.

To intercept the actual value generation, the decorator's wrapper must iterate through the generator and yield the values.

#### Example: Yield Logger

Here is a decorator that logs every time a generator yields a value:

```python theme={null}
def log_yields(func):
    def wrapper(*args, **kwargs):
        generator = func(*args, **kwargs)
        print(f"[Decorator] Generator '{func.__name__}' initialized.")
        
        # We must iterate and yield to preserve lazy evaluation
        for index, value in enumerate(generator, start=1):
            print(f"[Decorator] Yielding item #{index}: {value}")
            yield value
            
    return wrapper

@log_yields
def count_to_three():
    yield "One"
    yield "Two"
    yield "Three"

# Test the decorated generator
gen = count_to_three()
print("--- Starting Iteration ---")
for item in gen:
    pass
```

**Output**

```text theme={null}
[Decorator] Generator 'count_to_three' initialized.
--- Starting Iteration ---
[Decorator] Yielding item #1: One
[Decorator] Yielding item #2: Two
[Decorator] Yielding item #3: Three
```

### Practice

### Exercise 1

Predict the output of the following code:

```python theme={null}
def print_call(func):
    def wrapper(*args, **kwargs):
        print("Decorator Wrapper Called")
        return func(*args, **kwargs)
    return wrapper

@print_call
def my_generator():
    print("Generator Body Started")
    yield 10

g = my_generator()
print("Generator Created")
print(next(g))
```

<Accordion title="Solution">
  ```text theme={null}
  Decorator Wrapper Called
  Generator Created
  Generator Body Started
  10
  ```

  Calling `my_generator()` immediately triggers the wrapper `print_call` which prints `"Decorator Wrapper Called"` and returns the generator object. The code inside `my_generator()` (printing `"Generator Body Started"`) runs only when `next(g)` is called.
</Accordion>

***

## Pattern 2: Stateful Iterators with Closures

Closures can be used to dynamically configure custom iterators without defining a full class structure. By combining a closure with a generator, we can maintain configuration and iteration state cleanly.

### Example: Configurable Fibonacci Generator

```python theme={null}
def make_fibonacci_generator(max_value):
    # Enclosed configuration state
    limit = max_value
    
    def fib():
        a, b = 0, 1
        while a <= limit:
            yield a
            a, b = b, a + b
            
    return fib

# Create two independent generators with different limits
fib_up_to_10 = make_fibonacci_generator(10)()
fib_up_to_50 = make_fibonacci_generator(50)()

print("Fibonacci to 10:", list(fib_up_to_10))
print("Fibonacci to 50:", list(fib_up_to_50))
```

**Output**

```text theme={null}
Fibonacci to 10: [0, 1, 1, 2, 3, 5, 8]
Fibonacci to 50: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
```

### Practice

### Exercise 1

What is the main benefit of using a closure to create configured generators?

<Accordion title="Solution">
  It allows you to enclose configuration values (like `limit` or `multiplier`) in the outer scope, keeping the generator function signature clean and modular.
</Accordion>

***

## Pattern 3: Decorated Data Pipelines

In real-world data science and machine learning applications, generators are often chained together to process large streams of data. Decorators can be used to monitor metrics (like execution time, memory usage, or record count) across the entire pipeline.

### Example: Monitoring a Pipeline

```python theme={null}
import time

def monitor_step(func):
    def wrapper(data_stream):
        start_time = time.time()
        count = 0
        
        # Process the incoming generator stream
        for item in func(data_stream):
            count += 1
            yield item
            
        end_time = time.time()
        duration = end_time - start_time
        print(f"[Pipeline] {func.__name__} processed {count} items in {duration:.4f}s.")
        
    return wrapper

@monitor_step
def extract_numbers(limit):
    for i in range(1, limit + 1):
        yield i

@monitor_step
def square_numbers(numbers):
    for num in numbers:
        yield num * num

# Run pipeline
pipeline = square_numbers(extract_numbers(5))
print("Pipeline initialized. Fetching values...")
print(list(pipeline))
```

**Output**

```text theme={null}
Pipeline initialized. Fetching values...
[Pipeline] extract_numbers processed 5 items in 0.0000s.
[Pipeline] square_numbers processed 5 items in 0.0001s.
[1, 4, 9, 16, 25]
```

***

## Check Your Understanding

**Question 1**

Why does a standard decorator that simply executes a wrapped function not work out-of-the-box for logging yields inside a generator?

<Accordion title="Solution">
  Because a generator function returns a generator object immediately when called instead of running the code block. To log yields, the decorator's wrapper function must actively iterate over the generator object and yield each item.
</Accordion>

**Question 2**

True or False: Using closures to configure generators helps keep local variables separated across multiple instances.

<Accordion title="Solution">
  **True.** Each outer function call creates a new scope (closure), ensuring that configuration states and variables do not leak or interfere with other generator instances.
</Accordion>

**Question 3**

How do decorators help when building modular streaming pipelines using generators?

<Accordion title="Solution">
  They allow you to add auxiliary tasks (like performance profiling, error handling, rate limiting, and item auditing) to each pipeline step cleanly without polluting the core data-transformation logic.
</Accordion>