Iterator

We already know that the following data types can directly be used in a for loop:

1. Collection data types, such as list, tuple, dict, set, str, etc.
2. Generators, including generator expressions and generator functions with yield.

Objects that can be directly used in a for loop are collectively called iterables (Iterable).

You can use isinstance() to check if an object is an Iterable:

>>> from collections.abc import Iterable
>>> isinstance([], Iterable)
True
>>> isinstance({}, Iterable)
True
>>> isinstance('abc', Iterable)
True
>>> isinstance((x for x in range(10)), Iterable)
True
>>> isinstance(100, Iterable)
False

A generator can not only be used in a for loop but can also be continuously called by the next() function to return the next value until a StopIteration error is finally raised, indicating that no more values can be returned.

Objects that can be called by the next() function and continuously return the next value are called iterators (Iterator).

You can use isinstance() to check if an object is an Iterator:

>>> from collections.abc import Iterator
>>> isinstance((x for x in range(10)), Iterator)
True
>>> isinstance([], Iterator)
False
>>> isinstance({}, Iterator)
False
>>> isinstance('abc', Iterator)
False

All generators are Iterator objects, but list, dict, and str, while being Iterable, are not Iterator.

To convert Iterable objects like list, dict, and str into Iterator, you can use the iter() function:

>>> isinstance(iter([]), Iterator)
True
>>> isinstance(iter('abc'), Iterator)
True

You might ask, why aren’t data types like list, dict, and str Iterator?

This is because a Python Iterator object represents a data stream. An Iterator object can be called by the next() function to continuously return the next piece of data until a StopIteration error is raised when there is no more data. This data stream can be thought of as an ordered sequence, but we cannot know the length of the sequence in advance; we can only continuously calculate the next piece of data on demand through the next() function. Therefore, the calculation of an Iterator is lazy evaluation; it only calculates when it needs to return the next piece of data.

An Iterator can even represent an infinitely large data stream, such as all natural numbers. It is impossible to store all natural numbers using a list.

Reference Source Code

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from collections.abc import Iterable, Iterator


def g():
    yield 1
    yield 2
    yield 3


print("Iterable? [1, 2, 3]:", isinstance([1, 2, 3], Iterable))
print("Iterable? 'abc':", isinstance("abc", Iterable))
print("Iterable? 123:", isinstance(123, Iterable))
print("Iterable? g():", isinstance(g(), Iterable))

print("Iterator? [1, 2, 3]:", isinstance([1, 2, 3], Iterator))
print("Iterator? iter([1, 2, 3]):", isinstance(iter([1, 2, 3]), Iterator))
print("Iterator? 'abc':", isinstance("abc", Iterator))
print("Iterator? 123:", isinstance(123, Iterator))
print("Iterator? g():", isinstance(g(), Iterator))

Summary

• All objects that can be used in a for loop are of type Iterable.
• All objects that can be used with the next() function are of type Iterator; they represent a lazily evaluated sequence.
• Collection data types like list, dict, and str are Iterable but not Iterator; however, you can obtain an Iterator object using the iter() function.

The essence of Python’s for loop is to continuously call the next() function. For example:

for x in [1, 2, 3, 4, 5]:
    pass

is actually completely equivalent to:

# First, obtain the Iterator object:
it = iter([1, 2, 3, 4, 5])
# Loop:
while True:
    try:
        # Obtain the next value:
        x = next(it)
    except StopIteration:
        # Exit the loop when StopIteration is encountered
        break