CPython API Guide: Solving PyObject_SelfIter Errors 2025

What is PyObject_SelfIter?

In Python, the iterator protocol consists of two methods: `__iter__()` and `__next__()`. An object is considered iterable if it has an `__iter__` method that returns an iterator. An object is an iterator if it has a `__next__` method that returns the next item or raises `StopIteration`.

Crucially, an iterator's `__iter__` method should return itself. This allows an iterator to be used where an iterable is expected, for example, in a `for` loop.

At the C-API level, this is managed by slots in a `PyTypeObject` struct:

• tp_iter: The C equivalent of `__iter__()`.
• tp_iternext: The C equivalent of `__next__()`.

`PyObject_SelfIter` is a simple C function provided by the CPython API. Its only job is to receive a Python object (`PyObject*`), increment its reference count, and return it.

You assign `PyObject_SelfIter` to the `tp_iter` slot of a custom C type that is designed to be an iterator. This correctly implements the rule that an iterator's `__iter__` method returns `self`.

The Anatomy of a `TypeError: '...' object is not iterable`

When you write `for item in my_object:`, Python internally calls `iter(my_object)`. At the C level, this attempts to call the function pointer in the `my_object->ob_type->tp_iter` slot. If this slot is `NULL`, the interpreter has no way to get an iterator, and it immediately raises the `TypeError`.

The error involving `PyObject_SelfIter` is more subtle. It arises from a logical error in your type definition. You might have an object that you think is an iterator, but you've failed to correctly wire up its `tp_iter` and `tp_iternext` slots. The most common mistake is defining a type that should be an iterator but leaving its `tp_iter` slot as `NULL`.

Common Scenarios Leading to PyObject_SelfIter Errors

Let's examine the three most frequent mistakes that lead to iteration-related `TypeError`s in C extensions.

Incorrect Iterator Type Definition

This is the classic case. You have created a custom iterator type but forgot to tell Python that it is its own iterator.

Incorrect Code:

// MyIterator's type definition
static PyTypeObject MyIteratorType = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "my_module.MyIterator",
    .tp_basicsize = sizeof(MyIteratorObject),
    .tp_itemsize = 0,
    .tp_flags = Py_TPFLAGS_DEFAULT,
    .tp_new = PyType_GenericNew,
    .tp_iter = NULL, // <-- THE MISTAKE!
    .tp_iternext = (iternextfunc)myiterator_next, 
};

Here, even though you've implemented the `tp_iternext` logic, the `tp_iter` slot is `NULL`. When Python tries to iterate, it fails before it even gets a chance to call `tp_iternext`.

Missing `tp_iternext` Implementation

Conversely, you might correctly set `tp_iter` but forget `tp_iternext`. An object with a valid `tp_iter` but `NULL` `tp_iternext` is not a valid iterator. While this might not always cause the *exact* same initial `TypeError`, it will fail during iteration.

Incorrect Code:

// MyIterator's type definition
static PyTypeObject MyIteratorType = {
    // ... other fields
    .tp_iter = PyObject_SelfIter,
    .tp_iternext = NULL, // <-- THE MISTAKE!
};

Python expects an iterator to have both methods. The combination of `tp_iter` and `tp_iternext` is what defines an object as an iterator at the C level.

Confusing Containers and Iterators

This is a more advanced conceptual error. Some objects are containers (like a list), and others are iterators (like a list iterator). A container's `tp_iter` should create and return a new iterator object. An iterator's `tp_iter` should return itself (using `PyObject_SelfIter`).

If you have a container type and mistakenly set its `tp_iter` to `PyObject_SelfIter`, you're telling Python that the container itself is the iterator. But if the container doesn't have a `tp_iternext` method, iteration will fail instantly with a `TypeError`.

A Step-by-Step Guide to Fixing the Error

Resolving this `TypeError` is a systematic process of inspection and correction.

Step 1: Pinpoint the Problematic Object

First, ensure you know which object is causing the error. If your Python code is `for item in my_custom_obj:`, then `my_custom_obj` is the starting point. If you are using a debugger like GDB with Python's debugging extensions (`py-bt`, `py-print`), you can inspect the object's type at the point of failure.

Step 2: Inspect the C-Level Type Definition

Go to the C source code for your extension module. Find the `PyTypeObject` struct that defines the type of `my_custom_obj`. Look specifically at two slots:

• tp_iter
• tp_iternext

Ask yourself: Is this type supposed to be a container or an iterator? Your answer determines what these slots should contain.

Step 3: Implement the Correct Iterator Protocol

Based on your type's role, apply the correct fix.

Case A: The Type is an Iterator

If your object is an iterator, it must have both `tp_iter` and `tp_iternext` defined. The fix is to set `tp_iter` to `PyObject_SelfIter`.

Correct Iterator Code:

// Forward declaration for the next function
static PyObject* myiterator_next(MyIteratorObject *self);

// Correct type definition for an iterator
static PyTypeObject MyIteratorType = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "my_module.MyIterator",
    .tp_basicsize = sizeof(MyIteratorObject),
    .tp_flags = Py_TPFLAGS_DEFAULT,
    .tp_new = PyType_GenericNew,
    .tp_iter = PyObject_SelfIter, // <-- CORRECT
    .tp_iternext = (iternextfunc)myiterator_next, // <-- CORRECT
};

Case B: The Type is a Container

If your object is a container, it should not use `PyObject_SelfIter`. Instead, its `tp_iter` slot must point to a function that creates and returns a separate iterator object.

Correct Container Code:

// Function that creates and returns a new iterator
static PyObject* mycontainer_iter(MyContainerObject *self) {
    MyIteratorObject *iterator = PyObject_New(MyIteratorObject, &MyIteratorType);
    if (!iterator) {
        return NULL;
    }
    // Initialize the iterator, perhaps with a reference to the container
    iterator->container = self;
    Py_INCREF(self);
    iterator->index = 0;
    return (PyObject *)iterator;
}

// Correct type definition for a container
static PyTypeObject MyContainerType = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "my_module.MyContainer",
    // ... other fields
    .tp_iter = (getiterfunc)mycontainer_iter, // <-- CORRECT
    .tp_iternext = NULL, // Containers do not have a next method
};

Comparison: Container vs. Iterator C-API Implementation

This table summarizes the key differences in the `PyTypeObject` definition for containers and their corresponding iterators.

Best Practices for CPython Extensions in 2025

As Python evolves, so do the best practices for writing robust C extensions.

• Reference Counting is King: Always be meticulous with `Py_INCREF` and `Py_DECREF`. When `PyObject_SelfIter` returns `self`, it correctly increments the reference count. Your custom `tp_iter` and `tp_iternext` functions must do the same for any objects they return.
• Embrace Modern Tools: For many use cases, writing raw C-API code is no longer necessary. Tools like Cython and pybind11 abstract away much of this boilerplate, including iterator protocol implementation. They generate the correct C code for you from more Python-like syntax, drastically reducing the chance of errors.
• Stay Updated: Keep an eye on the "What's New" documents for recent Python versions (e.g., Python 3.12, 3.13). While the core iterator protocol is stable, related APIs and performance characteristics can change.
• Use Type Flags Correctly: Always start with `Py_TPFLAGS_DEFAULT`. This flag ensures your type participates in garbage collection and has other sensible defaults, which can prevent subtle bugs.