CPython Internals: How Python Optimizes and Caches Small Integers

In Python, everything is an object—including primitive data types like integers. While treating numbers as first-class objects provides tremendous flexibility (such as arbitrary-precision arithmetic), it introduces non-trivial overhead. Creating a new heap-allocated object for every loop index, counter, or status code would lead to massive memory fragmentation and runtime allocation overhead.

To mitigate this, CPython implements a built-in optimization: small integer caching. Python pre-allocates and caches all integers in the range [-5, 256] as global singletons. Whenever your program requests an integer within this boundary, CPython returns a reference to a pre-existing object rather than allocating a new one.

Here is an examination of the CPython source code to see exactly how this optimization is architected and executed.

---

1. Verifying Integer Caching in Python

In Python, the == operator compares equality of values, whereas the is operator compares object identity (verifying whether both references point to the exact same memory address).

>>> a = 256
>>> b = 256
>>> a is b
True

>>> a = 257
>>> b = 257
>>> a is b
False

>>> a = -5
>>> b = -5
>>> a is b
True

>>> a = -6
>>> b = -6
>>> a is b
False

Note: If you run a = 257; b = 257 in a single line or inside a script/module, Python’s constant folding and code block compiler may reuse the same code object constant. However, in an interactive REPL or when computed dynamically across distinct scopes, the identity check clearly demonstrates that 256 resolves to an existing singleton while 257 forces a fresh allocation.

---

2. The Core Mechanism in CPython Source Code

To see how this is achieved under the hood, we dive into CPython’s Objects/longobject.c and Include/cpython/longobject.h.

The Entry Point: PyLong_FromLong

Whenever CPython converts a native C long into a Python integer (PyLongObject), it invokes PyLong_FromLong(long ival):

PyObject *
PyLong_FromLong(long ival)
{
    PyLongObject *v;
    unsigned long abs_ival;
    int sign = 1;

    /* Check if the integer falls in the small integer cache */
    if (IS_SMALL_INT(ival)) {
        return get_small_int((sdigit)ival);
    }

    /* Fallback: allocate a new PyLongObject dynamically */
    ...
}

If the target integer satisfies IS_SMALL_INT(ival), the allocator completely bypasses standard memory allocation and calls get_small_int(), returning a reference to an already allocated singleton.

---

3. Demystifying the Range: [-5, 256]

Why does CPython cache exactly -5 through 256? Let’s check how the boundaries and the check macro are defined:

#define NSMALLPOSINTS           257
#define NSMALLNEGINTS           5

#define IS_SMALL_INT(ival) \n    (-NSMALLNEGINTS <= (ival) && (ival) < NSMALLPOSINTS)

Notice the design choice here:

• NSMALLNEGINTS is defined as a positive constant 5.
• NSMALLPOSINTS is defined as 257.
• The range check evaluates: -5 <= ival && ival < 257.
• Because < 257 is exclusive, the upper bound is 256.
• Because -5 <= ival is inclusive, the lower bound is -5.

This makes the cached domain exactly [-5, 256], representing a total of 262 cached integers ($5 + 1 + 256 = 262$).

Index:     0     1     2     3     4     5     6   ...   261
Value:   [-5]  [-4]  [-3]  [-2]  [-1]   [0]   [1]  ...  [256]

Why is NSMALLNEGINTS Stored as +5?

By storing NSMALLNEGINTS as a positive integer, computing the array offset becomes a single addition operation without sign manipulation or multiplication:

$\text{Index} = \text{ival} + \text{NSMALLNEGINTS}$

• If $\text{ival} = -5 \implies -5 + 5 = 0$
• If $\text{ival} = 0 \implies 0 + 5 = 5$
• If $\text{ival} = 256 \implies 256 + 5 = 261$

Prepending a - sign during the range comparison is trivial at compile time, while keeping the constant positive optimizes runtime array indexing.

---

4. Retrieving from the Cache: get_small_int

The retrieval function fetches the pre-allocated reference from the interpreter’s state:

static inline PyObject *
get_small_int(sdigit ival)
{
    assert(IS_SMALL_INT(ival));
    return _PyLong_GetSmallInt_Internal(ival);
}

PyObject *
_PyLong_GetSmallInt_Internal(sdigit ival)
{
    PyInterpreterState *interp = _PyInterpreterState_GET();
    size_t index = (size_t)(ival + NSMALLNEGINTS);
    
    PyObject *obj = (PyObject *)interp->small_ints[index];
    Py_INCREF(obj);
    return obj;
}

Key steps in the retrieval flow:

1. Index Computation: Maps ival in [-5, 256] directly to [0, 261].
2. Array Access: Accesses interp->small_ints[index], an array of PyLongObject* pointers held directly inside the interpreter state (PyInterpreterState).
3. Reference Count Increment: Calls Py_INCREF(obj). Because Python uses reference counting for memory management, even singletons must have their reference counts tracked correctly.
4. Return: Returns the pointer to the caller, bypassing PyObject_Malloc entirely.

flowchart TD
    A[PyLong_FromLong(ival)] --> B{IS_SMALL_INT(ival)?}
    B -- Yes: -5 <= ival <= 256 --> C[index = ival + NSMALLNEGINTS]
    C --> D[Fetch interp->small_ints[index]]
    D --> E[Py_INCREF(obj)]
    E --> F[Return Singleton Pointer]
    B -- No --> G[Allocate new PyLongObject on Heap]
    G --> H[Initialize Digits & Sign]
    H --> I[Return New Pointer]

---

5. Lifecycle: Initialization and Destruction

Where do these 262 integer objects come from? They are initialized when the Python runtime boots up and cleared when it shuts down.

Initialization: _PyLong_Init

During interpreter initialization, CPython invokes _PyLong_Init:

int
_PyLong_Init(PyInterpreterState *interp)
{
    for (size_t i = 0; i < NSMALLNEGINTS + NSMALLPOSINTS; i++) {
        sdigit ival = (sdigit)i - NSMALLNEGINTS;
        PyLongObject *v = _PyLong_New(1);
        if (v == NULL) {
            return -1;
        }
        Py_SET_REFCNT(v, 1);
        v->ob_digit[0] = (digit)abs(ival);
        /* Set sign and flags based on ival */
        ...
        interp->small_ints[i] = v;
    }
    return 0;
}

The interpreter iterates from 0 to 261, creates permanent PyLongObject instances representing -5 through 256, and stores their pointers in interp->small_ints.

Cleanup: _PyLong_Fini

When the interpreter terminates, _PyLong_Fini is called to prevent memory leaks:

void
_PyLong_Fini(PyInterpreterState *interp)
{
    for (size_t i = 0; i < NSMALLNEGINTS + NSMALLPOSINTS; i++) {
        Py_CLEAR(interp->small_ints[i]);
    }
}

Py_CLEAR decrements the reference count and sets the pointer to NULL, cleanly releasing the pre-allocated pool.

---

6. Universal Usage Across CPython APIs

The small integer optimization is not limited to PyLong_FromLong. Any CPython API function that converts numeric types into Python integers checks the cache:

• PyLong_FromUnsignedLong
• PyLong_FromLongLong
• PyLong_FromUnsignedLongLong
• PyLong_FromSize_t
• PyLong_FromSsize_t

Whenever a numeric conversion takes place inside the CPython runtime, the system first verifies if the resulting value can be served directly from small_ints.

---

7. Architectural Rationale: Why Cache [-5, 256]?

CPython core developers selected this specific range based on statistical empirical analysis of real-world codebases:

1. High Reusability of Small Positive Numbers: Positive integers in [0, 256] appear constantly as collection lengths, loop bounds, array indices, ASCII byte conversions, HTTP status codes, and bitwise flags.
2. Common Negative Offsets: Small negative numbers like -1 and -2 are frequently used in slice indexing (list[-1]), error return codes, and offset arithmetic.
3. Negligible Overhead: Pre-allocating 262 objects consumes only a few kilobytes of RAM. In return, the interpreter avoids millions of dynamic heap allocations and garbage collection cycles over the lifespan of a typical process.

---

Key Takeaways

• Pre-allocated Singletons: CPython allocates an array of 262 PyLongObject singletons covering the range [-5, 256] during interpreter startup.
• Fast Array Lookup: The cache lookup uses ival + NSMALLNEGINTS to perform an $O(1)$ direct array index lookup into interp->small_ints without hash maps or search trees.
• Reference Counted: Even though small integers are singletons, their references are actively tracked via Py_INCREF and Py_DECREF to maintain interpreter consistency.
• Identity Semantics: Because these objects are shared globally, comparing small integers using is evaluates to True, but relying on is for value equality in production code is an anti-pattern.