Address of a buffer in Python

Python has the Buffer Protocol. Objects that support this protocol allow for memory-sharing across Python objects. This can be very useful to speed up processing as it can eliminate unnecessary copying of data. Unfortunately, it’s also a potentially fantastic source of super-difficult to resolve bugs.

One thing Python does if you’re not very careful is to copy the underlying data of an object. For example, if you request a slice of a memoryview, no copy is made. If however it’s a bytes or bytearray, a copy is indeed made on slicing.

How can you check that you haven’t unintentionally copied the data you were trying not to copy? In C, you could compare the starting address of the two blocks of memory. The Buffer Protocol includes a struct Py_buffer, whose member buf contains a pointer to the beginning of the block of memory. Unfortunately, although Python does have this Buffer Protocol, it’s only the C-side of Python that knows anything about this; you can’t detect from within Python that an object supports the protocol.

However, you can use the ctypes module to advantage. First create a ctypes array class that is of the same size as the original array. Then, use the class method from_buffer() to have the instance reference the original buffer’s memory. Next, use the ctypes function addressof() to obtain the address of the original buffer.

>>> # Create a writable array in memory
>>> x = bytearray(b'\x01\x02\x03')

>>> # Create a ctypes shadow object for the same memory
>>> import ctypes
>>> Buffer = ctypes.c_char * len(x)
>>> buf = Buffer.from_buffer(x)

>>> # Get address of the shadow object
>>> ctypes.addressof(buf)
4449460176

>>> # Obtain a memoryview of the buffer
>>> m = memoryview(x)

>>> # Create another shadow copy
>>> buf2 = Buffer.from_buffer(m)

>>> # Get address of second shadow object
>>> # (note the addresses are identical)
>>> ctypes.addressof(buf2)
4449460176