Efficient memory usage in Python with Zero-copy

Python has a lot of magic with memory under the hood and, in most cases, Inefficiently.

A simple task that reads a file in chunks is inefficient if you don’t take some things in the count.

Let’s say we have this code:

The reading operation allocates 10 MB, and the second operation, content_to_write, creates a memory copy of the first 1024 elements in memory. Remember the Python magic? Here it is :)

We can use Zero-copy to improve these kinds of tasks.

Zero-copy is a method to copy data from the disk/network to the memory without passing through the CPU. So, it’s speedy and efficient.

In Python, you can use Zero-copy by using memoryview.

Consider the above code using zero-copy:

In this case, the reading operation allocates 10 MB and reuses it because it doesn’t copy data ton content_to_write. Instead, it copies a reference.