equivalent of getbuffer for BytesIO in Python 2

Question

In Python 3, I can get the size of a ByteIO object via object.getbuffer().nbytes (where object = ByteIO()), but what would be the best equivalent for getbuffer() in Python 2? Doing some exploring, I found out I can use len(object.getvalue()) or sys.getsizeof(object), but I don't know if Python 2 will accept them.

Answer 1

After digging in python 2.7 source code I found a simple solution: because io.BytesIO() returns a file descriptor, it has a standard set of functions including tell().

Note that indirect methods such as len(fd.getvalue()) or fd.getbuffer().nbytes copy buffer out and then compute buffer size. In my case, when the buffer holds 1/2 of the memory, this ends up as an application crash :/

Contrary fd.tell() just reports a current position of the descriptor and do not need any memory allocation!

Note that both sys.getsizeof(fd), fd.__sizeof__() do not return correct bufer size.

>>> from io  import BytesIO
>>> from sys import getsizeof
>>> with BytesIO() as fd:              
...  for x in xrange(200):
...   fd.write(" ")
...   print fd.tell(), fd.__sizeof__(), getsizeof(fd)
1 66 98
2 66 98
3 68 100
4 68 100
5 70 102
6 70 102
.....
194 265 297
195 265 297
196 265 297
197 265 297
198 265 297
199 265 297
200 265 297

Answer 2

You can use getvalue()

Example:

from io import BytesIO
if __name__ == "__main__":
    out = BytesIO()
    out.write(b"test\0")
    print len(out.getvalue())

See: https://docs.python.org/2/library/io.html#io.BytesIO.getvalue