The use case: Python class stores large numpy arrays (large, but small enough that working with them in-memory is a breeze) in a useful structure. Here's a cartoon of the situation:
main class: Environment
; stores useful information pertinent to all balls
"child" class: Ball
; stores information pertinent to this particular ball
Environment
member variable: balls_in_environment
(list of Ball
s)
Ball
member variable: large_numpy_array
(NxN numpy array that is large, but still easy to work with in-memory)
I would like to preferably persist Environment
as whole.
Some options:
pickle
: too slow, and it produces output that takes up a LOT of space on the hard drivedatabase: too much work; I could store the important information in the class in a database (requires me to write functions to take info from the class, and put it into the DB) and later rebuild the class by creating a new instance, and refilling it with data from the DB (requires me to write functions to do the rebuilding)
JSON: I am not very familiar with JSON, but Python has a standard library to deal with it, and it is the recommended solution of this article -- I don't see how JSON would be more compact than
pickle
though; more importantly, doesn't deal nicely withnumpy
MessagePack: another recommended package by the same article mentioned above; however, I have never heard of it, and don't want to strike out into the unknown with what seems to be a standard problem
numpy.save
+ something else: store the numpy arrays associated with eachBall
, usingnumpy.save
functionality, and store the non-numpy stuff separately somehow (tedious)?
What is the best option for my use case?