I am running a django app that includes matplotlib and allows the user to specify the axes of the graph. This can result in 'Overflow Error: Agg complexity exceeded'
When that happens up to 100MB of RAM get tied up. Normally I free that memory up using fig.gcf()
, plot.close()
, and gc.collect()
, but the memory associated with the error does not seem to be associated with the plot object.
Does anyone know how I can release that memory?
Thanks.
Here is some code that gives me the Agg Complexity Error.
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
import gc
a = np.arange(1000000)
b = np.random.randn(1000000)
fig = plt.figure(num=1, dpi=100, facecolor='w', edgecolor='w')
fig.set_size_inches(10,7)
ax = fig.add_subplot(111)
ax.plot(a, b)
fig.savefig('yourdesktop/random.png') # code gives me an error here
fig.clf() # normally I use these lines to release the memory
plt.close()
del a, b
gc.collect()
I assume you can run the code you posted at least once. The problem only manifests itself after running the posted code many times. Correct?
If so, the following avoids the problem without really identifying the source of the problem.
Maybe that is a bad thing, but this works in a pinch: Simply use multiprocessing
to run the memory-intensive code in a separate process. You don't have to worry about fig.clf()
or plt.close()
or del a,b
or gc.collect()
. All memory is freed when the process ends.
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
import multiprocessing as mp
def worker():
N=1000000
a = np.arange(N)
b = np.random.randn(N)
fig = plt.figure(num=1, dpi=100, facecolor='w', edgecolor='w')
fig.set_size_inches(10,7)
ax = fig.add_subplot(111)
ax.plot(a, b)
fig.savefig('/tmp/random.png') # code gives me an error here
if __name__=='__main__':
proc=mp.Process(target=worker)
proc.daemon=True
proc.start()
proc.join()
You don't have to proc.join()
either. The join
will block the main process until the worker
completes. If you omit the join
, then the main process simply continues with the worker
process working in the background.
I find here
http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg11809.html
, it gives an interesting answer that may help
try replacing :
import matplotlib.pyplot as plt
fig = plt.figure()
with
from matplotlib import figure
fig = figure.Figure()