Memory profiling/monitoring (python) on Google App

Question

I've been working with app engine for quite some time, I know that there is appstats but those only show datastore/memcache and other not related to actual memory usage stats.

I've tried to integrate with AppTrace (with all the components latest versions) but I can't continue, since I get this error:

ImportError: dlopen(../apptrace/guppy/sets/setsc.so, 2): Symbol not found: __PyLong_AsScaledDouble
Referenced from: ../apptrace/guppy/sets/setsc.so
Expected in: flat namespace

So my question is: what is the best way (on latest appengine sdk) to profile/monitor memory/catch memory leaks and other python-memory related stuff (either on local or Google server)?

btw, we use Python 2.7 and we're working on Mac OS X (10.7.4)

Answer 1

I think this is best utility

appengine-profiler - Google App Engine profiler in Python - Google Project Hosting -> http://code.google.com/p/appengine-profiler/

08-13 12:40AM 04.586 /camstore/upload 200 508ms 351cpu_ms 293api_cpu_ms 0kb libwww-perl/5.825,gzip(gfe)
11.222.111.222 - - [13/Aug/2010:00:40:05 -0700] "POST /camstore/upload HTTP/1.1" 200 181 - "libwww-perl/5.825,gzip(gfe)"
"example.appspot.com:443" ms=508 cpu_ms=352 api_cpu_ms=293 cpm_usd=0.018512

[I] 08-13 12:40AM 05.021
  Request summary (uptime=161, ID=6C0D1DD1:1.999999999 : Google App Engine/1.3.6 @ na5):
  ms         =  425.66 (api_datastore_v3 =  98%, other =   2%)
  cpu_ms     =  326.67 (api_datastore_v3 =  95%, other =   5%)
  api_cpu_ms =  293.33 (api_datastore_v3 = 100%, other =   0%)

also you can use memcacheApi and other staf

Answer 2

This post is 3 years old, however I thought this answer is still worth sharing to help others. I have premium Google App Engine support and contacted them regarding this same issue.

The Google engineer advised me that the Google App Engine runtime API is deprecated, but still functions. It provides a method called memory_usage.

from google.appengine.api.runtime import runtime
import logging

logging.info(runtime.memory_usage())

This will output memory usage statistics, where the numbers are expressed in MB. For example:

current: 464.0859375
average1m: 464
average10m: 379.575

By placing the logging statement at key points in your code, you can work out which part is causing a memory leak.

Answer 3

For alternatives see Best way to profile/optimize a website on google's appengine.

For fixing this particular issue, this post (although old) http://sourceforge.net/tracker/?func=detail&aid=3047282&group_id=105577&atid=641821 suggest to reinstall/update guppy.

ps: next time can you post the full traceback, and the versions of the relevant libraries

Answer 4

I think there are no tools to monitor memory usage in Google App Engine, you could profile program, monitor module import times, code coverage. So not tools to detect small memory leaks.

Answer 5

Pympler: https://github.com/pympler/pympler Currently have to remove the 'ImportError' portion of the except block at line 1330 in asizeof.py (i.e. so it catches all exceptions) as statvfs isn't usable on GAE:

try:
    from os import statvfs
    _typedef_both(type(statvfs(curdir)), refs=_statvfs_refs,  # statvfs_result
                  item=_sizeof_Cvoidp, leng=_len)
except:  # ImportError: <- Comment out, or add an OSError except as well
    pass

Otherwise, works perfectly:

import logging, traceback

try:
  from pympler.asizeof import asizeof
  for variables in [locals(), globals()]:
    logging.debug(str({k: asizeof(variables[k]) for k in variables})

except Exception as e:
  logging.warning('Could not perform memory check: %s\n%s' % (str(e), str(traceback.format_exc())))