Where does GoogleAppEngineLauncher keep the local

Question

GoogleAppEngineLauncher can display the local log file of my app while it is running on my Mac during development. However, I can't change the font size there so I would like to use the tail command to watch the log file myself.

It's a shame but I can't find the log files. They are not under /var/log/, ~/Library/Logs or /Library/Logs. Do you know where they are?

(Maybe there are no physical files, just the stdout of the python development environment and so the log is only available in the launcher application.)

Answer 1

If you just want to see the logs at runtime, they're printed on the command-line along with the HTTP calls.

logging.debug() and logging.error() are not printed, but calls info upwards are.

See this answer for a little more detail.

Answer 2

As you surmise, and can confirm by studying the source file /usr/local/google_appengine/google/appengine/tools/dev_appserver.py, the logs are not being written to disk (a cStringIO.StringIO instance is used to keep them in memory, as the rest of the code is oriented to writing them "to a file-like object").

What I would recommend is writing your own app server script, which imports dev_appserver, subclasses dev_appserver.ApplicationLoggingHandler, and overrides just one method:

from google.appengine.tools import dev_appserver

class MyHandler(dev_appserver.ApplicationLoggingHandler):

    def __init__(self, *a, **k):
        dev_appserver.ApplicationLoggingHandler.__init__(self, *a, **k)
        self.thefile = open('/tmp/mylog.txt', 'w')

    def emit(self, record):
        dev_appserver.ApplicationLoggingHandler(self, record)
        self.thefile.write(str(record) + '\n')
        self.thefile.flush()

You also need to ensure this class is used instead of the standard one, e.g. by subclassing the dispatcher or ensuring you use its dependency-injection capability. (dev_appserver_main.py lets you control this better, I think).

I think this customization approach is far more cumbersome than it should be (it's perfectly normal to want the logs written to file, after all -- either to display them differently, as you desire, or to process them later with some auxiliary script), and so I'd also recommend putting a feature request on app engine's tracker: dev_appserver.py should accept one more flag, which, if specified, gives the path on which to write logs to disk.

And, to be honest, if I needed this feature right now, myself, I'd do it the dirty way: editing that .py file (and its related _main.py) to add said flag and its use. That should be a dozen lines in all, much easier than the "canonical" way I just outlined. Of course, it is dirty because every time there's a new SDK you'll have to apply the patch again, and again, and again... which is why one should also propose the patch on GAE's tracker, as part of the feature request I suggested, hoping it gets accepted soon!-)

Answer 3

A simple, and also dirty, fix is to add the following code into your dev_appserver.py file, towards the top:

import logging
logging.basicConfig( filename='/tmp/gae.log', filemode='a' )

Obviously, change your logfile to what you want. This requires the least amount of change, and is the easiest to commit into your repo and diff when there's a new release.

Update: Slightly better might be to make it into a command-line option:

def start_logging():
  import logging

  logfile=''
  for item in sys.argv[1:]:
    if re.match('--log_file=', item):
      logfile=item.split('=')[1]
      # Remove this item from sys.argv
      sys.argv.remove(item)
      break

  if logfile:
    print "Please monitor the log file (with tail -f %s)" % logfile
    logging.basicConfig( filename=logfile, filemode='a' ) 

Answer 4

Many of these answers are now outdated. :)

In today's devappserver, use --logs_path=LOGS_FILE if you want to log to a file (in its native sqlite database format). Or as suggested in a comment, simply pipe the output if that's too complicated.

Since there's a log API, it actually now stores log entries in a file in --storage_path if not set. I have noticed a few bugs myself, though. (I'll assume they don't exist now, it's been a while since I used this.)