Log output to file and system log at the same time

Question

I am trying to diagnose an issue in an app I have written. The issue is a sporadic one, and occurs only under real-world conditions: in the field, away from my PC, and when I’m in the middle of something else, with no resources to spare for immediate debugging. Therefore, my best bet is collecting and analyzing log data.

Unfortunately, by the time I realize the issue has struck again and get around to debugging it, any log data has already rotated out of the Android log as I frequently have other chatty apps running at the same time. Increasing the size of the log buffer has not helped (either Android does not honor it or other apps are still too chatty) so I have abandoned this route.

For this reason, I am now considering having my app log to a separate text file in addition to the regular log.

Now I could easily double every call like

Log.i(TAG, "something happened");

adding another call that writes the same thing to the log file—but that does not seem very elegant to me.

Another option would be to replace all calls to Log with a wrapper that writes the event both to the Android log and the log file.

Question: Does the Android API provide a built-in mechanism for this, i.e. telling Log to write its data to the default log and a text file at the same time? Or do I need to code this by myself?

Edit:

Assumptions:

• I know where in my code I need to generate log output (which can happen anywhere, which may or may not involve an exception) and what I want to be written to the log.
• Getting log data from the device to my PC is also not a concern (one-man show, I just plug my phone into my PC and transfer the log file).

If you know the current Android API has no built-in mechanism to achieve what I want, then ”no, Android does not support this” is a perfectly acceptable answer. In that case the solution is clear—I would fall back to the wrapper function. I am specifically not looking for a different approach to the problem.

Answer 1

You have not specified if some exception are thrown but you don't handle. In case, take a look at this answer: Android Handling Unhandled Exception

If you must look at a bunch of variables and objects, I'd suggest two choices:

• Write a copy of your logs on a file. When your problem occurs, just ask the user to send the file to you. This is ideal during tests with self-aware users.
• Obtain statistics about usage, like commercial software do. Just log user operations and send the data to your server (you would need one for this). This is the most transparent way to do remote logging.

In the case of writing log to a file, you can read and write what you want in internal memory (inside the app's sandbox) or external memory (in this case, write permission is required and explicit permission must have been granted at runtime if you are targeting Android 6 and above).

Answer 2

After doing some more research, it seems the Android API does not provide a standard way to do this. There are two possible workarounds:

Mirror output at the source

• System.out and System.err output, which is written to the console in desktop systems, writes to the log on Android. These two can be redirected into any PrintStream of your choice, which would give you all Java console output. You can subclass PrintStream to duplicate its input, feeding it into the default stream as well as into a file of your choice.
• Create a class which exposes the same methods as android.util.Log. In each method, call through to the respective android.util.Log method and additionally log the data to a file. If you call your class Log (but with a different package name, e.g. org.example.Log), then all you need to do is replace imports of android.util.Log with an import of your class, and any Log method calls will go to your class.

Caveats: This will only give you data explicitly logged by your code (i.e. for which you have the source files), as well as anything that goes to System.out or System.err. It will not include log output from JAR libraries (if you cannot modify their source code), nor any output generated by the system (such as stack traces from default exception handlers) or by other processes (some of which may be system processes and report conditions related to your process).

Read the logs from the command line

This article explains how to read the logs from within Android. In a nutshell:

• Android includes a command line utility called logcat on the device, which will give you a continuous feed of log messages until stopped. (Try it by adb shelling into your device and running it. It has a bunch of command-line options to control its behavior. Not sure if it is present on all distributions, though.)
• Launch this command via Runtime.getRuntime().exec("logcat"), then obtain the input stream of the process returned. This will give you an input stream of log messages.
• According to the article, your app needs the android.permission.READ_LOGS permission to read logs.

I have read statements that certain versions of Android (4.2 was mentioned) do not allow this permission to be granted to non-system apps, though. According to my own tests, behavior without this permissions differ: Anbox will return the full logcat, while LineageOS (tested on 15.1) will only show log entries from the app which called it (including previous instances, presumably everything associated with the same Linux user). This can be a limitation or a welcome filter feature. YMMV.

logcat conveniently has a command line option, -f, to specify an output file. I tried

Runtime.getRuntime().exec("logcat -f " + absolutePathToLogFile);

and logcat keeps logging as long as the app’s process runs. Killing the app (by clicking the X in the title bar on Anbox) apparently also terminated the child process.

Now you can either run this code when your app starts up, or you can turn this functionality into a separate app which starts on boot and continuously collects logs for all apps.

Caveats: This may fill up your storage space quickly if you have some chatty apps running (which is why entries rotate out of the logcat so quickly in the first place). It is recommended to make log mirroring configurable (e.g. via Preferences) and/or ensure old files are deleted regularly. Also, if you keep the logcat process running until your app terminates, you will not be able to access the file over MTP as there is no easy way to run the media scanner (if you scan the file while it is still written to, it will appear truncated over MTP until another media scan runs).