Extract notification text from parcelable, content

Question

So I got my AccessibilityService working with the following code:

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    if (event.getEventType() == AccessibilityEvent.TYPE_NOTIFICATION_STATE_CHANGED) {
        List<CharSequence> notificationList = event.getText();
        for (int i = 0; i < notificationList.size(); i++) {
            Toast.makeText(this.getApplicationContext(), notificationList.get(i), 1).show();
        }
    }
}

It works fine for reading out the text displayed when the notifcation was created (1).

The only problem is, I also need the value of (3) which is displayed when the user opens the notification bar. (2) is not important for me, but it would be nice to know how to read it out. As you probably know, all values can be different.

So, how can I read out (3)? I doubt this is impossible, but my notificationList seems to have only one entry (at least only one toast is shown).

Thanks a lot!

/edit: I could extract the notification parcel with

if (!(parcel instanceof Notification)) {
            return;
        }
        final Notification notification = (Notification) parcel;

However, I have no idea how to extract the notifcation's message either from notification or notification.contentView / notification.contentIntent.

Any ideas?

/edit: To clarify what is asked here: How can I read out (3)?

Answer 1

I've wasted a few hours of the last days figuring out a way to do what you (and, me too, by the way) want to do. After looking through the whole source of RemoteViews twice, I figured the only way to accomplish this task is good old, ugly and hacky Java Reflections.

Here it is:

    Notification notification = (Notification) event.getParcelableData();
    RemoteViews views = notification.contentView;
    Class secretClass = views.getClass();

    try {
        Map<Integer, String> text = new HashMap<Integer, String>();

        Field outerFields[] = secretClass.getDeclaredFields();
        for (int i = 0; i < outerFields.length; i++) {
            if (!outerFields[i].getName().equals("mActions")) continue;

            outerFields[i].setAccessible(true);

            ArrayList<Object> actions = (ArrayList<Object>) outerFields[i]
                    .get(views);
            for (Object action : actions) {
                Field innerFields[] = action.getClass().getDeclaredFields();

                Object value = null;
                Integer type = null;
                Integer viewId = null;
                for (Field field : innerFields) {
                    field.setAccessible(true);
                    if (field.getName().equals("value")) {
                        value = field.get(action);
                    } else if (field.getName().equals("type")) {
                        type = field.getInt(action);
                    } else if (field.getName().equals("viewId")) {
                        viewId = field.getInt(action);
                    }
                }

                if (type == 9 || type == 10) {
                    text.put(viewId, value.toString());
                }
            }

            System.out.println("title is: " + text.get(16908310));
            System.out.println("info is: " + text.get(16909082));
            System.out.println("text is: " + text.get(16908358));
        }
    } catch (Exception e) {
        e.printStackTrace();
    }

This code worked fine on a Nexus S with Android 4.0.3. However, I didn't test if it works on other versions of Android. It's very likely that some values, especially the viewId changed. I think there should be ways to support all versions of Android without hard-coding all possible ids, but that's the answer to another question... ;)

PS: The value you're looking for (referring to as "(3)" in your original question) is the "text"-value.

Answer 2

I've spent the last week working with a similar problem and can propose a solution similar to Tom Tache's (using reflection), but might be a little bit easier to understand. The following method will comb a notification for any text present and return that text in an ArrayList if possible.

public static List<String> getText(Notification notification)
{
    // We have to extract the information from the view
    RemoteViews        views = notification.bigContentView;
    if (views == null) views = notification.contentView;
    if (views == null) return null;

    // Use reflection to examine the m_actions member of the given RemoteViews object.
    // It's not pretty, but it works.
    List<String> text = new ArrayList<String>();
    try
    {
        Field field = views.getClass().getDeclaredField("mActions");
        field.setAccessible(true);

        @SuppressWarnings("unchecked")
        ArrayList<Parcelable> actions = (ArrayList<Parcelable>) field.get(views);

        // Find the setText() and setTime() reflection actions
        for (Parcelable p : actions)
        {
            Parcel parcel = Parcel.obtain();
            p.writeToParcel(parcel, 0);
            parcel.setDataPosition(0);

            // The tag tells which type of action it is (2 is ReflectionAction, from the source)
            int tag = parcel.readInt();
            if (tag != 2) continue;

            // View ID
            parcel.readInt();

            String methodName = parcel.readString();
            if (methodName == null) continue;

            // Save strings
            else if (methodName.equals("setText"))
            {
                // Parameter type (10 = Character Sequence)
                parcel.readInt();

                // Store the actual string
                String t = TextUtils.CHAR_SEQUENCE_CREATOR.createFromParcel(parcel).toString().trim();
                text.add(t);
            }

            // Save times. Comment this section out if the notification time isn't important
            else if (methodName.equals("setTime"))
            {
                // Parameter type (5 = Long)
                parcel.readInt();

                String t = new SimpleDateFormat("h:mm a").format(new Date(parcel.readLong()));
                text.add(t);
            }

            parcel.recycle();
        }
    }

    // It's not usually good style to do this, but then again, neither is the use of reflection...
    catch (Exception e)
    {
        Log.e("NotificationClassifier", e.toString());
    }

    return text;
}

Because this probably looks a bit like black magic, let me explain in more detail. We first pull the RemoteViews object from the notification itself. This represents the views within the actual notification. In order to access those views, we either have to inflate the RemoteViews object (which will only work when an activity context is present) or use reflection. Reflection will work in either circumstance and is the method used here.

If you examine the source for RemoteViews here, you will see that one of the private members is an ArrayList of Action objects. This represents what will be done to the views after they are inflated. For example, after the views are created, setText() will be called at some point on each TextView that is a part of the hierarchy to assign the proper Strings. What we do is obtain access to this list of actions and iterate through it. Action is defined as follows:

private abstract static class Action implements Parcelable
{
    ...
}

There are a number of concrete subclasses of Action defined in RemoteViews. The one we're interested in is called ReflectionAction and is defined as follows:

private class ReflectionAction extends Action
{
    String methodName;
    int type;
    Object value;
}

This action is used to assign values to views. A single instance of this class would likely have the values {"setText", 10, "content of textview"}. Therefore, we're only interested in the elements of mActions that are "ReflectionAction" objects and assign text in some way. We can tell a particular "Action" is a "ReflectionAction" by examining the "TAG" field within the Action, which is always the first value to be read from the parcel. TAGs of 2 represent ReflectionAction objects.

After that, we just have to read the values from the parcel according to the order in which they were written (see the source link, if you're curious). We find any string that is set with setText() and save it in the list. (setTime() is also included, in case the notification time is also needed. If not, those lines can be safely deleted.)

While I typically oppose the use of reflection in instances like this, there are times when it is necessary. Unless there is an activity context available, the "standard" method won't work properly, so this is the only option.

Answer 3

To answer your question: This does not seem possible in your case. Below I explain why.

"The main purpose of an accessibility event is to expose enough information for an AccessibilityService to provide meaningful feedback to the user." In cases, such as yours:

an accessibility service may need more contextual information then the one in the event pay-load. In such cases the service can obtain the event source which is an AccessibilityNodeInfo (snapshot of a View state) which can be used for exploring the window content. Note that the privilege for accessing an event's source, thus the window content, has to be explicitly requested. (See AccessibilityEvent)

We can request this privilege explicitly by setting meta data for the service in your android manifest file:

<meta-data android:name="android.accessibilityservice" android:resource="@xml/accessibilityservice" />

Where your xml file could look like:

<?xml version="1.0" encoding="utf-8"?>
 <accessibility-service
     android:accessibilityEventTypes="typeNotificationStateChanged"
     android:canRetrieveWindowContent="true"
 />

We explicitly request the privilige for accessing an event's source (the window content) and we specify (using accessibilityEventTypes) the event types this service would like to receive (in your case only typeNotificationStateChanged). See AccessibilityService for more options which you can set in the xml file.

Normally (see below why not in this case), it should be possible to call event.getSource() and obtain a AccessibilityNodeInfo and traverse through the window content, since "the accessibility event is sent by the topmost view in the view tree".

While, it seems possible to get it working now, further reading in the AccessibilityEvent documentation tells us:

If an accessibility service has not requested to retrieve the window content the event will not contain reference to its source. Also for events of type TYPE_NOTIFICATION_STATE_CHANGED the source is never available.

Apparently, this is because of security purposes...

---

To hook onto how to extract the notifcation's message either from notification or notification.contentView / notification.contentIntent. I do not think you can.

The contentView is a RemoteView and does not provide any getters to obtain information about the notification.

Similarly the contentIntent is a PendingIntent, which does not provide any getters to obtain information about the intent that will be launched when the notification is clicked. (i.e. you cannot obtain the extras from the intent for instance).

Furthermore, since you have not provided any information on why you would like to obtain the description of the notification and what you would like to do with it, I cannot really supply you with a solution to solve this.

Answer 4

If you've tried TomTasche's solution on Android 4.2.2, you'll notice it doesn't work. Expanding on his answer, and user1060919's comment... Here is an example that works for 4.2.2:

Notification notification = (Notification) event.getParcelableData();
RemoteViews views = notification.contentView;
Class secretClass = views.getClass();

try {
    Map<Integer, String> text = new HashMap<Integer, String>();

    Field outerField = secretClass.getDeclaredField("mActions");
    outerField.setAccessible(true);
    ArrayList<Object> actions = (ArrayList<Object>) outerField.get(views);

    for (Object action : actions) {
        Field innerFields[] = action.getClass().getDeclaredFields();
        Field innerFieldsSuper[] = action.getClass().getSuperclass().getDeclaredFields();

        Object value = null;
        Integer type = null;
        Integer viewId = null;
        for (Field field : innerFields) {
            field.setAccessible(true);
            if (field.getName().equals("value")) {
                value = field.get(action);
            } else if (field.getName().equals("type")) {
                type = field.getInt(action);
            }
        }
        for (Field field : innerFieldsSuper) {
            field.setAccessible(true);
            if (field.getName().equals("viewId")) {
                viewId = field.getInt(action);
            }
        }

        if (value != null && type != null && viewId != null && (type == 9 || type == 10)) {
            text.put(viewId, value.toString());
        }
    }

    System.out.println("title is: " + text.get(16908310));
    System.out.println("info is: " + text.get(16909082));
    System.out.println("text is: " + text.get(16908358));
} catch (Exception e) {
    e.printStackTrace();
}

Answer 5

Achep's AcDisplay project has provided a solution, check the code from Extractor.java

Answer 6

Adding on Remi's answer, to identify different notification types and extract data, use the below code.

Resources resources = null;
try {
    PackageManager pkm = getPackageManager();
    resources = pkm.getResourcesForApplication(strPackage);
} catch (Exception ex){
    Log.e(strTag, "Failed to initialize ids: " + ex.getMessage());
}
if (resources == null)
    return;

ICON = resources.getIdentifier("android:id/icon", null, null);
TITLE = resources.getIdentifier("android:id/title", null, null);
BIG_TEXT = resources.getIdentifier("android:id/big_text", null, null);
TEXT = resources.getIdentifier("android:id/text", null, null);
BIG_PIC = resources.getIdentifier("android:id/big_picture", null, null);
EMAIL_0 = resources.getIdentifier("android:id/inbox_text0", null, null);
EMAIL_1 = resources.getIdentifier("android:id/inbox_text1", null, null);
EMAIL_2 = resources.getIdentifier("android:id/inbox_text2", null, null);
EMAIL_3 = resources.getIdentifier("android:id/inbox_text3", null, null);
EMAIL_4 = resources.getIdentifier("android:id/inbox_text4", null, null);
EMAIL_5 = resources.getIdentifier("android:id/inbox_text5", null, null);
EMAIL_6 = resources.getIdentifier("android:id/inbox_text6", null, null);
INBOX_MORE = resources.getIdentifier("android:id/inbox_more", null, null);