Correct way to use a matrix to zoom an imageview a

Let me start off by saying I've read through a fair amount of zoom questions and recalculating coordinates on here. But I can't seem to apply them to my situation.

I have a custom ImageView class that can zoom and scroll. The problem I am having comes from doing the following in this order:

Zooming in on a point, and using that point as a focal point (this works)
Scrolling while still zoomed in
Trying to zoom either in/out on a different point (doesn't work)

Here is a example of the custom view. There are two circles: red and green. red is where the focal point should be and green is where the focal point is close to where it actually is.

The problem arises when I try to zoom in on the area circled in orange. In the below picture, I tried to zoom in on the orange circled area (not very much, just to show where the focal point is calculated to be). As you can see below, the red circle correctly calculates the new focal point, but for some reason the green circle is actually where it is zooming in/out around.

I am using the following code to map the zoomed coordinates with the actual imageview coordinates

public boolean onScale(ScaleGestureDetector detector) {
        //update the current scale
    scaleFactor *= detector.getScaleFactor();
    scaleFactor = Math.max(ZOOM_LEVEL_4, Math.min(scaleFactor, ZOOM_LEVEL_0));

    //applying the scaleFactor to the focal point as determined in onScaleBegin
    transformMatrix.setScale(scaleFactor, scaleFactor, 
                  newFocalPoints[0], newFocalPoints[1]);

    //apply the matrix to the child
    child.transform(transformMatrix, newFocalPoints[0],  newFocalPoints[1],
                oldFocalPoints[0], oldFocalPoints[1]);
    return true;
}

@Override
public boolean onScaleBegin(ScaleGestureDetector detector){
    //when a new scaling process begins, get the current imageMatrix and 
    //map the points to account for the current zoom level

    //the initial points. based on screen location and current scroll pos
    float startX = detector.getFocusX() + getScrollX();
    float startY = detector.getFocusY() + getScrollY();
    oldFocalPoints = new float[]{startX, startY};

    //map oldFocalPoints to coordinates of the imageView based on current zoom
    Matrix inverseTransformMatrix = new Matrix();

    if(transformMatrix.invert(inverseTransformMatrix))
        inverseTransformMatrix.mapPoints(newFocalPoints, oldFocalPoints);

    return true;
}

The green dot in the above pictures is set to be at {oldCoordinates[0], oldCoordinates[1]} for debug purposes, and while it isn't exactly the focal point, it is pretty darn close. So it seems that although I appear to be calculating the new focal point correctly (red circle), it doesn't seem to be applied correctly. Can anyone spot something wrong? Thanks in advance!

After much pain, I have discovered the solution. Here is the code:

@Override
public boolean onScale(ScaleGestureDetector detector) {

    scaleFactor *= detector.getScaleFactor();
    scaleFactor = Math.max(ZOOM_4, Math.min(scaleFactor, ZOOM_LEVEL_0));

    float xDiff = initialFocalPoints[0] - currentFocalPoints[0];
    float yDiff = initialFocalPoints[1] - currentFocalPoints[1];

    transformMatrix.setScale(scaleFactor, scaleFactor, 
                                 currentFocalPoints[0], currentFocalPoints[1]);
    transformMatrix.postTranslate(xDiff, yDiff);    
    child.setImageMatrix(transformMatrix);

    return true;
}

@Override
public boolean onScaleBegin(ScaleGestureDetector detector){

    float startX = detector.getFocusX() + getScrollX();
    float startY = detector.getFocusY() + getScrollY();

    initialFocalPoints = new float[]{startX, startY};

    if(transformMatrix.invert(inverseTransformMatrix))
    inverseTransformMatrix.mapPoints(currentFocalPoints, initialFocalPoints);
    return true;
}

The lines that made the difference were the following:

float xDiff = initialFocalPoints[0] - currentFocalPoints[0];
float yDiff = initialFocalPoints[1] - currentFocalPoints[1];
transformMatrix.postTranslate(xDiff, yDiff);

The answer was as simple as figuring out the difference between the two points and translating the imageview everytime the image is scaled.