Accessing html response content in Xamarin browser

I have what seems to be an unusual requirement that my colleagues and I haven't been able to implement in our Xamarin project. What we are trying to do is dynamically process the contents of any page that a browser navigates to rather than simply the content returned from the initial URL it accesses.

I have seen solutions for accessing the content returned this way:

Xamarin: How to get HTML from page in WebView?

However this code will only run when the browser receives the first html response from the initial URL it is pointed to. We need to dynamically process any content the browser receives even if it is redirected to a different URL or if the user clicks on links in the content. We have implemented this in our .NET native Windows clients using the WebBrowser.DocumentCompleted event, which is raised whenever the control loads content. We have not found any event raised or method called when the Xamarin WebViewRendered loads content except on the first load, so we have not been able to get the content of subsequent resources.

To summarize: how can I dynamically get the content of any URL that a web browser navigates to? The answer can include anything accessible in a .NET Xamarin project deployable to Android and iOS including third party open source code.

Both these device-dependent techniques use JavaScript evaluation/execution after the HTML page has finished loading in order for to receive the entire html contents as a JavaScript result (edit the JavaScript in the examples if your requirements to not require the entire page to be return).

Xamarin.Android

You can capture the loaded html within a WebView by assigning a subclassed WebViewClient that implements IValueCallback. The OnReceiveValue method will contain the html content after the page has finished loading:

Example `WebViewClient` Class:

public class EmbeddedWebViewClient : WebViewClient, IValueCallback
{
    public EmbeddedWebViewClient(WebView view)
    {
        view.Settings.JavaScriptEnabled = true;
    }

    public override void OnPageFinished(WebView view, string url)
    {
        base.OnPageFinished(view, url);
        Log.Info("SO", $"Page loaded from: {url}");
        view.EvaluateJavascript("(function() { return ('<html>'+document.getElementsByTagName('html')[0].innerHTML+'</html>'); })();", this);
    }

    public void OnReceiveValue(Java.Lang.Object value)
    {
        // "value" holds the html contents of the loaded page
        Log.Debug("SO", ((string)value).Substring(0, 40));
    }
}

Example WebViewClient usage:

WebView webview = FindViewById<WebView>(Resource.Id.webview);
webview.SetWebViewClient(new EmbeddedWebViewClient(webview));
webview.LoadUrl("https://xamarin.com");

Xamarin.iOS

You can capture the loaded html within a WKWebView by assigning a IWKNavigationDelegate to it. Either implement it within its own class or the controller that contains the WKWebView

Example `IWKNavigationDelegate` Implementation:

public class NavigationDelegate : NSObject, IWKNavigationDelegate
{
    [Export("webView:didFinishNavigation:")]
    public async void DidFinishNavigation(WKWebView webView, WKNavigation navigation)
    {
        var content = await webView.EvaluateJavaScriptAsync("(function() { return ('<html>'+document.getElementsByTagName('html')[0].innerHTML+'</html>'); })();");
        var html = FromObject(content);
        Console.WriteLine((html.ToString()).Substring(0, 40));
    }
}

Example WKWebView usage:

wkwebview = new WKWebView(UIScreen.MainScreen.Bounds, new WKWebViewConfiguration());
wkwebview.NavigationDelegate = new NavigationDelegate();
Add(wkwebview);
wkwebview.LoadRequest(new NSUrlRequest(new Uri("https://xamarin.com")));

Note: This same basic logic can be done with a UIWebView, but WKWebView is so much faster that if you do not need to support iOS 7 clients, I would personally focus on using WKWebView