HttpWebRequest.GetResponse throws WebException on

Question

When a web server responds to HttpWebRequest.GetResponse() with HTTP 304 (Not Modified), GetResponse() thows a WebException, which is so very weird to me. Is this by design or am I missing something obvious here?

Answer 1

This is really a frustrating problem, and can be alternatively worked around by using the following extension method class and calling request.BetterGetResponse()

//-----------------------------------------------------------------------
//
//     Copyright (c) 2011 Garrett Serack. All rights reserved.
//
//
//     The software is licensed under the Apache 2.0 License (the "License")
//     You may not use the software except in compliance with the License.
//
//-----------------------------------------------------------------------

namespace CoApp.Toolkit.Extensions {
    using System;
    using System.Net;

    public static class WebRequestExtensions {
        public static WebResponse BetterEndGetResponse(this WebRequest request, IAsyncResult asyncResult) {
            try {
                return request.EndGetResponse(asyncResult);
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }

        public static WebResponse BetterGetResponse(this WebRequest request) {
            try {
                return request.GetResponse();
            }
            catch (WebException wex) {
                if( wex.Response != null ) {
                    return wex.Response;
                }
                throw;
            }
        }
    }
}

You read more about it in my blog post on this subject at http://fearthecowboy.com/2011/09/02/fixing-webrequests-desire-to-throw-exceptions-instead-of-returning-status/

Answer 2

I also came across to this issue with code:

try
{
    ...
    var webResponse = req.GetResponse();
    ...
}
catch (WebException ex)
{
    Log.Error("Unknown error occured", ex);
    //throw; 
}

And it appears that if Remote Server returns 304 status it must be passed to Browser by throwing this error or returning custom 304 so the Browser could return cached response. Otherwise you will probably get empty Response from the Remote Server.

So in my case for normal behaviour with correct Cache handling it should be like:

try
{
    ...
    var webResponse = req.GetResponse();
    ...
}
catch (WebException ex)
{
    if (((HttpWebResponse)ex.Response).StatusCode == HttpStatusCode.NotModified)
        throw;
    Log.Error("Unknown error occured", ex);
}

Answer 3

Just as an FYI, this is an update to Anton Gogolev's answer that uses the C#6 (VS2015) when clause. It's a little less annoying when using a debugger as it removes one catchpoint:

public static HttpWebResponse GetHttpResponse(this HttpWebRequest request)
{
    try
    {
        return (HttpWebResponse) request.GetResponse();
    }
    catch (WebException ex)
        when (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
    {
        return (HttpWebResponse) ex.Response;
    }
}

Answer 4

Ok, this seems to be a by-design behavior and a perfect example of a vexing exception. This can be solved with this:

public static HttpWebResponse GetHttpResponse(this HttpWebRequest request)
{
    try
    {
        return (HttpWebResponse) request.GetResponse();
    }
    catch (WebException ex)
    {
        if(ex.Response == null || ex.Status != WebExceptionStatus.ProtocolError)
            throw; 

        return (HttpWebResponse)ex.Response;
    }
}

Answer 5

The way to avoid this System.WebException is to set AllowAutoRedirect property to false. This disables the automatic redirection logic of the WebRequest. It seems to be broken for 304 redirection requests, as it is not a real redirection in the strictest sense. Of course that means that the other redirection requests 3xx have to be handled manually.