Correct response to HTTP HEAD Request on HTTPS onl

Question

We have an ASP.Net MVC3 site only accessible over HTTPS, by using the RequireHTTPS attribute on the controller.

We are receiving numerous HTTP HEAD method requests, mainly from what appear to be Twitter bots. The default ASP.Net/MVC3 response is a '500 Internal Server Error', and are being caught/logged by elmah and log4net (now filtered out!).

I could write a specific controller and route to handle these non-HTTPS requests as per this question - Responding to HEAD Request in asp.NET MVC 3.

But, from the bots perspective what would be the best response? 200 to show the server is alive, a 302 redirect to the HTTPS url, or stick with the 500 as the site isn't accessible over HTTP?

Answer 1

You could respond with

405 Method Not Allowed

which means

The method specified in the Request-Line is not allowed for the resource identified by the Request-URI. The response MUST include an Allow header containing a list of valid methods for the requested resource.

or with

501 Not Implemented

which means

The server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.

Personally, I would go with the 405 since it's an error on the client side, a "Hey man, we don't serve that stuff here." seems more appropriate to me than "What the hell are you talking about? I don't understand it." one, the latter is suggested by the the server does not recognize the request method bit of the 501 description.

All the HTTP status codes: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Answer 2

In my case, I was only getting HEAD requests on the root of the site / which seems like bots probing. So, I was a bit worried about returning a 500 or 404.

More on 405

405 may be OK as per Albireo's answer, but you need to return the accepted verbs, something like:

// 405 must include allowable methods.
// https://tools.ietf.org/html/rfc2616#section-14.7
httpContext.Response.StatusCode = (int)HttpStatusCode.MethodNotAllowed;
httpContext.Response.AddHeader( "Allow", "GET" );

302 option

Looking at the comment in the MVC code which does not redirect the HEAD request:

//only redirect for GET requests, otherwise the browser might not propagate the verb and request
//body correctly.

It seems like another option is to send a 302. It should be reasonably safe to return a 302 to the HTTPS site for bot HEAD requests to root (which is what MVC does for a GET). So, I implemented the following which is based on the way that MVC does it:

if( isHead == true && isRoot == true )
{
    httpContext.ClearError();
    httpContext.Response.Clear();                    
    httpContext.Response.StatusCode = 302;
    string url = "https://" + httpContext.Request.Url.Host + httpContext.Request.RawUrl;
    httpContext.Response.Redirect(url, endResponse: false);                    
    return;
}

Implement in global.asax.cs:

protected void Application_Error( object sender, EventArgs e )
{
     //Your code here
}