HttpURLConnnection request failures on Android 6.0

I am using Google's Volley library for my application project , that targets minimum api level 14.So, Volley library uses HttpURLConnection as the NetworkClient.

Therefore , there should not be any issue related to Removal of Apache HTTPClient. However, I have done the configuration required for 6.0 Sdk i.e compileSdkVersion 23, build-tool-version 23.0.1 and build:gradle:1.3.1' and even tried adding useLibrary 'org.apache.http.legacy'. Have updated the same for Volley library project in my application.

Recently ,I tried to run my app on Android 6.0 (MarshMallow), my project compiles and runs. But those requests that require authentication headers are failing on MarshMallow with:

BasicNetwork.performRequest: Unexpected response code 401 com.android.volley.AuthFailureError

However the same is running on all Api level below 23.

I have checked the headers many times.Strangely, those requests that do not require authentication are giving response with 200 OK.

Right now I am totally clueless what is breaking the requests, does anybody have any idea what has changed in new Version that HttpURLConnection request fails for only Api level 23? Is anybody else using Volley and facing similar issue?

Here is my CustomRequest Class

public class CustomRequest extends Request<Void> {

int id, cmd;
Map<String, String> params;
BaseModel model;

public CustomRequest(int method, int cmd, String url, Map<String, String> params, int id, BaseModel model) {
    super(method, url, null);
    this.id = id;
    this.cmd = cmd;
    this.params = params;
    this.model = model;

    if (method == Method.GET) {
        setUrl(buildUrlForGetRequest(url));
    }

    Log.v("Volley", "Making request to: " + getUrl());
}

private String buildUrlForGetRequest(String url) {
    if (params == null || params.size() == 0) return url;

    StringBuilder newUrl = new StringBuilder(url);
    Set<Entry<String, String>> paramPairs = params.entrySet();
    Iterator<Entry<String, String>> iter = paramPairs.iterator();

    newUrl.append("?");
    while (iter.hasNext()) {
        Entry<String, String> param = iter.next();
        newUrl
            .append(param.getKey())
            .append("=")
            .append(param.getValue());
        if (iter.hasNext()) newUrl.append("&");
    }

    return newUrl.toString();
}

@Override
public Map<String, String> getHeaders() throws AuthFailureError {
    Map<String, String> headers = new HashMap<String, String>();

    headers.put("X-Api-Version", Contract.API_VERSION);
    headers.put("X-Client", "android");
    String accessToken = APP.getInstance().getToken();
    if (!accessToken.equals("")) {
        headers.put("Authorization", "Bearer " + accessToken);    
    }

    return headers;
}

@Override
protected Response<Void> parseNetworkResponse(NetworkResponse response) {
    Exception ex;
    try {
        String jsonString = new String(
                response.data, HttpHeaderParser.parseCharset(response.headers));
        JsonNode json = new ObjectMapper().readTree(jsonString);
        if (model != null) model.parse(id, json);
        EventBus.getDefault().post(new EventResponse(cmd, true, null));
        return Response.success(null, HttpHeaderParser.parseCacheHeaders(response)); //Doesn't return anything. BaseModel.parse() does all the storage work.
    } catch (NoMoreDataException e) {
        ex = e;
        EventBus.getDefault().post(new NoMoreDataModel(cmd, e));
        EventBus.getDefault().post(new EventResponse(cmd, false, null));
        return Response.success(null, HttpHeaderParser.parseCacheHeaders(response));
    } catch (Exception e) {
        ex = e;
        Log.e("CustomRequest", Log.getStackTraceString(e));

        String message = APP.getInstance().getString(R.string.failedRequest);
        if (!TextUtils.isEmpty(e.getMessage()))
            message = e.getMessage();
        EventBus.getDefault().post(new ErrorEventModel(cmd, message, e));
        return Response.error(new ParseError(ex));
    }
}

@Override
protected Map<String, String> getParams() throws AuthFailureError {
    return params;
}

@Override
protected void deliverResponse(Void response) {
    Log.v("Volley", "Delivering result: " + getUrl());
}

@Override
public void deliverError(VolleyError error) {
    Log.e("CustomRequest", "Delivering error: Request=" + getUrl() + " | Error=" + error.toString());

    String message = APP.getInstance().getString(R.string.failedRequest);
    EventBus.getDefault().post(new ErrorEventModel(cmd, message, error));
}

}

Only difference I found between Api 23 and others is the HostNameVerifier. For Api level 23 : com.android.okhttp.internal.tls.OkHostnameVerifier For Api level <23 : javax.net.ssl.DefaultHostnameVerifier.

After checking the Server side of my application, found the reason behind the issue. For Android 6.0(MarshMallow) the headers were becoming empty and this was not the case with other versions.

So the fix that worked for me:

Created and Added a new custom header X-Proxy-No-Redirect => 1 and passed along with other headers.

Theory behind it:

There is a API server to which we send request, then the API server redirects the request to corresponding site based on the oAuth token While redirecting For the redirection to happen, there are two ways to do that

1 - We just send a response back to the caller stating to redirect to a certain page. Then the caller(Networking library) takes care of redirection

2 - API server will itself makes the redirect request and get the response and then pass to caller

Earlier we were using the Method 1.

On Android 6.0 - The networking lib(Volley) doesn't seem to set all the headers while making the redirection request. Once this new header is set, Method 2 will come into effective.

P.S This fix was applicable for my application project , maybe not for others.Just providing what was wrong and what helped me.