Here is my code:
DefaultHttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
This works for every url I have tried so far, except for some urls that contain an anchor. Some of these anchored urls return a 400. The weird thing is that it isn't all links that contain an anchor, a lot of them work just fine.
Unfortunately, I have to be really general as I can't provide the specific urls here.
The links are completely valid and work just fine in any browser, but the HttpClient returns a 400 when trying the link. If I remove the anchor it will work.
Any ideas what to look for?
For example: http://www.somedomain.com/somedirectory/somepage#someanchor
Sorry again for the generics
EDIT: I should mention this is for Android.
As @Greg Sansom says, the URL should not be sent with an anchor / fragment. The fragment part of the URL is not relevant to the server.
Here's the expected URL syntax from relevant part of the HTTP 1.1 specification:
http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
Note: there is no fragment
part in the syntax.
What happens if you do send a fragment
clearly is server implementation specific. I expect that you will see a variety of responses:
- Some servers will silently strip / ignore the fragment part. (This is what you are expecting to happen).
- Some servers might treat this as a request error and respond with a 400.
- Some servers might mistakenly treat the fragment as part of the path or query, and give you a 404 or some other response, depending on how "confused" the fragment makes the server.
- Some servers might actually imbue the fragment with a specific meaning. (This strikes me as a stupid thing to do, but you never know ...)
IMO, the most sensible solution is to strip it from the URL before instantiating the HttpGet
object.
FOLLOWUP
The recommended way to remove a fragment from a URL string is to turn it into a java.net.URL
or java.net.URI
instance, extract the relevant components, use these to create a new java.net.URL
or java.net.URI
instance (leaving out the fragment of course), and finally turn it back into a String.
But I think that the following should also work, if you can safely assume that your URLs are all valid absolute HTTP or HTTPS URLs.
int pos = url.indexOf("#");
String strippedUrl = (pos >= 0) ? url.substring(0, pos) : url;
Your usage of the anchor in the url is incorrect.
When we perform a "Get", we need to get the entire resource (page). The anchor is just a tag marking a location, normally your browser will scroll to the position of the anchor once the page is loaded. It does not make sense to "Get" the page at a specific anchor - the entire page must be fetched.
It is possible your inconsistent results are because some web servers are ignoring the anchor component, and others are correcting your error.
The solution is just to remove the #anchor portion of the url before running your code.
String user_url2="uhttp://www.somedomain.com/somedirectory/somepage#someanchor";
HttpClient client = new DefaultHttpClient();
HttpGet siteRequest = new HttpGet(user_url2);
StringBuilder sb = new StringBuilder();
HttpResponse httpResponse;
try {
httpResponse = client.execute(siteRequest);
HttpEntity entity = httpResponse.getEntity();
InputStream in = entity.getContent();
String line = null;
BufferedReader reader = new BufferedReader(
new InputStreamReader(in));
while ((line = reader.readLine()) != null)
{
sb.append(line);
}
result = sb.toString();
result string will display url value
There is a bug in Android HttpClient that was fixed in HttpClient 1.2 but not backported to Android
https://issues.apache.org/jira/browse/HTTPCLIENT-1177
https://github.com/apache/httpclient/commit/be6347aef0f7450133017b775113a8f3fadd2f1c
I have opened a bug report at:
https://code.google.com/p/android/issues/detail?id=65909