Some requests with special character in this with the french accents.
var client = new HttpClient();
var data0 = await client.GetAsync("http://fr.wikipedia.org/wiki/Monastère_d'Arkadi");
This simple code yields:
StatusCode: 301, ReasonPhrase: 'Moved Permanently', Version: 1.1
Any ideas what is happening? Downloading the French article of New York works in fact. I´ve even try to encode the the name but nothing works.
Wikipedia sends an HTTP 301 indicating that the permanent home of
http://fr.wikipedia.org/wiki/Monastère_d'Arkadi
is now
https://fr.wikipedia.org/wiki/Monast%C3%A8re_d'Arkadi
If you look at the response header, you will see that the Location header has the updated URL.
It succeeds in downloading the page in a browser because the browser automatically uses the new Location.
UPDATE
If I enter https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi
(the value in the Location header of the 301 response) in IE, I get an HTTP 200. However, I also get an HTTP 301 using HttpClient.
If I use WebClient
, I get the error "Too many redirections were attempted." It looks like there is some sort of redirect loop happening here. I'm exploring this further.
UPDATE 2
I downloaded the page using wget, with verbose logging. Note that wget was unable to validate the SSL certificate for Wikipedia, and that there were two redirects: From the http URL to the https URL, then again to the URL with escaped characters.
E:\Software\GnuWin32\bin>wget -v --no-check-certificate http://fr.wikipedia.org/wiki/Monastère_d'Arkadi
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = E:\Software\GnuWin32/etc/wgetrc
--2015-10-17 23:22:43-- http://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi
Resolving fr.wikipedia.org... 208.80.154.224
Connecting to fr.wikipedia.org|208.80.154.224|:80... connected.
HTTP request sent, awaiting response... 301 TLS Redirect
Location: https://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi [following]
--2015-10-17 23:22:44-- https://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi
Connecting to fr.wikipedia.org|208.80.154.224|:443... connected.
WARNING: cannot verify fr.wikipedia.org's certificate, issued by `/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Va
lidation CA - SHA256 - G2':
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi [following]
--2015-10-17 23:22:44-- https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi
Connecting to fr.wikipedia.org|208.80.154.224|:443... connected.
WARNING: cannot verify fr.wikipedia.org's certificate, issued by `/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Va
lidation CA - SHA256 - G2':
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `MonastA"re_d'Arkadi'
[ <=> ] 171,252 463K/s in 0.4s
2015-10-17 23:22:45 (463 KB/s) - `MonastA"re_d'Arkadi' saved [171252]