Why Wikipedia returns 301 response code with certa

2020-06-26 06:05发布

问题:

Some requests with special character in this with the french accents.

var client = new HttpClient();
var data0 = await client.GetAsync("http://fr.wikipedia.org/wiki/Monastère_d'Arkadi");

This simple code yields:

StatusCode: 301, ReasonPhrase: 'Moved Permanently', Version: 1.1

Any ideas what is happening? Downloading the French article of New York works in fact. I´ve even try to encode the the name but nothing works.

回答1:

Wikipedia sends an HTTP 301 indicating that the permanent home of

http://fr.wikipedia.org/wiki/Monastère_d'Arkadi

is now

https://fr.wikipedia.org/wiki/Monast%C3%A8re_d'Arkadi

If you look at the response header, you will see that the Location header has the updated URL.

It succeeds in downloading the page in a browser because the browser automatically uses the new Location.

UPDATE

If I enter https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi (the value in the Location header of the 301 response) in IE, I get an HTTP 200. However, I also get an HTTP 301 using HttpClient.

If I use WebClient, I get the error "Too many redirections were attempted." It looks like there is some sort of redirect loop happening here. I'm exploring this further.

UPDATE 2

I downloaded the page using wget, with verbose logging. Note that wget was unable to validate the SSL certificate for Wikipedia, and that there were two redirects: From the http URL to the https URL, then again to the URL with escaped characters.

E:\Software\GnuWin32\bin>wget -v --no-check-certificate http://fr.wikipedia.org/wiki/Monastère_d'Arkadi
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = E:\Software\GnuWin32/etc/wgetrc
--2015-10-17 23:22:43--  http://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi
Resolving fr.wikipedia.org... 208.80.154.224
Connecting to fr.wikipedia.org|208.80.154.224|:80... connected.
HTTP request sent, awaiting response... 301 TLS Redirect
Location: https://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi [following]
--2015-10-17 23:22:44--  https://fr.wikipedia.org/wiki/Monast%E8re_d'Arkadi
Connecting to fr.wikipedia.org|208.80.154.224|:443... connected.
WARNING: cannot verify fr.wikipedia.org's certificate, issued by `/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Va
lidation CA - SHA256 - G2':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi [following]
--2015-10-17 23:22:44--  https://fr.wikipedia.org/wiki/Monast%C3%A8re_d%27Arkadi
Connecting to fr.wikipedia.org|208.80.154.224|:443... connected.
WARNING: cannot verify fr.wikipedia.org's certificate, issued by `/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Va
lidation CA - SHA256 - G2':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `MonastA"re_d'Arkadi'

    [  <=>                                                                          ] 171,252      463K/s   in 0.4s

2015-10-17 23:22:45 (463 KB/s) - `MonastA"re_d'Arkadi' saved [171252]