C# How can I check if a URL exists/is valid?

问题:

I am making a simple program in visual c# 2005 that looks up a stock symbol on Yahoo! Finance, downloads the historical data, and then plots the price history for the specified ticker symbol.

I know the exact URL that I need to acquire the data, and if the user inputs an existing ticker symbol (or at least one with data on Yahoo! Finance) it works perfectly fine. However, I have a run-time error if the user makes up a ticker symbol, as the program tries to pull data from a non-existent web page.

I am using the WebClient class, and using the DownloadString function. I looked through all the other member functions of the WebClient class, but didn\'t see anything I could use to test a URL.

How can I do this?

回答1:

You could issue a \"HEAD\" request rather than a \"GET\"?

(edit) - lol! Looks like I\'ve done this before!; changed to wiki to avoid accusations of rep-garnering. So to test a URL without the cost of downloading the content:

// using MyClient from linked post
using(var client = new MyClient()) {
    client.HeadOnly = true;
    // fine, no content downloaded
    string s1 = client.DownloadString(\"http://google.com\");
    // throws 404
    string s2 = client.DownloadString(\"http://google.com/silly\");
}

You would try/catch around the DownloadString to check for errors; no error? It exists...

---

With C# 2.0 (VS2005):

private bool headOnly;
public bool HeadOnly {
    get {return headOnly;}
    set {headOnly = value;}
}

and

using(WebClient client = new MyClient())
{
    // code as before
}

回答2:

Here is another implementation of this solution:

using System.Net;

///
/// Checks the file exists or not.
///
/// The URL of the remote file.
/// True : If the file exits, False if file not exists
private bool RemoteFileExists(string url)
{
    try
    {
        //Creating the HttpWebRequest
        HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
        //Setting the Request method HEAD, you can also use GET too.
        request.Method = \"HEAD\";
        //Getting the Web Response.
        HttpWebResponse response = request.GetResponse() as HttpWebResponse;
        //Returns TRUE if the Status code == 200
        response.Close();
        return (response.StatusCode == HttpStatusCode.OK);
    }
    catch
    {
        //Any exception will returns false.
        return false;
    }
}

From: http://www.dotnetthoughts.net/2009/10/14/how-to-check-remote-file-exists-using-c/

回答3:

These solutions are pretty good, but they are forgetting that there may be other status codes than 200 OK. This is a solution that I\'ve used on production environments for status monitoring and such.

If there is a url redirect or some other condition on the target page, the return will be true using this method. Also, GetResponse() will throw an exception and hence you will not get a StatusCode for it. You need to trap the exception and check for a ProtocolError.

Any 400 or 500 status code will return false. All others return true. This code is easily modified to suit your needs for specific status codes.

/// <summary>
/// This method will check a url to see that it does not return server or protocol errors
/// </summary>
/// <param name=\"url\">The path to check</param>
/// <returns></returns>
public bool UrlIsValid(string url)
{
    try
    {
        HttpWebRequest request = HttpWebRequest.Create(url) as HttpWebRequest;
        request.Timeout = 5000; //set the timeout to 5 seconds to keep the user from waiting too long for the page to load
        request.Method = \"HEAD\"; //Get only the header information -- no need to download any content

        using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
        {
            int statusCode = (int)response.StatusCode;
            if (statusCode >= 100 && statusCode < 400) //Good requests
            {
                return true;
            }
            else if (statusCode >= 500 && statusCode <= 510) //Server Errors
            {
                //log.Warn(String.Format(\"The remote server has thrown an internal error. Url is not valid: {0}\", url));
                Debug.WriteLine(String.Format(\"The remote server has thrown an internal error. Url is not valid: {0}\", url));
                return false;
            }
        }
    }
    catch (WebException ex)
    {
        if (ex.Status == WebExceptionStatus.ProtocolError) //400 errors
        {
            return false;
        }
        else
        {
            log.Warn(String.Format(\"Unhandled status [{0}] returned for url: {1}\", ex.Status, url), ex);
        }
    }
    catch (Exception ex)
    {
        log.Error(String.Format(\"Could not test url {0}.\", url), ex);
    }
    return false;
}

回答4:

If I understand your question correctly, you could use a small method like this to give you the results of your URL test:

WebRequest webRequest = WebRequest.Create(url);  
WebResponse webResponse;
try 
{
  webResponse = webRequest.GetResponse();
}
catch //If exception thrown then couldn\'t get response from address
{
  return 0;
} 
return 1;

You could wrap the above code in a method and use it to perform validation. I hope this answers the question you were asking.

回答5:

Try this (Make sure you use System.Net):

public bool checkWebsite(string URL) {
   try {
      WebClient wc = new WebClient();
      string HTMLSource = wc.DownloadString(URL);
      return true;
   }
   catch (Exception) {
      return false;
   }
}

When the checkWebsite() function gets called, it tries to get the source code of the URL passed into it. If it gets the source code, it returns true. If not, it returns false.

Code Example:

//The checkWebsite command will return true:
bool websiteExists = this.checkWebsite(\"https://www.google.com\");

//The checkWebsite command will return false:
bool websiteExists = this.checkWebsite(\"https://www.thisisnotarealwebsite.com/fakepage.html\");

回答6:

Here is another option

public static bool UrlIsValid(string url)
{
    bool br = false;
    try {
        IPHostEntry ipHost = Dns.Resolve(url);
        br = true;
    }
    catch (SocketException se) {
        br = false;
    }
    return br;
}

回答7:

This solution seems easy to follow:

public static bool isValidURL(string url) {
    WebRequest webRequest = WebRequest.Create(url);
    WebResponse webResponse;
    try
    {
        webResponse = webRequest.GetResponse();
    }
    catch //If exception thrown then couldn\'t get response from address
    {
        return false ;
    }
    return true ;
}

回答8:

i have a more simple way to determine weather a url is valid.

if (Uri.IsWellFormedUriString(uriString, UriKind.RelativeOrAbsolute))
{
   //...
}

回答9:

Web servers respond with a HTTP status code indicating the outcome of the request e.g. 200 (sometimes 202) means success, 404 - not found etc (see here). Assuming the server address part of the URL is correct and you are not getting a socket timeout, the exception is most likely telling you the HTTP status code was other than 200. I would suggest checking the class of the exception and seeing if the exception carries the HTTP status code.

IIRC - The call in question throws a WebException or a descendant. Check the class name to see which one and wrap the call in a try block to trap the condition.

回答10:

Following on from the examples already given, I\'d say, it\'s best practice to also wrap the response in a using like this

    public bool IsValidUrl(string url)
    {
         try
         {
             var request = WebRequest.Create(url);
             request.Timeout = 5000;
             request.Method = \"HEAD\";

             using (var response = (HttpWebResponse)request.GetResponse())
             {
                response.Close();
                return response.StatusCode == HttpStatusCode.OK;
            }
        }
        catch (Exception exception)
        { 
            return false;
        }
   }

回答11:

I have always found Exceptions are much slower to be handled.

Perhaps a less intensive way would yeild a better, faster, result?

public bool IsValidUri(Uri uri)
{

    using (HttpClient Client = new HttpClient())
    {

    HttpResponseMessage result = Client.GetAsync(uri).Result;
    HttpStatusCode StatusCode = result.StatusCode;

    switch (StatusCode)
    {

        case HttpStatusCode.Accepted:
            return true;
        case HttpStatusCode.OK:
            return true;
         default:
            return false;
        }
    }
}

Then just use:

IsValidUri(new Uri(\"http://www.google.com/censorship_algorithm\"));

回答12:

WebRequest request = WebRequest.Create(\"http://www.google.com\");
try
{
     request.GetResponse();
}
catch //If exception thrown then couldn\'t get response from address
{
     MessageBox.Show(\"The URL is incorrect\");`
}