C# How can I check if a URL exists/is valid?

Question

I am making a simple program in visual c# 2005 that looks up a stock symbol on Yahoo! Finance, downloads the historical data, and then plots the price history for the specified ticker symbol.

I know the exact URL that I need to acquire the data, and if the user inputs an existing ticker symbol (or at least one with data on Yahoo! Finance) it works perfectly fine. However, I have a run-time error if the user makes up a ticker symbol, as the program tries to pull data from a non-existent web page.

I am using the WebClient class, and using the DownloadString function. I looked through all the other member functions of the WebClient class, but didn't see anything I could use to test a URL.

How can I do this?

Answer 1

These solutions are pretty good, but they are forgetting that there may be other status codes than 200 OK. This is a solution that I've used on production environments for status monitoring and such.

If there is a url redirect or some other condition on the target page, the return will be true using this method. Also, GetResponse() will throw an exception and hence you will not get a StatusCode for it. You need to trap the exception and check for a ProtocolError.

Any 400 or 500 status code will return false. All others return true. This code is easily modified to suit your needs for specific status codes.

/// <summary>
/// This method will check a url to see that it does not return server or protocol errors
/// </summary>
/// <param name="url">The path to check</param>
/// <returns></returns>
public bool UrlIsValid(string url)
{
    try
    {
        HttpWebRequest request = HttpWebRequest.Create(url) as HttpWebRequest;
        request.Timeout = 5000; //set the timeout to 5 seconds to keep the user from waiting too long for the page to load
        request.Method = "HEAD"; //Get only the header information -- no need to download any content

        using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
        {
            int statusCode = (int)response.StatusCode;
            if (statusCode >= 100 && statusCode < 400) //Good requests
            {
                return true;
            }
            else if (statusCode >= 500 && statusCode <= 510) //Server Errors
            {
                //log.Warn(String.Format("The remote server has thrown an internal error. Url is not valid: {0}", url));
                Debug.WriteLine(String.Format("The remote server has thrown an internal error. Url is not valid: {0}", url));
                return false;
            }
        }
    }
    catch (WebException ex)
    {
        if (ex.Status == WebExceptionStatus.ProtocolError) //400 errors
        {
            return false;
        }
        else
        {
            log.Warn(String.Format("Unhandled status [{0}] returned for url: {1}", ex.Status, url), ex);
        }
    }
    catch (Exception ex)
    {
        log.Error(String.Format("Could not test url {0}.", url), ex);
    }
    return false;
}

Answer 2

Following on from the examples already given, I'd say, it's best practice to also wrap the response in a using like this

    public bool IsValidUrl(string url)
    {
         try
         {
             var request = WebRequest.Create(url);
             request.Timeout = 5000;
             request.Method = "HEAD";

             using (var response = (HttpWebResponse)request.GetResponse())
             {
                response.Close();
                return response.StatusCode == HttpStatusCode.OK;
            }
        }
        catch (Exception exception)
        { 
            return false;
        }
   }

Answer 3

I have always found Exceptions are much slower to be handled.

Perhaps a less intensive way would yeild a better, faster, result?

public bool IsValidUri(Uri uri)
{

    using (HttpClient Client = new HttpClient())
    {

    HttpResponseMessage result = Client.GetAsync(uri).Result;
    HttpStatusCode StatusCode = result.StatusCode;

    switch (StatusCode)
    {

        case HttpStatusCode.Accepted:
            return true;
        case HttpStatusCode.OK:
            return true;
         default:
            return false;
        }
    }
}

Then just use:

IsValidUri(new Uri("http://www.google.com/censorship_algorithm"));

Answer 4

i have a more simple way to determine weather a url is valid.

if (Uri.IsWellFormedUriString(uriString, UriKind.RelativeOrAbsolute))
{
   //...
}

Answer 5

Here is another option

public static bool UrlIsValid(string url)
{
    bool br = false;
    try {
        IPHostEntry ipHost = Dns.Resolve(url);
        br = true;
    }
    catch (SocketException se) {
        br = false;
    }
    return br;
}

Answer 6

If I understand your question correctly, you could use a small method like this to give you the results of your URL test:

WebRequest webRequest = WebRequest.Create(url);  
WebResponse webResponse;
try 
{
  webResponse = webRequest.GetResponse();
}
catch //If exception thrown then couldn't get response from address
{
  return 0;
} 
return 1;

You could wrap the above code in a method and use it to perform validation. I hope this answers the question you were asking.