cURL couldn't resolve host How to deal with th

2019-06-12 01:02发布

问题:

I am working on a C++ program, which uses cURL lib. When I try to connect to URL, that is incorrect or the page is down I get following error :

  • getaddrinfo(3) failed for www.wp.pl#:80
  • Couldn't resolve host 'www.wp.pl#'

And the program terminates. And that is the problem, because I it to go futher and connect to other pages. I just won't parse and analyze the broken page I don't want to end my program after come across such one.

I posted similar topic, but it contained only snippets of code so it was diffcult for you to say what is wrong, now I will post whole code.

  int PageHandler::getCleanAndRepairedPage(string pageURL, string& outDoc)
    throw (exception) {

if (find(visitedPages.begin(), visitedPages.end(), pageURL)
        != visitedPages.end()) {

    // url has already been visited
    outDoc = "";
    return VISITED_PAGE_ERROR;

} else { // new url

    visitedPages.push_back(pageURL);

    char *charURL;

    charURL = const_cast<char*> (pageURL.c_str());

    CURL *curl;

    char curl_errbuf[CURL_ERROR_SIZE];

    TidyBuffer output = { 0 };
    TidyBuffer errbuf = { 0 };
    TidyBuffer docbuf = { 0 };

    TidyDoc tdoc = tidyCreate(); // Initialize "document"
    tidyOptSetBool(tdoc, TidyForceOutput, yes);
    tidyOptSetInt(tdoc, TidyWrapLen, 4096);
    tidyBufInit(&docbuf);

    int rc = -1;
    Bool ok;

    curl = curl_easy_init();

    curl_easy_setopt( curl, CURLOPT_URL, charURL );

    curl_easy_setopt( curl, CURLOPT_ERRORBUFFER, curl_errbuf );
    curl_easy_setopt( curl, CURLOPT_WRITEFUNCTION, write_cb );
    curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt( curl , CURLOPT_WRITEDATA, &docbuf );

    int err;

    err = curl_easy_perform(curl);

    cout << "curl_easy_perfom return value = " << err << endl;

    if (!err) {

        ok = tidyOptSetBool(tdoc, TidyXhtmlOut, yes); // Convert to XHTML

        if (ok) {

            rc = tidySetErrorBuffer(tdoc, &errbuf); // Capture diagnostics

            if (rc >= 0) {

                rc = tidyParseBuffer(tdoc, &docbuf); // parse the buffer
            }
            if (rc >= 0) {

                rc = tidyCleanAndRepair(tdoc); // Tidy it up!
            }
            if (rc >= 0) {

                rc = tidyRunDiagnostics(tdoc); // Kvetch
            }
            if (rc > 1) { // If error, force output.

                rc = (tidyOptSetBool(tdoc, TidyForceOutput, yes) ? rc : -1);
            }
            if (rc >= 0) {

                rc = tidySaveBuffer(tdoc, &output); // Pretty Print
            }
        }

        if (rc >= 0) {

            if (rc > 0) {

                //printf("\nDiagnostics:\n\n%s", errbuf.bp);
            }

        } else {

            printf("A severe error (%d) occurred.\n", rc);
        }

    } else {

        printf("%s\n", curl_errbuf);
    }

    if (err == NO_ERROR) {
        string tmp(reinterpret_cast<char const*> (output.bp));
        outDoc = tmp;
    } else {
        outDoc = "";
    }

    curl_easy_cleanup(curl);

    tidyBufFree(&docbuf);
    tidyBufFree(&output);
    tidyBufFree(&errbuf);
    tidyRelease(tdoc);

    return err; // err == 0 <=> everything ok!

}

}

And the console output :

EDIT : I forgot to say what is going on on console output. I connect to many pages and the invalid URL is just one of them. At the beginning you can see a succesful connection and cURL messages about it, the result of curl_easy_perform equals 0, which indicate that everything went good. Next message is about connection to the invalid URL, as u can see return value of curl_easy_perform is 6, which is not good.

回答1:

"www.wp.pl#" is not a legal URL and it causes libcurl to (mistakenly) use the entire string as a host name, and you don't have any such and the operation fails.



标签: c++ curl libcurl