How do you parse and process HTML/XML in PHP?

2楼-- · 2018-12-31 00:32

phpQuery and QueryPath are extremely similar in replicating the fluent jQuery API. That's also why they're two of the easiest approaches to properly parse HTML in PHP.

Examples for QueryPath

Basically you first create a queryable DOM tree from an HTML string:

 $qp = qp("<html><body><h1>title</h1>..."); // or give filename or URL

The resulting object contains a complete tree representation of the HTML document. It can be traversed using DOM methods. But the common approach is to use CSS selectors like in jQuery:

 $qp->find("div.classname")->children()->...;

 foreach ($qp->find("p img") as $img) {
     print qp($img)->attr("src");
 }

Mostly you want to use simple #id and .class or DIV tag selectors for ->find(). But you can also use XPath statements, which sometimes are faster. Also typical jQuery methods like ->children() and ->text() and particularly ->attr() simplify extracting the right HTML snippets. (And already have their SGML entities decoded.)

 $qp->xpath("//div/p[1]");  // get first paragraph in a div

QueryPath also allows injecting new tags into the stream (->append), and later output and prettify an updated document (->writeHTML). It can not only parse malformed HTML, but also various XML dialects (with namespaces), and even extract data from HTML microformats (XFN, vCard).

 $qp->find("a[target=_blank]")->toggleClass("usability-blunder");

.

phpQuery or QueryPath?

Generally QueryPath is better suited for manipulation of documents. While phpQuery also implements some pseudo AJAX methods (just HTTP requests) to more closely resemble jQuery. It is said that phpQuery is often faster than QueryPath (because of fewer overall features).

For further information on the differences see this comparison on the wayback machine from tagbyte.org. (Original source went missing, so here's an internet archive link. Yes, you can still locate missing pages, people.)

And here's a comprehensive QueryPath introduction.

Advantages

Simplicity and Reliability
Simple to use alternatives ->find("a img, a object, div a")
Proper data unescaping (in comparison to regular expression grepping)

0人赞添加讨论(0) 举报

宁负流年不负卿

3楼-- · 2018-12-31 00:32

We have created quite a few crawlers for our needs before. At the end of the day, it is usually simple regular expressions that do the thing best. While libraries listed above are good for the reason they are created, if you know what you are looking for, regular expressions is a safer way to go, as you can handle also non-valid HTML/XHTML structures, which would fail, if loaded via most of the parsers.

0人赞添加讨论(0) 举报

路过你的时光

4楼-- · 2018-12-31 00:32

This sounds like a good task description of W3C XPath technology. It's easy to express queries like "return all href attributes in img tags that are nested in <foo><bar><baz> elements." Not being a PHP buff, I can't tell you in what form XPath may be available. If you can call an external program to process the HTML file you should be able to use a command line version of XPath. For a quick intro, see http://en.wikipedia.org/wiki/XPath.

0人赞添加讨论(0) 举报

倾城一夜雪

5楼-- · 2018-12-31 00:33

I have written a general purpose XML parser that can easily handle GB files. It's based on XMLReader and it's very easy to use:

$source = new XmlExtractor("path/to/tag", "/path/to/file.xml");
foreach ($source as $tag) {
    echo $tag->field1;
    echo $tag->field2->subfield1;
}

Here's the github repo: XmlExtractor

0人赞添加讨论(0) 举报

笑指拈花

6楼-- · 2018-12-31 00:34

Try Simple HTML DOM Parser

A HTML DOM parser written in PHP 5+ that lets you manipulate HTML in a very easy way!
Require PHP 5+.
Supports invalid HTML.
Find tags on an HTML page with selectors just like jQuery.
Extract contents from HTML in a single line.
Download

Examples:

How to get HTML elements:

// Create DOM from URL or file
$html = file_get_html('http://www.example.com/');

// Find all images
foreach($html->find('img') as $element)
       echo $element->src . '<br>';

// Find all links
foreach($html->find('a') as $element)
       echo $element->href . '<br>';

How to modify HTML elements:

// Create DOM from string
$html = str_get_html('<div id="hello">Hello</div><div id="world">World</div>');

$html->find('div', 1)->class = 'bar';

$html->find('div[id=hello]', 0)->innertext = 'foo';

echo $html;

Extract content from HTML:

// Dump contents (without tags) from HTML
echo file_get_html('http://www.google.com/')->plaintext;

Scraping Slashdot:

// Create DOM from URL
$html = file_get_html('http://slashdot.org/');

// Find all article blocks
foreach($html->find('div.article') as $article) {
    $item['title']     = $article->find('div.title', 0)->plaintext;
    $item['intro']    = $article->find('div.intro', 0)->plaintext;
    $item['details'] = $article->find('div.details', 0)->plaintext;
    $articles[] = $item;
}

print_r($articles);

0人赞添加讨论(0) 举报

梦醉为红颜

7楼-- · 2018-12-31 00:35

With FluidXML you can query and iterate XML using XPath and CSS Selectors.

$doc = fluidxml('<html>...</html>');

$title = $doc->query('//head/title')[0]->nodeValue;

$doc->query('//body/p', 'div.active', '#bgId')
        ->each(function($i, $node) {
            // $node is a DOMNode.
            $tag   = $node->nodeName;
            $text  = $node->nodeValue;
            $class = $node->getAttribute('class');
        });

https://github.com/servo-php/fluidxml

0人赞添加讨论(0) 举报