Scraping with Meteor.js

2019-03-09 04:32发布

Can I scrape with meteor.js? Just discovered cheerio which works excellent combined with request. Can I use these with meteor, or is there something similar?

Do you have an working example?

4条回答
\"骚年 ilove
2楼-- · 2019-03-09 05:04

now you should use meteorhacks npm package https://github.com/meteorhacks/npm and require this with :

var cheerio = Meteor.npmRequire('cherio');

worked for me :)

查看更多
看我几分像从前
3楼-- · 2019-03-09 05:15

You can have a look to http://casperjs.org/ which is very useful. You can also do screenshots, automated test, etc...

查看更多
一夜七次
4楼-- · 2019-03-09 05:23

The following code is used in this project to scrape a tweetstorm:

if (Meteor.isClient) {

  Meteor.call('getTweets', function (error, result) {
    if (error) {
      console.log("error", error);
    };

    Session.set("tweets", result);
  });

  Template.tweets.helpers({
    rant: function () {
      return Session.get("tweets");
    }
  });

}

Server side

  if (Meteor.isServer) {
      Meteor.startup(function () {
        var cheerio = Meteor.npmRequire('cheerio');

    Meteor.methods({
      getTweets: function () {
        result = Meteor.http.get("https://twitter.com/Royal_Arse/status/538330380273979393");
        $ = cheerio.load(result.content);
        var body = $('#stream-items-id > li:nth-child(n) > div > div > p').text();
        return body;
      },

    })

  });
}
查看更多
啃猪蹄的小仙女
5楼-- · 2019-03-09 05:25

Of course! Its hard to imagine what meteor can't do! First you need something to handle the remote http requests. In your meteor directory in the terminal run meteor add http to add the Meteor.http package, also npm install cheerio (have a look at another SO question on how to install npm modules to see exactly where to install external npm modules.

Here is an example that might help you out a bit, it scrapes the current time.

Server js

require = __meteor_bootstrap__.require; //to use npm require must be exposed.
var cheerio = require('cheerio');

Meteor.methods({
    getTime: function () {
        result = Meteor.http.get("http://www.timeanddate.com/worldclock/city.html?n=136");
        $ = cheerio.load(result.content);
        CurrentTime = $('#ct').html();
        return CurrentTime;
    }
});

Client side script:

Meteor.call("getTime", function(error, result) {
    alert("The current time is " + result); 
});

I hope this is helpful. amongst with Cheerio there are also other node frameworks such as node.io

查看更多
登录 后发表回答