What parameters will Twitter lib's fetchTweets

问题:

I've created a twitter bot for retweeting and liking tweets using Agarwal's tutorial which uses the Twitter lib library for searching tweets. I'm using Twitterlib version 21 in my project.

It seems to work for the most part, but I have one particular problem. When I include the "min_retweets:X" parameter in my search, it doesn't seem to be recognized. This is not an officially documented search parameter but it does work when you use it in normal twitter searches on the site and returns only tweets that have been retweeted X times.

If I make "min_retweets:X" the first search term then the search will return no results. If I make it the last search term I get results but they are not limited to tweets that have been retweeted X times.

I've poked around in the fetchtweets() method inside Twitterlib a little but it's beyond me currently to figure out where the issue might be here. Version 21 of the library says it was updated to make searches with the ":" work properly, and that seems accurate as far as some other search terms I've used, just not this one. There's also a "min_faves:X" parameter but I haven't tested to see if it works with Twitterlib's search.

If anyone knows of a workaround to get this working, I'd appreciate it. Below is the code I use to call the function and the code for the function itself from Twitter lib:

var tweets = twit.fetchTweets( 
  TWITTER_SEARCH_PHRASE, function(tweet) {
    if (!tweet.possibly_sensitive) {
      return tweet.id_str;
    }
  }, {
    multi: true,
    lang: "en",
    count: 5,
    since_id: props.getProperty("SINCE_TWITTER_ID")
  });

---

OAuth.prototype.fetchTweets = function(search, tweet_processor, options) {

  var tweets, response, result = [], data, i, candidate;  
  var phrase = encodeString('lang:' + (options && options.lang || 'en') + ' ' + encodeString(search).replace(/%3A/g, ":")); // English language by default

  this.checkAccess();

  if(options == null) {
    options = {};
  }

  var url = [
    "https://api.twitter.com/1.1/search/tweets.json?count=", 
    (options.count || "5"),
    options.filter ? ("&filter=" + encodeString(options.filter)) : "",
    "&include_entities=",
    options.include_entities ? encodeString(options.include_entities) : "false",
    "&result_type=",
    options.result_type ? encodeString(options.result_type) : "recent",
    "&q=",
    phrase,
    options.since_id ? "&since_id=" + encodeString(options.since_id) : ""
    ].join("");
  var request_options =
  {
    "method": "get"
  };

try {

    response = this.fetch(url, request_options);

    if (response.getResponseCode() === 200) {

      data = JSON.parse(response.getContentText());
      if (data) {

        tweets = data.statuses;

        if(!tweet_processor) {
          return options && options.multi ? tweets : tweets[tweets.length - 1];
        }
        for (i=tweets.length-1; i>=0; i--) {
          candidate = tweet_processor(tweets[i]);
          if(candidate === true) candidate = tweets[i];
          if(candidate) {
            if(options && options.multi) {
              result.push(candidate);
            } else {
              return candidate;
            }
          }
        }
        if(result.length) {
          return result;
        }
        if(i < 0) {
          Logger.log("No matching tweets this go-round");
        }
      }
    } else {
      Logger.log(response);
    }
  } catch (e) {
    Logger.log(JSON.stringify(e));
    throw e;
  }
  return result;
}

回答1:

Yes, it looks like you've uncovered a legit bug on that one. If you look at the execution transcript after running fetchTweets() with a min_retweets search query, it comes out looking like this:

https://api.twitter.com/1.1/search/tweets.json?count=5&include_entities=false&result_type=recent&q=lang%3Aen%20min_retweets%3A100%2520swag

That extra %25 is the percent sign itself getting re-escaped, and it's breaking the query. I'm going to have to go back and comprehensively reexamine all of the special characters; I'm pretty sure some have to be double-escaped and some must not. In the meantime, here's a workaround. You can call twit.fetch and feed it the URL directly, like this:

var tweets = JSON.parse(
  twit.fetch(
     "https://api.twitter.com/1.1/search/tweets.json?"
     + "count=5&result_type=recent&q=" 
     + Twitterlib.encodeString("lang:en min_retweets:100 #swag"),     
    {method: "get"}).getContentText("UTF-8")
).statuses;

I'll also come back and let you know when v22 drops and fixes your issue. Thanks for using my library!

回答2:

I checked this for other characters (/#@;$,?=+) and got their respective execution transcript as:

%252F%2523%2540%253B%2524%252C%253F%253D%252B

So the percent sign is getting re-escaped.

Only for ":" it is working properly.

In the fetchtweets function:

var phrase = encodeString('lang:' + (options && options.lang || 'en') + ' ' + encodeString(search).replace(/%3A/g, ":"));

Here ":" is replaced with %3A which comes out to be correct. Maybe adding the same for other characters will work too?

Also, the workaround script takes too long to execute and doesn't always return recent tweets. I got some tweets that had been tweeted around 3 months back!