Tweepy Truncated Status

Question

I was mining user timeline data with tweepy and, have faced some difficulties in understanding the following:

1. Is the 'retweeted' and 'truncated' attribute referring to the same thing (i.e., status text beyond 140 characters)?
2. If not, what is the difference?
3. I came across a stackoverflow question where someone asked how to retrieve status text which has been 'chopped' due the the length being over 140 characters. It suggested that there is a retweeted attribute in the _json dictionary which will be true if that is the case and the full status text will under status->retweeted_status->text. However, I have not been able to find it and, the only status text was under status->text ending with '...'. Did I get this wrong and if so, how do I get the full text?

Thanks for your help in advance.

Answer 1

I am also troubled due to this problem.

When the tweet is retweeted, there are two "full_text" tags in .json file. The real full_text can be extracted by calling tag of 'retweet_status'. The solution is as below:

for tweet in tweepy.Cursor(api.search, q = "social", tweet_mode='extended',
                           wait_on_rate_limit = True, wait_on_rate_limit_notify = True, include_entities=True).items(10):
    try:
        print tweet.retweet.im_self._json['retweeted_status']['full_text']
    except:
        print tweet.retweet.im_self._json['full_text']

Answer 2

Is the 'retweeted' and 'truncated' attribute referring to the same thing (i.e., status text beyond 140 characters)? If not, what is the difference?

No, tweets can be truncated not only after a retweet (forward), but also after a reply or a mention (see my example below which is not a retweet). If it is the case, 'truncated' will be set to 'True' (and 'retweeted' will be True or False). It is also possible to have 'retweeted' equals to True while 'truncated' being equals to False, if the tweet is not truncated because its size is well below 140 characters.

It suggested that there is a retweeted attribute in the _json dictionary which will be true if that is the case and the full status text will be under status->retweeted_status->text.

This is true only if the tweet is a genuine retweet. Actually the retweeted_status could itself be truncated if it is coming from another truncated tweet. The best way is to use the tweet_mode='extended' parameter in Tweepy to retrieve the full text (unfortunately not documented in the Tweepy doc). For instance:

(not extended)

print api.get_status('862328512405004288')._json['text']

@tousuncotefoot @equipedefrance @CreditAgricole @AntoGriezmann @KMbappe @layvinkurzawa @UmtitiSam J'ai jamais vue d… https://tco/kALZ2ki9Vc

(extended)

print api.get_status('862328512405004288', tweet_mode='extended')._json['full_text']

@tousuncotefoot @equipedefrance @CreditAgricole @AntoGriezmann @KMbappe @layvinkurzawa @UmtitiSam J'ai jamais vue de match de foot et cela ferait un beau cadeau pour mon copain !!

查看更多

0人赞 添加讨论(0) 举报

Answer 3

Regarding question 3)

Below is a way to access the extended status text from a Tweepy stream listener, given that the tweet is more than 140 characters (in this case the status object will have an extended_tweet dictionary). If the tweet is not more than 140 characters, then just get the status text as usual:

class MyStreamListener(tweepy.StreamListener):
    def on_status(self, status):
        try:
            text = status.extended_tweet["full_text"]
        except AttributeError:
            text = status.text

Answer 4

The simplest way worked for me is simply get the status and search for the retweeted full text.

status = api.get_status(id, tweet_mode="extended")
try:
    print(status.retweeted_status.full_text)
except AttributeError:  # Not a Retweet
    print(status.full_text)

this works even in search

tweets=[]
for tweet in api.search(q=keyword, lang="en", count=count, tweet_mode='extended'):
    try:
       tweets.append(tweet.retweeted_status.full_text)
    except AttributeError:  # Not a Retweet
       tweets.append(tweet.full_text)