Twitter4j slow status stream

Question

I was able to stream trending tweets based on keywords using the onStatus() method. Initially, the tweets were coming in at a fast rate. Now they appear to be streaming very slow.Has anyone else had this issue and have a solution?

Answer 1

From here:

Scaling

Twitter streaming volume is not constant. Throughout the course of a 24 hour period, there is a natural ebb and flow to the number of Tweets delivered per second. In addition, the amount of data grows steadily month-over-month, and significant world or cultural events may cause traffic spikes of 3 or more times the current daily peak volume. Clients which want to maintain their connections must provision and test for these cases.

The best practice for ingesting Tweets and other streaming messages is to decouple collection and processing of high volume streams. For example, collect the raw text of messages in one process, passing each message into a message queue, rotated flatfile, or database. A second process or set of processes should parse the messages and extract any necessary fields for storage or further manipulation.