What algorithm does twitter use to determine the 10 topics that you can see at search.twitter.com? I would like to implement that algorithm and I would also like to show the 50 most popular topics (instead of 10). Can you describe the most efficient algorithm?
Thanks!
(Twitters API can be found at- http://apiwiki.twitter.com/REST%20API%20Documentation)
Also, I would like to be able to implement the algorithm by searching through the public timeline- http://twitter.com/statuses/public_timeline.rss
Twitter's trending algorithm is not just volume of keywords. That's part of it, but there's also a decay factor so that "justin beiber" isn't top trending forever.
This post on quora backs this up. http://www.quora.com/Trending-Topics-Twitter/What-is-the-basis-of-Twitters-current-Trending-Topics-algorithm?q=trending+algorithm
decay is typically done by using the relative age of the post in the algorithm, giving more weight to newer topics/posts/etc.
see also http://www.quora.com/What-tools-algorithms-or-data-structures-would-you-use-to-build-a-Trending-Topics-algorithm-for-a-high-velocity-stream?q=trending+algorithm
So what Twitter probably does is it counts the number of mentions of a particular term minus stop words (stop words like : do, me, you, I, not, on etc) So "the cat is out of the bag" and "my dog ate my cat" would mean that cat ,dog and bag would be the terms it extracted (the rest are all stop words) And it then counts 'cat' as 2 references, so 'cat' would be a trending topic in this case.