Twitter4j on java - query search only gives me the

Question

I tried to construct a method that gives me the results from the search of a text in twitter, but only gives me the 6 most recent results, and I need more, 50~100~200... really... Some more..

I tried a lot of for ordo while bucles tryin' only to get more results. Failed.

Is there some method to do it? I tried with getMentions(), but for a error and bad results and another uses, I need to do it with twitter.query(Search).

Is there someone that knows about it?

I'm playin' with TWITTER4J - 4.0.2 Version.

Here's my code :

List<Status> estados = new ArrayList<Status>();

    BufferedReader br = null;
    FileWriter fichero = null;
    PrintWriter pw = null;
    File file = new File("dbmenciones.txt");

    try {

        Query query = new Query();
        query.setQuery("#tits");
        query.setResultType(Query.MIXED);
        query.setCount(100);

        QueryResult qr = null;
        long lastid = Long.MAX_VALUE;

        int vv = 0;

        do {

            vv++;
            if (512 - estados.size() > 100){
                query.setCount(100);
            } else {
                query.setCount(512 - estados.size());
            }

            qr = twitter.search(query);
            estados.addAll(qr.getTweets());

            System.out.println("Obtenidos "+estados.size()+" tweets.");

            for (Status stt : estados){
                if (stt.getId()<lastid) lastid = stt.getId();
            }           



            query.setMaxId(lastid-1);

        } while (vv < 14);

        for (Status status : estados){

            System.out.println(status.getId()+" :  "+status.getText());

            System.out.printf("Publicado : %s, @%-20s ,dijo: %s\n", 
                    status.getCreatedAt().toString(), 
                    status.getUser().getScreenName(), cleanText(status.getText())); 

            listaUsuarios.add("\""+status.getUser().getId()+"\";\""+status.getUser().getScreenName()+"\";\""+status.getUser().getFollowersCount()+"\"");

            fichero = new FileWriter("C:\\Documents and Settings\\a671\\Escritorio\\"+file, true);
            pw = new PrintWriter(fichero);

            System.out.println("Added : "+listaUsuarios.get(listaUsuarios.size()-1));

            pw.println(listaUsuarios.get(listaUsuarios.size()-1));

            if (null != fichero){
                  fichero.close();
             } 
        }

        } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                   try {
                   if (null != fichero)
                      fichero.close();
                   } catch (Exception e2) {
                      e2.printStackTrace();
                   }
                   try {
                        if (br != null) br.close();
                    } catch (IOException ex) {
                        ex.printStackTrace();
                    }
                }

            }
}

First I start the query, and then I use do while to get a lot of times the results. At the last time, I enter the results in a .txt file. Nothing awful. Help. Thanks.

Answer 1

I have been playing with Twitter4Java and I simplified it. You just need to use the library called twitter4j-core-4.0.1.jar.

If you let me to give you an advice, I will strongly recommend you to split the code into classes, so the code will be easier to understand. Here is the program that will search the word you are looking for, you just need to change the IO library for your preferred one (or just System.out.println):

Main.java:

    public class Main implements Constants{

    /*Use the menu Run-> Run configurations...->Arguments -> Program arguments to give a term to be used as parameter*/
    public static void main(String[] args) {
        if (args.length < 1) {
            System.out.println(USAGE);
            System.exit(NO_OK);
        }

        TermFinder termFinder = new TermFinder(args[0]);
        new Timer(termFinder);

        String tweetQuery = "Madrid";
        ResultsAnalyzer resultsAnalyzer = new ResultsAnalyzer(tweetQuery);
        System.out.println("Tweets which contains the word "+tweetQuery+":");
        System.out.println(resultsAnalyzer.findInTweet());
        System.out.println("-----------------------------------------------");

        String usernameQuery = "@Linus__Torvalds";
        resultsAnalyzer.setQuery(usernameQuery);
        System.out.println("Searching if user "+usernameQuery+" exists:");
        System.out.println(resultsAnalyzer.findInUserName());
        System.out.println("-----------------------------------------------");

        String hashtagQuery = "#rock";
        resultsAnalyzer.setQuery(hashtagQuery);
        System.out.println(resultsAnalyzer.findInHashtag());
        System.out.println("-----------------------------------------------");

        String hashtagQuery2 = "#aupa_atleti_campeon";
        resultsAnalyzer.setQuery(hashtagQuery2);
        System.out.println(resultsAnalyzer.findInHashtag());
        System.out.println("-----------------------------------------------");
    }

}

ResultsAnalyzer.java:

import java.security.InvalidParameterException;

public class ResultsAnalyzer implements Constants{

    private String query;
    private IO io;

    public ResultsAnalyzer(String query){
        this.setQuery(query);
        this.setIo(new IO());
    }

    public String getQuery() {
        return query;
    }

    public void setQuery(String query) {
        if(query.length()<=TWEET_MAX_LENGTH){
            this.query = query;
        }
        else{
            System.out.println("The term can't be longer than a tweet (140 characters)");
            System.exit(NO_OK);
        }
    }

    public IO getIo() {
        return io;
    }

    public void setIo(IO io) {
        this.io = io;
    }

    /**
     * 
     * @param code: int which can be: 0 to search query in the user name, 1 for search query in tweet
     * @return coincidentData: String which will be empty if there are no matches. Otherwise, it will contain the matches
     */
    public String find(int code){
        if(code != SEARCH_ON_USERNAME && code != SEARCH_ON_TWEET){
            throw new InvalidParameterException("Code must be " + SEARCH_ON_USERNAME + " or "+ SEARCH_ON_TWEET + ".");
        }
        String results = this.getIo().in.readFile(FILE_NAME);
        String[]tweets = results.split("\n");
        if(tweets.length==0){
            return "";//No matches
        }else{
            String aux = "", coincidentData = "";
            for (String tweet : tweets) {
                aux = tweet.split(LINE_SEPARATOR)[code];
                if(aux.contains(this.getQuery())){
                    coincidentData += aux;
                }
            }
            return coincidentData;
        }
    }

    public String findInTweet(){
        String results = find(SEARCH_ON_TWEET);
        if(results.equals("")){
            return "Term \"" + this.getQuery() + "\" not found on any tweet.";
        }
        else{
            return "Term \"" + this.getQuery() + "\" found on the tweet: \""+results+"\"";
        }
    }

    public String findInUserName(){
        String results = find(SEARCH_ON_USERNAME);
        if(results.equals("")){
            return "Username \"" + this.getQuery() + "\" not found.";
        }
        else{
            return "Username \"" + this.getQuery() + "\" found.";
        }
    }

    public String findInHashtag(){
        String possibleMatches = find(SEARCH_ON_TWEET);
        if(possibleMatches.contains(LINE_HEADER)){//At least more than a tweet with the certain hashtag
            String[]tweets = possibleMatches.split(LINE_HEADER);
            String results = "";
            for (String tweet : tweets) {
                if(tweet.contains(HASHTAG + this.getQuery())){
                    results += tweet;
                }
            }
            String finalResult = "";
            if(results.equals("")){
                finalResult = "Hashtag \""+this.getQuery()+"\" not found.";
            }else{
                finalResult = "Hashtag \""+this.getQuery()+"\" found on tweet: \""+results+"\"";
            }
            return finalResult;
        }
        else{//Just one tweet contains the certain hashtag
            String finalResult = "";
            if(possibleMatches.equals("")){
                finalResult = "Hashtag \""+this.getQuery()+"\" not found.";
            }else{
                finalResult = "Hashtag \""+this.getQuery()+"\" found on tweet: \""+possibleMatches+"\"";
            }
            return finalResult;
        }

    }
}

TermFinder.java:

import twitter4j.*;
import twitter4j.auth.AccessToken;

import java.util.List;

public class TermFinder implements Constants{
    /**
     * Usage: java TermFinder [query]
     *
     * @param args: query to search
     */

    private IO io;
    private String word;

    public TermFinder(String word){
        this.setWord(word);
        this.setIo(new IO());
    }

    public IO getIo() {
        return this.io;
    }

    public void setIo(IO io) {
        this.io = io;
    }

    public String getWord() {
        return word;
    }

    public void setWord(String word) {
        if(word.length()<=TWEET_MAX_LENGTH){
            this.word = word;
        }
        else{
            System.out.println("The term can't be longer than a tweet (140 characters)");
            System.exit(NO_OK);
        }
    }

    public void find() {
        System.out.println("Starting the search of tweets with the word \""+this.getWord()+"\"...");
        Twitter twitter = new TwitterFactory().getInstance();

        twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_SECRET);
        twitter.setOAuthAccessToken(new AccessToken(ACCESS_TOKEN, TOKEN_SECRET));

        try {
            Query query = new Query(this.getWord());
            QueryResult result;
            do {
                result = twitter.search(query);
                List<Status> tweets = result.getTweets();
                String currentTweet = "";
                for (Status tweet : tweets) {
                    System.out.println("@" + tweet.getUser().getScreenName() + " - " + tweet.getText());
                    currentTweet += LINE_HEADER + tweet.getUser().getScreenName() + LINE_SEPARATOR + tweet.getText()+"\n";
                    this.getIo().out.writeStringOnFile(FILE_NAME, currentTweet);
                }
            } while ((query = result.nextQuery()) != null);
            System.exit(OK);
            System.out.println("End of the search");
        } catch (TwitterException te) {
            te.printStackTrace();
            System.out.println("Failed to search tweets: " + te.getMessage());
            System.exit(NO_OK);
        }
    }
}

Timer.java:

public class Timer implements Runnable, Constants{

    private Thread thread;
    private TermFinder termFinder;

    public Timer(TermFinder termFinder){
        this.termFinder = termFinder;
        this.thread = new Thread(this);
        this.thread.start();
    }

    @Override
    public void run() {
        try{
            while(true){
                this.termFinder.find();
                Thread.sleep(Constants.PERIOD);
            }
        }catch(Exception e){
            System.out.println(e.getMessage());
        }
    }
}

Constants.java:

public interface Constants {

    //OAuth fields
    public static final String CONSUMER_KEY = "XXXXXUSiC5rLRsH0eP4475MJB";
    public static final String CONSUMER_SECRET = "XXXXXBhrVFqoGCcCWjf3vJygE4gnoYuAjJtJvDAm2bLODiQaBj";
    public static final String ACCESS_TOKEN = "XXXXX21983-O92srOl4EtnwYU4bNBTSSj2Dn7tJssucyCjjwJx";
    public static final String TOKEN_SECRET = "XXXXXHFltlISnuUd4TS1q5PTKXdi9uZLEsmziUxtnA7I7";

    //Usage pattern
    public static final String USAGE = "java TermFinder [query]";

    //Results file name, format values and Twitter constants
    public static final String FILE_NAME = "results.txt";
    public static final String LINE_HEADER = "@";
    public static final String LINE_SEPARATOR = "-";
    public static final String HASHTAG = "#";

    //Numeric values and constants
    public static final int PERIOD = 3600000;//milliseconds of an hour (for class Timer)
    public static final int TWEET_MAX_LENGTH = 140;
    public static final int OK = 0;
    public static final int NO_OK = -1;
    public static final int SEARCH_ON_USERNAME = 0;
    public static final int SEARCH_ON_TWEET = 1;

}

In.java:

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;

public class In {

    public In() {
    }

    /**
     * Tranforms the content of a file (i.e: "origin.txt") in a String
     * 
     * @param fileName: Name of the file to read
     * @return String which represents the content of the file
     */
    public String readFile(String fileName) {

        FileReader fr = null;
        try {
            fr = new FileReader(fileName);
        } catch (FileNotFoundException e1) {
            e1.printStackTrace();
        }

        BufferedReader bf = new BufferedReader(fr);

        String file = "", aux = "";
        try {
            while ((file = bf.readLine()) != null) {
                aux = aux + file + "\n";
            }
            bf.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return aux;
    }
}

Out.java:

import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;

public class Out {

    public Out() {}

    /**
     * Writes on the file named as the first parameter the String which gets as second parameter.
     * 
     * @param fileName: Destiny file
     * @param message: String to write on the destiny file
     */
    public void writeStringOnFile(String fileName, String message) {
        FileWriter w = null;
        try {
            w = new FileWriter(fileName);
        } catch (IOException e) {
            e.printStackTrace();
        }
        BufferedWriter bw = new BufferedWriter(w);
        PrintWriter wr = new PrintWriter(bw);

        try {
            wr.write(message);
            wr.close();
            bw.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

IO.java:

public class IO {

    public Out out;
    public In in;

    /**
     * Object which provides an equivalent use to Java 'System' class
     */
    public IO() {
        this.setIn(new In());
        this.setOut(new Out());
    }

    public void setIn(In in) {
        this.in = in;
    }

    public In getIn() {
        return this.in;
    }

    public void setOut(Out out) {
        this.out = out;
    }

    public Out getOut() {
        return this.out;
    }
}

Hope it helps. Please let me know if you have any issue.

Clemencio Morales Lucas.