使用线程的块来处理的文件块(Use threading to process file chunk

2019-08-03 11:47发布

我有包括5000个IP地址的数组列表。 对于每一个IP地址,我想执行一个SNMPGET请求和FTPDownload命令。 我要实现它的方式,其中,在时间2个不同的线程同时运行前五IP地址。 这些IP地址的执行后,接下来的2个IP地址将在这些线程执行。 谁能帮助该怎么办呢?

在这里,连接是其延伸的螺纹,也可以实现被写入其run()方法的工作的类。 请帮忙。

Connection newConnection =new Connection(0);
Connection newConnection1 =new Connection(1);

for(int i = 0; i < NE_list.getRowCount(); i=i+2)
{
if(NE_list.getValueAt(i, 0).toString().equals("true")) //Some condition here for the IP Address
{

            newConnection.i=i;
            newConnection1.i=i+1;
            newConnection.runprogram();
            newConnection1.runprogram();
 }


    } 

class Connection extends Thread{
int  i;
Connection(int val){
  i=val;
}
void runprogram(){
start();
}
public void run(){
//SNMP and FTP Code here for IP Address in index i of NE_list
}
}

Answer 1:

执行人框架将是您的解决方案最适合。 我创建了一个例子在这里。 您可以增加线程的数量按照您的要求。

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

class SomeRunnable implements Runnable {
    int threadNo = -1 ;
    List<String> list = new ArrayList<String>();
    public SomeRunnable(List list, int threadNo ) {
        this.list.addAll(list);
        this.threadNo =threadNo;
    }
    @Override
    public void run() {
        for (String element : list) {
            System.out.println("By Thread:" + threadNo+", Processed Element:" +element);
        }
    }

}

public class ExecutorDemo {
    public static void main(String[] args) {
        List<String> list = new ArrayList<String>();
        for (int i = 0; i < 100; i++) {
            list.add("Elem:"+i);
        }
        // Divide list 
        int divideIndex = list.size()/2;
        //Create objects of Runnable
        SomeRunnable obj1 = new SomeRunnable(list.subList(0, divideIndex),1);
        SomeRunnable obj2 = new SomeRunnable(list.subList(divideIndex,list.size()),2);

        //Create fixed Thread pool, here pool of 2 thread will created
        ExecutorService pool = Executors.newFixedThreadPool(2);

        pool.execute(obj1);
        pool.execute(obj2);

        pool.shutdown();
    }

}


Answer 2:

兹将具有5个线程工作示例。 只要把test.txt的在您的应用程序CLASS_PATH。

class MyRunnable implements Runnable {
List<List<String>> records;
MyRunnable(List<List<String>> records){
    this.records = records;
}
public void run(){
    for(List list : records){
        System.out.println(Thread.currentThread().getName() + " : "+list.toString());
    }
}}

主类 -

public class FileProcessThreads {
public List<List<String>> process(String fileName) throws IOException {
    List<List<String>> records = new ArrayList<>();
    BufferedReader br = new BufferedReader(new FileReader(fileName));
    String line = null;
    while((line = br.readLine()) != null){
        List<String> listValues = Arrays.asList(line.split(" "));
        records.add(listValues);
    }
    System.out.println(records.size());
    return records;
}
public static void main(String[] args) throws IOException {
    FileProcessThreads fp = new FileProcessThreads();
    List<List<String>> records = fp.process("test.txt");
    ExecutorService es = Executors.newFixedThreadPool(5);
    int recordsInEachThread = (int) (records.size()/5);
    System.out.println(recordsInEachThread);

    MyRunnable my1 = new MyRunnable(records.subList(0, recordsInEachThread));
    MyRunnable my2 = new MyRunnable(records.subList(recordsInEachThread+1, recordsInEachThread*2));
    MyRunnable my3 = new MyRunnable(records.subList(recordsInEachThread*2 + 1, recordsInEachThread*3));
    MyRunnable my4 = new MyRunnable(records.subList(recordsInEachThread*3 + 1, recordsInEachThread*4));
    MyRunnable my5 = new MyRunnable(records.subList(recordsInEachThread*4 + 1, records.size() - 1));
    es.execute(my1);
    es.execute(my2);
    es.execute(my3);
    es.execute(my4);
    es.execute(my5);
    es.shutdown();
}}


文章来源: Use threading to process file chunk by chunk