Apache POI XSSFWorkbook memory leak

2019-08-23 23:18发布

问题:

So I'm making a large-scale prime number generator in Java (with the help of JavaFX).

It uses the Apache POI library (I believe I'm using v3.17) to output the results to Excel spreadsheets.

The static methods for this exporting logic are held in a class called ExcelWriter. Basically, it iterates through an Arraylist arguments and populates a XSSFWorkbook with it's contents. Afterwords, a FileOutputStream is used to actually make it an excel file. Here are the relevant parts of it:

public class ExcelWriter {

//Configured JFileChooser to make alert before overwriting old files
private static JFileChooser fileManager = new JFileChooser(){
@Override
public void approveSelection(){
    ...
}        
};


private static FileFilter filter = new FileNameExtensionFilter("Excel files","xlsx");
private static boolean hasBeenInitialized = false;



//Only method that can be called externally to access this class's functionality
public static <T extends Object> void makeSpreadsheet
    (ArrayList<T> list,  spreadsheetTypes type, int max, String title, JFXProgressBar progressBar) 
            throws IOException, InterruptedException{
    progressBar.progressProperty().setValue(0);
    switch (type){
        case rightToLeftColumnLimit:
            makeSpreadsheetRightToLeft(list, false, max, title, progressBar);
            break;
       ...
    }
}


static private <T extends Object> void makeSpreadsheetRightToLeft
    (ArrayList<T> list,  boolean maxRows, int max, String title, JFXProgressBar progressBar) 
            throws IOException, InterruptedException{
    initializeChooser();
    XSSFWorkbook workbook = new XSSFWorkbook();
    XSSFSheet sheet = workbook.createSheet("Primus output"); 
    int rowPointer = 0;
    int columnPointer = 0;
    double progressIncrementValue = 1/(double)list.size();

    //Giving the spreadsheet an internal title also
    Row row = sheet.createRow(0);
    row.createCell(0).setCellValue(title);

    row = sheet.createRow(++rowPointer);

    //Making the sheet with a max column limit
    if (!maxRows){            
        for (T number: list){ 
            if (columnPointer == max){
                columnPointer = 0;
                row = sheet.createRow(++rowPointer);
            }
            Cell cell = row.createCell(columnPointer++);
            progressBar.setProgress(progressBar.getProgress() + progressIncrementValue);
            cell.setCellValue(number.toString());             
        }
    }else {
        //Making the sheet with a max row limit
        int columnWrapIndex = (int)Math.ceil(list.size()/(float)max);
        for (T number: list){ 
            if (columnPointer == columnWrapIndex){
                columnPointer = 0;
                row = sheet.createRow(++rowPointer);
            }
            Cell cell = row.createCell(columnPointer++);
            progressBar.setProgress(progressBar.getProgress() + progressIncrementValue);
            cell.setCellValue(number.toString());
        }         
    }
    writeToExcel(workbook, progressBar);


}


static private void writeToExcel(XSSFWorkbook book, JFXProgressBar progressBar) throws IOException, InterruptedException{
    //Exporting to Excel
    int returnValue = fileManager.showSaveDialog(null);

    if (returnValue == JFileChooser.APPROVE_OPTION){
        File file = fileManager.getSelectedFile();

        //Validation logic here


        try{

            FileOutputStream out = new FileOutputStream(file);
            book.write(out);
            out.close();
            book.close();
        }catch (FileNotFoundException ex){

        } 

    }
}
}

Afterwards, my FXML document controller has a buttonListerner which calls:

longCalculationThread thread = new longCalculationThread(threadBundle);
thread.start();

The longcalculationthread creates a list of about a million prime numbers and Exports them to the ExcelWriter using this code:

private void publishResults() throws IOException, InterruptedException{
    if (!longResults.isEmpty()){
        if (shouldExport) {
            progressText.setText("Exporting to Excel...");
            ExcelWriter.makeSpreadsheet(longResults, exportType, excelExportLimit, getTitle(), progressBar);

    }
   }

The problem is, even though the variable holding the workbook in the XSSF workbook is a local variable to the methods it is used in, it doesn't get garbage collected afterwards.

It takes up like 1.5GB of RAM (I don't know why), and that data is only reallocated when another huge export is called (not for small exports). My problem isn't really that the thing takes a lot of RAM, it's that even when the methods are completed the memory isn't GCed. Here are some pictures of my NetBeans profiles:

Normal memory usage when making array of 1000000 primes:

Huge heap usage when making workbook

Memory Isn't reallocated when workbook ins't accessible anymore

Fluctuation seen when making a new workbook using the same static methods

回答1:

I found the answer! I had to prompt the GC with System.gc(). I remember trying this out earlier, however I must have put it in a pace where the workbook was still accessible and hence couldn't be GCed.