iText 7.0.5 - Forms - NullPointerException when ad

I used Bruno's example (https://developers.itextpdf.com/examples/form-examples/clone-create-fields-table) to add form fields to tables in a newly created document. The example itself works without any problems, but if I want to create 50 tables I get the following exception:

Exception in thread "main" java.lang.NullPointerException
    at com.itextpdf.kernel.pdf.PdfDictionary.get(PdfDictionary.java:552)
    at com.itextpdf.kernel.pdf.PdfDictionary.getAsArray(PdfDictionary.java:156)
    at com.itextpdf.kernel.pdf.PdfPage.getAnnotations(PdfPage.java:743)
    at com.itextpdf.kernel.pdf.annot.PdfAnnotation.getPage(PdfAnnotation.java:427)
    at com.itextpdf.forms.fields.PdfFormField.regenerateField(PdfFormField.java:1781)
    at com.itextpdf.forms.fields.PdfFormField.setValue(PdfFormField.java:1043)
    at com.itextpdf.forms.fields.PdfFormField.setValue(PdfFormField.java:1004)
    at com.itextpdf.forms.fields.PdfFormField.setValue(PdfFormField.java:999)
    at com.itextpdf.forms.fields.PdfFormField.createText(PdfFormField.java:469)
    at com.itextpdf.forms.fields.PdfFormField.createText(PdfFormField.java:410)
    at com.itextpdf.forms.fields.PdfFormField.createText(PdfFormField.java:391)
    at CreateFormInTable$MyCellRenderer.draw(CreateFormInTable.java:62)
    at com.itextpdf.layout.renderer.TableRenderer.drawChildren(TableRenderer.java:1023)
    at com.itextpdf.layout.renderer.AbstractRenderer.draw(AbstractRenderer.java:458)
    at com.itextpdf.layout.renderer.TableRenderer.draw(TableRenderer.java:948)
    at com.itextpdf.layout.renderer.DocumentRenderer.flushSingleRenderer(DocumentRenderer.java:138)
    at com.itextpdf.layout.renderer.RootRenderer.processRenderer(RootRenderer.java:349)
    at com.itextpdf.layout.renderer.RootRenderer.shrinkCurrentAreaAndProcessRenderer(RootRenderer.java:338)
    at com.itextpdf.layout.renderer.RootRenderer.addChild(RootRenderer.java:236)
    at com.itextpdf.layout.RootElement.add(RootElement.java:109)
    at com.itextpdf.layout.Document.add(Document.java:143)
    at CreateFormInTable.manipulatePdf(CreateFormInTable.java:45)
    at CreateFormInTable.main(CreateFormInTable.java:25)

I use Java 1.8.0_121 on Windows 8 Enterprise.

The modified manipulatePdf method looks like this:

protected void manipulatePdf(String dest) throws Exception {
    PdfDocument pdfDoc = new PdfDocument(new PdfWriter(dest));
    Document doc = new Document(pdfDoc);

    for (int i = 0; i < 50; i++) {
        System.out.println(i);
        Table table = new Table(2);
        Cell cell;
        cell = new Cell().add("Name:");
        table.addCell(cell);
        cell = new Cell();
        cell.setNextRenderer(new MyCellRenderer(cell, "name" + i));
        table.addCell(cell);
        cell = new Cell().add("Address");
        table.addCell(cell);
        cell = new Cell();
        cell.setNextRenderer(new MyCellRenderer(cell, "address" + i));
        table.addCell(cell);
        doc.add(table);
    }
    doc.close();
}

The complete example that produces the NPE can be found here: https://github.com/trettstadtnlb/itext-table-form-fields

Has anyone experienced this kind of behaviour or can confirm that this is a bug?

It is easy to reproduce this behavior in iText 7.0.x using much less content:

PdfDocument pdfDoc = new PdfDocument(new PdfWriter(dest));
Document doc = new Document(pdfDoc);
doc.add(new AreaBreak(AreaBreakType.NEXT_PAGE));
doc.add(new AreaBreak(AreaBreakType.NEXT_PAGE));

PdfTextFormField field = PdfFormField.createText(pdfDoc, new Rectangle(100, 100, 300, 20), "name", "");
PdfAcroForm form = PdfAcroForm.getAcroForm(pdfDoc, true);
form.addField(field);

doc.close();

The problem behind this is that for a form field (widget) without an associated page, iText tries to determine the page that widget is on by iterating all document pages and looking up their respective annotation arrays. And it already does so during createText, while setting the field value (and updating widget appearances)...

In the code above, though, the previous newly created pages, at least the first one, has already been flushed to the writer and de-initialized in memory. Thus, the loop over the pages tries to retrieve a value from a de-initialized page dictionary, i.e. from a null Map, hence the NullPointerException.

Whether this is a bug in iText or merely an anti-pattern, I don't know.

Inspired by @SamuelHuylebroeck's comment I tried both the OP's original code and the simplified code above with immediateFlush set to false. Indeed, this prevents the exception and the result of the simplified code looks ok. The result of the OP's original code, though, is not as he expects it, the table is built across four pages but all the fields turns up on the last page!

So, in case of form elements created directly (i.e. not via events) setting immediateFlush to false may be considered a work-around (but not a fix as the early flushing is an important feature of iText). The general case of form creation is broken, though, unless someone can point out that the code here is an iText 7 anti-pattern and can show how it should be done.