PDFBox 1.8.10: Fill and Sign Document, Filling aga

In my previous SO question PDFBox 1.8.10: Fill and Sign PDF produces invalid signatures I explained, how I failed to fill and afterwards sign a PDF-Document, using PDFBox 1.8.10. After this got sorted out with some kind help, I now continue to work on the same topic. Starting with doc_v2.pdf (links to the file are below!), I fill and sign it, resulting in doc_v2_fillsigned.pdf (doing it in one go, saving it incrementally). Again I open the edited document (using again PDFBox) and try to fill another field.

Then saving the document leads to the following stack trace:

    Exception in thread "main" java.lang.NullPointerException
        at org.apache.pdfbox.pdmodel.interactive.form.PDAppearance.calculateFontSize(PDAppearance.java:930)
        at org.apache.pdfbox.pdmodel.interactive.form.PDAppearance.setAppearanceValue(PDAppearance.java:359)
        at org.apache.pdfbox.pdmodel.interactive.form.PDVariableText.setValue(PDVariableText.java:131)
        at com.c10n.scalibur.ehealthdemo.examples.PdfEditor.fill(PdfEditor.java:100)
        at com.c10n.scalibur.ehealthdemo.examples.SignPdf_ProfileLayer.start(SignPdf_ProfileLayer.java:66)
        at com.c10n.scalibur.ehealthdemo.examples.SignPdf_ProfileLayer.main(SignPdf_ProfileLayer.java:28)

What I do in the failing fill run:

    File curentDocument new File("doc_v2_fillsigned.pdf);
    File newDocument = new File("doc_v2_fillsigned_filled.pdf);

    String fieldName ="New Emergency Contact";
    String value="test";
    PDDocument doc = null;

    try(FileOutputStream fos = new FileOutputStream(newDocument)){

        try(FileInputStream fis = new FileInputStream(currentDocument);){
            int c;
            while ((c = fis.read(buffer)) != -1) {
                fos.write(buffer, 0, c);
            }
        }

        doc = PDDocument.load(currentDocument);
        PDDocumentCatalog catalog = doc.getDocumentCatalog();

        catalog.getCOSObject().setNeedToBeUpdate(true);
        catalog.getPages().getCOSObject().setNeedToBeUpdate(true);

        PDAcroForm form = catalog.getAcroForm();

        form.getCOSObject().setNeedToBeUpdate(true);
        form.getDefaultResources().getCOSObject().setNeedToBeUpdate(true);

        PDField field = form.getField(fieldName);
        field.setValue(value); // here the exception occurs.

        // What should happen afterwards:
        field.getCOSObject().setNeedToBeUpdate(true);
        field.getAcroForm().getCOSObject().setNeedToBeUpdate(true);

        ((COSDictionary) field.getDictionary().getDictionaryObject("AP")).getDictionaryObject("N").setNeedToBeUpdate(true);

        try(FileInputStream fis = new FileInputStream(newDocument)){
            doc.saveIncremental(fis, fos);
        }
    }finally{
        if(null != doc){
            doc.close();
            doc=null;
        }
    }

Files:

the empty document: https://www.dropbox.com/s/xf5pb0ng8k9zd4i/doc_v2.pdf?dl=0

filled and signed instance: https://www.dropbox.com/s/s8295tfyjpe1l4l/doc_v2_fillsigned.pdf?dl=0

Again, any help to resolve this is welcome!

Update:

mkl asked in the comments for the code to create the fillsigned pdf. So far I learned, that only signing suffices, to let the fill-code above fail afterwards. So here are some excepts from my signing code:

@Override
public byte[] sign(InputStream data) throws SignatureException, IOException {
    CMSTypedDataInputStream input = new CMSTypedDataInputStream(data);
    try {

        CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
        InputStream in = new ByteArrayInputStream(certData);

        X509Certificate signCert = (X509Certificate)certFactory.generateCertificate(in);

        ContentSigner signer = new MyContentSigner(profile);

        SignerInfoGenerator i = new JcaSignerInfoGeneratorBuilder(
                new JcaDigestCalculatorProviderBuilder().setProvider("BC").build())

        .build(signer,signCert);

        Store<?> certStore = new JcaCertStore(Collections.singletonList(signCert));

        CMSSignedDataGenerator cmsGen = new CMSSignedDataGenerator();

        cmsGen.addCertificates(certStore);
        cmsGen.addSignerInfoGenerator(i);

        CMSSignedData signedData = cmsGen.generate(input);

        byte[] result =signedData.getEncoded(); 
        return result;
    } catch (Exception e) {
        e.printStackTrace();
        throw new SignatureException(e);
    }
}

this creates the signature, using BouncyCastle 1.52, the code is called from

public void sign(SignatureInterface signer, String signatureFieldName, int pageNumber, String location, String reason, boolean lock) throws IOException, SignatureException{
    PDSignature signature = new PDSignature();
    signature.setFilter(PDSignature.FILTER_ADOBE_PPKLITE); // default filter
    signature.setSubFilter(PDSignature.SUBFILTER_ADBE_PKCS7_DETACHED); // for visible sigs!

    signature.setLocation(location);
    signature.setReason(reason);

    signature.setSignDate(Calendar.getInstance());

    SignatureOptions options = makeSignatureVisible(signature,signatureFieldName, pageNumber, lock );

    doc.addSignature(signature, signer, options);
}

which uses the following method, to produce a visible signature in some signature field, adding a picture there and placing it correctly:

SignatureOptions makeSignatureVisible( PDSignature signature, String fieldName, int pageNumber, boolean lock) throws IOException{
    PDDocumentCatalog catalog = doc.getDocumentCatalog();

    catalog.getCOSObject().setNeedToBeUpdate(true);
    catalog.getPages().getCOSObject().setNeedToBeUpdate(true);

    PDAcroForm form = catalog.getAcroForm();

    form.getCOSObject().setNeedToBeUpdate(true);
    form.getDefaultResources().getCOSObject().setNeedToBeUpdate(true);

    PDSignatureField field = (PDSignatureField) form.getField(fieldName);

    field.setSignature(signature);      
    field.setReadonly(lock);

    FileInputStream image = new FileInputStream("MUniverse_Signature.jpg");

    PDVisibleSignDesigner visibleSig =  new PDVisibleSignDesigner(newDocument.getName(), image, 1);

    PDRectangle area = getFieldArea(field);

    float max_width = area.getWidth();
    float max_height = area.getHeight();

    float scale = 1;

    if(max_height < visibleSig.getHeight()){
        scale = max_height / visibleSig.getHeight();
        System.out.println("scale: "+scale);
    }

    if(max_width < scale*visibleSig.getWidth()){
        scale = max_width / visibleSig.getWidth();
        System.out.println("scale: "+scale);
    }

    float zoom = ((scale-1)*100);

    visibleSig.zoom(zoom);

    PDPage page = (PDPage) doc.getDocumentCatalog().getAllPages().get(pageNumber);
    visibleSig.coordinates(area.getLowerLeftX(),page.getMediaBox().getHeight()-area.getUpperRightY());
    visibleSig.signatureFieldName(fieldName);

    PDVisibleSigProperties signatureProperties = new PDVisibleSigProperties();

    signatureProperties.signerName("name").signerLocation("location").signatureReason("Security")
    .visualSignEnabled(true).setPdVisibleSignature(visibleSig).buildSignature();

    SignatureOptions options = new SignatureOptions();
    options.setVisualSignature(signatureProperties);

    return options;
}

I suspect these fragment are no neccessary, and applying the signing examples, which come with PDFBox results in the same conflict, when trying to fill the pdf with incrementle saving afterwards.

regards,

daniel

The cause of the problem is that somehow during the original filling and signing the fonts in the default resources of the interactive form dictionary got lost.

PDFBox while filling in the form tries to access the font definition to create an appearance stream. It doesn't find it and, therefore, eventually fails.

In detail

In the original document doc_v2.pdf the interactive form dictionary looks like this:

You can clearly see the entries for ZaDb and Helv in the Font dictionary in the default resources DR dictionary.

In contrast the interactive form dictionary of the filled and signed document doc_v2_fillsigned.pdf looks like this:

As you see the the Font dictionary in the default resources DR dictionary is missing.

The cause

The OP observed further:

I played around some further and it suffices to sign. Afterwards, filling it fails. I assume, my signature creation is more or less the same as in PDFBox' example describing, how to ad a visible signature.

Based on this I simply applied the PDFBox example CreateVisibleSignature to the OP's original doc_v2.pdf file. Indeed, this already removed the Font default resource dictionary.

Thus, this definitively looks like a PDFBox bug.

PS: On the Apache Jira...

Looking around on the PDFBox Jira one actually already can find issues in this regard:

PDFBOX-2816 - PDFBox makes disallowed changes when signing a signed document
... I notice these changes in structure after signing:
1. Default resources (/DR) were droped from AcroForm dictionary;
PDFBOX-3017 - Improve document signing
Improve signing code: ...
- prepareNonVisualSignature clears the AcroForm DR (acroForm.setDefaultResources(null)) which is not good if there are other form fields
This primarily relates to non-visual signatures, so it is the pendant of the issue at hand. Thus, DR seems to get lost in signing with PDFBox in general

Thus, this is a known issue. I'm not sure whether PDFBox development takes issue votes into account when prioritizing, but if you are interested in this issue being resolved, a vote won't hurt... ;)