ItextSharp font color issue in ver 5.5.4+

I have some code that creates a red "stamp" using a red font color:

    string StampDate = DateTime.Now.ToString("MM/dd/yyyy");
    string FontPath = Server.MapPath("/assets/Fonts");
    string OrigFile = Server.MapPath("/test.pdf");
    const int OpacityPercent = 80;
    const float PDFPaidFontSize = 28;
    const float PDFCopyX = 170;
    const float PDFPaidX = 385;
    const float PDFY = 20;
    const float PDFDateXOffset = 7;
    const float PDFDateYOffset = 12;
    const float PDFDateFontSize = 10;
    const string PaidStampTxt = "PAID";
    const string CopyStampTxt = "COPY";
    const string ArialFilename = "arialbd.ttf";
    PdfStamper stamper = null;
    PdfReader reader = null;
    PdfReader.unethicalreading = true;
    MemoryStream streamPDF;
    try
    {
        reader = new PdfReader(OrigFile);
        streamPDF = new MemoryStream();
        stamper = new PdfStamper(reader, streamPDF);
        for (int i = 1; i <= reader.NumberOfPages; i++)
        {
            PdfGState gstate = new PdfGState();
            gstate.FillOpacity = gstate.StrokeOpacity = OpacityPercent / 100F;
            PdfContentByte overContent = stamper.GetOverContent(i);
            overContent.SaveState();
            overContent.SetGState(gstate);
            overContent.SetColorFill(BaseColor.RED);
            overContent.BeginText();
            BaseFont font = BaseFont.CreateFont(BaseFont.TIMES_BOLD, BaseFont.CP1252, BaseFont.NOT_EMBEDDED);
            overContent.SetFontAndSize(font, PDFPaidFontSize);
            overContent.ShowTextAligned(PdfContentByte.ALIGN_LEFT, CopyStampTxt, PDFCopyX, PDFY, 0);
            overContent.ShowTextAligned(PdfContentByte.ALIGN_LEFT, PaidStampTxt, PDFPaidX, PDFY, 0);
            overContent.SetColorFill(BaseColor.BLACK);
            font = BaseFont.CreateFont(Path.Combine(FontPath, ArialFilename), BaseFont.CP1252, BaseFont.NOT_EMBEDDED);
            overContent.SetFontAndSize(font, PDFDateFontSize);
            overContent.ShowTextAligned(PdfContentByte.ALIGN_LEFT, StampDate, PDFPaidX + PDFDateXOffset, PDFY - PDFDateYOffset, 0);
            overContent.EndText();
            overContent.RestoreState();
        }
    }
    finally
    {
        if (stamper != null)
        {
            stamper.Close();
        }
        if (reader != null)
        {
            reader.Close();
        }
    }
    byte[] pdf = streamPDF.ToArray();
    Response.Cache.SetCacheability(HttpCacheability.NoCache);
    Response.Buffer = false;
    Response.Clear();
    Response.ClearContent();
    Response.ClearHeaders();
    Response.Charset = string.Empty;
    Response.ContentType = "application/pdf";
    Response.AddHeader("content-length", pdf.Length.ToString());
    Response.AddHeader("Content-Disposition", "inline;filename=test.pdf;");
    Response.BinaryWrite(pdf);
    Response.Close();

The resulting pdf text has a gray color instead of red. When i reverted back to ver 5.5.3 it shows up as red again. I have tried 5.5.4 and 5.5.5 and they both appear to have the same issue.

My question is: Is this a bug or do I need to change my code to something newer?

Edit: This appears to only be an issue with certain pdf files. Changing the fonts and pdf file versions did not seem to have an effect.

Comparing a pdf that does work and a pdf that does not work (neither i can share publicly) I noticed that the pdf that does NOT work is a tagged pdf, has fast web view enabled, and was produced by adobe pdf library. A pdf that DOES work is NOT a tagged pdf, does NOT have fast web view enabled, and was created by itextsharp.

Since I cannot control what the source pdf files will be, I reverted to an earlier version of itextsharp that seems to work all the time.

A first attempt to reproduce the issue

I just executed your code for these values:

        int OpacityPercent = 70;
        int PDFPaidFontSize = 20;
        string CopyStampTxt = "COPY";
        string PaidStampTxt = "PAID";
        int PDFCopyX = 100;
        int PDFPaidX = 250;
        int PDFY = 500;
        string FontPath = @"C:\Windows\Fonts";
        string ArialFilename = "ariali.ttf";
        int PDFDateFontSize = 30;
        string StampDate = "TODAY";
        int PDFDateXOffset = 0;
        int PDFDateYOffset = 35;

with a simple source PDF, and the result PDF looks like this:

In contrast to your observation

The resulting pdf text has a gray color instead of red.

the resulting text color is reddish (a partially transparent red on white).

I tested using iTextSharp 5.5.5.

To get a gray color instead of red, therefore, there must be something special about your variable values or source PDF, it is not a general iTextSharp issue.

Reproducing the issue using the file provided by the OP

After the first attempt to reproduce the issue, the OP provided a sample file Test.pdf, and indeed, for this file the result of the same code is:

Thus, there indeed is an issue.

Analysis

A comparison of the added content stream operations in both cases shows:

For the first attempt using my sample PDF:

/Xi0 gs
1 0 0 rg
BT
/Xi1 20 Tf
1 0 0 1 100 500 Tm
(COPY) Tj

for the second attempt using the OP's sample file:

/Xi0 gs
1 0 0 rg
BT
0 g
/Xi1 20 Tf
1 0 0 1 100 500 Tm
(COPY) Tj

Thus, in spite of the same code been used, there is an additional 0 g operation in the latter case, and that operations selects black as fill color.

This came as quite a surprise. Thus, I looked through the iText code and code history for an explanation (I chose the iText/Java code as that's where the original development takes place and changes can be inspected more thoroughly).

And indeed, PdfContentByte.beginText ends with:

if (isTagged()) {
    try {
        restoreColor();
    } catch (IOException ioe) {

    }
}

Backgrounds

Thus, in case of tagged PDFs this method "resets the color". Why that?

Looking through the code history gives some hints

Revision 5499 Tagged PDF support: keeping graphics and text in one canvas. Not yet ready...
Revision 5515 Now iText can write both text and graphics into 1 canvas. For that you should set PdfDocument.putTextAndGraphicsTogether to true.

Here the block above first appears with a slight difference
```
if (autoControlTextBlocks) {
    try {
        restoreColor();
    } catch (IOException ioe) {

    }
}
```
i.e. here the color is restored only if autoControlTextBlocks is true.
...
Revision 5533 writer.isTagged property now decides whether to write all content in one canvas or into separate ones.

Here the flag autoControlTextBlocks is replaced by a isTagged call of the associated writer.

My interpretation:

To properly support tagged PDFs, it was necessary or at least advantageous to keep graphics and text in one canvas (formerly they were created in different canvasses which eventually were concatenated, so related graphics and texts were distant from each other in the content).
To keep graphics and text together with minimal overhead in the highlevel code, a new autoControlTextBlocks mode has been added to PdfContentByte which starts and stops text objects automatically where needed and saves and restores a separate color set for texts.
This mode seems to have been chosen as means for support of tagged content in iText while it seems to not have been considered useful for other contexts. Thus, this mode now automatically is used for tagged files.

In my opinion this choice is not optimal. PdfContentByte is part of the publicly available iText API and it is publicly advertised (as "over content") for low-level tweaking of generated or pre-existing PDFs. Introducing such side-effects violates the API contract, at least it is a nuisance keeping people from upgrading.

Work-around

Simply switch the order of the color setting and text object starting operations, using

...
overContent.BeginText();
overContent.SetColorFill(BaseColor.RED);
...

results in

Resolution

If I interpret the final check-ins correctly, this issue should be fixed in iText version 5.5.6.

commit 301a45b57dcef37ae0ec3625fbdd6caaf4004a3a

Removed deprecated logic of saving and restoring color for tagged pdf documents in PdfContentByte class (DEV-1371).