How to use pre-multiplied during image convolution

问题:

i'm trying to apply a box blur to an transparent image, and i'm getting a "dark halo" around the edges.

Jerry Huxtable has a short mention of the problem, and a very good demonstration showing the problem happen:

But i, for the life of me, cannot understand how "pre-multiplied alpha" can fix the problem. Now for a very simple example. i have a 3x3 image, containing one red and one green pixel:

In reality the remaining pixels are transparent:

Now we will apply a 3x3 Box Blur to the image. For simplicities sake, we'll only calculate the new value of the center pixel. The way a box blur works is that since we have a 9 position square (3x3, called the kernel) we take 1/9th of each pixels in the kernel, and add it up:

So

finalRed =   1/9 * red1 + 1/9 * red2 + 1/9 * red3+ ... + 1/9 * red9
finalGreen = 1/9*green1 + 1/9*green2 + 1/9*green3+ ... + 1/9*green9
finalBlue =  1/9* blue1 + 1/9* blue2 + 1/9* blue3+ ... + 1/9* blue9
finalAlpha = 1/9*alpha1 + 1/9*alpha2 + 1/9*alpha3+ ... + 1/9*alpha9

In this very simplified example, the calculations become very simple:

finalRed =   1/9 * 255
finalGreen = 1/9 * 255
finalBlue =  0
finalAlpha = 1/9*255 + 1/9*255

This gives me a final color value of:

finalRed =   28
finalGreen = 28
finalBlue =  0
finalAlpha = 56 (22.2%)

This color is too dark. When i perform a 3px Box blur on the same 3x3 pixel image in Photoshop, i get what i expect:

Which is clearer when displayed over white:

---

In reality i'm performing a box blur on a bitmap containing transparent text, and the text gains the tell-tale dark around the fringes:

i'm starting with a GDI+ Bitmap that is in PixelFormat32bppARGB format

---

How do i use "pre-multiplied alpha" when applying 3x3 convolution kernel?

Any answer will have to include new forumla, since:

final = 1/9*(pixel1+pixel2+pixel3...+pixel9)

Is getting me the wrong answer.

---

Edit: A simpler example is:

i'll perform this math with color and alpha values in the range of 0..1:

i'm going to apply the box blur convolution filter to the middle pixel:

ARGB'
      = 1/9 * (0,1,0,1) + 1/9 * (0,0,0,0) + 1/9 * (0,0,0,0) + 
        1/9 * (0,1,0,1) + 1/9 * (0,0,0,0) + 1/9 * (0,0,0,0) + 
        1/9 * (0,1,0,1) + 1/9 * (0,0,0,0) + 1/9 * (0,0,0,0);

      = (0, 0.11, 0, 0.11) + (0,0,0,0) + (0,0,0,0) +
        (0, 0.11, 0, 0.11) + (0,0,0,0) + (0,0,0,0) +
        (0, 0.11, 0, 0.11) + (0,0,0,0) + (0,0,0,0)

      = (0, 0.33, 0, 0.33)

Which gives a fairly transparent dark green.

Which is not what i expect to see. And by comparison Photoshop's Box Blur is:

If i assume (0, 0.33, 0, 0.33) is pre-multiplied alpha, and un-multiply it, i get:

(0, 1, 0, 0.33)

Which looks right for my all-opaque example; but i don't know what to do when i begin to involve partially transparent pixels.

See also

• Texture filtering: Alpha cutouts
• Premultiplied alpha

回答1:

tkerwin has already provided the correct answer, but it seems to need further explanation.

The math you've shown in your question is absolutely correct, right up until the end. It is there that you're missing a step - the results are still in a pre-multiplied alpha mode, and must be "unmultiplied" back to the PixelFormat32bppARGB format. The opposite of a multiply is a divide, thus:

finalRed = finalRed * 255 / finalAlpha;
finalGreen = finalGreen * 255 / finalAlpha;
finalBlue = finalBlue * 255 / finalAlpha;

You've expressed a concern that the divide might create a result that is wildly out of range, but that won't happen. If you trace the math, you'll notice that the red, green, and blue values can't be greater than the alpha value, because of the pre-multiplication step. If you were using a more complicated filter than a simple box blur it might be a possibility, but that would be the case even if you weren't using alpha! The correct response is to clamp the result, turning negative numbers into 0 and anything greater than 255 into 255.

回答2:

Following the advice in your link, you pre-multiply before blurring and un-pre-multiply after blurring. In your example, pre-multiplying actually does nothing, since there are no semi-transparent pixels. You did the blur, then you need you un-pre-multiply by doing (assuming normalized color values from 0 to 1):

RGB' = RGB/A  (if A is > 0)
A' = A

This will get you a non-pre-multiplied blurred final image.

回答3:

Correct Math, Wrong Operation

Both answers appear wrong here, and you got the math right in your example, just not the over operation.

Use the proper blending formula, which according to Porter Duff is:

FG.RGB + (1.0 - FG.Alpha)*BG.RGB

Not sure where the rest of the answers are coming from, but wow.

The alpha encoding dictates the over operation.

When Properly Encoded, RGBA Represents Emission and Occlusion

The reason that you must have associated alpha in an image is because the RGB values always represent the emission. In this case, the RGB represents the emission of pixel, while the alpha represents the degree of occlusion.

That is, if you linearly interpolate, much like your box blur, the values must be indicative of the emissions. In the most simple case, consider a pixel emitting 100% and we linearly interpolate down by 25% at each increment.

Here, the emission for each RGB should decrease by 25%, such that we'd have 100%, 75%, 50%, 25%, and finally 0%. Note that at each increment the emission would scale with the degree of occlusion, such that the alpha would remain perfectly in sync at the same values. Simply apply your convolution across RGBA as one unit of emission and occlusion. Most importantly, the blending formula must be the first one in this answer.

Now consider the unassociated alpha case, which isn't actually encoded at all. Here the emission is completely unassociated with the degree of occlusion, so the over operation bundles that scaling of the emission up into the first step, where the unassociated emission is multiplied by the degree of occlusion.

FG.RGB * FG.A + (1.0 - FG.A) BG.RGB

But what happens if someone foolishly tries to perform the exact same linear interpolation, rotation, blur, smudge, or any other manipulation on such completely not-encoded values? Well let's see...

In the unassociated alpha example, our emission is 100%, yet our degree of occlusion is 75%. Now interpolate down by 25%. Notice how the fundamental math of light transport fails here. We would end up with the alpha representation completely out-of-sync with the emission. 100%RGB-75%A, 75%RGB-56.25%A, 50%RGB-28.125%A, etc.

But Wait, There's More Darkness...

The final piece of the puzzle is that you are using nonlinearly encoded display referred values. That is, when using values encoded for an sRGB display, they are nonlinearly compressed, which in turn is undone by the display's EOTF back to linear light output from the display. That means that our code values do not represent the radiometric-like ratios that are emitted from the display. As such, the linear math of emission and occlusion will fail, in much the same way performing audio math on compressed MP3 values will fail.

So in our simple linear interpolation example, we can see that the math fundamentally falls apart quickly.

Our first increment would be 100% emission, and our second would be 75%, and so on. But if we linearly interpolate down from 100% to 25% using sRGB nonlinearly encoded values, the actual emitted amount of light is disconnected from even this simple math, and our alpha will fringe in a very similar manner to how the unassociated alpha encoding fell apart. The only way around this is to make our process three steps:

1. Undo the nonlinearly compressed sRGB transfer function on our pixels.
2. Perform our math.
3. Reapply the nonlinear sRGB encoding according to the destination transfer function, in this case sRGB.

Right Answer, Wrong Over Formula

Again though, your box blur math was correct enough in the first instance, with your semi-transparent green being at 33% emission and the alpha occlusion being 33%. The fundamental problem was he over operation, which would be the above formula. That is, in the case of your green over full emission background:

  FG.RGB + (1.0 - FG.A) * BG.RGB
  [0.0 0.33 0.0] + ((1.0 - 0.33) * [1.0 1.0 1.0])
  [0.0 0.33 0.0] + [0.77 0.77 0.77]
  [0.77 1.0 0.77]

Note however that if you were to composite your example over a solid 100% sRGB red emission that even with the proper math, you'd get a darker result due to the non-radiometric encoded values. Your choice of sRGB green and red are perfect examples that will exacerbate the nonlinear math darkening that will happen, especially if you perform the blur on the red object over top of a fully emissive green background. The following is composited entirely correctly, albeit using the nonlinearly compressed sRGB code values for the math:

Additional Reading

A few relevant quotes. First from Zap Andersson of Autodesk in the infamous Adobe Alpha thread, confirmed by Alvy Ray Smith, the creator of the alpha channel:

The error here is that Chris is interpreting the word "Premultiplied" literally. Many people do, incorrectly. I really hate the word, actually, because it leads you to believe it's literal meaning, that something has been multiplied with something "before" (I.E. "Pre" something). That is wrong. It is a total misrepresentation of what "Premultiplied" means.

Academy Scientific and Technical Achievement winner Larry Gritz:

At this point, I'm going to go on a bit of a religious rant and just say that associated alpha (premultiplied color) is the only choice that makes sense. It's the only thing that renderers are likely to produce naturally, it's the only way that compositing and any other image processing math works, and in my opinion it's the only sensible form to store any image.

It is also worth noting that associated alpha can express RGBA values that unassociated alpha (un-premultipled color) CANNOT. It's just a fact of life

(An example of what Mr. Gritz is referring to here by his CANNOT statement is the case of something like a flame or a reflection, which has no occlusion and only emission, or RGB with zero Alpha. This is fundamentally impossible to express using an unassociated alpha operation.)

Academy Scientific and Technical Achievement winner Jeremy Selan:

First off, I'm 'pro' premultiplication. To the best of my understanding, premultiplication is the natural representation of pixels with alpha; the big confusing thing about it is the name.