FFMPEG: Making a specifc range of color pixels tra

Question

I am trying to overlay one video on top of another using ffmpeg, but couldn't quite understand the error. I based on the existing command from here

More specifically, I want to replace the all colors close to a specific color (say brown r:82,g:44,b:11), and then have them set as transparent.

ffmpeg -i moonmen.mp4 -i transparent_overlay.mp4 -filter_complex
"[1]split[m][a];
 [a]geq='if(between(r(X,Y), 77, 87)*between(g(X,Y), 39, 49)*between(b(X,Y), 06, 16) ,255:255:255,0:0:0)';
 [m][al]alphamerge[ovr];
 [0][ovr]overlay"
output.mp4

but I got error:

[Parsed_geq_1 @ 0x7fc8e2e08400] Either YCbCr or RGB but not both must be specified
[AVFilterGraph @ 0x7fc8e2e07c60] Error initializing filter 'geq' with args 'if(between(r(X,Y), 77, 87)*between(g(X,Y), 39, 49)*between(b(X,Y), 06, 16) ,255:255:255,0:0:0)'
Error initializing complex filters.
Invalid argument

Answer 1

The geq filter can work on both RGB and YUV format input, so one of the expressions has to be labeled to indicate that. But more importantly, your input may not be RGB, or have alpha, so that has to be fixed first.

ffmpeg -i moonmen.mp4 -i transparent_overlay.mp4 -filter_complex
"[1]format=rgba,geq=r='r(X,Y)':a='if(between(r(X,Y),77,87)*between(g(X,Y),39,49)*between(b(X,Y),6,16),0,255)'[ovr];
 [0][ovr]overlay"
output.mp4