Getting Bitmap from Video decoded with Nestream Ap

Question

I am wondering if someone who has handled NetStream.appendBytes in Flash knows how to get the bitmapData from a decoded video frame? I have already looked at this question but that is from 3 years ago and the more recent comment/answer seems to be gone. In 2014 has anyone managed to turn those bytes into a bitmap? I am working with Flash Player 11.8 and this is not a desktop/AIR app.

In the image below I can do steps 1) and 2) but there's a brick wall at step 3)

The problem is that simply using bitmapdata.draw(video_container); does not work but instead it throws a Policy File error even though I am using a byteArray (from local video file in the same directory as the SWF). No internet is even involved but Flash tells me that "No Policy File granted permission from the server" or such nonsense. I think the error is just a bail-out insteading of straight up saying "You are not allowed to do this.."

I have tried: trying to appease this Crossdomain.xml issue anyway and looking into all known security/domain settings. I came to the conclusion that the error is not the problem but a side effect of the issue.. The issue here being that: Flash Player is aware of the SWF's location and of any files local to it. That's okay when you pass a String as URL etc but when the Netstream data is not local to the SWF domain then it becomes a Policy File issue. Problem is my data is in the Memory not in a folder like the SWF and therefore cannot alllow bitmapData.draw since it cannot "police" an array of bytes, any known fixes for this?... (I can't even say the words I really wanted to use).

What I am trying to achieve: Is to essentially use Netstream as an H.263 or H.264 image decoder in the same way Loader is a JPEG-to-Bitmap decoder or LoadCompressed.. is an MP3-to-PCM decoder. You know, access the raw material (here RGB pixels), apply some effects functions and then send to screen or save to disk.

Answer 1

I know it is a little late, but I think I found a solution for your problem.

So, to avoid the Security sandbox violation #2123 Error, you have just to do like this :

// ...

net_stream.play(null);

net_stream.play('');

// ...

Hope that can help.

Answer 2

I know this question is a couple months old, but I wanted to post the correct answer (because I just had this problem as well and other will too).

Correct answer:

It's a bug that has been open at adobe for almost 2 years

Link to the bug on Adobe

Work Around until the bug gets fixed (I am using this and it works great):

Workaround using Sprite and graphics

Answer 3

To take a snapshot from a video stream we don't need NetStream.appendBytes which inject data into a NetStream object.

For that we can use BitmapData.draw which has some security constraints. That's why in many times we get a flash security error. About that, Adobe said :

"... This method is supported over RTMP in Flash Player 9.0.115.0 and later and in Adobe AIR. You can control access to streams on Flash Media Server in a server-side script. For more information, see the Client.audioSampleAccess and Client.videoSampleAccess properties in Server-Side ActionScript Language Reference for Adobe Flash Media Server. If the source object and (in the case of a Sprite or MovieClip object) all of its child objects do not come from the same domain as the caller, or are not in a content that is accessible to the caller by having called the Security.allowDomain() method, a call to the draw() throws a SecurityError exception. This restriction does not apply to AIR content in the application security sandbox. ...".

For crossdomain file creation and some other security config for AMS server, you can take a look on this post : Crossdomain Video Snapshot - Fixing BitmapData.draw() Security Sandbox Violation.

After allowing our script to get data from our video stream, we can pass to the code.

I wrote a code that play a video stream ( rtmp or http ) and take a snapshot to show it in the stage or save it as a file after applying a pixel effect :

const server:String = null; //'rtmp://localhost/vod'
const stream:String = 'stream'; // 'mp4:big_buck_bunny_480p_h264.mp4';
var nc:NetConnection;
var ns:NetStream;
var video:Video;
const jpg_quality:int = 80;
const px_size:int = 10;

nc = new NetConnection();
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR, function(e:AsyncErrorEvent):void{});
nc.addEventListener(NetStatusEvent.NET_STATUS, function(e:NetStatusEvent):void{
    if(e.info.code == 'NetConnection.Connect.Success'){
        ns = new NetStream(nc);
        ns.addEventListener(NetStatusEvent.NET_STATUS, function(e:NetStatusEvent):void{});
        ns.addEventListener(AsyncErrorEvent.ASYNC_ERROR, function(e:AsyncErrorEvent):void{});
        video = new Video(320, 180);
        video.x = video.y = 10;
        video.attachNetStream(ns);
        addChild(video);
        ns.play(stream);
    }
})
nc.connect(server);

btn_show.addEventListener(
    MouseEvent.CLICK,
    function(e:MouseEvent): void{
        var bmp:Bitmap = pixelate(video, px_size);
            bmp.x = 10;
            bmp.y = 220;
        addChild(bmp);
    }
)
btn_save.addEventListener(
    MouseEvent.CLICK,
    function(e:MouseEvent): void{

        var bmp:Bitmap = pixelate(video, px_size);

        var jpg_encoder:JPGEncoder = new JPGEncoder(80);
        var jpg_stream:ByteArray = jpg_encoder.encode(bmp.bitmapData);

        var file:FileReference = new FileReference();
            file.save(jpg_stream, 'snapshot_'+int(ns.time)+'.jpg');

    }
)

function pixelate(target:DisplayObject, px_size:uint):Bitmap {

    var i:uint, j:uint = 0;
    var s:uint = px_size;
    var d:DisplayObject = target;

    var w:uint = d.width;
    var h:uint = d.height;

    var bmd_src:BitmapData = new BitmapData(w, h);
        bmd_src.draw(d);

    var bmd_final:BitmapData = new BitmapData(w, h);

    var rec:Rectangle = new Rectangle();
        rec.width = rec.height = s;

    for (i = 0; i < w; i += s){
        for (j = 0; j < h; j += s){
            rec.x = i;
            rec.y = j;                  
            bmd_final.fillRect(rec, bmd_src.getPixel32(i, j));
        }
    }

    bmd_src.dispose();
    bmd_src = null;

    return new Bitmap(bmd_final);
}

Of course, this is just a simple example to show the manner to get a snapshot from a video stream, you should adapt and improve it to your needs ...

I hope all that can help you.