HTML5 video draw into canvas2D issue on Android Ch

2019-09-05 00:02发布

站内问答 / JavaScript
1424 1 6

I am developing VR html5 page in javascript (no jQuery or other frameworks) that uses WebGL to render a sphere and has texture to which streamed video is rendered into.

All that works fine on iPhone 6, 6+, however on Android I've hit the wall - simply, video is not being transferred into texture (gl.texSubImage2D). Texture remains black. No WebGL errors thrown.

So I have created a test, without WebGL, that just tries to play a video and draw it's frames into canvas2D, so I can at least verify that frames are indeed extracted from streamed video.

Video is played on user interaction (touch) and when canplay event is triggered, I start frame loop (window.requestAnimationFrame) to draw video into canvas (canvas.drawImage( video, 0,0 ))

Both video and canvas are on the document body visible, next to each other.

Result: On desktop it works as expected, two screens, left is video with native controls, right is canvas. When I click play, video starts, and canvas gets refreshed at the same time. On Android Chrome 48.0.2564.106 no pixels drawn - canvas is totally empty.

I have installed Android Chrome Beta (50.0.2661.57) and it works there, but on Android Chrome 48.0.2564.106 it does not.

Codes to setup video and canvas:

<!doctype html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="format-detection" content="telephone-no" />
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
    <title>Video Canvas Test</title>
    <script>

    var video;
    var canvas;
    var ctx;
    var info;
    var pass = 0;

    window.onload = function()
    {
        video = document.createElement("video");
        video.oncanplay = function(){ initializeCanvas(); }
        video.onerror = function(e){ console.error("video problem"); }
        video.loop = true;
        video.controls = "true";
        video.src = "video/big-buck-bunny_trailer.webm";
        video.style.width = "400px";
        video.style.height = "300px";
        video.style.position = "absolute";
        video.style.left = "20px";
        video.style.top = "20px";
        video.style.backgroundColor = "#8080FF";

        canvas = document.createElement("canvas");
        canvas.style.backgroundColor = "#8080FF";
        canvas.style.width = "400px";
        canvas.style.height = "300px";
        canvas.style.position = "absolute";
        canvas.style.left = "420px";
        canvas.style.top = "20px";

        ctx = canvas.getContext("2d");

        info = document.createElement("p");
        info.innerHTML = window.navigator.userAgent;
        info.style.position = "absolute";
        info.style.width = "200px";
        info.style.height = "auto";
        info.style.position = "absolute";
        info.style.left = "20px";
        info.style.top = "320px";

        document.body.appendChild(video);
        document.body.appendChild(canvas);
        document.body.appendChild(info);
    }

    function initializeCanvas()
    {
        console.log("Video ready to play. Init canvas.");
        ctx.canvas.width = 640; // I am 100% sure this is correct video size.
        ctx.canvas.height = 360;
        console.log("Video size: " + video.videoWidth + "x" + video.videoHeight);
        ctx.font = "30px Arial";
        updateCanvas();
    }

    function updateCanvas()
    {
        pass ++;

        ctx.clearRect(0,0,ctx.canvas.width,ctx.canvas.height);
        ctx.drawImage(video,0,0);
        ctx.fillText("Pass: " + pass,30,120);

        window.requestAnimationFrame( updateCanvas );
    }
    </script>
</head>
<body>
</body>
</html>

Can anyone confirm if canvas.drawImage on Chrome 48 and older cannot accept video as drawing source? This looks impossible to me considering WebGL has been in Chrome's support for ages and with plenty of experiments with video textures out there.

I have tried other samples that copy video into WebGL texture or Canvas2D and they do not work as well.

Is there anything I am missing here?

1条回答
再贱就再见
2楼-- · 2019-09-05 00:27

After thorough research it appears that:

  1. Canvas2D.drawImage( video, 0,0 ) was broken since July 2015 (at least as I found bug reports that date from then) in Android Chrome and was kinda fixed-not-fixed-fixed-again-and-again-not-fixed - but I can confirm that it is fixed in Android Chrome 49 and in Android Chrome 50 (Beta). Same issue with decoding pixels from video to be drawn into Canvas2D has affected drawing video into WebGL texture.

  2. The only workaround is to have either: a) custom-made video streaming server using Websocket + JPG sequence streamer in JavaScript (images received through opened websocket connection) or b) JPG sequence streamer in JavaScript (downloads images one by one and takes care of loading new - deleting old).

I went with option 2.b and with CDN kicked in it really works great for 1024x512 video size, which was good enough solution.

I would first preload mp3 file using http request, then load it into Audio object. Once that's done, JPG streamer would kick in and would then commence playback using sound currentTime value. It was 100% synchronised with sound!

However, issue is fixed in Chrome 49, and Chrome 50 (beta), so this likely will be obsolete in 2-3 months.

查看更多
0人赞 添加讨论(0) 举报
登录 后发表回答
相关问题
查看全部
相关文章
查看全部
收藏的人(6)