Output of ffmpeg into HDFS directly

2019-05-29 00:22发布

问题:

I have a requirement in which I have to convert a RTSP stream into mp4 video or frames (as the case may be) & want to save them in HDFS (Hadoop Filesystem).

For this I have tried using ffmpeg to convert the RTSP stream into mp4 video/frames & saving the video/frames in local filesystem. Like below -
ffmpeg -i rtsp://10.0.37.150:8554/big_bunny.mkv -r 1 -f image2 frames/big_frame-%3d.bmp
ffmpeg -i rtsp://10.0.37.150:8554/big_bunny.mkv big_bunny.mp4

And then saving the video/frames (stored in local filesystem) into hdfs using put command -
hadoop fs -put frames/ /user/maddy/
hadoop fs -put big_bunny.mp4 /user/maddy/

This is working. But I want to do this directly (that is in one step without saving the required files in local filesystem)

I tried something (like below command) but it is not working
ffmpeg -i rtsp://10.0.37.150:8554/big_bunny.mkv hdfs://localhost:9000/user/maddy/big_bunny.mp4

I get this error -
hdfs://localhost:9000/user/maddy/big_bunny.mp4: Protocol not found

So is there any way in which using ffmpeg I can directly save these files into hdfs without first saving in my local filesystem.
Or is there any other tool through which I can achieve this?

EDIT :

Tried ffmpeg -i rtsp://10.0.37.150:8554/big_bunny.mkv - | hadoop fs -put - /user/maddy/
as suggested by @incBrain

But received this error -
[NULL @ 0xce37a0] Unable to find a suitable output format for 'pipe:' pipe:: Invalid argument

回答1:

You can redirect the output of ffmpeg to a pipe and use hadoop fs -put - /user/... to read the input from stdin like this:

ffmpeg -i rtsp://10.0.37.150:8554/test.mkv -f avi - | hadoop fs -put - /user/maddy/test.avi

Note that you will need -f <format> option since ffmpeg can not guess the output format from the file extension since we use our pipe.