gstreamer with multiple cameras: how can I split t

2019-08-12 17:47发布

问题:

I am trying to build a GStreamer pipeline which interleaves images from multiple cameras into a single data flow which can be passed through a neural network and then split into separate branches for sinking. I am successfully using the appsrc plugin and the Basler Pylon 5 - USB 3.0 API to create the interleaved feed. However, before I go through the work to write the neural network GStreamer element, I want to get the splitting working.

Currently, I am thinking of tagging the images with an "ID" indicating which camera it came from. Then I thought I could split the data flow using this tag. However, I have not been able to find any subject matter dealing with this issue exactly. I have seen that you can use tee plugin to branch the pipeline, but I haven't seen it used to split based on tags. Is it possible to use tee to do this?

I have seen people use tee to split a feed based on the source with something like this:

gst-launch-1.0 -vvv \
    tee name=splitter \
    $VSOURCE \
    ! $VIDEO_DECODE \
    ! $VIDEO_SINK splitter. \
    $VSOURCE1 \
    ! $VIDEO_DECODE \
    ! $VIDEO_SINK splitter.

However, this does not allow me to have a single path through the neural network element.

If it helps, here is a diagaram of the pipeline I envision:

cam1 ---\                                  /---> udpsink/appsink                              
         \                                / 
          appsrc-->neural_network-->tee---
         /                                \
cam2 ---/                                  \---> udpsink/appsink

回答1:

The tee element just forwards the same data to both branches. You should write another element which takes the input and only outputs the data of the stream you are interested in.

You should also place a queue element behind each branch to provide separate threads for each branch. I called the element to split up the streams camfilter which has a property id:

cam1 ---\                                  /---> queue --> camfilter id=1 --> udpsink/appsink                              
         \                                / 
          appsrc-->neural_network-->tee---
         /                                \
cam2 ---/                                  \---> queue --> camfilter id=2 --> udpsink/appsink