python picamera, gstreamer and twisted: the ultimate combo!

[packetservo]

Dear All,

I am using picamera python module to capture frames from RPi and then feed it to a gstreamer pipeline.

The working principle is to capture frames in a "custom output" object that has its write method. In this method, the buffer is pushed in to a gstreamer pipeline using an "appsrc" element. The code goes like this:

Code: Select all

class CamStreamer(object):
    def __init__(self,videoFormat,resolution,framerate):
        self.size = 0
        self.capstring = 'video/x-'+videoFormat+',width='+str(resolution[0]) \
                         +',height='+str(resolution[1])+',framerate=' \
                         +str(framerate)+'/1'
        self.playing = False
        self.paused = False
        self.make_pipeline()
        self.play_pipeline()

    def make_pipeline(self):
        CLI = [
                  'appsrc name="source" ! ',
                  'h264parse ! video/x-h264,stream-format=avc ! ',
                  'h264parse ! video/x-h264,stream-format=byte-stream ! ',
                  'mpegtsmux ! filesink name="sink"'
                  ]
        gcmd = "".join(CLI)
        self.pipeline = Gst.parse_launch(gcmd)
        self.filesink = self.pipeline.get_by_name("sink")
        self.filesink.set_property("location","custom.ts")
        self.appsrc = self.pipeline.get_by_name("source")
        self.appsrc.set_property("is-live",True)
        self.gstcaps = Gst.Caps.from_string(self.capstring)
        self.appsrc.set_property("caps",self.gstcaps)

    def play_pipeline(self):
        self.pipeline.set_state(Gst.State.PLAYING)

    def stop_pipeline(self):
        self.pipeline.set_state(Gst.State.READY)
        	    
    def write(self,s):
        gstbuff = Gst.Buffer.new_wrapped(s)
        ret = self.appsrc.emit("push-buffer",gstbuff)

    def flush(self):
        self.stop_pipeline()

and the RPi camera is accessed using another python class:

Code: Select all

class RPiCam(object):
    def __init__(self):
        self.resolution = (640,480)
        self.bitrate = 1000000
        self.format = "h264"
        self.framerate = 30
        self.rotation = 180
        
        self.camera = picamera.PiCamera()
        self.camera.resolution = self.resolution
        self.camera.framerate = self.framerate

    def startCam(self):
        self.camera.start_recording(CamStreamer(self.format,self.resolution,
                                                     self.framerate),format=self.format, bitrate=self.bitrate)

    def recordCam(self,duration):
        self.camera.wait_recording(duration)

    def stopCam(self):
        self.camera.stop_recording()

    def closeCam(self):
        self.camera.close()

I wish to use the gstreamer pipeline to stream a video over TCP at later stage. For this, there is no better choice than "twisted" module of python. So, my intention is to use twisted reactor thread to carry out capturing of RPi video.

the picamera is run in a background thread using threads.defertoThread

Code: Select all

def Record():
    myCam = RPiCam()
    myCam.startCam()
    myCam.recordCam(5)
    myCam.stopCam()
    myCam.closeCam()
    return True

def Complete(status):
    print "Recording completed {0}".format(status)

if __name__ == '__main__':
    d = threads.deferToThread(Record)
    d.addCallback(Complete)

    reactor.run()

Other necessary imports that are needed are:

Code: Select all

import gi
gi.require_version('Gst','1.0')
from gi.repository import GObject, Gst, GstVideo
GObject.threads_init()

#dt.datetime.now().strftime('%d-%b-%Y %H:%M:%S')

from twisted.internet import gireactor # for non-GUI apps
gireactor.install()
Gst.init(None)

from twisted.internet import reactor, threads
import picamera
import picamera.array

With this, I am able to capture the video frames using rpicamera and feed it successfully to a gstreamer pipeline.

And now come the question part.. I wish to generate a key frame (intra frame) request at a regular interval using mmal interface.

Any idea, how this can be achieved?

[tonywaite]

Hi,
I don't know the answer to your question, but thank you for posting the code.
I am very interested in understanding examples of 'threaded' picamera streams - and you've got the lot!
Good luck!

[jward]

The PiCamera function start_recording() supports the "intra_period" option.

[DvdZmd]

This code is a excelent example of what I was looking for.
But I have a problem when CamStreamer must create the pipeline
The code reaches this line and never respond anything

self.pipeline = Gst.parse_launch("gcmd")

I don't know what to debug