Posts: 1
Joined: Thu Aug 09, 2018 3:18 pm

raspivid -save_pts and MMAL_TIME_UNKNOWN

Thu Aug 09, 2018 4:23 pm


I am using save_pts options of raspivid, with Camera board V2.1, and it looks like lots of frames don't get timestamped.
I can see they get MMAL_TIME_UNKNOWN when I remove this raspivid test
below is a normal test shell output also with then a weird mkv generated

I would like to use these timestamps to synchronize camera frames with compass input eventually
Is this expected behavior or could I have a wrong configuration somewhere ?

Thanks !

Code: Select all

$ uname -rvm
4.14.56-1-ARCH #1 SMP Wed Jul 18 13:22:12 UTC 2018 armv7l
$ vcgencmd version
Jul 17 2018 13:28:45 
Copyright (c) 2012 Broadcom
version 4da89b2ba95c8826c492b1d734296c8a1a2c7d2b (clean) (release)
$ /opt/vc/bin/raspivid -pts ts -o test.h264 -v

raspivid Camera App v1.3.12

Width 1920, Height 1080, filename test.h264
bitrate 17000000, framerate 30, time delay 5000
H264 Profile high
H264 Level 4
H264 Quantisation level 0, Inline headers No
H264 Intra refresh type (null), period -1
Wait method : Simple capture
Initial state 'record'

Preview Yes, Full screen Yes
Preview window 0,0,1024,768
Opacity 255
Sharpness 0, Contrast 0, Brightness 50
Saturation 0, ISO 0, Video Stabilisation No, Exposure compensation 0
Exposure Mode 'auto', AWB Mode 'auto', Image Effect 'none'
Flicker Avoid Mode 'off'
Metering Mode 'average', Colour Effect Enabled No with U = 128, V = 128
Rotation 0, hflip No, vflip No
ROI x 0.000000, y 0.000000, w 1.000000 h 1.000000
Camera component done
Encoder component done
Starting component connection stage
Connecting camera preview port to preview input port
Starting video preview
Connecting camera video port to encoder input port
Opening output file "test.h264"
Opening output file "ts"
Enabling encoder output port
Starting video capture
Finished capture
Closing down
Close down completed, all components disconnected, disabled and destroyed
$ cat ts
# timecode format v2
$ ffprobe -v error -count_frames -show_entries stream=nb_read_frames test.h264 
$ mkvmerge -o t.mkv --timecodes 0:ts test.h264 
mkvmerge v25.0.0 ('Prog Noir') 32-bit
'test.h264': Using the demultiplexer for the format 'AVC/h.264'.
'test.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
The file 't.mkv' has been opened for writing.
Warning: 'test.h264' track 0: The number of external timestamps 15 is smaller than the number of frames in this track. The remaining frames of this track might not be timestamped the way you intended them to be. mkvmerge might even crash.
Progress: 100%
The cue entries (the index) are being written...
Multiplexing took 1 second.

Posts: 569
Joined: Wed Oct 02, 2013 12:28 pm

Re: raspivid -save_pts and MMAL_TIME_UNKNOWN

Sat Aug 11, 2018 11:20 pm

I see the problem, too.
The reason for this is the following:
Usually the frames were all delivered in one buffer. In this case you get a buffer with flags:

But if the size of the buffer exceeds 65536 it is split in two buffer,
first size 65536 with MMAL_BUFFER_HEADER_FLAG_NAL_END
second with the flags above.

Only the first buffer has a valid PTS, but because of the test

Code: Select all

              if(pData->pstate->save_pts &&
                  (buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END ||
                   buffer->flags == 0 ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME) &&
                  !(buffer->flags & MMAL_BUFFER_HEADER_FLAG_CONFIG))
                  if(buffer->pts != MMAL_TIME_UNKNOWN && buffer->pts != pData->pstate->lasttime)
only the second buffer enters the if condition and is then filtered out by the second if.

I'm still not sure how to do this correctly, you can modify it to be:

Code: Select all

               if(pData->pstate->save_pts &&
                  (buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_NAL_END ||
                   buffer->flags == 0 ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME) &&
                  !(buffer->flags & MMAL_BUFFER_HEADER_FLAG_CONFIG))
                  if(buffer->pts != MMAL_TIME_UNKNOWN && buffer->pts != pData->pstate->lasttime)
This will at least store the correct number of frames in pts file.

I can't really remember, why I originally wrote the if condition like this. I think when I wrote it the first buffer would get a flag of 0 and would have been captured by "buffer->flags == 0" but at some point the buffer has changed to "buffer->flags = MMAL_BUFFER_HEADER_FLAG_NAL_END".

Return to “Camera board”