yam213
Posts: 1
Joined: Thu Aug 09, 2018 3:18 pm

raspivid -save_pts and MMAL_TIME_UNKNOWN

Thu Aug 09, 2018 4:23 pm

Hello,

I am using save_pts options of raspivid, with Camera board V2.1, and it looks like lots of frames don't get timestamped.
I can see they get MMAL_TIME_UNKNOWN when I remove this raspivid test
below is a normal test shell output also with then a weird mkv generated

I would like to use these timestamps to synchronize camera frames with compass input eventually
Is this expected behavior or could I have a wrong configuration somewhere ?

Thanks !

Code: Select all

$ uname -rvm
4.14.56-1-ARCH #1 SMP Wed Jul 18 13:22:12 UTC 2018 armv7l
$ vcgencmd version
Jul 17 2018 13:28:45 
Copyright (c) 2012 Broadcom
version 4da89b2ba95c8826c492b1d734296c8a1a2c7d2b (clean) (release)
$ /opt/vc/bin/raspivid -pts ts -o test.h264 -v

raspivid Camera App v1.3.12

Width 1920, Height 1080, filename test.h264
bitrate 17000000, framerate 30, time delay 5000
H264 Profile high
H264 Level 4
H264 Quantisation level 0, Inline headers No
H264 Intra refresh type (null), period -1
Wait method : Simple capture
Initial state 'record'


Preview Yes, Full screen Yes
Preview window 0,0,1024,768
Opacity 255
Sharpness 0, Contrast 0, Brightness 50
Saturation 0, ISO 0, Video Stabilisation No, Exposure compensation 0
Exposure Mode 'auto', AWB Mode 'auto', Image Effect 'none'
Flicker Avoid Mode 'off'
Metering Mode 'average', Colour Effect Enabled No with U = 128, V = 128
Rotation 0, hflip No, vflip No
ROI x 0.000000, y 0.000000, w 1.000000 h 1.000000
Camera component done
Encoder component done
Starting component connection stage
Connecting camera preview port to preview input port
Starting video preview
Connecting camera video port to encoder input port
Opening output file "test.h264"
Opening output file "ts"
Enabling encoder output port
Starting video capture
Finished capture
Closing down
Close down completed, all components disconnected, disabled and destroyed
$ cat ts
# timecode format v2
0.000
33.326
66.653
99.979
133.306
166.639
199.960
233.290
266.614
1033.135
1999.621
2066.273
2099.599
2966.100
3999.240
$ ffprobe -v error -count_frames -show_entries stream=nb_read_frames test.h264 
[STREAM]
nb_read_frames=140
[/STREAM]
$ mkvmerge -o t.mkv --timecodes 0:ts test.h264 
mkvmerge v25.0.0 ('Prog Noir') 32-bit
'test.h264': Using the demultiplexer for the format 'AVC/h.264'.
'test.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
The file 't.mkv' has been opened for writing.
Warning: 'test.h264' track 0: The number of external timestamps 15 is smaller than the number of frames in this track. The remaining frames of this track might not be timestamped the way you intended them to be. mkvmerge might even crash.
Progress: 100%
The cue entries (the index) are being written...
Multiplexing took 1 second.

ethanol100
Posts: 537
Joined: Wed Oct 02, 2013 12:28 pm

Re: raspivid -save_pts and MMAL_TIME_UNKNOWN

Sat Aug 11, 2018 11:20 pm

I see the problem, too.
The reason for this is the following:
Usually the frames were all delivered in one buffer. In this case you get a buffer with flags:
p-frame: MMAL_BUFFER_HEADER_FLAG_NAL_END & MMAL_BUFFER_HEADER_FLAG_FRAME_END
i-frame: MMAL_BUFFER_HEADER_FLAG_NAL_END & MMAL_BUFFER_HEADER_FLAG_FRAME_END & MMAL_BUFFER_HEADER_FLAG_KEYFRAME

But if the size of the buffer exceeds 65536 it is split in two buffer,
first size 65536 with MMAL_BUFFER_HEADER_FLAG_NAL_END
second with the flags above.

Only the first buffer has a valid PTS, but because of the test

Code: Select all

              if(pData->pstate->save_pts &&
                  (buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END ||
                   buffer->flags == 0 ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME) &&
                  !(buffer->flags & MMAL_BUFFER_HEADER_FLAG_CONFIG))
               {
                  if(buffer->pts != MMAL_TIME_UNKNOWN && buffer->pts != pData->pstate->lasttime)
                  {
only the second buffer enters the if condition and is then filtered out by the second if.

I'm still not sure how to do this correctly, you can modify it to be:

Code: Select all

               if(pData->pstate->save_pts &&
                  (buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_NAL_END ||
                   buffer->flags == 0 ||
                   buffer->flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME) &&
                  !(buffer->flags & MMAL_BUFFER_HEADER_FLAG_CONFIG))
               {
                  if(buffer->pts != MMAL_TIME_UNKNOWN && buffer->pts != pData->pstate->lasttime)
                  {
This will at least store the correct number of frames in pts file.

I can't really remember, why I originally wrote the if condition like this. I think when I wrote it the first buffer would get a flag of 0 and would have been captured by "buffer->flags == 0" but at some point the buffer has changed to "buffer->flags = MMAL_BUFFER_HEADER_FLAG_NAL_END".

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 8 guests