DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Fri Jun 19, 2020 10:16 am

Hi,

The trouble is the firmware drivers are the only ones that work with RaspiVid and RaspiStill. I would use my older software based on your yavta example (using dtoverlay drviers which are much more preferable) but I couldn't get it to pipe out the video to my rtsp application. If I could get it to do that then all my issues are solved and I'm good to go.

I also found out that building the userland tree with ./buildme --debug didn't work and I had to add the "all" option for it to build. Then I noticed after a lot of time that my builds weren't applying the changes I'd made. Also the breakpoint was always bypassed no matter what I did and after googling I was none the wiser. So debugging on the whole didn't help which is a real pity and I was left with adding in checks and fprintf's to stderr.

Is there anyway I can easily get the v4l2_mmal example to pipe out to stdout to my rtsp test application? It's located here by the way: https://github.com/danriches/RaspiVid and the Makefile is for the test rtsp application which would be located in the live folder in home which is the current stable repo from LiveMedia. Works great with RaspiVid, just not v4l2_mmal...

BTW: RaspiStill works with the TC358743 too, I ran raspistill -o test.jpg -q 100 -f -k and when I press enter I get a snapshot. So it all will work once this problem is sorted out. I've compared the mmal debug output and can't see any differences except when I press enter on raspistill. I've corrected the encoder pool and now I don't get a seg fault but it all just freezes until I press ctrl+c. Argh!!

Hope you're keeping well and thank you for your responses.

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 22, 2020 9:15 am

Hi 6by9,

I noticed that the image filename handle to capture to wasn't set correctly so I sorted that out and now I'm getting an error when trying to send a buffer to the encoder output port as seen in the code below a few lines from the end where it calls mmal_port_send_buffer(), any ideas??:

Code: Select all

//Open file - set the jpg_filename, make sure to reserver memory using malloc
         char jpg[20] = "/home/pi/test.jpg";
         int lenf = strlen(jpg);
         state.jpg_filename = malloc(lenf+1);
         vcos_assert(state.jpg_filename);
         if(state.jpg_filename)
            strncpy(state.jpg_filename, jpg, lenf);
         fprintf(stderr, "Filename is: %s", state.jpg_filename);
         state.callback_data_image.file_handle = open_filename(&state, state.jpg_filename);

         fprintf(stderr, "Setting up buffers for image encoder callback\n");
         if (state.callback_data_image.file_handle)
         {
            //setup the buffers for the stills image encoder
            int num = mmal_queue_length(state.encoder_pool_image->queue);
            if (state.common_settings.verbose)
               fprintf(stderr, "Number of queued buffers %d\n", num);
            int q;
            for (q = 0; q < num; q++)
            {
               MMAL_BUFFER_HEADER_T *buffer = mmal_queue_get(state.encoder_pool_image->queue);

               if (!buffer)
                  vcos_log_error("Unable to get a required still buffer %d from pool queue", q);
               
               if (mmal_port_send_buffer(encoder_output_port_image, buffer)!= MMAL_SUCCESS)
                  vcos_log_error("Unable to send a buffer to stills encoder output port (%d)", q);
               else
                  fprintf(stderr, "Created buffer pool for still image encoder\n");
            }
         }
Kind Regards,

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 22, 2020 9:38 am

Hi 6by9,

I finally have it working!! My problems were down to a file handle not being set and sending the buffers to the image encoder before enabling it. Once I'd reversed that it just worked!

Thank you , thank you, thank you for all your help!!

Kind Regards,

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 22, 2020 2:33 pm

Ok, it seems that if there is a video output capture set in RaspiVid args then the image capture fails. Ie if I use the following line:

Code: Select all

raspivid -w 1920 -h 1080 -fps 50 -n -b 6000000 -t 0
then a 10 second timeout sets the capture parameter and I get a 1.2Mb jpeg, fantastic! However, if I use this:

Code: Select all

raspivid -w 1920 -h 1080 -fps 50 -n -b 6000000 -t 0 -o - | ./streamserver
I get nothing but an empty zero byte jpg file (I can omit the pipe and keep the "-o -" to output to stdout, same happens). Also the same regardless of preview on or off...

So should I pause the video stream (maybe disable video output port on camera), take capture, resume (enable video output port on camera) or is there another way I should be handling this? It's only happening if I set RaspiVid to output to stdout and I've turned off all my debug output too.

Kind Regards,

Dan
BTW: I've updated the github code to show where I'm at so far... (github/danriches/RaspiVid)

EDIT: I think the code isn't getting to the part that sends the capture parameter... So maybe ignore me for the moment, or comment if you like....

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Tue Jun 23, 2020 8:47 am

Hi 6by9 and anyone that can assist,

Right, I have a big problem with this at the moment. I've managed to locate where the code loops normally when capturing video in "running = wait_for_next_change(&state);" so I changed the code to ignore that line and removed the "vcos_sleep(ABORT_INTERVAL);" and placed a loop that checks for the gpio pin. At the moment I've left out the gpio check and currently it loops around until 20 seconds have elapsed and then triggers a single frame capture. Line 3504 is the start of the while(1) loop.

The debug mmal output shows everything is set up correctly and after 20 seconds the code freezes until I connect to my streaming server application. Only after this is a capture done and then the stream output freezes even though the code is still in it's loop and I can't reconnect to the stream, there are no mmal buffer messages either. I've placed the code on github here: https://github.com/danriches/RaspiVid

Should I be pausing the video output somehow whilst capturing and then unpausing it so the video stream resumes on stdout? I'm using this command to start everything off: "raspivid -w 1920 -h 1080 -fps 50 -n -b 6000000 -t 0 -v -o - | ~/live/raspi/testRaspi"

Please help!!

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Reading single frames piped from Raspivid in C/C++

Tue Jun 23, 2020 9:48 am

Video encode (and preview) will pause automatically when you capture a still.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Tue Jun 23, 2020 10:10 am

Hi 6by9,

They do indeed pause but they never resume after the capture. Should I be setting a flag to pause the video, capture image and then resume after? Or should I be resetting the flag for the image capture once done? EDIT: Now tried both of these and no joy. I was writing to the sd card and then changed to the ramdisk but again no joy the same output which is that it looks like the video port resumes but only briefly before falling over!

raspivid -w 1920 -h 1080 -fps 50 -n -b 6000000 -t 0 -v -o -
does the same thing as my original command:
raspivid -w 1920 -h 1080 -fps 50 -n -b 6000000 -t 0 -v -o - | ~/live/raspi/testRaspi
in that you get the 5 or 6 lines of mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324) which i take to be the camera -> video encoder mmal messages.
So it's not my testRaspi RTSP stream application at fault, it's something to do with the video processing...

Currently the image capture won't trigger until the stdout stream is being consumed but we can ignore that for the moment as I could always locally consume it. However, I start vlc up on my pc to watch the stream straight after starting RaspiVid and then wait 20 seconds for the image capture to trigger, which it does but straight after the capture is complete everything freezes. Debugging properly doesn't give me anything at all so I'm stuck with using fprintf to stderr. This is what I get, any ideas or pointers of where to look would be great:

Code: Select all

mmal: mmal_port_enable: vc.ril.image_encode:out:0(JPEG) port 0x155b140, cb 0x1a838, buffers (1/1/1,81920/81920/81920)
Enable Done
Stills encoder output port enabled
Setting up buffers for image encoder callback
Number of queued buffers 1
Created buffer pool for still image encoder
Start to setup buffers for video callback
Setup buffers for video callback done
Number of queued buffers 1
Created buffer pool for video encoder port
bCapturing flag 1
mmal: mmal_port_parameter_set: vc.ril.camera(3:1) port 0x1544880, param 0x7ee9626c (10011,12)
Starting video capture
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
Starting capture 0
mmal: mmal_port_parameter_set: vc.ril.camera(3:1) port 0x1544880, param 0x7ee9626c (10011,12)
mmal: mmal_port_parameter_set: vc.ril.camera(3:2) port 0x1544b90, param 0x7ee9626c (10011,12)
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 81920
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
Image Callback - Got port userdata!
Image Callback - if pData!
Image Callback - buffer->length 37990
Image Callback - got here 1!
Image Callback - got here 2!
Image Callback - Should have written a file by now
Image Callback - set complete flag
Image Callback - release buffer header
Image Callback - create new buffer
Image Callback - got new buffer
mmal: mmal_port_parameter_set: vc.ril.camera(3:1) port 0x1544880, param 0x7ee9626c (10011,12)
Finished capture 0
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)
mmal: mmal_port_parameter_set: vc.ril.camera(1:0) port 0x1543e90, param 0x73af9bd8 (10049,324)

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Wed Jun 24, 2020 8:12 am

I really don't understand how the video just works for ages but then dies after I take a still capture, not during but after! I'm really really stuck at this point. Should I be looking at some parameter settings? The trouble is those have zero documentation though so I'm shooting in the dark. Is it buffers? I have let the code use the "recommended" buffers but I could increase them maybe? I have tried setting the cma setting to 96m and 128m with no change at all.

I've updated my code on github if anyone would be kind enough to take a peek and say "hey you've not got enough buffers" or "you've not done this" etc. I'll be updating the application on github with a working version if I ever get there for others to use...

I've run this command: sudo vcdbg log msg and I get:

Code: Select all

3996760.241: mmal: mmal_vll_load: could not load VLL 'videnc.vll': 
4026902.395: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
4026904.808: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
What does this mean?

I've also seen your post about setting the logging level to 0xc0 and this is what I get running "sudo vcdbg log msg" after running the app:

Just the last few lines or so which I hope is useful, but the odd thing is the seg fault at the end. Is this the videocore crashing??

Code: Select all

3753695.455: video_encode:54:RIL: Callback request for 76768084596
3753695.470: is_eos:0 valid_lines:1080
3753695.482: free Q	 rc:527 wc:643
3753695.494: input Q	 rc:524 wc:527
3753695.508: sent Q	 rc:523 wc:523
3753695.520: job obtained:3f8df7d4
3753695.530: updated free Q	 rc:528 wc:643
3753695.549: writing job 3f8df7d4 to input Q
3753695.564: isNewFrame:1 isPrevJobComplete:1 valid_lines:0
3753695.584: new_job->timestamp:-541326732 new_job->img_high_res:11 new_job->img_low_res:3f750920
3753695.600: updated input Q	 rc:524 wc:528
3753695.643: cameraRIL: ca_process_frame passed image retcode 0 on port 71, timestamp 76768084596
3753695.685: cameraRIL: extract_focus_info: focus_status 0, af_status 0, caf status 4294967295
3753695.702: ca_release_lines: lowest_line_count is 1080
3753695.719: ca_release_lines: release stripe 0x3F750920 and low res 0x3F751840
3753695.737: cameraRIL: ca_finished_frame 0x3F750920 and lo_res 0x3F751840
3753695.764: cameraRIL:try_update_and_start_capture enter: current_state=2, pending_state=2
3753695.790: cameraRIL: initiating new grab
3753695.806: cameraRIL:next frame timestamped at 0 by clock (last one was 0)
3753695.825: cameraRIL:try_frame_start starting a grab - res is 1920x1080 xfm=0
3753695.840: cameraRIL: calling camplus grab_stripe() with flags 0x0
3753695.858: cameraRIL:try_update_and_start_capture exit: current_state=2, pending_state=2
3753709.235: video_encode:54:RIL:ve_worker_thread:done enc_frame3 in 22 ms
3753709.259: adding current_job 3f8df614 to sent Q
3753709.272: updated sent Q	 rc:523 wc:524
3753709.294: video_encode:54:RIL:ve_worker_thread(event 0x1,input.r/w=524/528)
3753709.310: video_encode:54:RIL:ve_process_thread:job 3f8df614(1,0)
3753709.319: job picked from input Q:3f8df684
3753709.345: video_encode:54:RIL:ve_process_thread:enc_getbytes status 2147438600, flags 0x0 bytes 0 (0x0) data 0x0
3753709.358: updated input Q	 rc:525 wc:528
3753709.366: writing job 3f8df614 to fifo
3753709.376: video_encode:54:RIL:ve_worker_thread:calling enc_frame3
3753712.855: video_encode:54:RIL:ve_process_thread:job 3f8df614(1,0)
3753712.904: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x200 bytes 9 (0x9) data 0xb8142f75
3753712.927: writing job 3f8df614 to fifo
3753712.972: video_encode:54:RIL: wrote 9 bytes for ts(76768001179) md(0) flags 0x0 into fifo, pData b8142f75, frame complete No
3753713.003: video_encode:54:RIL:ve_process_thread:job 3f8df614(1,0)
3753713.048: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x221 bytes 13758 (0x35be) data 0xb8142f7e
3753713.066: writing job 3f8df614 to fifo
3753713.110: video_encode:54:RIL: wrote 13758 bytes for ts(76768001179) md(0) flags 0x410 into fifo, pData b8142f7e, frame complete Yes
3753713.146: updated free Q after freeing job 3f8df614 rc:528 wc:644
3753714.647: cameraRIL: stripe callback (video) hi:3f750960(handle 25) lo:3f751700 offset 0x0 height 512
3753714.676: cameraRIL:CB:cp_stripe_callback captured stripe of height 512, image lines 0-512. Hi-res=0x3F750960, lo_res=0x3F751700
3753718.420: cameraRIL: stripe callback (video) hi:3f750960(handle 25) lo:3f751700 offset 0x6B00000 height 528
3753718.448: cameraRIL:CB:cp_stripe_callback captured stripe of height 528, image lines 512-1040. Hi-res=0x3F750960, lo_res=0x3F751700
3753718.723: cameraRIL: stripe callback (video) hi:3f750960(handle 25) lo:3f751700 offset 0xD958000 height 40
3753718.752: cameraRIL:CB:cp_stripe_callback captured stripe of height 40, image lines 1040-1080. Hi-res=0x3F750960, lo_res=0x3F751700
3753718.767: cameraRIL:CB:cp_stripe_callback frame completed
3753718.822: cameraRIL: passing image 3f750960 via callback port 71. Flags 10. Timestamp 76768117963
3753718.836: cameraRIL: passing image via encoder_func
3753718.884: video_encode:54:RIL: Callback request for 76768117963
3753718.899: is_eos:0 valid_lines:1080
3753718.909: free Q	 rc:528 wc:644
3753718.922: input Q	 rc:525 wc:528
3753718.934: sent Q	 rc:524 wc:524
3753718.944: job obtained:3f8df844
3753718.957: updated free Q	 rc:529 wc:644
3753718.974: writing job 3f8df844 to input Q
3753718.992: isNewFrame:1 isPrevJobComplete:1 valid_lines:0
3753719.013: new_job->timestamp:-541293365 new_job->img_high_res:11 new_job->img_low_res:3f750960
3753719.026: updated input Q	 rc:525 wc:529
3753719.065: cameraRIL: ca_process_frame passed image retcode 0 on port 71, timestamp 76768117963
3753719.105: cameraRIL: extract_focus_info: focus_status 0, af_status 0, caf status 4294967295
3753719.121: ca_release_lines: lowest_line_count is 1080
3753719.138: ca_release_lines: release stripe 0x3F750960 and low res 0x3F751700
3753719.153: cameraRIL: ca_finished_frame 0x3F750960 and lo_res 0x3F751700
3753719.177: cameraRIL:try_update_and_start_capture enter: current_state=2, pending_state=2
3753719.204: cameraRIL: initiating new grab
3753719.220: cameraRIL:next frame timestamped at 0 by clock (last one was 0)
3753719.237: cameraRIL:try_frame_start starting a grab - res is 1920x1080 xfm=0
3753719.248: cameraRIL: calling camplus grab_stripe() with flags 0x0
3753719.264: cameraRIL:try_update_and_start_capture exit: current_state=2, pending_state=2
3753731.473: video_encode:54:RIL:ve_worker_thread:done enc_frame3 in 21 ms
3753731.496: adding current_job 3f8df684 to sent Q
3753731.510: updated sent Q	 rc:524 wc:525
3753731.532: video_encode:54:RIL:ve_worker_thread(event 0x1,input.r/w=525/529)
3753731.544: job picked from input Q:3f8df6f4
3753731.554: updated input Q	 rc:526 wc:529
3753731.567: video_encode:54:RIL:ve_worker_thread:calling enc_frame3
3753731.613: video_encode:54:RIL:ve_process_thread:job 3f8df684(1,0)
3753731.648: video_encode:54:RIL:ve_process_thread:enc_getbytes status 2147438600, flags 0x0 bytes 0 (0x0) data 0x0
3753731.660: writing job 3f8df684 to fifo
3753733.293: video_encode:54:RIL:ve_process_thread:job 3f8df684(1,0)
3753733.332: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x200 bytes 9 (0x9) data 0xb814653c
3753733.359: writing job 3f8df684 to fifo
3753733.406: video_encode:54:RIL: wrote 9 bytes for ts(76768017862) md(0) flags 0x0 into fifo, pData b814653c, frame complete No
3753733.442: video_encode:54:RIL:ve_process_thread:job 3f8df684(1,0)
3753733.482: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x221 bytes 2751 (0xabf) data 0xb8146545
3753733.508: writing job 3f8df684 to fifo
3753733.548: video_encode:54:RIL: wrote 2751 bytes for ts(76768017862) md(0) flags 0x410 into fifo, pData b8146545, frame complete Yes
3753733.576: updated free Q after freeing job 3f8df684 rc:529 wc:645
3753737.204: cameraRIL: stripe callback (video) hi:3f7508a0(handle 46) lo:3f751740 offset 0x0 height 512
3753737.238: cameraRIL:CB:cp_stripe_callback captured stripe of height 512, image lines 0-512. Hi-res=0x3F7508A0, lo_res=0x3F751740
3753740.962: cameraRIL: stripe callback (video) hi:3f7508a0(handle 46) lo:3f751740 offset 0x6B00000 height 528
3753740.992: cameraRIL:CB:cp_stripe_callback captured stripe of height 528, image lines 512-1040. Hi-res=0x3F7508A0, lo_res=0x3F751740
3753741.239: cameraRIL: stripe callback (video) hi:3f7508a0(handle 46) lo:3f751740 offset 0xD958000 height 40
3753741.268: cameraRIL:CB:cp_stripe_callback captured stripe of height 40, image lines 1040-1080. Hi-res=0x3F7508A0, lo_res=0x3F751740
3753741.284: cameraRIL:CB:cp_stripe_callback frame completed
3753741.337: cameraRIL: passing image 3f7508a0 via callback port 71. Flags 10. Timestamp 76768134647
3753741.352: cameraRIL: passing image via encoder_func
3753741.400: video_encode:54:RIL: Callback request for 76768134647
3753741.413: is_eos:0 valid_lines:1080
3753741.426: free Q	 rc:529 wc:645
3753741.437: input Q	 rc:526 wc:529
3753741.449: sent Q	 rc:525 wc:525
3753741.461: job obtained:3f8df8b4
3753741.472: updated free Q	 rc:530 wc:645
3753741.490: writing job 3f8df8b4 to input Q
3753741.508: isNewFrame:1 isPrevJobComplete:1 valid_lines:0
3753741.526: new_job->timestamp:-541276681 new_job->img_high_res:11 new_job->img_low_res:3f7508a0
3753741.541: updated input Q	 rc:526 wc:530
3753741.599: cameraRIL: ca_process_frame passed image retcode 0 on port 71, timestamp 76768134647
3753741.638: cameraRIL: extract_focus_info: focus_status 0, af_status 0, caf status 4294967295
3753741.651: ca_release_lines: lowest_line_count is 1080
3753741.666: ca_release_lines: release stripe 0x3F7508A0 and low res 0x3F751740
3753741.679: cameraRIL: ca_finished_frame 0x3F7508A0 and lo_res 0x3F751740
3753741.702: cameraRIL:try_update_and_start_capture enter: current_state=2, pending_state=2
3753741.728: cameraRIL: initiating new grab
3753741.747: cameraRIL:next frame timestamped at 0 by clock (last one was 0)
3753741.767: cameraRIL:try_frame_start starting a grab - res is 1920x1080 xfm=0
3753741.777: cameraRIL: calling camplus grab_stripe() with flags 0x0
3753741.796: cameraRIL:try_update_and_start_capture exit: current_state=2, pending_state=2
3753754.235: video_encode:54:RIL:ve_worker_thread:done enc_frame3 in 22 ms
3753754.257: adding current_job 3f8df6f4 to sent Q
3753754.270: updated sent Q	 rc:525 wc:526
3753754.289: video_encode:54:RIL:ve_worker_thread(event 0x1,input.r/w=526/530)
3753754.306: video_encode:54:RIL:ve_process_thread:job 3f8df6f4(1,0)
3753754.315: job picked from input Q:3f8df764
3753754.341: video_encode:54:RIL:ve_process_thread:enc_getbytes status 2147438600, flags 0x0 bytes 0 (0x0) data 0x0
3753754.359: updated input Q	 rc:527 wc:530
3753754.367: writing job 3f8df6f4 to fifo
3753754.381: video_encode:54:RIL:ve_worker_thread:calling enc_frame3
3753757.757: video_encode:54:RIL:ve_process_thread:job 3f8df6f4(1,0)
3753757.802: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x200 bytes 9 (0x9) data 0xb8147004
3753757.822: writing job 3f8df6f4 to fifo
3753757.872: video_encode:54:RIL: wrote 9 bytes for ts(76768051229) md(0) flags 0x0 into fifo, pData b8147004, frame complete No
3753757.915: video_encode:54:RIL:ve_process_thread:job 3f8df6f4(1,0)
3753757.956: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x221 bytes 13672 (0x3568) data 0xb814700d
3753757.982: writing job 3f8df6f4 to fifo
3753758.023: video_encode:54:RIL: wrote 13672 bytes for ts(76768051229) md(0) flags 0x410 into fifo, pData b814700d, frame complete Yes
3753758.058: updated free Q after freeing job 3f8df6f4 rc:530 wc:646
3753759.829: cameraRIL: stripe callback (video) hi:3f7509a0(handle 24) lo:3f751780 offset 0x0 height 512
3753759.865: cameraRIL:CB:cp_stripe_callback captured stripe of height 512, image lines 0-512. Hi-res=0x3F7509A0, lo_res=0x3F751780
3753763.595: cameraRIL: stripe callback (video) hi:3f7509a0(handle 24) lo:3f751780 offset 0x6B00000 height 528
3753763.619: cameraRIL:CB:cp_stripe_callback captured stripe of height 528, image lines 512-1040. Hi-res=0x3F7509A0, lo_res=0x3F751780
3753763.893: cameraRIL: stripe callback (video) hi:3f7509a0(handle 24) lo:3f751780 offset 0xD958000 height 40
3753763.922: cameraRIL:CB:cp_stripe_callback captured stripe of height 40, image lines 1040-1080. Hi-res=0x3F7509A0, lo_res=0x3F751780
3753763.937: cameraRIL:CB:cp_stripe_callback frame completed
3753764.120: cameraRIL: passing image 3f7509a0 via callback port 71. Flags 10. Timestamp 76768151330
3753764.130: cameraRIL: passing image via encoder_func
3753764.176: video_encode:54:RIL: Callback request for 76768151330
3753764.189: is_eos:0 valid_lines:1080
3753764.202: free Q	 rc:530 wc:646
3753764.212: input Q	 rc:527 wc:530
3753764.224: sent Q	 rc:526 wc:526
3753764.232: job obtained:3f8df924
3753764.242: updated free Q	 rc:531 wc:646
3753764.259: writing job 3f8df924 to input Q
3753764.275: isNewFrame:1 isPrevJobComplete:1 valid_lines:0
3753764.293: new_job->timestamp:-541259998 new_job->img_high_res:11 new_job->img_low_res:3f7509a0
3753764.310: updated input Q	 rc:527 wc:531
3753764.352: cameraRIL: ca_process_frame passed image retcode 0 on port 71, timestamp 76768151330
3753764.390: cameraRIL: extract_focus_info: focus_status 0, af_status 0, caf status 4294967295
3753764.405: ca_release_lines: lowest_line_count is 1080
3753764.419: ca_release_lines: release stripe 0x3F7509A0 and low res 0x3F751780
3753764.437: cameraRIL: ca_finished_frame 0x3F7509A0 and lo_res 0x3F751780
3753764.463: cameraRIL:try_update_and_start_capture enter: current_state=2, pending_state=2
3753764.486: cameraRIL: initiating new grab
3753764.504: cameraRIL:next frame timestamped at 0 by clock (last one was 0)
3753764.522: cameraRIL:try_frame_start starting a grab - res is 1920x1080 xfm=0
3753764.534: cameraRIL: calling camplus grab_stripe() with flags 0x0
3753764.554: cameraRIL:try_update_and_start_capture exit: current_state=2, pending_state=2
3753777.362: video_encode:54:RIL:ve_worker_thread:done enc_frame3 in 22 ms
3753777.387: adding current_job 3f8df764 to sent Q
3753777.400: updated sent Q	 rc:526 wc:527
3753777.422: video_encode:54:RIL:ve_worker_thread(event 0x1,input.r/w=527/531)
3753777.442: video_encode:54:RIL:ve_process_thread:job 3f8df764(1,0)
3753777.450: job picked from input Q:3f8df7d4
3753777.476: video_encode:54:RIL:ve_process_thread:enc_getbytes status 2147438600, flags 0x0 bytes 0 (0x0) data 0x0
3753777.487: updated input Q	 rc:528 wc:531
3753777.495: writing job 3f8df764 to fifo
3753777.507: video_encode:54:RIL:ve_worker_thread:calling enc_frame3
3753781.663: video_encode:54:RIL:ve_process_thread:job 3f8df764(1,0)
3753781.714: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x200 bytes 9 (0x9) data 0xb814a575
3753781.733: writing job 3f8df764 to fifo
3753781.776: video_encode:54:RIL: wrote 9 bytes for ts(76768067913) md(0) flags 0x0 into fifo, pData b814a575, frame complete No
3753781.807: video_encode:54:RIL:ve_process_thread:job 3f8df764(1,0)
3753781.856: video_encode:54:RIL:ve_process_thread:enc_getbytes status 0, flags 0x221 bytes 17856 (0x45c0) data 0xb814a57e
3753781.874: writing job 3f8df764 to fifo
3753781.918: video_encode:54:RIL: wrote 17856 bytes for ts(76768067913) md(0) flags 0x410 into fifo, pData b814a57e, frame complete Yes
3753781.953: updated free Q after freeing job 3f8df764 rc:531 wc:647
3753783.541: cameraRIL: stripe callback (video) hi:3f7508e0(handle 40) lo:3f7517c0 offset 0x0 height 512
3753783.574: cameraRIL:CB:cp_stripe_callback captured stripe of height 512, image lines 0-512. Hi-res=0x3F7508E0, lo_res=0x3F7517C0
3753787.289: cameraRIL: stripe callback (video) hi:3f7508e0(handle 40) lo:3f7517c0 offset 0x6B00000 height 528
3753787.319: cameraRIL:CB:cp_stripe_callback captured stripe of height 528, image lines 512-1040. Hi-res=0x3F7508E0, lo_res=0x3F7517C0
3753787.589: cameraRIL: stripe callback (video) hi:3f7508e0(handle 40) lo:3f7517c0 offset 0xD958000 height 40
3753787.619: cameraRIL:CB:cp_stripe_callback captured stripe of height 40, image lines 1040-1080. Hi-res=0x3F7508E0, lo_res=0x3F7517C0
3753787.636: cameraRIL:CB:cp_stripe_callback frame completed
3753787.689: cameraRIL: passing image 3f7508e0 via callback port 71. Flags 10. Timestamp 76768184697
3753787.702: cameraRIL: passing image via encoder_func
3753787.754: video_encode:54:RIL: Callback request for 76768184697
3753787.769: is_eos:0 valid_lines:1080
3753787.781: free Q	 rc:531 wc:647
3753787.793: input Q	 rc:528 wc:531
3753787.807: sent Q	 rc:527 wc:527
3753787.817: job obtained:3f8df994
3753787.828: updated free Q	 rc:532 wc:647
3753787.846: writing job 3f8df994 to input Q
3753787.866: isNewFrame:1 isPrevJobComplete:1 valid_lines:0
3753787.886: new_job->timestamp:-541226631 new_job->img_high_res:11 new_job->img_low_res:3f7508e0
3753787.900: updated input Q	 rc:528 wc:532
3753787.940: cameraRIL: ca_process_frame passed image retcode 0 on port 71, timestamp 76768184697
3753787.981: cameraRIL: extract_focus_info: focus_status 0, af_status 0, caf status 4294967295
3753787.998: ca_release_lines: lowest_line_count is 1080
3753788.015: ca_release_lines: release stripe 0x3F7508E0 and low res 0x3F7517C0
3753788.033: cameraRIL: ca_finished_frame 0x3F7508E0 and lo_res 0x3F7517C0
3753788.057: cameraRIL:try_update_and_start_capture enter: current_state=2, pending_state=2
3753788.084: cameraRIL: initiating new grab
3753788.102: cameraRIL:next frame timestamped at 0 by clock (last one was 0)
3753788.123: cameraRIL:try_frame_start starting a grab - res is 1920x1080 xfm=0
3753788.133: cameraRIL: calling camplus grab_stripe() with flags 0x0
3753788.153: cameraRIL:try_update_and_start_capture exit: current_state=2, pending_state=2
3753788.878: deliver buffer 3f94fa84 in output_dir
3753788.895: output type is FRAME
3753788.909: extract output buffer if one available
3753788.922: got output buffer (3f94fa84)
3753788.952: video_encode:54:RIL: got FIFO header for (76762345459) with 15635 bytes
3753788.999: video_encode:54:RIL: copying 15635 bytes for (76762345459) into output buffer (3f94fa84), allocLen 65536, FilledLen 0, available_data 15635
037290.723: �8
805364.060: \�
303174.162: 
284079.371: 0�
3821462.342: ?7��8�Կ�³�ө?��3`��?EI�hc���l@����?
1516175.460: d
807199.136: ��
001703.937: ��
368206.289: �����zR��2���3��l��E������5�S�;�C� t.���
015043.856: a��Ġ�P��$̌����Ġ��
Segmentation fault
Kind Regards,

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Reading single frames piped from Raspivid in C/C++

Wed Jun 24, 2020 12:59 pm

No, the seg fault is vcdbg going crazy.
The weird characters are likely to be because the cache is stale. "vcgencmd cache_flush" to manually flush it.

"mmalsrv: send_buffer_to_host: tx failed" is generally that you've hit ctrl-c so the VPU can't send a buffer back to the host (ARM).

Sorry, I really don't want to provide support for the firmware drivers for tc358743. It's been explicitly stated many a time that they are unsupported, and as such they will remain. I can't get the tc358743 to run twice in a row with the firmware, and that I am definitely not investigating.

Testing with "yavta -c5000 -E - -m /dev/video0 | xxd > foo.txt" (xxd produces a hex dump) gives me:

Code: Select all

00000000: 5765 2772 6520 656e 636f 6469 6e67 2074  We're encoding t
00000010: 6f20 2d0a 4465 7669 6365 202f 6465 762f  o -.Device /dev/
00000020: 7669 6465 6f30 206f 7065 6e65 642e 0a44  video0 opened..D
00000030: 6576 6963 6520 6075 6e69 6361 6d27 206f  evice `unicam' o
00000040: 6e20 6070 6c61 7466 6f72 6d3a 6665 3830  n `platform:fe80
00000050: 3130 3030 2e63 7369 2720 2864 7269 7665  1000.csi' (drive
00000060: 7220 2775 6e69 6361 6d27 2920 6973 2061  r 'unicam') is a
00000070: 2076 6964 656f 2063 6170 7475 7265 2028   video capture (
00000080: 7769 7468 6f75 7420 6d70 6c61 6e65 7329  without mplanes)
00000090: 2064 6576 6963 652e 0a56 6964 656f 2066   device..Video f
000000a0: 6f72 6d61 743a 2055 5956 5920 2835 3935  ormat: UYVY (595
000000b0: 3635 3935 3529 2031 3238 3078 3732 3020  65955) 1280x720 
000000c0: 2873 7472 6964 6520 3235 3630 2920 6669  (stride 2560) fi
000000d0: 656c 6420 6e6f 6e65 2062 7566 6665 7220  eld none buffer 
000000e0: 7369 7a65 2031 3834 3332 3030 0a55 6e61  size 1843200.Una
000000f0: 626c 6520 746f 2067 6574 2066 7261 6d65  ble to get frame
00000100: 2072 6174 653a 2049 6e61 7070 726f 7072   rate: Inappropr
00000110: 6961 7465 2069 6f63 746c 2066 6f72 2064  iate ioctl for d
00000120: 6576 6963 6520 2832 3529 2e0a 7663 2e72  evice (25)..vc.r
00000130: 696c 2e69 7370 3a69 6e3a 3028 5559 5659  il.isp:in:0(UYVY
00000140: 2928 3078 3138 6130 6432 3029 7479 7065  )(0x18a0d20)type
00000150: 3a20 7669 6465 6f2c 2066 6f75 7263 633a  : video, fourcc:
00000160: 2055 5956 5920 6269 7472 6174 653a 2030   UYVY bitrate: 0
00000170: 2c20 6672 616d 6564 3a20 3020 6578 7472  , framed: 0 extr
00000180: 6120 6461 7461 3a20 302c 2028 6e69 6c29  a data: 0, (nil)
00000190: 2077 6964 7468 3a20 3132 3830 2c20 6865   width: 1280, he
000001a0: 6967 6874 3a20 3732 302c 2028 302c 302c  ight: 720, (0,0,
000001b0: 3132 3830 2c37 3230 2920 7069 7865 6c20  1280,720) pixel 
000001c0: 6173 7065 6374 2072 6174 696f 3a20 302f  aspect ratio: 0/
000001d0: 302c 2066 7261 6d65 2072 6174 653a 2030  0, frame rate: 0
000001e0: 2f30 0a62 7566 6665 7273 206e 756d 3a20  /0.buffers num: 
000001f0: 3828 6f70 7420 312c 206d 696e 2031 292c  8(opt 1, min 1),
00000200: 2073 697a 653a 2031 3834 3332 3030 286f   size: 1843200(o
00000210: 7074 2031 3834 3332 3030 2c20 6d69 6e3a  pt 1843200, min:
00000220: 2031 3834 3332 3030 292c 2061 6c69 676e   1843200), align
00000230: 3a20 300a 4372 6561 7465 6420 706f 6f6c  : 0.Created pool
00000240: 206f 6620 6c65 6e67 7468 2038 2c20 7369   of length 8, si
00000250: 7a65 2030 0a45 6e61 626c 6520 656e 636f  ze 0.Enable enco
00000260: 6465 722e 2e2e 2e0a 4372 6561 7465 2070  der.....Create p
00000270: 6f6f 6c20 6f66 2033 2062 7566 6665 7273  ool of 3 buffers
00000280: 206f 6620 7369 7a65 2030 2066 6f72 2072   of size 0 for r
00000290: 656e 6465 720a 4372 6561 7465 2070 6f6f  ender.Create poo
000002a0: 6c20 6f66 2033 2062 7566 6665 7273 206f  l of 3 buffers o
000002b0: 6620 7369 7a65 2030 2066 6f72 2065 6e63  f size 0 for enc
000002c0: 6f64 6520 6970 0a43 7265 6174 6520 706f  ode ip.Create po
000002d0: 6f6c 206f 6620 3320 6275 6666 6572 7320  ol of 3 buffers 
000002e0: 6f66 2073 697a 6520 3133 3832 3430 3020  of size 1382400 
000002f0: 666f 7220 656e 636f 6465 2f72 656e 6465  for encode/rende
00000300: 720a 0000 0001 2764 0028 ac2b 4028 02dd  r.....'d.(.+@(..
00000310: 00f1 226a 0000 0001 28ee 025c b000 0000  .."j....(..\....
00000320: 0125 8880 4fd9 740f ef7b cd03 f39e 4e7c  .%..O.t..{....N|
00000330: 0d59 4851 3d0c 3a5a 415b a232 a1a0 5dba  .YHQ=.:ZA[.2..].
00000340: 749d 5f3d 9423 5b82 661d 46d7 b991 f4f8  t._=.#[.f.F.....
00000350: fb01 9fc2 fccd 663b 2bfb bee3 38e8 788f  ......f;+...8.x.
00000360: 143a adfd 7ee5 2356 db28 10de 9f1b fb1c  .:..~.#V.(......
00000370: 0469 ec37 1762 19b9 ada5 aa27 9bda 3b90  .i.7.b.....'..;.
00000380: 884e aa96 2bca 816d b465 2852 d71f 08d4  .N..+..m.e(R....
00000390: f528 6141 122f f6aa 716c 93b6 e4dc 1c53  .(aA./..ql.....S
So all the debug text is also still going to stdout, same as the encoded data.
Change "int debug=1;" at line 58 to =0, all the debug text goes away, and I get

Code: Select all

00000000: 0000 0001 2764 0028 ac2b 4028 02dd 00f1  ....'d.(.+@(....
00000010: 226a 0000 0001 28ee 025c b000 0000 0125  "j....(..\.....%
00000020: 8880 4fd9 8696 bf97 108f e350 f918 f84f  ..O........P...O
00000030: 7fd7 63a9 9997 dbec 2333 fb76 5a18 be8b  ..c.....#3.vZ...
00000040: a142 73e8 600f 27b0 8173 0c65 b2bf bbb1  .Bs.`.'..s.e....
00000050: 6b01 3b0d 1351 e8d2 607c 6591 ed1d 6047  k.;..Q..`|e...`G
00000060: 75ea f708 932b 45d2 7624 92e2 c793 656f  u....+E.v$....eo
00000070: 368a f0b1 3c9b ce9a ae84 2579 8020 4c7b  6...<.....%y. L{
00 00 00 01 is an H264 start code, so that looks pretty good to me. You've got the headers of 18 and 9 byte (including the start codes), and then a start code for the first frame. I'm not digging further into that.

Selecting stdout as the default output does try setting debug to 0, there is some debug output that has already been produced at the point it is checked.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Wed Jun 24, 2020 1:34 pm

Hi 6by9,

Thanks for that, I'll switch off all debugging and check it before enabling the pipe to the stream server.

I appreciate that the TC358743 and ADV7280A-M unicam drivers are unsupported. I can get the TC358743 to run as many times as I like in a row by using a gpio pin to pull the reset line low and then high again and it works every time flawlessly (well with my 410 hours of streaming only testing it did). I personally like the v4l2 drivers but I'm having difficulty understanding the way v4l2 <-> mmal works, if I understood it better I think I'd be running along fine with it all.

Again I appreciate your time and effort in pointing me in the right directions.

Many thanks and kind regards,

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Thu Jun 25, 2020 1:01 pm

Hi 6by9,

I tried v4l2_mmal which gave an appalling result, the stream was frame glitchy and had errors in the image encoding. Then I noticed you said yavta and not v4l2_mmal and that works a treat so all I need to do now is add the image encoder pipeline and callbacks.

2 questions though:

1) Should I use a separate instance of the portdata struct for the image encoder or can I use the same one?
2) Should the image encoder have its own buffer pool? (I think it should)?


Many Thanks,

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Reading single frames piped from Raspivid in C/C++

Thu Jun 25, 2020 2:36 pm

isp_output_callback has a set of if(thing exists) { replicate and send the buffer}. You need an extra one of them, with an extra pool for the headers (no payload), and you also want some additional condition for "and I want to capture a still encode of this image too"

portdata struct? Do you mean port->userdata? In which case that is a member of MMAL_PORT_T and every port will have one. It's just an easy way to refer back to a client structure from within the callbacks.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Fri Jun 26, 2020 8:05 am

Hi 6by9,

Yes I meant the port->userdata sorry about that. You said
you also want some additional condition for "and I want to capture a still encode of this image too"
so does that mean I can't create a component of "vc.ril.image_encode" and set the trigger parameter just as I did in RaspiVid? Or do I have to intercept the UYVY images from the isp and convert them to jpeg or bitmaps?

EDIT: I think you've already said the answer in that the isp callback copies the buffer to the video splitter, so really I need to add the image encoder and just copy the isp output to that plus a callback for the image buffer data. Is that about right??

Kind Regards,

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Reading single frames piped from Raspivid in C/C++

Fri Jun 26, 2020 12:32 pm

DanR wrote:
Fri Jun 26, 2020 8:05 am
EDIT: I think you've already said the answer in that the isp callback copies the buffer to the video splitter, so really I need to add the image encoder and just copy the isp output to that plus a callback for the image buffer data. Is that about right??
There is no video splitter component, it uses the buffer_header_replicate function to create a cloned buffer header that can then be sent to multiple components. When all the cloned buffer headers are released, then the source buffer is returned for reuse.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 29, 2020 9:17 am

Hi 6by9,

I'm having difficulty determining what the difference is between the two functions video_save_image and save_thread in yavta as they both appear to do the same thing and have write calls. I'm guessing one saves the image or rather H264 stream out to a file so what does the other one do?

Many Thanks,

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 29, 2020 10:20 am

DanR wrote:
Mon Jun 29, 2020 9:17 am
I'm having difficulty determining what the difference is between the two functions video_save_image and save_thread in yavta as they both appear to do the same thing and have write calls. I'm guessing one saves the image or rather H264 stream out to a file so what does the other one do?
Have a look at the context they are called from, and the parameters.

video_save_image is part of the upstream yavta and saves a V4L2 buffer within the caller's context.
save_thread has a queue that it waits on and saves MMAL buffers from, within an independent thread context.

Blocking MMAL callbacks for an extended duration is generally a bad idea as it affects overall performance. The callback therefore stuffs the buffer into a queue for saving by an independent thread and allows the callback thread to continue.

(Technically to be generic, save_thread should be doing a mmal_buffer_header_release rather than mmal_port_send_buffer (and definitely not to a specific port), so that then the buffer release callback can do the right thing with the buffer. You could then end buffers from isp_output_callback to a copy of the thread, and they'll be handed back to the ISP once finished with. You could also do a queue length check so that if the save was taking longer than real-time and you had multiple buffers queued up, you can drop buffers from being saved).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Mon Jun 29, 2020 2:13 pm

Hi 6by9,

I've added a vc.ril.image_encode component to the "setup_mmal" function and added the buffer pool, set the input to the same as the vc.ril.video_encode input and then set the output port format to JPEG without any errors on startup. With these added I start the application and 2 buffers are processed (see output below) and then nothing for a few seconds and I then get a "select timeout" error. Is this related to this forum thread: viewtopic.php?t=228450

If so, how do I fix it as the post doesn't show what was done to remedy the issue?
Also my code changes are here: https://github.com/danriches/yavta

Code: Select all

sudo ./yavta --encode-to=/ramdisk/test.h264 --capture=2000 -n 3 -m /dev/video0
We're encoding to /ramdisk/test.h264
Device /dev/video0 opened.
Device `unicam' on `platform:unicam 3f801000.csi' (driver 'unicam') is a video capture (without mplanes) device.
Video format: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4177920
Unable to get frame rate: Inappropriate ioctl for device (25).
Created image encoder
vc.ril.isp:in:0(UYVY)(0x7ecd20)type: video, fourcc: UYVY bitrate: 0, framed: 0 extra data: 0, (nil) width: 1920, height: 1088, (0,0,1920,1080) pixel aspect ratio: 0/0, frame rate: 0/0
buffers num: 3(opt 1, min 1), size: 4177920(opt 4177920, min: 4177920), align: 0
Created pool of length 3, size 0
Unable to set JPEG restart interval
3 buffers requested, V4L2 returned 3 bufs.
length: 4177920 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x72700000.
Importing DMABUF 6 into VCSM...
...done. vcsm_handle 2813952
Exported buffer 0 to dmabuf 6, vcsm handle 2813952
Linking V4L2 buffer index 0 ptr 0x7f5d78 to MMAL header 0x7f57e0. mmal->data 0xC0000004
length: 4177920 offset: 4177920 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x72304000.
Importing DMABUF 7 into VCSM...
...done. vcsm_handle 2818048
Exported buffer 1 to dmabuf 7, vcsm handle 2818048
Linking V4L2 buffer index 1 ptr 0x7f5de8 to MMAL header 0x7f59b8. mmal->data 0xC0000005
length: 4177920 offset: 8355840 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x71f08000.
Importing DMABUF 8 into VCSM...
...done. vcsm_handle 2822144
Exported buffer 2 to dmabuf 8, vcsm handle 2822144
Linking V4L2 buffer index 2 ptr 0x7f5e58 to MMAL header 0x7f5b90. mmal->data 0xC0000002
0 (0) [-] none 0 4177920 B 365649.120117 365649.133392 -17543.860 fps ts mono/EoF
1 (1) [-] none 1 4177920 B 365649.133143 365649.155362 76.770 fps ts mono/EoF
select timeout
Kind Regards,

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Tue Jun 30, 2020 8:27 am

Hi 6by9,

So the image encode component does connect up and get configured correctly as when I deliberately set it to MJPEG I can see a configuration error output with: sudo vcdbg log msg

However with the image encoder parts added I get this only when I run the application in the vcdbg output

Code: Select all

1603899.814: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
1603899.917: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
I also see this output from the application:

Code: Select all

pi@imageportdev:~/yavta $ sudo ./yavta --encode-to=/ramdisk/test.h264 --capture=2000 -n 3 -m /dev/video0
We're encoding to /ramdisk/test.h264
Device /dev/video0 opened.
Device `unicam' on `platform:unicam 3f801000.csi' (driver 'unicam') is a video capture (without mplanes) device.
Video format: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4177920
Unable to get frame rate: Inappropriate ioctl for device (25).
Created image encoder
vc.ril.isp:in:0(UYVY)(0x8a6d20)type: video, fourcc: UYVY bitrate: 0, framed: 0 extra data: 0, (nil) width: 1920, height: 1088, (0,0,1920,1080) pixel aspect ratio: 0/0, frame rate: 0/0
buffers num: 3(opt 1, min 1), size: 4177920(opt 4177920, min: 4177920), align: 0
Created pool of length 3, size 0
Unable to set JPEG restart interval
3 buffers requested, V4L2 returned 3 bufs.
length: 4177920 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x726fd000.
Importing DMABUF 6 into VCSM...
...done. vcsm_handle 2949120
Exported buffer 0 to dmabuf 6, vcsm handle 2949120
Linking V4L2 buffer index 0 ptr 0x8afd78 to MMAL header 0x8af7e0. mmal->data 0xC0000005
length: 4177920 offset: 4177920 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x72301000.
Importing DMABUF 7 into VCSM...
...done. vcsm_handle 2953216
Exported buffer 1 to dmabuf 7, vcsm handle 2953216
Linking V4L2 buffer index 1 ptr 0x8afde8 to MMAL header 0x8af9b8. mmal->data 0xC0000004
length: 4177920 offset: 8355840 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x71f05000.
Importing DMABUF 8 into VCSM...
...done. vcsm_handle 2957312
Exported buffer 2 to dmabuf 8, vcsm handle 2957312
Linking V4L2 buffer index 2 ptr 0x8afe58 to MMAL header 0x8afb90. mmal->data 0xC0000002
0 (0) [-] none 0 4177920 B 432864.212606 432864.214839 -13157.895 fps ts mono/EoF
1 (1) [-] none 1 4177920 B 432864.214832 432864.234934 449.236 fps ts mono/EoF
select timeout
The part that falls over is when the select function is called to see if the /dev/video0 device is ready with more data, at least that's what I think it does. Is this a problem with buffers or do I need to do something else to get it past this issue? I think once this part is sorted it should all just work...

Kind Regards,

Dan

DanR
Posts: 79
Joined: Fri Jan 18, 2013 1:20 pm

Re: Reading single frames piped from Raspivid in C/C++

Tue Jun 30, 2020 9:33 am

Hi 6by9,

Well it was down to setting the MMAL_PARAMETER_JPEG_RESTART_INTERVAL parameter on the wrong encoder (video not the image encoder), what a wally I am!!! Just need to sort out saving the image data and adding a lock semaphore around the save bit so it's not running twice over ;)

Thank you ever so much for all your input, pointers and guidance. It is as ever very very much appreciated!

Very Kind Regards,

Dan

Return to “Interfacing (DSI, CSI, I2C, etc.)”