richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Play video and "record"

Sun Mar 12, 2017 2:57 pm

I am working on a project where I can playing live video from a camera on another Pi. O the client side, I have a Pi3 to display the video but I also would like to record, pause & stop recording.

The question is: if I am using tunneling, can I use a single input tunnel then fork the data so that I can either directly write the stream to HD or feed it into an encoder for saving to hd?

If there a specific tunnel fork or can I simple feed the data into the two tunnels then call the clear buffer command?

User avatar
dickon
Posts: 1933
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, in Towcester

Re: Play video and "record"

Sun Mar 12, 2017 3:15 pm

You could probably use either approach on a 3, given the extra CPU speed available. I'd go with the video_splitter component myself.

There's documentation under the firmware repository at https://github.com/raspberrypi/firmware ... components.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Tue Mar 14, 2017 7:17 am

Thanks, that got me going....working on the code now.

richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 4:46 pm

OK guys, I'm stumped.

I've been modifying the video.c sample code but I can't get it to work. I've sucessfully implemented the video_splitter component and can play the video on screen but when I try to write the video to storage, it crashes. I am stumped.

What I am trying to do it this:

1. video stream from ip camera -> video_decoder -> video_splitter -> stream 1 & stream 2

2. stream 1 -> video_render

3. stream 2 -> video_encoder -> output buffer -> file.h264

The issue seems to be in the output buffer. I am using tunnels between components but it seems that I need to output from the encoder to a buffer in order to write to the HD. There is a "write_media" component but there is no documentation or examples of how to use it. (All examples use the output buffer to write to a file.)

My code goes through the loop twice (get the first data block) then segfaults on the second loop when trying to fill the buffer a second time.

Can anyone help me with this? Some of you out there have done this so any help would be great.

User avatar
dickon
Posts: 1933
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, in Towcester

Re: Play video and "record"

Mon Mar 27, 2017 4:51 pm

Can't do much without the code to look at...

I don't believe the *_media components have been implemented, and, as you say, there's no documentation for them anyway.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 4:56 pm

I don't want to post the entire code but I could post parts of it or I could send you the code in a PM.

User avatar
dickon
Posts: 1933
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, in Towcester

Re: Play video and "record"

Mon Mar 27, 2017 5:04 pm

Bung it on github somewhere?

Worth noting that the only difference between what you're doing and what my omxtx transcoder does (in monitor mode) is use a webcam source instead of a file.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 5:34 pm

This is my hack...don't judge, I still have a lot to clean up

Here is the loop in question:

Code: Select all

//// open file to save ####################

//Create / Open Video Output file
   printf("0. create output file video.h264\n",0);
  int fd = fopen (FILENAME, "w");
  if (fd == -1){
    fprintf (stderr, "error: open\n");
    exit (1);
  }
       
   
   
//// 2. create components #####################
   
   // create video_decode
   if(ilclient_create_component(client, &video_decode, "video_decode", ILCLIENT_DISABLE_ALL_PORTS | ILCLIENT_ENABLE_INPUT_BUFFERS) != 0)
      status = -14;
   list[0] = video_decode; 
   printf("1. create video decode tunnel completed\n",0);
       
   
     // added by RCG - split input buffer so that it can be sent to both the video_render and video_encoder tunnels
   if(status == 0 && ilclient_create_component(client, &video_splitter, "video_splitter", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[1] = video_splitter;   
   printf("2. create video splitter tunnel completed\n",0);
     
  
   // create video_render
   if(status == 0 && ilclient_create_component(client, &video_render, "video_render", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[2] = video_render; 
   printf("3. create video render tunnel completed\n");
    // create clock
   if(status == 0 && ilclient_create_component(client, &clock, "clock", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[3] = clock; 
   printf("4. create video clock tunnel completed\n");
         
   // added by RCG - re-encode video - save to hd
   if(status == 0 && ilclient_create_component(client, &video_encode, "video_encode",  ILCLIENT_DISABLE_ALL_PORTS |
				 ILCLIENT_ENABLE_INPUT_BUFFERS |
				 ILCLIENT_ENABLE_OUTPUT_BUFFERS) != 0)
      status = -14;
   list[4] = video_encode; 
   printf("5. create video encode tunnel completed\n");
  


   
   memset(&cstate, 0, sizeof(cstate));
   cstate.nSize = sizeof(cstate);
   cstate.nVersion.nVersion = OMX_VERSION;
   cstate.eState = OMX_TIME_ClockStateWaitingForStartTime;
   cstate.nWaitMask = 1;
   if(clock != NULL && OMX_SetParameter(ILC_GET_HANDLE(clock), OMX_IndexConfigTimeClockState, &cstate) != OMX_ErrorNone)
      status = -13;
   printf("6. activate clock tunnel completed\n");

   // create video_scheduler
   if(status == 0 && ilclient_create_component(client, &video_scheduler, "video_scheduler", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[4] = video_scheduler;
   printf("7. setup video scheduler tunnel completed\n");

  
   set_tunnel(tunnel, video_decode, 131, video_splitter, 250);
   set_tunnel(tunnel+1, video_splitter, 251, video_scheduler, 10);
   set_tunnel(tunnel+2, video_splitter, 252, video_encode, 200.);  
   set_tunnel(tunnel+2, video_scheduler, 11, video_render, 90);
   set_tunnel(tunnel+3, clock, 80, video_scheduler, 12);
   
 
   // setup clock tunnel first
   if(status == 0 && ilclient_setup_tunnel(tunnel+3, 0, 0) != 0)
      status = -15;
   else
      ilclient_change_component_state(clock, OMX_StateExecuting);

  
   if(status == 0)
      ilclient_change_component_state(video_splitter, OMX_StateIdle);
   
   
//// enabling buffers in encoder
   if(status == 0)
      ilclient_change_component_state(video_decode, OMX_StateIdle);
   
    printf("encode to idle...\n");
    if(status == 0)
      ilclient_change_component_state(video_encode, OMX_StateIdle);   

 
   printf("enabling port buffers for 201...\n");
   if (ilclient_enable_port_buffers(video_encode, 201, NULL, NULL, NULL) != 0) {
      printf("enabling port buffers for 201 failed!\n");
      exit(1);
   }

   printf("encode to executing...\n");
   ilclient_change_component_state(video_encode, OMX_StateExecuting);
   
    printf("8. setup video clock tunnel completed\n");

   memset(&format, 0, sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE));
   format.nSize = sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE);
   format.nVersion.nVersion = OMX_VERSION;
   format.nPortIndex = 130;
   format.eCompressionFormat = OMX_VIDEO_CodingAVC;
   format.xFramerate = 60 << 16;

// ############## Add ability to set the window size Added by RCG #######################

   ilclient_change_component_state(video_render, OMX_StateIdle);

   OMX_CONFIG_DISPLAYREGIONTYPE configDisplay;
   memset(&configDisplay, 0, sizeof(OMX_CONFIG_DISPLAYREGIONTYPE));
   configDisplay.nSize = sizeof(OMX_CONFIG_DISPLAYREGIONTYPE);
   configDisplay.nVersion.nVersion = OMX_VERSION;
   configDisplay.nPortIndex = 90;
   configDisplay.fullscreen = OMX_FALSE;
   configDisplay.noaspect   = OMX_TRUE;
   configDisplay.set = (OMX_DISPLAYSETTYPE)(OMX_DISPLAY_SET_DEST_RECT|OMX_DISPLAY_SET_SRC_RECT|OMX_DISPLAY_SET_FULLSCREEN|OMX_DISPLAY_SET_NOASPECT);
   configDisplay.dest_rect.x_offset  = x;
   configDisplay.dest_rect.y_offset  = y;
   configDisplay.dest_rect.width     = w;
   configDisplay.dest_rect.height    = h;
   configDisplay.src_rect.x_offset   = 0;
   configDisplay.src_rect.y_offset   = 0;
   configDisplay.src_rect.width      = 1280;
   configDisplay.src_rect.height     = 720;

   int stat = 0;
   stat =  OMX_SetParameter(ILC_GET_HANDLE(video_render), OMX_IndexConfigDisplayRegion, &configDisplay);
   
   printf("9. setup video windows according to command line params completed\n");
  
// ############### ENDOF window size Added by RCG ####################################### 

   if(status == 0 &&
      OMX_SetParameter(ILC_GET_HANDLE(video_decode), OMX_IndexParamVideoPortFormat, &format) == OMX_ErrorNone &&
      ilclient_enable_port_buffers(video_decode, 130, NULL, NULL, NULL) == 0)
   {
       
      OMX_BUFFERHEADERTYPE *buf;
      OMX_BUFFERHEADERTYPE *out;
      OMX_ERRORTYPE r;
      int port_settings_changed = 0;
      int first_packet = 1;

      ilclient_change_component_state(video_decode, OMX_StateExecuting);
    
      printf("10. starting video loop\n");
      while((buf = ilclient_get_input_buffer(video_decode, 130, 1)) != NULL)
      {

         
        printf("11. video loop\n");	
         // feed data and wait until we get port settings changed
         unsigned char *dest = buf->pBuffer;

	     data_len = read(STDIN_FILENO, dest, buf->nAllocLen-data_len);

         if(port_settings_changed == 0 &&
            ((data_len > 0 && ilclient_remove_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1) == 0) ||
             (data_len == 0 && ilclient_wait_for_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1,
                                                       ILCLIENT_EVENT_ERROR | ILCLIENT_PARAMETER_CHANGED, 10000) == 0)))
              {
            port_settings_changed = 1;

            if(ilclient_setup_tunnel(tunnel, 0, 0) != 0)
            {
               status = -7;
               break;
            }

            ilclient_change_component_state(video_scheduler, OMX_StateExecuting);
            
            
             if(ilclient_setup_tunnel(tunnel+1, 0, 0) != 0)
            {
               status = -7;
               break;
            }
            // now setup the splitter
            ilclient_change_component_state(video_splitter, OMX_StateExecuting);
 
            // now setup tunnel to video_render
            if(ilclient_setup_tunnel(tunnel+2, 0, 1000) != 0)
            {
               status = -12;
               break;
            }

            ilclient_change_component_state(video_render, OMX_StateExecuting);
            
          }
         if(!data_len)
            break;

         buf->nFilledLen = data_len;
         data_len = 0;

         buf->nOffset = 0;
         if(first_packet)
         {
            buf->nFlags = OMX_BUFFERFLAG_STARTTIME;
            first_packet = 0;
         }
         else
            buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN;

         if(OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_decode), buf) != OMX_ErrorNone)
         {
            status = -6;
            break;
         }
         
   /// this section is blocking the loop on the second time around
         
         printf("GET output buffer\n");
	 out = ilclient_get_output_buffer(video_encode, 201, 1);
         printf("got output buffer\n");
         
         r = OMX_FillThisBuffer(ILC_GET_HANDLE(video_encode), out);
	 if (r != OMX_ErrorNone) 
         {
	    printf("Error filling buffer: %x\n", r);
	 }

	 if (out != NULL) 
         {
	    if (out->nFlags & OMX_BUFFERFLAG_CODECCONFIG) {
	       int i;
	       for (i = 0; i < out->nFilledLen; i++)
		  printf("%x ", out->pBuffer[i]);
	       printf("\n");
	    }

	    r = fwrite(out->pBuffer, 1, out->nFilledLen, fd);
	    if (r != out->nFilledLen) {
	       printf("fwrite: Error emptying buffer: %d!\n", r);
	    }
	    else {
	       printf("Writing frame file\n");
	    }
  
	   out->nFilledLen = 0;
         
          if (out = NULL)
          {
                 printf("Error buffer = NULL\n");   
          }
         }
  
      }
Fails on the second loop with "GET output buffer"

User avatar
dickon
Posts: 1933
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, in Towcester

Re: Play video and "record"

Mon Mar 27, 2017 6:04 pm

Ah. ilclient. I've not done anything with that.

Aside from the 'int fd' (which should be a FILE * as you're using libc file calls, not OS syscalls), the only thing that leaps out at me is the ilclient_get_output_buffer() call. Do you really need to call this each time around the loop? Just call it once outside the loop, then write the buffer out and pass it back with OMX_FillThisBuffer() each time. Having a look at your event handlers might be useful, too.

Looking back at omxtx, getting this working is much more work than I remember it being. Have a look at it, and see what you're missing. It's at https://github.com/dickontoo/omxtx.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

richardcgiroux
Posts: 35
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 6:42 pm

Thanks for your comments. I'll look at it immediately.

Return to “OpenMAX”