richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Play video and "record"

Sun Mar 12, 2017 2:57 pm

I am working on a project where I can playing live video from a camera on another Pi. O the client side, I have a Pi3 to display the video but I also would like to record, pause & stop recording.

The question is: if I am using tunneling, can I use a single input tunnel then fork the data so that I can either directly write the stream to HD or feed it into an encoder for saving to hd?

If there a specific tunnel fork or can I simple feed the data into the two tunnels then call the clear buffer command?

dickon
Posts: 216
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Play video and "record"

Sun Mar 12, 2017 3:15 pm

You could probably use either approach on a 3, given the extra CPU speed available. I'd go with the video_splitter component myself.

There's documentation under the firmware repository at https://github.com/raspberrypi/firmware ... components.

richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Tue Mar 14, 2017 7:17 am

Thanks, that got me going....working on the code now.

richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 4:46 pm

OK guys, I'm stumped.

I've been modifying the video.c sample code but I can't get it to work. I've sucessfully implemented the video_splitter component and can play the video on screen but when I try to write the video to storage, it crashes. I am stumped.

What I am trying to do it this:

1. video stream from ip camera -> video_decoder -> video_splitter -> stream 1 & stream 2

2. stream 1 -> video_render

3. stream 2 -> video_encoder -> output buffer -> file.h264

The issue seems to be in the output buffer. I am using tunnels between components but it seems that I need to output from the encoder to a buffer in order to write to the HD. There is a "write_media" component but there is no documentation or examples of how to use it. (All examples use the output buffer to write to a file.)

My code goes through the loop twice (get the first data block) then segfaults on the second loop when trying to fill the buffer a second time.

Can anyone help me with this? Some of you out there have done this so any help would be great.

dickon
Posts: 216
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Play video and "record"

Mon Mar 27, 2017 4:51 pm

Can't do much without the code to look at...

I don't believe the *_media components have been implemented, and, as you say, there's no documentation for them anyway.

richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 4:56 pm

I don't want to post the entire code but I could post parts of it or I could send you the code in a PM.

dickon
Posts: 216
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Play video and "record"

Mon Mar 27, 2017 5:04 pm

Bung it on github somewhere?

Worth noting that the only difference between what you're doing and what my omxtx transcoder does (in monitor mode) is use a webcam source instead of a file.

richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 5:34 pm

This is my hack...don't judge, I still have a lot to clean up

Here is the loop in question:

Code: Select all

//// open file to save ####################

//Create / Open Video Output file
   printf("0. create output file video.h264\n",0);
  int fd = fopen (FILENAME, "w");
  if (fd == -1){
    fprintf (stderr, "error: open\n");
    exit (1);
  }
       
   
   
//// 2. create components #####################
   
   // create video_decode
   if(ilclient_create_component(client, &video_decode, "video_decode", ILCLIENT_DISABLE_ALL_PORTS | ILCLIENT_ENABLE_INPUT_BUFFERS) != 0)
      status = -14;
   list[0] = video_decode; 
   printf("1. create video decode tunnel completed\n",0);
       
   
     // added by RCG - split input buffer so that it can be sent to both the video_render and video_encoder tunnels
   if(status == 0 && ilclient_create_component(client, &video_splitter, "video_splitter", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[1] = video_splitter;   
   printf("2. create video splitter tunnel completed\n",0);
     
  
   // create video_render
   if(status == 0 && ilclient_create_component(client, &video_render, "video_render", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[2] = video_render; 
   printf("3. create video render tunnel completed\n");
    // create clock
   if(status == 0 && ilclient_create_component(client, &clock, "clock", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[3] = clock; 
   printf("4. create video clock tunnel completed\n");
         
   // added by RCG - re-encode video - save to hd
   if(status == 0 && ilclient_create_component(client, &video_encode, "video_encode",  ILCLIENT_DISABLE_ALL_PORTS |
				 ILCLIENT_ENABLE_INPUT_BUFFERS |
				 ILCLIENT_ENABLE_OUTPUT_BUFFERS) != 0)
      status = -14;
   list[4] = video_encode; 
   printf("5. create video encode tunnel completed\n");
  


   
   memset(&cstate, 0, sizeof(cstate));
   cstate.nSize = sizeof(cstate);
   cstate.nVersion.nVersion = OMX_VERSION;
   cstate.eState = OMX_TIME_ClockStateWaitingForStartTime;
   cstate.nWaitMask = 1;
   if(clock != NULL && OMX_SetParameter(ILC_GET_HANDLE(clock), OMX_IndexConfigTimeClockState, &cstate) != OMX_ErrorNone)
      status = -13;
   printf("6. activate clock tunnel completed\n");

   // create video_scheduler
   if(status == 0 && ilclient_create_component(client, &video_scheduler, "video_scheduler", ILCLIENT_DISABLE_ALL_PORTS) != 0)
      status = -14;
   list[4] = video_scheduler;
   printf("7. setup video scheduler tunnel completed\n");

  
   set_tunnel(tunnel, video_decode, 131, video_splitter, 250);
   set_tunnel(tunnel+1, video_splitter, 251, video_scheduler, 10);
   set_tunnel(tunnel+2, video_splitter, 252, video_encode, 200.);  
   set_tunnel(tunnel+2, video_scheduler, 11, video_render, 90);
   set_tunnel(tunnel+3, clock, 80, video_scheduler, 12);
   
 
   // setup clock tunnel first
   if(status == 0 && ilclient_setup_tunnel(tunnel+3, 0, 0) != 0)
      status = -15;
   else
      ilclient_change_component_state(clock, OMX_StateExecuting);

  
   if(status == 0)
      ilclient_change_component_state(video_splitter, OMX_StateIdle);
   
   
//// enabling buffers in encoder
   if(status == 0)
      ilclient_change_component_state(video_decode, OMX_StateIdle);
   
    printf("encode to idle...\n");
    if(status == 0)
      ilclient_change_component_state(video_encode, OMX_StateIdle);   

 
   printf("enabling port buffers for 201...\n");
   if (ilclient_enable_port_buffers(video_encode, 201, NULL, NULL, NULL) != 0) {
      printf("enabling port buffers for 201 failed!\n");
      exit(1);
   }

   printf("encode to executing...\n");
   ilclient_change_component_state(video_encode, OMX_StateExecuting);
   
    printf("8. setup video clock tunnel completed\n");

   memset(&format, 0, sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE));
   format.nSize = sizeof(OMX_VIDEO_PARAM_PORTFORMATTYPE);
   format.nVersion.nVersion = OMX_VERSION;
   format.nPortIndex = 130;
   format.eCompressionFormat = OMX_VIDEO_CodingAVC;
   format.xFramerate = 60 << 16;

// ############## Add ability to set the window size Added by RCG #######################

   ilclient_change_component_state(video_render, OMX_StateIdle);

   OMX_CONFIG_DISPLAYREGIONTYPE configDisplay;
   memset(&configDisplay, 0, sizeof(OMX_CONFIG_DISPLAYREGIONTYPE));
   configDisplay.nSize = sizeof(OMX_CONFIG_DISPLAYREGIONTYPE);
   configDisplay.nVersion.nVersion = OMX_VERSION;
   configDisplay.nPortIndex = 90;
   configDisplay.fullscreen = OMX_FALSE;
   configDisplay.noaspect   = OMX_TRUE;
   configDisplay.set = (OMX_DISPLAYSETTYPE)(OMX_DISPLAY_SET_DEST_RECT|OMX_DISPLAY_SET_SRC_RECT|OMX_DISPLAY_SET_FULLSCREEN|OMX_DISPLAY_SET_NOASPECT);
   configDisplay.dest_rect.x_offset  = x;
   configDisplay.dest_rect.y_offset  = y;
   configDisplay.dest_rect.width     = w;
   configDisplay.dest_rect.height    = h;
   configDisplay.src_rect.x_offset   = 0;
   configDisplay.src_rect.y_offset   = 0;
   configDisplay.src_rect.width      = 1280;
   configDisplay.src_rect.height     = 720;

   int stat = 0;
   stat =  OMX_SetParameter(ILC_GET_HANDLE(video_render), OMX_IndexConfigDisplayRegion, &configDisplay);
   
   printf("9. setup video windows according to command line params completed\n");
  
// ############### ENDOF window size Added by RCG ####################################### 

   if(status == 0 &&
      OMX_SetParameter(ILC_GET_HANDLE(video_decode), OMX_IndexParamVideoPortFormat, &format) == OMX_ErrorNone &&
      ilclient_enable_port_buffers(video_decode, 130, NULL, NULL, NULL) == 0)
   {
       
      OMX_BUFFERHEADERTYPE *buf;
      OMX_BUFFERHEADERTYPE *out;
      OMX_ERRORTYPE r;
      int port_settings_changed = 0;
      int first_packet = 1;

      ilclient_change_component_state(video_decode, OMX_StateExecuting);
    
      printf("10. starting video loop\n");
      while((buf = ilclient_get_input_buffer(video_decode, 130, 1)) != NULL)
      {

         
        printf("11. video loop\n");	
         // feed data and wait until we get port settings changed
         unsigned char *dest = buf->pBuffer;

	     data_len = read(STDIN_FILENO, dest, buf->nAllocLen-data_len);

         if(port_settings_changed == 0 &&
            ((data_len > 0 && ilclient_remove_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1) == 0) ||
             (data_len == 0 && ilclient_wait_for_event(video_decode, OMX_EventPortSettingsChanged, 131, 0, 0, 1,
                                                       ILCLIENT_EVENT_ERROR | ILCLIENT_PARAMETER_CHANGED, 10000) == 0)))
              {
            port_settings_changed = 1;

            if(ilclient_setup_tunnel(tunnel, 0, 0) != 0)
            {
               status = -7;
               break;
            }

            ilclient_change_component_state(video_scheduler, OMX_StateExecuting);
            
            
             if(ilclient_setup_tunnel(tunnel+1, 0, 0) != 0)
            {
               status = -7;
               break;
            }
            // now setup the splitter
            ilclient_change_component_state(video_splitter, OMX_StateExecuting);
 
            // now setup tunnel to video_render
            if(ilclient_setup_tunnel(tunnel+2, 0, 1000) != 0)
            {
               status = -12;
               break;
            }

            ilclient_change_component_state(video_render, OMX_StateExecuting);
            
          }
         if(!data_len)
            break;

         buf->nFilledLen = data_len;
         data_len = 0;

         buf->nOffset = 0;
         if(first_packet)
         {
            buf->nFlags = OMX_BUFFERFLAG_STARTTIME;
            first_packet = 0;
         }
         else
            buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN;

         if(OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_decode), buf) != OMX_ErrorNone)
         {
            status = -6;
            break;
         }
         
   /// this section is blocking the loop on the second time around
         
         printf("GET output buffer\n");
	 out = ilclient_get_output_buffer(video_encode, 201, 1);
         printf("got output buffer\n");
         
         r = OMX_FillThisBuffer(ILC_GET_HANDLE(video_encode), out);
	 if (r != OMX_ErrorNone) 
         {
	    printf("Error filling buffer: %x\n", r);
	 }

	 if (out != NULL) 
         {
	    if (out->nFlags & OMX_BUFFERFLAG_CODECCONFIG) {
	       int i;
	       for (i = 0; i < out->nFilledLen; i++)
		  printf("%x ", out->pBuffer[i]);
	       printf("\n");
	    }

	    r = fwrite(out->pBuffer, 1, out->nFilledLen, fd);
	    if (r != out->nFilledLen) {
	       printf("fwrite: Error emptying buffer: %d!\n", r);
	    }
	    else {
	       printf("Writing frame file\n");
	    }
  
	   out->nFilledLen = 0;
         
          if (out = NULL)
          {
                 printf("Error buffer = NULL\n");   
          }
         }
  
      }
Fails on the second loop with "GET output buffer"

dickon
Posts: 216
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Play video and "record"

Mon Mar 27, 2017 6:04 pm

Ah. ilclient. I've not done anything with that.

Aside from the 'int fd' (which should be a FILE * as you're using libc file calls, not OS syscalls), the only thing that leaps out at me is the ilclient_get_output_buffer() call. Do you really need to call this each time around the loop? Just call it once outside the loop, then write the buffer out and pass it back with OMX_FillThisBuffer() each time. Having a look at your event handlers might be useful, too.

Looking back at omxtx, getting this working is much more work than I remember it being. Have a look at it, and see what you're missing. It's at https://github.com/dickontoo/omxtx.

richardcgiroux
Posts: 33
Joined: Sun May 15, 2016 1:54 pm

Re: Play video and "record"

Mon Mar 27, 2017 6:42 pm

Thanks for your comments. I'll look at it immediately.

Return to “OpenMAX”

Who is online

Users browsing this forum: No registered users and 1 guest