DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 10:36 am

Hi All,

I've come across an issue that I've no idea how to fix and after searching t'internet I'm none the wiser so I thought I'd ask the myriad of greatness, the forums here. So, when you pipe the output from an application such as "cat largefile.txt | readerapp" and the readerapp is busy, the cat command will stall waiting for the pipe to be read. In my instance it's yavta which is piping h264 video to an rtsp server application which times out if no one connects to the rtsp server stream within the timeout. No, I can't increase the timeout as the parent process has other things to do such as save an image on a gpio trigger which it won't do if the pipe has stalled. How on earth do I get the parent process to just keep sending data without stalling? Is there a method to call to check the pipe and not write to it if full or there's no reads?

Any ideas?

Kind Regards,

Dan

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26685
Joined: Sat Jul 30, 2011 7:41 pm

Re: Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 10:42 am

So you want to throw away part of the H264 bitstream? How do you propose to deal with a datastream that has chucks missing? Just rely on it resyncing at some point in the future, hopefully the next I-frame?

The sending process needs to check if the pipe is already full, if it is, then throw the data away.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

jahboater
Posts: 5799
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 10:49 am

You might want to look at select() where you can wait until an fd is ready for a write.
There is an optional timeout.

man 2 select

alternatively perhaps O_NONBLOCK on the open might be of interest:

man 2 open
Pi4 8GB running PIOS64

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 11:13 am

Thanks for the replies!

James wrote:
So you want to throw away part of the H264 bitstream? How do you propose to deal with a datastream that has chucks missing? Just rely on it resyncing at some point in the future, hopefully the next I-frame?

The sending process needs to check if the pipe is already full, if it is, then throw the data away.
Loosing the first part of the stream until the next I-Frame would be fine as long as the primary process of watching for a gpio trigger and saving an image was allowed to run. So how do I check if the pipe is full?

Kind Regards,

Dan

jahboater
Posts: 5799
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 11:36 am

DanR wrote:
Tue Jun 30, 2020 11:13 am
So how do I check if the pipe is full?
select()

see "man 2 select"

or possibly just check the return value from a non-blocking write().
Pi4 8GB running PIOS64

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Tue Jun 30, 2020 11:53 am

Thanks jahboater, I'll give that a go! sorry I missed your previous post.

Kind Regards,

Dan

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Thu Jul 02, 2020 10:30 am

Hmm, the problem with using select is that it holds up the flow somewhat even with a really small timeout value set which means the stream gets borked with dropped frames. Can I fopen the pipe with a non blocking flag set instead? I've seen this with "open" but not with "fopen", so my question is really what flag can I / do I set to accomplish this?

Cheers,

Dan

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Thu Jul 02, 2020 11:15 am

So I issued an fcntl to set the nonblocking mode after reading and or'ing the flags. That's not helped, actually it's worse! So I think my only option is to write a quick live555 app to connect locally to the RTSP stream server so the stdout pipe is consumed in a proper manner and prevents the corruption I'm seeing.

Thanks for all the answers though, I've learned something new and that's always a good thing.

Cheers,

Dan

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Fri Jul 10, 2020 7:55 am

Hi 6by9,

What does this error relate to and how can I fix it:

Code: Select all

assertion failure:/home/pi/userland/interface/mmal/vc/mmal_vc_client.c:204:mmal_vc_lookup_client_context():client_context_pool.contexts[CLIENT_CONTEXT_MAGIC_MASK(index)].inuse
Am I not giving the vc routines enough time to finish or is there something else I need to be doing??

I'm also getting this when I run sudo vcdbg log msg:

Code: Select all

2825279.844: mmal: mmal_port_event_send: event lost on port 1,0 (buffer header callback not defined)
2825281.775: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
2825281.857: mmalsrv: send_buffer_to_host: tx failed:size 292 st -1
Kind Regards,

Dan

DanR
Posts: 84
Joined: Fri Jan 18, 2013 1:20 pm

Re: Writing to a pipe stalls if it's not read

Tue Jul 21, 2020 8:52 am

Hi 6by9,

I'm having trouble with my version of yavta at the moment (added image encoder functionality) and I can't seem to find the cause of one of two errors listed below which happen around 57mins to 1hr20. Do you have any suggestions on where to look? What does the assertion failure error mean?

Error 1 and probably the most important one to fix

Code: Select all

assertion failure:/home/pi/userland/interface/mmal/vc/mmal_vc_client.c:204:mmal_vc_lookup_client_context():client_context_pool.contexts[CLIENT_CONTEXT_MAGIC_MASK(index)].inuse
Error 2 which only happens once in a while

Code: Select all

select timeout
Many Thanks,

Dan

Return to “C/C++”