jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: Strange segfault after several hours of running program

Wed Jul 31, 2019 7:08 am

Yeah it did sound too good to be true. Upon looking at OpenVG issue #68 it does not appear related. Sure, VideoCore can freeze up when stress-tested, but the userspace library shouldn't do a rogue memory read even when things are stuck.

6by9's main advice was to stop using IL and just use MMAL so you can still pursue that route, right? Glad you found some sample code. The example I was going to suggest is VLC's --vout=mmal_vout plugin.

Note that both MMAL and IL link with libvchiq_arm.so and may even call the same problematic completion_thread(), so removing all ilclient stuff isn't 100% guaranteed to avoid the crash. You've still got my earlier suggestion of instrumenting the crashing function in libvchiq_arm.so. The code for the library is less than 2000 lines and thus quick to compile. You could add printouts, build with -O0, and use gdb to see everything going on at the point of the bad memory reference. From there, even if you can't make a proper fix you could place in a workaround so that it avoids accessing garbage memory.

These tasks take a bit of investment but if your turnaround time for the crash is several hours then much of this is worthwhile compared to your other options!
ktb92677 wrote:
Mon Jul 29, 2019 5:52 pm
Can I run ilclient and SDL2 at the same time? To have them overlay eachother almost?
Good question. As I understand it KMS is designed for exclusive rendering so that likely would not work. For your use-case I think you'd have to write lower level code for a dispmanx overlay. Similarly, that's the plan for restoring subtitles (formerly OpenVG) in omxplayer. Of course if you haven't yet found sufficient benefit from removing OpenVG, and don't require running on a Pi 4 just yet, you can table this until your bigger problems are resolved first.

ktb92677
Posts: 28
Joined: Fri Sep 20, 2013 10:29 pm

Re: Strange segfault after several hours of running program

Wed Jul 31, 2019 8:44 pm

jdonald wrote:
Wed Jul 31, 2019 7:08 am
Yeah it did sound too good to be true. Upon looking at OpenVG issue #68 it does not appear related. Sure, VideoCore can freeze up when stress-tested, but the userspace library shouldn't do a rogue memory read even when things are stuck.

6by9's main advice was to stop using IL and just use MMAL so you can still pursue that route, right? Glad you found some sample code. The example I was going to suggest is VLC's --vout=mmal_vout plugin.

Note that both MMAL and IL link with libvchiq_arm.so and may even call the same problematic completion_thread(), so removing all ilclient stuff isn't 100% guaranteed to avoid the crash. You've still got my earlier suggestion of instrumenting the crashing function in libvchiq_arm.so. The code for the library is less than 2000 lines and thus quick to compile. You could add printouts, build with -O0, and use gdb to see everything going on at the point of the bad memory reference. From there, even if you can't make a proper fix you could place in a workaround so that it avoids accessing garbage memory.

These tasks take a bit of investment but if your turnaround time for the crash is several hours then much of this is worthwhile compared to your other options!
ktb92677 wrote:
Mon Jul 29, 2019 5:52 pm
Can I run ilclient and SDL2 at the same time? To have them overlay eachother almost?
Good question. As I understand it KMS is designed for exclusive rendering so that likely would not work. For your use-case I think you'd have to write lower level code for a dispmanx overlay. Similarly, that's the plan for restoring subtitles (formerly OpenVG) in omxplayer. Of course if you haven't yet found sufficient benefit from removing OpenVG, and don't require running on a Pi 4 just yet, you can table this until your bigger problems are resolved first.
Okay I think at this point I will take up your suggestion on rebuilding libvchiq_arm.so with debugging so I can really get to the bottom of this error.

I am currently in the process of finishing up the h264 decoder using MMAL (which is proving a bit more challenging than I'd like... the example only shows how to decode the h264 NAL units, it does not show how to render them to the screen. I probably will post the code here when I am done so other people can take a look too. Considering how poorly @6by9 spoke of IL that example really should be removed from hello_pi anyways).

jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: Strange segfault after several hours of running program

Thu Aug 01, 2019 2:56 pm

In another thread someone just mentioned this repo: https://github.com/6by9/drm_mmal
I didn't know about it but this sounds like exactly the example you were looking for. Be sure to use the meson build because that Makefile (to be deleted) is stale.
ktb92677 wrote:
Fri Jul 26, 2019 5:04 pm
if MMAL can do the exact same things as IL can and MMAL is well maintained, whereas IL has been abandoned why is the default example for hello_pi/hello_video still using IL?
Well, at this point a lot of the code under under hello_pi consists of deprecated APIs: OpenVG, brcmGLES won't even run on a Pi 4. It always takes some intiative to dig into old examples to do a proper restructuring and I guess as usual there's more priority on supporting modern stuff rather than taking care of legacy samples. Since you're interested in getting that straightened out, you may want to try filing cleanup pull requests on raspberrypi/firmware and I imagine they'd be welcome. Or at least add a note about this in hello_video/README.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7527
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Strange segfault after several hours of running program

Fri Aug 02, 2019 9:53 am

IL is such a pain in the neck as an API that I don't want to expend what is going to be a considerable amount of time to dig into the problem you're seeing.
MMAL is a far more maintainable and understandable API, therefore I would put the time in should you encounter an issue using it.

https://github.com/raspberrypi/userland ... s/mmalplay was the test app for video playback that was used back in the Broadcom days. A quick build implies that it still works, although without scheduling of the video frames.
Adding -es to enable scheduling blows up fairly impressively, so I'm guessing something has changed significantly in the firmware since this was last used. I'll try to remember to have a look when I next have the debugger connected.

You can do your own scheduling of video rendering from userspace if you bring the decoded video frame back to your application (preferably as zero copy), and then present them on the video_render. Actually I believe the VPU is only running the same scheduler component as https://github.com/raspberrypi/userland ... cheduler.c, therefore you can probably run that on the ARM.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ktb92677
Posts: 28
Joined: Fri Sep 20, 2013 10:29 pm

Re: Strange segfault after several hours of running program

Fri Aug 09, 2019 7:42 pm

6by9 wrote:
Fri Aug 02, 2019 9:53 am
IL is such a pain in the neck as an API that I don't want to expend what is going to be a considerable amount of time to dig into the problem you're seeing.
MMAL is a far more maintainable and understandable API, therefore I would put the time in should you encounter an issue using it.

https://github.com/raspberrypi/userland ... s/mmalplay was the test app for video playback that was used back in the Broadcom days. A quick build implies that it still works, although without scheduling of the video frames.
Adding -es to enable scheduling blows up fairly impressively, so I'm guessing something has changed significantly in the firmware since this was last used. I'll try to remember to have a look when I next have the debugger connected.

You can do your own scheduling of video rendering from userspace if you bring the decoded video frame back to your application (preferably as zero copy), and then present them on the video_render. Actually I believe the VPU is only running the same scheduler component as https://github.com/raspberrypi/userland ... cheduler.c, therefore you can probably run that on the ARM.
Okay I am going to add some closure to this conversation. Thanks to both @6by9 and @jdonald. After switching away from both openVG and iLclient and moving to OpenGLES2 and mmal everything is running smoothly and it has been a week with zero crashes. I am now having trouble overlaying opengles with the mmal h264 stream as a background. But that is a separate topic so I will make a post about that shortly here...

I think we can mark this issue as resolved!

User avatar
dividuum
Posts: 188
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Strange segfault after several hours of running program

Fri Aug 09, 2019 10:03 pm

Mind giving a few insights on how scheduling works now? Previously with OMX I've used the schedule component together with a clock to sync audio and video. How does a similar pipeline now work with mmal? Are you using the schedule component now?
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

Return to “Graphics programming”