Strange segfault after several hours of running program

[jdonald]

Yeah it did sound too good to be true. Upon looking at OpenVG issue #68 it does not appear related. Sure, VideoCore can freeze up when stress-tested, but the userspace library shouldn't do a rogue memory read even when things are stuck.

6by9's main advice was to stop using IL and just use MMAL so you can still pursue that route, right? Glad you found some sample code. The example I was going to suggest is VLC's --vout=mmal_vout plugin.

Note that both MMAL and IL link with libvchiq_arm.so and may even call the same problematic completion_thread(), so removing all ilclient stuff isn't 100% guaranteed to avoid the crash. You've still got my earlier suggestion of instrumenting the crashing function in libvchiq_arm.so. The code for the library is less than 2000 lines and thus quick to compile. You could add printouts, build with -O0, and use gdb to see everything going on at the point of the bad memory reference. From there, even if you can't make a proper fix you could place in a workaround so that it avoids accessing garbage memory.

These tasks take a bit of investment but if your turnaround time for the crash is several hours then much of this is worthwhile compared to your other options!

Can I run ilclient and SDL2 at the same time? To have them overlay eachother almost?

Good question. As I understand it KMS is designed for exclusive rendering so that likely would not work. For your use-case I think you'd have to write lower level code for a dispmanx overlay. Similarly, that's the plan for restoring subtitles (formerly OpenVG) in omxplayer. Of course if you haven't yet found sufficient benefit from removing OpenVG, and don't require running on a Pi 4 just yet, you can table this until your bigger problems are resolved first.

[ktb92677]

Yeah it did sound too good to be true. Upon looking at OpenVG issue #68 it does not appear related. Sure, VideoCore can freeze up when stress-tested, but the userspace library shouldn't do a rogue memory read even when things are stuck.

6by9's main advice was to stop using IL and just use MMAL so you can still pursue that route, right? Glad you found some sample code. The example I was going to suggest is VLC's --vout=mmal_vout plugin.

Note that both MMAL and IL link with libvchiq_arm.so and may even call the same problematic completion_thread(), so removing all ilclient stuff isn't 100% guaranteed to avoid the crash. You've still got my earlier suggestion of instrumenting the crashing function in libvchiq_arm.so. The code for the library is less than 2000 lines and thus quick to compile. You could add printouts, build with -O0, and use gdb to see everything going on at the point of the bad memory reference. From there, even if you can't make a proper fix you could place in a workaround so that it avoids accessing garbage memory.

These tasks take a bit of investment but if your turnaround time for the crash is several hours then much of this is worthwhile compared to your other options!

Can I run ilclient and SDL2 at the same time? To have them overlay eachother almost?

Good question. As I understand it KMS is designed for exclusive rendering so that likely would not work. For your use-case I think you'd have to write lower level code for a dispmanx overlay. Similarly, that's the plan for restoring subtitles (formerly OpenVG) in omxplayer. Of course if you haven't yet found sufficient benefit from removing OpenVG, and don't require running on a Pi 4 just yet, you can table this until your bigger problems are resolved first.

Okay I think at this point I will take up your suggestion on rebuilding libvchiq_arm.so with debugging so I can really get to the bottom of this error.

I am currently in the process of finishing up the h264 decoder using MMAL (which is proving a bit more challenging than I'd like... the example only shows how to decode the h264 NAL units, it does not show how to render them to the screen. I probably will post the code here when I am done so other people can take a look too. Considering how poorly @6by9 spoke of IL that example really should be removed from hello_pi anyways).

[jdonald]

In another thread someone just mentioned this repo: https://github.com/6by9/drm_mmal
I didn't know about it but this sounds like exactly the example you were looking for. Be sure to use the meson build because that Makefile (to be deleted) is stale.

if MMAL can do the exact same things as IL can and MMAL is well maintained, whereas IL has been abandoned why is the default example for hello_pi/hello_video still using IL?

Well, at this point a lot of the code under under hello_pi consists of deprecated APIs: OpenVG, brcmGLES won't even run on a Pi 4. It always takes some intiative to dig into old examples to do a proper restructuring and I guess as usual there's more priority on supporting modern stuff rather than taking care of legacy samples. Since you're interested in getting that straightened out, you may want to try filing cleanup pull requests on raspberrypi/firmware and I imagine they'd be welcome. Or at least add a note about this in hello_video/README.

[6by9]

IL is such a pain in the neck as an API that I don't want to expend what is going to be a considerable amount of time to dig into the problem you're seeing.
MMAL is a far more maintainable and understandable API, therefore I would put the time in should you encounter an issue using it.

https://github.com/raspberrypi/userland ... s/mmalplay was the test app for video playback that was used back in the Broadcom days. A quick build implies that it still works, although without scheduling of the video frames.
Adding -es to enable scheduling blows up fairly impressively, so I'm guessing something has changed significantly in the firmware since this was last used. I'll try to remember to have a look when I next have the debugger connected.

You can do your own scheduling of video rendering from userspace if you bring the decoded video frame back to your application (preferably as zero copy), and then present them on the video_render. Actually I believe the VPU is only running the same scheduler component as https://github.com/raspberrypi/userland ... cheduler.c, therefore you can probably run that on the ARM.

[ktb92677]

IL is such a pain in the neck as an API that I don't want to expend what is going to be a considerable amount of time to dig into the problem you're seeing.
MMAL is a far more maintainable and understandable API, therefore I would put the time in should you encounter an issue using it.

https://github.com/raspberrypi/userland ... s/mmalplay was the test app for video playback that was used back in the Broadcom days. A quick build implies that it still works, although without scheduling of the video frames.
Adding -es to enable scheduling blows up fairly impressively, so I'm guessing something has changed significantly in the firmware since this was last used. I'll try to remember to have a look when I next have the debugger connected.

You can do your own scheduling of video rendering from userspace if you bring the decoded video frame back to your application (preferably as zero copy), and then present them on the video_render. Actually I believe the VPU is only running the same scheduler component as https://github.com/raspberrypi/userland ... cheduler.c, therefore you can probably run that on the ARM.

Okay I am going to add some closure to this conversation. Thanks to both @6by9 and @jdonald. After switching away from both openVG and iLclient and moving to OpenGLES2 and mmal everything is running smoothly and it has been a week with zero crashes. I am now having trouble overlaying opengles with the mmal h264 stream as a background. But that is a separate topic so I will make a post about that shortly here...

I think we can mark this issue as resolved!

[dividuum]

Mind giving a few insights on how scheduling works now? Previously with OMX I've used the schedule component together with a clock to sync audio and video. How does a similar pipeline now work with mmal? Are you using the schedule component now?