6by9's main advice was to stop using IL and just use MMAL so you can still pursue that route, right? Glad you found some sample code. The example I was going to suggest is VLC's --vout=mmal_vout plugin.
Note that both MMAL and IL link with libvchiq_arm.so and may even call the same problematic completion_thread(), so removing all ilclient stuff isn't 100% guaranteed to avoid the crash. You've still got my earlier suggestion of instrumenting the crashing function in libvchiq_arm.so. The code for the library is less than 2000 lines and thus quick to compile. You could add printouts, build with -O0, and use gdb to see everything going on at the point of the bad memory reference. From there, even if you can't make a proper fix you could place in a workaround so that it avoids accessing garbage memory.
These tasks take a bit of investment but if your turnaround time for the crash is several hours then much of this is worthwhile compared to your other options!
Good question. As I understand it KMS is designed for exclusive rendering so that likely would not work. For your use-case I think you'd have to write lower level code for a dispmanx overlay. Similarly, that's the plan for restoring subtitles (formerly OpenVG) in omxplayer. Of course if you haven't yet found sufficient benefit from removing OpenVG, and don't require running on a Pi 4 just yet, you can table this until your bigger problems are resolved first.