Given that your serial connections are quite short it should be perfectly possible to connect a serial Tx to the Rx on four other devices. Be sure they are all working at 3.3v else you will probably damage something otherwise.
You will need all the ground pins connected together as well.
There is no way this kind of connectivity should cause a kernel panic. Sounds like you made an error in your connections or something is using 5v or worse still RS232 signal levels. Or you have some other problem in your system.
I presume you have configure the serial ports on your Pi s correctly and have tested this works with just one Tx and one Rx.
A more robust and professional way to so this would be to use some serial line driver chips. Perhaps a multi-drop RS422/RS485 set up. https://www.youtube.com/watch?v=9O_NgoU1CUc
this will help protect your Pi from damage and allow faster data rates over longer distances.
Memory in C++ is a leaky abstraction .