An update: Apparently I came across another post elsewhere which seemed to imply that busybox needs to be statically linked and not dynamic, which seemed to be the issue for me as well. I changed the build config to make it static linking and then the kernel panic disappeared.
I now am able to reach the Kernel console successfully, but I have a strange problem now. I reach /sbin/init and then the UART output to the host PC freezes. But when I connect HDMI to the Raspberry Pi, all is good and I get the console on the HDMI display.
Do you know of any solution to this problem? This seems to be reported here already but no one seems to have a conclusive solution for it .(viewtopic.php?t=270723