I am very new in this Raspberry matter, although have some experience with Buildroot. As much as I can see, you have bult text mode only and kernel altogether. Also, I think kernel compiling results in uImage file. So, where did you put it (instead of kernel.img ?) ? Have you tried (if it is possible) to boot via NFS - I consider it to be good for development. If so, can it be done without u-boot ?
I hope I can add Qt to your config to achieve graphic.