I put a new version of the kernel source and binaries on github. This is based on the foundation tree rather then the upstream source. I still don't have sound or accelerated video working yet. Also the builtin wifi and bluetooth isn't working yet either. The number of changes from the foundation tree is very small.
1. Most of the kernel modules that are included in the standard distros are now included.
2. Reboot and shutdown work now.(sudo reboot actually works)
3. The foundation MMC driver is used rather then the upstream driver. No more wierd lockups or severe performance issues.
4. The device tree is built from the same source as the 32 bit version. The source(dtsi) has a few "#ifdef" lines for the differences for 64 bit.
In both cases the correct branch is rpi-4.5.y+rpi364.
Depending on how much interest I find, I might post another version in a week or two.