- Pi4 2GB with JMicron(eh) M-key SSD adapter and 0.5TB Intel's SSD (short USB3-A extension cable is on the way) acting as tftp and nfs server
- 2x Pi4 4GB (two for now - another 2-3 allowed) as PXE net booted(*) workers
[*[all three running 64bit Raspberry Pi OS (**)
- all three have snap version of microk8s installed(**) (server has scheduling switched off - it is 'cordoned')
- Microk8s's metallb happily talking to my Draytek server and assigning (static as requested in yaml) IP to 'pods' (read containers) (- not IPs associated with real interfaces, it uses BGP to talk to my router)
- cluster has its own private docker repo (working of nfs persisted volume)
- and my containers can happily use nfs persisted volumes, too
(*) speaking of PXE boot - it worked like a charm: in tftp root I had directories that match mac addresses of workers to contain their boot dir which only has cmdline.txt amended to match IP given to them by my rooter. Also, it order to avoid any messing up with secondary DHCP server and such malarkeys, I've added TFTP_IP in firmware config (with IP my router assigns to the server Pi4) and that was it. No other stuff was needed for PXE boot!
(**) in order for Microk8s to work on workers I had to recompile kernel (it was far, far less of an issue than I originally was worried might be - see here viewtopic.php?f=63&t=281061&p=1702371#p1702371)
I've been pleasantly surprised with BGP working just out of box and quite glad I avoided need for proxying all through the server.
Picture is work in progress - couple of bracket needed to be made and all hanged on the wall somewhere - only three cables are going away from the cluster: power for smart switch and USB power supply and cat 6 ethernet cable.
And to conclude: Raspberry Pi (in this case version 4!) is really a lots of fun!