The K8S Cube runs on 16 Raspberry Pi single board computers. However, there are other considerations for networking, power, storage and casing that I want to go through here.
Raspberries
The Raspberries are a mix of models 3B+, 4B and there may be 1-2 Model 3 boards, too.
BitScope Blade Rack
Running up to 20 Raspberry Pis from 20 powerbricks is not a choice I wanted to make. Back in 2019 when I started my original project, I ordered a Blade Rack from the Australian company BitScope. Their name indicates their background in designing oscilloscopes that work with Raspberry Pis. They have also delivered some clusters with 144 Raspberry nodes in full-length 6U racks.
The BR20 is a 5U rack mount chassis that consists of 10 Duo Pi boards that each can power two Raspberries. The rack provides 5V power through the Raspberry pin header. The power budget may be low for a full set of Model 4Bs, but so far I have been okay with my ragtag mix of nodes.
The power budget is 5V at 30A continuous and 40A peak. I have an old laptop power brick that can deliver that.
Custom Shift Register MOSFET controller
The Blade Rack is going to be tucked away with my network appliances. That involves a ladder and powering Raspberries on and off requires removing a jumper behind the front panel of the Blade Rack. In other words, not nice and not sustainable. The other alternative is switching power on the entire cluster and that is not happening either.
I have designed and soldered a board with two 74HC595 shift registers and a matching amount of MOSFETs to replace the power jumper on the boards. By hooking that into a RP Pico W microcontroller, I can switch 16 nodes on and off at will.
PiKVMv3
I have a PiKVM for remote access to the keyboard, mouse and HDMI of the nodes. By itself, the KVM can access one device, but with the help of a Tesmart 16 port KVM Switch controlled through LAN, I can access all nodes from my laptop.
HP 1920-24G switch
The main features of this 1U rack mount switchs are cheap price and some manageability. There is a confusing web interface and a semi-useless RS232 management port that can be used for mostly nothing.
It does meet my needs though: 24 1Gbps ports, VLAN support.
EdgeRouter PoE (Router, plan A)
My DHCP and DNS server is running on my home router and I have managed to boot Model 3B+ nodes from PXE successfully with some extensive hackery.
EdgeOS running on the device is a fork of Vyatta that is a Linux-based router virtual appliance. So, anything I do, should be possible with a Linux box.
Futro S920 with OPNsense (Router, plan B)
I have an old thin client computer beefed up with 2 NICs and a third with 4 ports. If the EdgeRouter DHCP quirks disturb my normal use of Internet, I can isolate the K8S Cube to another network.
The thin client runs OPNsense nicely.
NAS
I have a NAS that can act as a bigger storage for the cluster. It can run as the initial or secondary location for boot images but I want the cluster to be self-contained and rather serve boot images from the units with SSD.