It started with a single RTX 3090. I bought it about two years ago. After a few weeks I realized I needed more VRAM, so I bought a second one. That's how it always starts. Two years later I have 32 RTX 3090s across 4 servers, 768GB of VRAM total, connected with InfiniBand and running on solar power. This is how it came together, including the parts that went wrong and the points where I almost stopped. Two GPUs, then four The real turning point was four GPUs. Once I had four cards running, I knew I wanted more, so I bought two more. At six cards the problem became PCIe lanes. I switched my motherboard from a Supermicro H12SSL-i to an ASUS ROMED8-2T to get more slots, and built my first 6-GPU server. As soon as that server was done, I knew I wanted a second one, and I wanted to connect them into a cluster. Finding InfiniBand Connecting two servers is harder than it sounds. I looked at llama.cpp RPC over Ethernet, but the latency was too high for what I needed. So I started reading about how large datacenters build their clusters. That's where I first saw InfiniBand. At first it looked like something only big datacenters use, but the more I read, the more I understood it. I started checking eBay for Mellanox switches and network cards, and after some research I went with an SB7800 switch and ConnectX-6 cards. Installing, updating, and configuring all of it was a challenge. I might write about that separately. One thing I'd recommend: clone your SSD right after you get your switch. Make the clone first, then experiment, so you can always roll back. The other issue with ConnectX-6 is heat. These cards run very hot. I spent time designing and 3D printing covers for them and testing different fan setups. The software stack Two servers, 12 GPUs, InfiniBand working. Then I hit the hardest part: the software. For my use cases I needed vLLM + Ray. I spent many days and nights testing and rebuilding. This was one of the points where I was close to giving up. I kept digging. After probably 100+ different configurations and tests, all running in Docker, it finally worked. It was a small model, but it started and gave real output. That was the moment I decided it was actually possible and I was halfway to my goal. I started building the third server right away. Risers and PSU problems With three servers and 18 GPUs on InfiniBand, the risers became the problem. I'd been using all kinds of gamer risers, from 20cm to 60cm, and they weren't stable. I kept getting errors. I looked for something better and found MCIO. I started replacing the gamer risers with MCIO adapters from ADT-Link. They came with SATA power, which I resoldered to the GPU power connector. That made managing GPU and MCIO power much easier. The next problem was the server PSUs. I'd been using Dell D2400E-S0 units, also with a lot of soldering. Sometimes, completely unpredictably and not related to load, a PSU would reset power on a GPU. It was hard to debug because I couldn't reproduce it. I replaced them with Super Flower Leadex 2000W units. After the risers and PSUs were sorted, the cluster finally ran stable and predictable. Going to 32 Models kept getting bigger, and 18 GPUs wasn't going to be enough. MCIO gave me extra PCIe, so I added two more GPUs to each server and built the final server as a full 8-GPU machine. Rebuilding the older rigs and assembling the last server took some time. When it was done: 4 servers, 32 RTX 3090, 768GB VRAM, one cluster. Solar power At full load the cluster pulls up to about 10kW, and I didn't want all of that on the grid. So it runs on its own solar setup: 20 solar panels on the main array (~12kW peak), plus 2.5kW and 4kW additional arrays Deye SUN-16K-SG01LP1-EU 16kW hybrid inverter 2× Seplos Mason batteries, 16.076 kWh each, 51.2V, LiFePO4 On sunny days it's fully covered by solar. What it's for I run models locally with full privacy and full control, processing a lot of data every day. Right now it's running Kimi K2.6, and I plan to run every top open-weight model that fits in VRAM. The cost reason is simple too: once you process billions of tokens a month, running your own hardware is cheaper than paying for an API, and your data stays with you. Cost The question I get most: roughly 30k just for the GPUs and server components. I bought most of it right before the memory crisis, which helped. Solar, batteries, and cooling are on top of that. What's next Proper benchmarks are coming, with real numbers, and I'll publish them. There's also a v2 rig planned with some changes based on what I learned. PS: Sorry for the photo quality. If this is interesting for you, I'll keep writing about more technical details, configs, and the solar setup.