HOME GAMES APPS CODE DESIGNS ELECTRONICS DOWNLOADS LEGACY

PI-Pythia

GOAL

An AI Test bed named after the High Priestess of the Oracle of Delphi.

CONCEPT

Must run Ollama

Must be local

Must be able to interpret documents

PURCHASES

Controller	Raspberry Pi 5 (8GB)	£76.80
Hard disk	ADATA LEGEND 800 500GB PCIe Gen4 x4	£45.95
Case	Argon ONE V3 M.2 NVME Raspberry Pi 5 Case	£46.00
Power	Raspberry Pi 27W USB-C Power Supply	£11.60

TOTAL		£180.35

Processor	Intel Xeon W-2135 - 6-Core 12-Threads 3.70GHz (4.50GHz Boost, 8.25MB Cache, 140W TDP)	£35.00
Heatsink	Lenovo ThinkStation Heatsink	£5.00
RAM	2 x 16GB - DDR4 3200MHz (PC4-25600, 2Rx8)	£30.00
Hard Drive	1 x 512GB - NVMe SSD	£30.00
Hard Drive	NVMe Caddy Heatsink	£5.00
Graphics Card	Gigabyte nVidia GeForce RTX3060 Gaming OC - 12GB GDDR6 FH PCIe4.0-x16 GPU	£200.00
Case	P520 [TPM V2.0, 900W PSU] B Grade	£140.00
Power cable	UK Plug to C13 (Kettle Lead) Power Cable	£2.00

TOTAL (upgrades)		£447.00

IMPLEMENTATION

So I went with a Raspberry PI to try out some basics. The case proved tricky to get working with no power and plenty of unscrewing and trying different things before it worked.

Installing Ollama was a breeze and the WebUI is an amazing tool. Performance was slow though. I was looking for a graphics card upgrade to improve my pc when I found one as part of a re-conditioned server which was cheaper than buying new and came with a warranty. I therefore built up a custom made server using a Xeon processor. I loaded up a command line Linux installation with no UI and installed Ollama and the extra NVidea drivers. The results were really, really good with Llama 3.1 8B able to return very quickly.