Ryzen 5900X, 64 gig DDR4-3200, 2tb ssd,10tb hdd and an RTX2070. Hosting Stable Diffusion, various llama.cpp instances with python bindings, jellyfin, sonarr, multiple modded minecraft servers, and a network file share.
Ryzen 5900X, 64 gig DDR4-3200, 2tb ssd,10tb hdd and an RTX2070. Hosting Stable Diffusion, various llama.cpp instances with python bindings, jellyfin, sonarr, multiple modded minecraft servers, and a network file share.
Nice, you must be into deep learning with such a setup, any particular reason the deep deep learning models and GPU run in your server rather than in a powerful desktop system? Maybe you’re actively offering AI services to the outside world?
Yup, mostly running pretrained models for text embedding and some generative stuff. No real fine tuning.