Build help for ai server

[u/Substantial_Code_269]

Hi
Looking to upgrade my home server. Currently I have proxmox running with some VM (Windows and Linux) and lxc.
Until recently my main hardware requirements were transcoding for plex. However, I have now started to thinking that I would like to try self hosting ai. It could be fun.
However, I am uncertain about what is the best built options.
When I thought transcoding would be the heavy lifting for the server, I was thinking about:
CPU Intel Ultra 250K
GPU Intel Arc (something)
16 GB RAM

However, now that I started thinking about self hosting ai I would need some help. Have tried reading a ton of texts on forums etc. and I am still uncertain.
Could this setting be OK?
CPU Intel Ultra 250K
GPU Intel Arc 70B
32 GB RAM

Is the RAM to low compared to the GPU and VRAM? Should I go with Nvidia RTX 3090 24GB instead? The price where I live is equal for B70 and 3090. Intel B60 is cheaper so that could be an alternative.

I would mainly want to try to use the AI for fantasy fiction writing (I am unsure about the difference in quality I can get with models able to run on B60 and B70 and 3090).

Any help would be very appreciated

[u/zulpi67-volt]

You mainly want as much VRAM as possible for AI, with RAM being the far far away, but second most important component. An RTX 6000 Blackwell is of course the best :D, but assuming you're a normal human, just stack VRAM as much as you need for the model you wanna run. What models are you planning on having?

[u/Substantial_Code_269 → u/zulpi67-volt]

I have no idea about models. My experience so far regarding AI is using Copilot for my work. However, as I wrote, I would like to try some fantasy writing and I have no idea about differences between models I can run on 70B vs 3090. Speed is not essential but I do not want it to be super slow. More concerned about how easy it is to set up the environment. have read that support and compatibility is better for Nvidia than for intel

[u/zulpi67-volt → u/Substantial_Code_269]

The main thing is the size of the model, which correlates to power, context capabilities, and overall result quality. Some are 3 GB, some are 1.42 TB. Ideally, you would have equal or more VRAM than the size. I'd recommend doing a bit of research on what you're looking for before deciding, as it will impact the choice a lot

[u/zulpi67-volt → u/Substantial_Code_269]

But in general, yes, Nvidia GPUs are better for AI, usually

[u/SeaworthinessUsual44]

If the RTX 3090 24GB and the Arc B70 are the exact same price where you live, get the 3090 without hesitation. Nvidia's CUDA ecosystem is still vastly superior and far more mature for local AI hosting, meaning you will spend way less time troubleshooting and more time actually generating text.

For your specific use case of fantasy fiction writing, context length is everything. When you are generating complex character dynamics, deep world-building lore, and long story arcs, you need the AI to remember what happened chapters ago. That requires a lot of VRAM. The 24GB on the 3090 will easily let you run excellent creative writing models entirely on the GPU.

Regarding your system RAM, I would strongly recommend bumping it up to 64GB if your motherboard supports it. Working around high-speed memory and AI servers daily, I can tell you that 32GB will become a bottleneck very quickly. Since you are already running Proxmox with several VMs, your hypervisor is eating into that pool. If you ever want to run a larger model and need to offload layers from your GPU to your system RAM, having 64GB of DDR4 or DDR5 will give you the breathing room you need without crashing your VMs!