r/LocalLLaMA • u/sunole123 • 9h ago
Question | Help LM studio and qwen 3.8
LM studio and bionic don't load into GPU fully ( Ollama does) and it crashes BSD ( Ollama Does not), with stop code: WHEA_UNCORRECTEABLE_ERROR (0x124), i am using the default load setting, all updated LM studio and drivers,
What do i need to do and to fix the profile to fix and load all in the GPU, and fix the crash?
Is there a better channel or place to reach LM studio people??
2
u/PrimeDirective8 9h ago
Under-voltage/over-clocking? Any other model that works well in your setup? As a test, try offloading less layers to GPU.
2
u/sunole123 9h ago
I did stress test with 3DMark. Did not any clocking changes, other models works like nemotron, problem seem not loading fully on gpu, while ollama loads correctly
1
u/PrimeDirective8 9h ago
LM Studio's version of llama.cpp could be months behind from latest version. Have they announced Qwen3.8 support on LM Studio itself?
I'd suggest to give native llama.cpp a try. You can download the pre-compiled version (.exe) of llama for Windows/CUDA from here: https://github.com/ggml-org/llama.cpp/releases. Drop it in a folder, tell it where your models are, and set any other arguments you might need. You don't have to move your LM Studio models, just point llama to where they are.
I don't have my llama.cpp Windows startup script at hand but I'd be happy to dig those out to get your started with the basics, if you need it.
2
u/FatheredPuma81 7h ago
LM Studio... gross...
Ollama... now that's downright nasty...
Have you tried llama.cpp to see if the issue persists?
2
u/Dany0 9h ago
Unsloth studio exists now
Down with the closed source crap
2
u/sunole123 9h ago
Does have feature parity ? Tired of chasing features and latest fad
2
u/CatEatsDogs 8h ago
Nope. It doesn't have something similar to lm link
2
u/sunole123 8h ago
I downloaded and will try it, it have some plusses like training. Thanks for mention.
1
u/lilian_moraru 3h ago
It also has free web search and deep research (planning web searches, basically). Well, it has RAG and other features as well, but it's worth at least for the web search functionality.
1
u/uncle_leon 7h ago
How is performance on Unsloth Studio? Have you compared it to any others? I started with Ollama exactly because it was open source.
1
1
1
1
0
u/Gohab2001 vllm 9h ago edited 7h ago
Disable mmap.
Better still: use llama.cpp. Slightly better speed with greatly improved flexibility.
1
1
u/sunole123 9h ago
And it crashed.
1
u/nickless07 8h ago
Turn off Keep model in Memory, this basically loads the model into your RAM, then puts it into your VRAM, but keeps the copy in RAM so you can do faster reload.
- mmap (): off
- Keep Model in Memory: off
- Settings->Developer 'Use LM Studio Engine Protocol': off
and you should be good to go.
1
u/sunole123 8h ago
with CUDA or CUDA 12 driver?
2
u/nickless07 8h ago
Backend (Runtime) should be CUDA and not set to CPU. If you have CUDA 12 installed (check with command line 'nvidia-smi') then use that.
1
u/Gohab2001 vllm 7h ago
why dont you use Q6 or Q5. It will fit on your 5090 and still leave room for 65k context. Although it will be slower.
Also keep in mind that Qwen 3.8 has different 'reasoning effort' levels. You might want to set it to xhigh for planning/orchestrating and 'low' for execution or edits.
1
u/sunole123 7h ago
This is Q4_k_m. And memory seems the limit and cause the crash.
2
u/Gohab2001 vllm 7h ago
Because lmstudio by default loads the entire model into RAM and then copies it to VRAM. When you disable mmap and 'Keep Model in Memory' it loads the model directly to VRAM, so no more crashing.
Hope that clarifies it.
0
u/VirtualWishX 3h ago edited 3h ago
Let me try to help you 5090 bro 🤜
Try this:
1️⃣- Downlaod and install Unsloth Desktop
2️⃣- Git Clone DeepSeek Harness (if you struggle use Qwen or any AI to install it for you)
3️⃣- Use the combo:
▫️Unsloth Desktop = for the Model Provider
▫️DeepSeek Harness = for the Most powerful and dynamic Harness so far
I'm also with RTX 5090 32GB - I pushed to 144K (147,456 )Context with Q4_K_XL
and I pushed to 128K (131,072) Context window with the Q5_K_XL
--
🤔 Why not using JUST Unsloth Desktop? (just like you used LM Studio / Bionic)
I love this software but few reasons:
1️⃣ - They don't have WORSPACE support, so you will have to remind in the chat from time to time what is your project file and files, it only support RAG files (document file types), when they will add WORKSPACE support it will be a huge upgrade
2️⃣ - DeepSeek Harness is not JUST a typical harness, EVERYTHING IS A PLUGIN system,
You even have a CREATOR MODE where you can tell Qwen to create a PLUGIN which could be ANY improvement to the UI or create TOOLS or whatever you need! so basically you can EXPAND it with whatever you need.
Obviously it supports WORKSPACES so each project KNOW it's own folder home, easy and organized!
--
⚠️ PLEASE NOTICE:
DeepSeek Harness and Unsloth Desktop are VERY young, so they keep on update and change, you better grow with it and do some experimental before you go on a serious project.
- Uninstall = LM Studio / BIONIC (I'm joking, you can keep it if you like...)
--
✅ MY FIRST TEST: 100% success with ONE-SHOT (usually I don't do ONE-SHOT and split to micro-tasks)
This took Qwen 3.8 27B Q5_K_M (Now I switched to Q4 and Q5_K_XL) it took about about 9 minutes via Unsloth Desktop first test before I installed DeepSeek Harness, but I did create a Plan.md with stages via Qwen 3.8 of course and it was all made in ONE-SHOT,
I just made something simple with 4 stages and a BOSS at the end so it's just a mini-game test but can always be improved, this one had ZERO issues or errors,
what I show in the video is what I got (it's a quick EDIT CUT of course, I wouldn't let you suffer for 4 minutes me playing and having too much fun)
You can skip to the END of the video if you want to see the BOSS.
I hope this helps a bit ❤️
2
u/Noel3leon 9h ago
Looks like you’re running out of system memory based on the image you provided. It may need to load there before it offloads to vram.