NEW Free Open Source AI Platform

[wardmundy]

From your favorite billionaire...

Muse Glimmer | Meta

An open 30B model for always-on local agents. Runs on a single GPU, Apache 2.0 licensed. Tuned for tool use, long tasks, and failure recovery.

developer.meta.com

muse-glimmer

Meta's latest open model built for always-on local agents. 30B parameters, licensed under Apache 2.0 and runs on a single GPU — tuned for tool use, long tasks, and failure recovery.

ollama.com

Download Ollama on macOS

Download Ollama for macOS

ollama.com

Ollama documentation - Ollama

docs.ollama.com

 

[KNERD]

There is also Kimi K3 which is said to be as good as Claude Fable, but you need several hundred GB of vRAM to run it

moonshotai/Kimi-K3 · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

 

[kenn10]

you need several hundred GB of vRAM

A bit more ram than most of us have.

 

[Halea]

On a Linux desktop with 32GB RAM and a 8 core / 16 thread AMD CPU Muse Glimmer is quite slow even with the Vulkan driver installed and the integrated GPU actively used.
But the gpt-oss-20B model works quite well and it's very responsive. My test/learning environment is LMStudio, not Ollama although I do my integration work on the latter.
On a mini-pc with a recent generation CPU (intel or AMD) just make sure that the integrated GPU is enabled and recognized. That improves responsiveness and allows enabling higher levels of reasoning.
Also, a web-search plugin is useful for situations where you need to do a generic search or pull data from a specific website. For that "npacker/web-tools" works quite well while it limits random web exposure.

 

[KNERD]

On a Linux desktop with 32GB RAM and a 8 core / 16 thread AMD CPU Muse Glimmer is quite slow even with the Vulkan driver installed and the integrated GPU actively used.
But the gpt-oss-20B model works quite well and it's very responsive. My test/learning environment is LMStudio, not Ollama although I do my integration work on the latter.
On a mini-pc with a recent generation CPU (intel or AMD) just make sure that the integrated GPU is enabled and recognized. That improves responsiveness and allows enabling higher levels of reasoning.
Also, a web-search plugin is useful for situations where you need to do a generic search or pull data from a specific website. For that "npacker/web-tools" works quite well while it limits random web exposure.

This is why all the Mac Minis are being bought up. The have what is called "unified memory" where it not only shares the system RAM with the GPU, but you can manually set how much can be shared with each space. With 128GB of RAM, people are setting 96GB for the GPU and 32GB for the operating system. As far as I know, the only other company doing this is AMD with their Ryzen AI MAX+ systems, but at a premium cost.

For a CPU based option, Google has been developing their open-source Gemma models for CPU usage. Gemma 4 26B-A4B can run on CPU using llama.cpp by offloading Mixture-of-Experts (MoE) layers via the --n-cpu-moe flag

 

[Halea]

... The have what is called "unified memory" where it not only shares the system RAM with the GPU, but you can manually set how much can be shared with each space. With 128GB of RAM, people are setting 96GB for the GPU and 32GB for the operating system. As far as I know, the only other company doing this is AMD with their Ryzen AI MAX+ systems, ...

Not sure if what I am going to say is 100% accurate but I noticed something interesting on LMStudio's hardware status page. Once the Vulkan driver is installed and it starts reporting on the GPU, it also shows how the 32GB RAM is being split between the CPU and the GPU. And to my surprise it somehow manages to assign 5GB to the GPU an 26 to the CPU (and 1GB for the video buffer) while it's running the gpt-oss-20b model. When I switch to Gemma-4 with like 7B parameters, according to the same status window the RAM is split equally between the GPU and the CPU (including the video buffer) right in the middle.
Under linux I have never been able to adjust that split. That's why I am wondering if the status reported is accurate or the result of some sort of "hallucination" (ha ha). On this machine typically 2GB of RAM is assigned to the GPU at boot time and about 1GB to the video controller and the balance to the CPU (~29GB). There is a BIOS setting for the GPU memory but once Linux boots it's always overwritten with the default values.
And no, my Ryzen 9 is definitely not a AI model.