- đ meta dropped muse glimmer â a 30 billion parameter open agent model, apache licensed, weights on hugging face.
- đŤ apple’s mlx engine had never seen the architecture and refused to load it, so i added support myself.
- đŹ checked against meta’s official version: 5 out of 5 next words matched, 0.9965 similarity to the reference numbers.
- đŞ opened a pull request to mlx-lm so anyone on a mac can run it the day it came out.
meta dropped muse glimmer yesterday, a 30 billion parameter open model built to run locally and act like an agent, apache licensed, weights right there on hugging face. the local ai crowd lit up fast, and i wanted to run it on my mac the way i run everything else, through apple’s mlx engine. one problem. the engine had never seen this model before, so it refused to load it. brand new architecture, zero support yet.
the easy move is to wait a few days for someone else to add it. i decided to add it myself.
đŠthe handful of things this model does differently
here is the thing nobody tells you about adding a new model to an engine like this. it is not magic and it is not a weekend of guessing. glimmer is close to models the engine already understands, so most of the work is adapting something that already exists and then fixing the handful of things this particular model does differently. glimmer had three of those. it gates its attention through a little sigmoid valve before writing the result out. it normalizes its attention math in an unusual scaleless way. and it splits its layers into local ones that track word position and global ones that deliberately ignore position entirely. get any of those wrong and the model turns into noise.
đŹi wanted proof, not looks fine
the part i care about most is how i checked it. it is easy to write something that produces sentences that look fine and call it done. i did not want looks fine. i wanted proof. so i ran meta’s own official version of the model and mine on the exact same prompts and compared the raw numbers coming out of both. five out of five next words matched, and the overall similarity of the internal numbers came out to 0.9965 out of a possible 1.0. that is not coherent looking. that is matching the reference. the difference from a perfect score is just the rounding you get from running a compressed copy, not a mistake in the math.
đŞsomeone builds the ladder, everybody climbs
then i opened a pull request to mlx-lm, the actual apple project, so anyone on a mac can run muse glimmer the day it came out instead of waiting. that is the whole point of open source. someone hits a wall, someone else builds the ladder, and everybody climbs.
you can see the pull request here: github.com/ml-explore/mlx-lm/pull/1710
this is the kind of work i love.