🔩the handful of things this model does differently
here is the thing nobody tells you about adding a new model to an engine like this. it is not magic and it is not a weekend of guessing. glimmer is close to models the engine already understands, so most of the work is adapting something that already exists and then fixing the handful of things this particular model does differently. glimmer had three of those. it gates its attention through a little sigmoid valve before writing the result out. it normalizes its attention math in an unusual scaleless way. and it splits its layers into local ones that track word position and global ones that deliberately ignore position entirely. get any of those wrong and the model turns into noise.
🔬i wanted proof, not looks fine
the part i care about most is how i checked it. it is easy to write something that produces sentences that look fine and call it done. i did not want looks fine. i wanted proof. so i ran meta’s own official version of the model and mine on the exact same prompts and compared the raw numbers coming out of both. five out of five next words matched, and the overall similarity of the internal numbers came out to 0.9965 out of a possible 1.0. that is not coherent looking. that is matching the reference. the difference from a perfect score is just the rounding you get from running a compressed copy, not a mistake in the math.
🪜someone builds the ladder, everybody climbs
then i opened a pull request to mlx-lm, the actual apple project, so anyone on a mac can run muse glimmer the day it came out instead of waiting. that is the whole point of open source. someone hits a wall, someone else builds the ladder, and everybody climbs.
you can see the pull request here: github.com/ml-explore/mlx-lm/pull/1710
this is the kind of work i love.
🤗the weights are up if you want them
i quantized muse glimmer 30b for apple silicon and put it on hugging face the day after this. it has been pulled a couple hundred times in its first two days, which for a brand new upload is more than i expected: huggingface.co/divinetribe/Muse-Glimmer-30B-Abliterated-MM-bf16
everything else i quantize for macs lives on one page with live download counts: all my open weight models.