I find 35B A3B viable as well, but your harness and runtime really matters to get tool calling and such dialed in. In fact, I would encourage you to experiment with it some as I find I get more reliable output from 35B A3B, though 27B is still generally smarter. A3B with a review cycle or two from 27B is great for me.
One of the reasons is, with good specs and design, A3B is just so fast. It isn't as smart as the 27B model, but it is close enough it can usually figure it out with the right tools.
Usually something like dsv4 running them. Depends in the horizon. It’s kind of an overnight thing for the project. Use matt pocock skills or similar. Build good spec. Iterate on it with a big model or your brain. Break it down into pieces and features like you would for a human. Tell them to work on tickets. Have other agent review. Repeat. Stuff gets built. You can keep a smarter agent in a loop (think bash) to review and have a simple decision tree: implement, review, mark done, pick next ticket. When no tickets stop. If error or pathology detected, touch a stop file. That is what I run overnight. A3B likes a simple harness as does 27B, mostly unmodified Pi with a proxy to fix model bugs. It takes some investment. It isn’t batteries included. These small models are not very smart. They need a pretty narrow task domain.
Same! The only reason I'm not using it more is because it's summertime. I'm not in any hurry.
Setting the memory to "fast timings" is good for 8-12% more tokens/second if you haven't tried yet. I miss the slightly older days of AMD when powerplay tables were unlocked and we could configure the timings and voltages manually, there's another 30% being left on the table ez
What do you mean by 'Setting the memory to "fast timings"'? The only runtime I can get working for my GPUs is llama.cpp, which I haven't seen anything like that in its argument set. My perusal of the options for vllm and sglang didn't suggest anything similar either before failing miserably.
I think they are referring to the AMD drivers on windows, under the overclocking section you can enable fast memory timings. Not sure if this sort of thing is exposed on linux.
