I've been running it on a 64GB M2. My favorite models to run tend to be about 20...

		simonw 9 months ago \| parent \| context \| favorite \| on: Tiny-LLM – a course of serving LLM on Apple Silico... I've been running it on a 64GB M2. My favorite models to run tend to be about 20GB to download (eg Mistral Small 3.1) and use about 20GB of RAM while they are running. I don't have a token/second figure to hand but it's fast enough that I'm not frustrated by it.