It's strange that my iPhone 14 is at regular temperature when using the E2B model. But also it's a lot slower (not sure how to measure the exact tokens per second, ~12 if I had to guess)
Under 15 is too slow for conversation personally. I guess 5 tokens per second is nice if you're one of the people who likes letting coding agents run overnight
I'd rather see a distill on the 26B model that uses only 3.8B parameters at inference time. Seems like it will be wildly productive to use for locally-hosted stuff
There can't be any interesting discussion about AI programming. Every conversation boils down to what skill files you use, or how Opus 4.6 compares to Codex, or how well you can manage 16 parallel agents.
There genuinely is a lot of interesting discussion to be had about LLMs, and I know this is true because I discuss things with my coworkers daily and learn a lot. I do admit that conversation online about LLMs is frequently lacking. I think it's a bit like politics - everyone has an opinion about it, so unfortunately online discourse devolves to the lowest common denominator. Hey guys, have you noticed that if you use LLMs frequently it's possible you'll forget to think critically?
But "there can't be any interesting discussion about AI programming" is completely false.
For me, almost every single time a conversation like this happens in real life it boils down to the one side claiming that "This is the future" and "Don't get left behind" followed by a torrent of hype and buzzwords. So no, there is no interesting conversations to be had about LLM programming anymore.
Some interesting conversation one can have with coworkers specifically:
1. How should code review and responsibility for code be updated to a) increase velocity, b) keep quality and c) keep reviewers from burnout. There are plenty scenarios in which vibe coding a component in an afternoon is the correct choice, even if it is buggy, insecure, and no one really understands it.
2. Which parts of the codebase work well with code assistants, which don't? Why? What could be changed to make it easier? In my experience, Claude Code sometimes loses its mind on infra topics. It is also not very good at complex, interconnected services (humans aren't either).
3. Which tasks could be offloaded to agents to save everyone time and sanity? - Creatig Jira Tickets from meeting transcripts is an obvious one, collecting and curating bug reports another one.
4. How should we design systems to better work for coding agents? Does it influence our tech choices? Should it influence them?
5. Is AI a net positive or negative for security?
And so much more. The last topic in particular is incredibly important, and things are developing so fast that you can probably have a new conversation on it every two weeks.
Maybe you struggle to have good conversations because I just provided an anecdote and you immediately stated that my anecdote is false? If this is how you typically interact with people I’m not surprised you’re not having interesting conversations.
He didn't state your anecdote is false, the first two words of his comment are "for me". That means in his experience, not yours.
Ironically, in your crusade to wave the "I'm being censored!" flag, the only person who is trying to do any censoring... is you!
And to top it off, as if that wasn't enough, you're also incredibly snarky and basically implying that this person who just vaguely disagrees with you must be unlikable or something. Which, in another twist of irony, actually makes YOU appear unlikable, because what well-adjusted adult would feel the need to throw someone under the bus for slightly disagreeing with them?
The problem is that mrcsharp added the last sentence: "So no, there is no interesting conversations to be had about LLM programming anymore." So they are definitely trying to turn their anecdote into a universal.
Most of your comments about johnfn are still apropos, though...
> "there is no interesting conversations to be had about LLM programming anymore"
The first sentence was in his experience, but this is a universal assertion. He is claiming no one, in the world, is having interesting conversations about LLMs.
Is my response really so off-base? Imagine you say "I like Rust because it made my app go fast" and someone replies "There is no one who has used Rust to improve performance." Do you really think that's a normal way to respond to someone sharing an anecdote?
But that's not how he responded because there's a whole ass comment before that.
Okay sure, if you read the comment and then use a Men in Black mind wiper thingy before reading the last sentence, then it might seem brazen or universal. But that's not what you did.
The only way that last comment can reasonably be taken to mean "for everyone on Earth" is if you did not read the lines before it. Because, in that context, to me, it's clear he is only talking about his experience.
This is a phenomena I've noticed lately where everyone feels the need to add a disclaimer for everything and not doing so is seen as an "aha gotcha!" type thing. But we're not algorithms. You do not read one line at a time and then digest it.
You're human, he's human, and there's context. You know that it would be extremely unreasonable for someone to think that nobody, anywhere, has anything to say about LLMs right? Okay. That doesn't mean that this person is being unreasonable.
Read his comment in context. The comment thread, condensed, is:
mudkipdev says "There can't be any interesting discussion about AI programming". We agree that is a universal claim. (Right? I mean, your comment says "You know that it would be extremely unreasonable for someone to think that nobody, anywhere, has anything to say about LLMs right" but isn't this a clear example?)
I say "There can be". We agree that is anecdotal.
mrcsharp says "For me, [stuff that supports mudkipdev]. Therefore, there is no interesting discussion".
He is re-asserting mudkipdev's point. mudkipdev says A, I say !A, he says, actually, A.
Your interpretation has him read the back-and-forth between me and mudkipdev, and respond to "A", "!A" with "B". If you only read mrcsharp and nothing else in the thread I can understand this reading, but the context changes things.
My pet peeve with all LLM discourse is whenever someone mentions any problem they experience with LLMs or any mistake they make, someone comments that humans make the same mistake.
I disagree and you could reduce basically anything to this: 'there can‘t be any interesting discussion about React. Every conversation boils down to which framework you use or how you manage state or whether you use typescript or javascript‘
All of those are opinions about programming. Which framework, which language, etc.
Conversations about which model to use aren’t conversations about programming.
A better analogy would be some topic that you can’t discuss without it boiling down to which text editor you should use. It’s related to programming, a little. But it’s not programming.
That is exactly why I left reddit. r/javascript had almost completely abandoned JavaScript discussions for React and Angular while r/programming was half filled with irrational JavaScript fear nonsense.
That isn't why /r/programming banned it. They banned it because every discussion about LLMs inevitably devolves into discussions about AI slop in varying levels of civility, and the rare good LLM submissions/discussions do not offset it.
Other tech-adjacent subreddits such as /r/rust have banned LLM discussion for similar, more pragmatic reasons.
Genuine question: how to distinguish yourself from the stream of slop?
I am also annoyed by the endless stream of articles and projects related to LLM-assisted coding. Not because I dislike LLM-assisted coding as an idea, but because it's all more of the same (as you said). I think that there are still a lot of low-hanging fruit in improving LLM harnesses that no one is working on because everyone seems to be chasing the latest trends ("agentic", "multiagentic", "skills") without thinking bigger.
But I'm afraid that if I finally invest time and implement some of my ideas on making LLM-assisted coding better (reliable, safer, easier for humans to interpret and understand generated code), I won't be able to gather any feedback. People will simply dismiss it as "yet another slop for creating more slop" and that's it.
Like saying theres no interesting discussions about programming. Just whether OOP is overhyped, python is slow, how well you can convert a c codebase to rust
You have not seen my recent WhatsApp chats. Me and a pal are talking about what we're doing with Claude code, and it's quite interesting!
Just like discussions about traditional programming never were only about syntax and type systems, AI discussions aren't only about prompts and harnesses. I find there's quite a bit of overlap actually! "How do you approach this problem?" Is a question that is valid in both discussions, for example.
It simply means the tokenizer's training corpus may have included a massive amount of German literature or accidentally oversampled a web page where that word was frequently repeated. Look up "glitch tokens" to learn more.
Yes, I agree that the value of Apple's chips is hard to beat, but there's still a massive bottleneck with hardware accessibility. The 128 GB MacBook described costs over $5,000 on their website, and in the consumer space, even the most often recommended GPU which is a 3090 with 24 GB VRAM, you can find used going for $700 at minimum. This effectively prices out all non-professional users who don't have money to spend to simply have parity with the $20 subscription on their phones. (and even for those who do, they have to cope with the fact that the model will always be dumber than ChatGPT, and that their hardware will grow outdated very quickly)
Also a noticeable disconnect between the hardware we have and the primary focus of open-source labs (scale up and cater to their enterprise customers, just look at GLM-5's increase to 744B parameters, double from GLM-4.5's 355B). We really just need some kind of Cambrian explosion in cheap hardware for local models to be feasible.
Curious if you agree that local is where Apple's bet is long term - it's out of reach now, but I found the jump in capability for the top line laptop interesting. Presumably chip development hasn't focused all that much on running LLM's for all that long, I'm wondering what kind of jump we'll see two or three releases down the line.
reply