Why is that better than just requiring the SAT, which while imperfect does a great job at weeding out students who have A’s in math but can’t solve 8th grade algebra problems.
I remember at Google at around 2007 - 2009, as Google was massively expanding its data centers, there was a lot of unused capacity, especially during off-hours. Any engineer could run as many jobs as they wanted at zero priority, which means the job would be first in line to be killed if a more important task needed the resource.
I did so many interesting experiments with MapReduces that would run overnight.
For a while, I would even build internal services that were basically "free" because I'd just run them all at priority 0.
Over time those services got less and less reliable as overall usage started to increase, so I was forced to either justify the resources or scale back - but that was a good thing.
I feel like something similar would be a good model for AI token use: big tech companies ought to have their own self-hosted LLM data centers to power their own needs, then let employees use off-hours capacity to experiment.
Outside of experimentation, we should be encouraging token efficiency for everyday tasks. Rather than having a certain number of tokens, engineers should be evaluated based on how much they actually get done.
Using a lot of tokens to automate a process that used to require hours of human labor every week? Good use of tokens, should be encouraged.
Using a lot of tokens to debug an easy frontend bug that could have been fixed by hand, and still took you 4 hours to complete? Waste of tokens, should be discouraged.
"Using a lot of tokens to debug an easy frontend bug that could have been fixed by hand, and still took you 4 hours to complete? Waste of tokens, should be discouraged."
Hahahah good luck with that!
For many of us, what is happening now was super obvious. Telling a new formed crack addict (who you wanted to become addicted) to be more thoughtful about their consumption of crack... yeah not gonna work is it.
Don’t think we’ll see similarly logically behaviour from LLM users tbh. A sizeable portion of the user base seems to insist on through opus at every trivial task
Most AI front ends seem to be designed for interactive jobs, so they make it hard to define a job that should be done eventually with zero priority. It makes much more sense to do that with spec-driven development (have work done with the human on the loop rather in the loop), but as far as I know that just isn’t well supported by any front end yet (would be happy to be proven wrong, my experience is with Google front ends).
my money is on: eventually frontier model dev and training becomes basic research funded by governments, and LLM operators become essentially private utilities a la ISPs, competing mostly on data center operational costs and occasionally new chip tech to run models cheaper
and governments will keep running massive data centers with classified frontier models for intelligence and propaganda purposes
I'm using a brand-new MacBook Pro with a high-end M5 processor, and this site is extremely unresponsive for me. Huge latency between clicking and getting feedback.
It also breaks accessibility.
The QR code use case seems far more reasonable to me, you're generating a static image.
It definitely wasn’t “working great”, I’ve now fixed a bug in the event system that was causing loops. The performance should be improved now.
Yes, accessibility is a key concern of mine, I’m keen to explore html-in-canvas as a way to have an accessibility tree combined with a rendered interaction.
I’m personally at a stage where React and CSS have pushed to DOM to extremes of complexity and difficultly in optimization that I desire simpler ARIA-based HTML combined with custom rendering. I’m hopeful that it will be easier to test, lighter for users, and faster for everyone.
There are periods of time where I might spend 80% of my time "coding", meaning I have minimal meetings and other responsibilities.
However, even out of that 80% of my time, what fraction is actually spent "writing code"?
AI can be an enormous accelerator for the time I'd normally spend writing lines of code by hand, but it doesn't really help with the rest of the work:
- Understanding the problem
- Waiting for the build system and tests to run
- Manually testing the app to make sure it behaves as I'd like
- Reviewing the diff to make sure it's clear
- Uploading the PR and writing a description
- Responding to reviewer feedback
There are times when AI can do the "write the code" portion 10x faster than I could, but if it's production code that actually matters, by the time I actually review the code, I doubt it's more than 2x.
>AI can be an enormous accelerator for the time I'd normally spend writing lines of code by hand, but it doesn't really help with the rest of the work:
- Understanding the problem - Waiting for the build system and tests to run - Manually testing the app to make sure it behaves as I'd like - Reviewing the diff to make sure it's clear - Uploading the PR and writing a description - Responding to reviewer feedback
What part of those you think it doesn't help with?
They can make it unnecessary for you to understand.
Consider hash tables. Nobody implements a hash table by hand any more.
I've written some, but not in this century.
Optimal hash table design is a specialist subject. Do you know about robin hood algorithms? Changing the random number generator's seed to discourage collision attacks?
A basic hash table starts to slow down around 70% full. Modern hash tables can get above 90% full before they have to expand.
Who keeps Knuth's Fundamental Algorithms handy any more? I own both the original edition and the revised edition. They're boxed up in the garage. I once read that book cover to cover. That was a long time ago.
That's not AI. That's solving the problem and putting it in a black box. That's how technology progresses.
That's obviously not what I'm talking about. If you're asking an AI to write an optimal hash table algorithm, something is clearly wrong. I'm talking specifically about understanding the business domain and problem you are trying to solve.
> That's not AI. That's solving the problem and putting it in a black box. That's how technology progresses.
The key word is solving. Meaning someone, after coming up with the solution, has taken times to prove that it works well in all usual and most extreme cases. With their reputation on the line.
That’s why you trust curl, ffmpeg, Knuth’s books,… but you don’t trust random cat on the internet. We don’t trust AI and the cost to review its output is not a great tradeoffs compared to just think and solve the problem.
Also it's highly multithreaded / multiprocess - you can run subagents that can communicate with each other, you can interrupt it while it's in the middle of thinking and it handles it gracefully without forgetting what it was doing
