SpaceX is headed by a person who is a strong ally of a politician who openly challenges Denmark's sovereignty over Greenland. Guess you wouldn't mind selling your organs to the same group of powerful people for a few bucks because you're not virtue signalling?
Right, you could disagree on which things to prioritize over dollar profits. My main point is that these preferences are not irrational like was asserted. At the scale of a sovereign wealth fund or pensions, you need to care about externalities; in the case of Denmark vs SpaceX you have something relatively concrete, in other cases we need to keep in mind that the goal of these funds are to improve the welfare of who they serve, and see past the dollar signs to take into account the consequences of the investments.
An article with a title saying tokens per second throughput without any qualifier e.g. what size the model is should immediately be classified as spam.
Google needs to catch up on what? Devs mindshare? The latest Opus 4.8 carefully selected benchmarks made sure to pick Gemini 3.1 Pro and not Gemini 3.5 Flash: 3.5 Flash is beating Opus 4.8 on several of the benchmarks Anthropic posted but simply was ignored.
I don't think SOTA-wise Google has a lot of catch up to do.
Gemini 3.5 Flash is not good at coding in practice. Gemini 3.1 Pro too, in particular is known to be bad at tool calls. Many companies would love to have alternatives to Claude Code (as it's a significant risk to depend on one vendor), so far most of the buzz is about moving to Codex but much fewer talk about moving to Gemini. All these benchmarks are not very informative, the Chinese labs do better on these benchmarks than in practice, for example.
Lots of things that are wrong are also imposed on parents. Not to say involuntarily childless people are to be blamed for anything, or that two wrongs make a right, but society is immensely misaligned against having children, and forced charity already exists in various forms whether you like it.
But honestly, developed countries not having children itself isn't that bad a thing. I feel that our existence and the hedonistic treadmill drains too many scarce resources, and population growth should not last long. On the other hand, it seems societies still gain productivity in spite of the slow population growth. There should be plenty of slack for everyone, so that middle-class parents don't feel like they are constantly in a deathmarch, and voluntarily childless people don't need to be pressured. There's an immense misallocation of resources that is hard to solve, and you end up seeing proposals like this.
Unsurprising he'd be cheered for saying what they wanted to hear.
But perhaps whether or not his stance is correct, the students needed to hear this. They (we) have to believe human brains still have value and find a way out; for otherwise there'd be no point to try anymore.
> They (we) have to believe human brains still have value and find a way out; for otherwise there'd be no point to try anymore.
Our value isn't predicated on our utility. The simple fact that we are sentient beings, capable of joy and suffering, gives us value. This is why we continue to support and care for the elderly and the disabled - we value them regardless of any practical utility we may derive from them.
If you go through life believing that your value depends on your practical utility, then things like AI are an incredibly scary existential threat. But denial is not a healthy way to cope with this threat. The solution is to recognize the value inherent in us as humans, and to demand public policies that reflect this fact.
A view that is not shared always by LLM cheerleaders. Part of Sam Altman's defence of the environmental impact of AI is that it is less than that of a human life.
"He said it was unreasonable to focus on "how much energy it takes to train an AI model, relative to how much it costs a human to do one inference query."
"It takes like 20 years of life and all of the food you eat during that time before you get smart," he said. "And not only that, it took the very widespread evolution of the 100 billion people that have ever lived and learned not to get eaten by predators and learned how to figure out science and whatever, to produce you."
It took a 100 billion people and their knowledge,experience to generate the data to train an AI. So that cost also comes under the environmental costs to build his version of AI.
unless he plans to freeze the training data at this point and use that for another billion years, the cost of building AI will always be more than the cost of humanity.
It's odd that whenever someone discovers a way to generate value from public noise, costs already paid, that they feel like they are being stolen from even though PPP for the average person will rise due to AI, not fall.
I think it's a little disingenuous to describe the notoriously-amoral (pathologically dishonest, accused of being a sociopath) CEO of the major AI company as a mere "AI cheerleader". He's not exactly a central example of the category.
Altman is closer to a storybook evil CEO than he is to an average "AI cheerleader".
"A cynic is a man who knows the price of everything, and the value of nothing." - Oscar Wilde
Or one I prefer, though unattributed: "If the only lens through which you can view life is value in currency, that which is priceless becomes worthless."
> The simple fact that we are sentient beings, capable of joy and suffering, gives us value.
People will understandably ask, what is the actual value of being capable of joy and suffering?
I frame it another way. There is value in affording all beings dignity, respect, and the opportunity to thrive. The question of our individual value as a being is undignified. People can be more or less valuable to a particular effort, but there should be no question about their worth as a person. It should not be a part of how we understand people and ourselves.
It is a healthy conclusion that your value doesn't depend on your practical utility, because that will come and go and is sometimes beyond your control. Your value isn't a question at all.
There's no value in life, but life should be allowed to exist. Who's to say otherwise?
The lifeless dust and rock of the moon is an simpler value proposition to quantify than the messy intrinsic value of overlapping, ever-changing life here on Earth.
Life is valuable to life. Or to say it another way, there is no concept of value beyond the reckoning of living things, even the value of dust and rock on the moon.
Every year half a million children die of diarrhea. There are so, so many people in the world, who are capable of joy and suffering, who "we" don't care much at all. However I have a feeling that "we" might be joining that group eventually.
Still, it’s vastly fewer now relative to the total number of children born than any previous time in human history. It could be even fewer had birth rates begin to drop instantly as a response to child mortality dropping dramatically even in most developing countries, rather than with a few-generation delay.
Yea, and I will take it a step further; it is really easy to start to worry about the “worth of a human life” when it’s yours. When we are in the position to not care about the worth of a human for our gain(such as children working to make iPhones for us to use cheaper) we call it economics.
But our modern day capitalist hellscape has made it extremely clear that if you aren't capable of providing value for shareholders, your life literally has no value. That's the reason the US government keeps cutting welfare programs, why union suppression exists.
The fact of the matter is that unless you are producing value for shareholders, you don't get to participate in society and are left to starve to death. No amount of flowery language is going to feed and house the unemployed. And we are running full speed into a situation with the explicit and overt goal of cresting as many unemployed people as possible while simultaneously ensuring that there are no resources or help offered to those unemployed people.
Flowery language will cover up the starving bodies in the streets the same way a can of febreeze will cover up a landfill. This is an enormous problem and if we don't fix it, people will die. Whether or not a human has intrinsic moral value by simply existing, we require money to survive in this society. A human life may be a mystical beautiful and valuable concept, but our society has determined that if you don't have money, you literally do not deserve to live.
That's what these students are so angry about. They're being pushed into a world that refuses to employ them and which delivers a death sentence for the crime of unemployment.
You're conflating society with the white collar job at hand. Yes, if you don't provide value for shareholders, your life is worthless _to that company_. The company is in the business of making money. The businesses goals are a microcosm; a small subset of society. There are many other ways to live (and live well, I might add).
Our joy and suffering depends on our connection with other people, and being useful to other people has since the dawn of humanity been one of the ways we connect.
Our joy and suffering depends on our ability to fulfil our basic needs like shelter and food, and in current socioeconomic environment that ability intimately depends on the ability to create wealth, which in turn depends on our ability to provide value to others.
I can see both being undermined. What public policies reflecting this do you envision?
Yes it is. If human life was inherently valuable then the concept of poverty wouldn't exist because the entity that sees it as valuable would be willing to spend resources on maintaining it.
> The solution is to recognize the value inherent in us as humans, and to demand public policies that reflect this fact.
Most social programs keep expanding until they become unsustainably expensive. You can't just make a law "everyone gets free money" and expect this to have no negative consequences.
>If human life was inherently valuable then the concept of poverty wouldn't exist
This only proves that injustice exists. Surprise: injustice still exists.
I'm hoping that you're still young and primarily motivated by survival, which can lure you into this cold world view. I think the reality is an inversion of that old "if you're not liberal at 20 you have no heart, but if you're not conservative by 30 you have no brain" chestnut.
Hopefully once you've made it past the raw basics of survival and the feelings of a dog-eat-dog world, you can look back and realize that compassionate people helped you over and over throughout your life, maybe without you even realizing it at the time. The next step is to realize that you can extend that same compassion to others.
If human lives weren't inherently valuable, the concept of charity wouldn't exist. Where does that leave us? I think probably the line of argument doesn't work in either direction.
Likewise, most of the time you don't have social programs, somebody will introduce social programs. You can't just say "no social programs" and expect this to have no positive consequences... okay this is falling apart a bit, but the point is, what makes 'not expanding UBI' so much harder than 'not introducing UBI'? If you can convince people that introducing UBI will lead to expanding UBI and that that is bad, what's stopping you from just convincing them of the latter?
For the past 500 years humans were existing with their labor value in the very center.
Do you expect the enlightenment project to be resumed and finished in a year? Or do you imply some religion-based worldview where the value is derived from God, flesh, nature?
No sir, grim times are coming. We can only hope that the current AI wave bites more than it can swallow.
"Value" is a word with many meanings. Your value as a human or a living being may be very different from your value to your employer or your value to the taxman or anywhere else.
It is very easy to get lost in between them, especially when listening to a good speaker who can flitter between those meanings at will.
What is worse is that those values interact. We indeed we continue to support and care for the elderly and the disabled, but only up to a point, and there is a reasonable discussion how exactly should countries divide their limited resources between vulnerable groups, including families with young kids. In that context, the future economic and societal value of a 5 y.o. vs. a 85 y.o. inevitably creeps up.
>If you go through life believing that your value depends on your practical utility
My value definitely depends on my practical utility. Everyone is capable of joy and suffering. A chicken is capable of joy and suffering.
Tell me this: in a hypothetical situation where you are going to end up on an uninhabited island and you can choose one person to go with you, who would you peek? An elderly disabled one or a person that has necessary skills to survive? Maybe utility matters?
I don't think that's quite right, unless you personally value joy for its own sake. I value knowledge, and joy is useful to creating knowledge, and suffering is harmful to it. But I don't want to have some futile joy, and I don't need to avoid some irrelevant suffering.
Otherwise you get effects like;
* Just take drugs, feel meaningless "joy" because that's what you value,
* Don't do anything less "joyful" even though it's more meaningful.
I'll admit that knowledge isn't practical, and you can't always identify when you're creating it, and a lot of people don't think in these terms and there's a lot of intuition involved, along with societal mores about caring for people which help the growth of knowledge as general rules without getting all bean-counting about it. But I think it matters that hedonism is an incoherent motivation and that creating knowledge is a far clearer one (and hedonism tends to turn into creating knowledge, anyway, if you like meaning). Hedonism, utilitarianism, same difference.
> Otherwise you get effects like;
> * Just take drugs, feel meaningless "joy" because that's what you value,
> * Don't do anything less "joyful" even though it's more meaningful.
These are entirely valid positions to take though. Obtaining knowledge for knowledge's sake isn't objectively more meaningful, even if it may be subjectively more valuable to you.
You could make the point that teaching, and thus furthering the collective knowledge of our species, may be somewhat objectively meaningful, because you impact the trajectory of humanity. But unless you draw joy from that specific fact alone, the joy from creating knowledge is just as selfish as taking drugs to attain a state of bliss (which, again, I don't oppose either.)
Also, I'd even challenge the notion that knowledge alone, at its face value, automatically equates to a benefit for humanity. Harari has made that point far more eloquently than I in Nexus.
Oh, the Sapiens guy. I read Sapiens, thought it was OK, then other people picked holes in it and persuaded me that it was worse than that. But I suppose that doesn't preclude this Nexus book being good.
But anyway I agree: motivations are arbitrary. Why you even got to do a thing? Just sit and be sessile and die. (This is not a personal attack, or recommended.)
I rely heavily on an assumption that we do all have more or less the same set of values - but this might be cultural, not biological: it's hard to get inside the head of, say, Aztecs, with whatever strange non-modern values they had.
I also make an assumption about knowledge being central among those values, although it's definitely not all that, and some people will say they don't even consider it. But I think they are doing anyway, if they live in the world as we know it.
Side comment: you've made "joy" separate from "bliss" and "meaning" separate from "knowledge", and then there's some undefined "benefit for humanity" that might not be any of those things, along with the apparent value of "impacting the trajectory of humanity" - is that good, just impacting it, in any non-specific way? lol terminology.
I think you missed my point. The distinction I made in terminology was on purpose: I used "joy" to describe the inherent motivation for trying to accomplish something, and "bliss" for the state some may try to reach by using drugs.
And I also made a distinction between knowledge and meaning, which you sort of seem to imply is a universally shared value, while I seriously doubt that is the case. There are many ways to derive meaning from existence that do not involve amassing knowledge - even just passively profiting off of the knowledge of others, but taking no curiosity in that at all.
And as you pointed out, I carefully phrased impacting the trajectory of humanity to avoid implying any moral judgement. People have many reasons for wanting to leave something behind that outlasts them, which may be good or bad or anything in between.
Obviously you'd want to name "joy" as a separate thing, to propose it as the thing we're motivated to do, but the problem is that you didn't describe it. So now I'm at: the thing we're motivated to do is the thing we're motivated to do, and it's not whatever I say it is, but apart from that you haven't told me anything about it. Of course I'm open to some pluralism, like it can be a string bag of mixed motivations, but I do think the motivations in our culture all agree with creating knowledge, and become vacuous without that element. What is "gain", "pleasure", etc., without meaning? (I don't know what you mean by meaning. I mean the process of explaining and learning and creating ideas.) Without it those are mechanical processes of the "number go up" type. Yes, I am skeptical that anything of that kind is anybody's deep motivation, though it may be a superficial one.
Why are you trying to avoid morality? That seems like a good way to never find out anything important, since importance is a moral judgement.
> the thing we're motivated to do is the thing we're motivated to do, and it's not whatever I say it is, but apart from that you haven't told me anything about it.
That’s just what I mean: I oppose your notion of a universally shared motivation of deriving meaning from creating knowledge. I don’t have an alternative to offer, because I believe no such objective motivation exists. Instead, it occurs to me you project your own belief system onto humanity (or at least your society) as a whole.
Yes, our world might be shaped most dramatically by those with a desire to create knowledge, but that still doesn’t support the generalisation that humans universally consider the creation of knowledge as a way to give their life meaning.
> Why are you trying to avoid morality?
I try to avoid bringing it into the question of what is and what isn’t a valid motivation for a continued existence, because that is one of the most fundamentally subjective aspects of being a sentient creature. Who am I to make a judgement?
....I think it's a fairly widespread view to value joy for its own sake. In fact, I would say that's pretty much how normal people would say they view joy.
So normal people don't make sense, what else is new.
The problem here is that joy-in-itself isn't anything. Say you're a huge hedonist, and you try to maximize your pleasure. Maybe you start with some notion involving a speedboat and cocaine. Then you might ask, how can you maximize your pleasure even more? That means you have to ask why you like things. You like things for reasons, and reasons have meaning, and meaning is knowledge. So maybe your next step is to add music or something. But in doing this your activity isn't just having pleasure, it's finding things out. The more you work at maximizing pleasure, the more you're finding things out, and the less of a cliche the things you enjoy are, and pleasure-in-itself becomes less real, because it never really meant anything. The alternate path is to stick closely to the cliches, ride around coked-up on your speedboat forever, and fail to really have a good time because mechanical behavior isn't genuinely enjoyable and trying to maximise pleasure is self-defeating.
Your examples make it clear that you do not understand normal people in the slightest. And when I say "normal" here, I'm not talking about the Blazing Saddles-style "salt of the earth; you know, morons." I mean roughly the middle 50-60% (by width) of the bell curve. The vast majority of humanity.
Normal people don't self-reflect, realize they like joy, and then try to maximize this with speedboats and cocaine. If they realize they like joy and want to maximize it, they do things like spend more time with their families.
THIS is going to be the limitation of capitalism. Capitalism isn't compassionate. It's a really good economic framework though, so it will be interesting how that's reconciled in the coming years
Money and monetary systems aren't compassionate -- people are.
Historically (in the USA), capitalism was paired with charity and supporting those around you (primarily for religious reasons).
One of the greatest downsides of the welfare system is that people don't give the money to others themselves (it's instead stripped from them and doled out without their input). They don't get to experience the good feelings that come from helping another person (only negative feelings about the government taking their money).
This removes the habits of practicing selflessness and it's positive feedback loop. As a result, we get all the downsides of capitalism with a trained selfish cohort who have no charitable feelings to counterbalance things.
There's also the other side. The beneficiaries of the system never feel the need to be grateful, as the money is legally theirs. So you end up with selfish/uncharitable on one side and ungrateful/envious on the other.
What do you win at the end of the race? I've never heard it concisely put. 'Dominance' is the word that comes to my mind, but I don't want to put words in your mouth and don't really know why that would inherently be a valuable trophy, so that's probably not what you were thinking of, right?
You are right. I'm not saying I like the way things are trending. But I also think setting up our children means making sure they're prepared for the future. I don't think AI will ever replace the importance of education.
Why should we care about that? Even if you wanted to argue our individual fates are tied to our country's, we don't all live in the same country, so how, actually, could we all care? Are you really convinced its so zero sum like this?
We collectively spend decades and decades creating a sophisticated global capitalism, huge networks and infrastructures of trade and travel, just to find ourselves in some dark forest-esque race with everyone else anyway? Is this really consistent to you? What was the point of anything in the last, like 40 years to you if we just need to act like we are still in a cold war, except this time its a war with everyone?
"Other countries" means China here, I think. China got a little on board with the global capitalism (and lifted 800 million people out of extreme poverty along the way, if we're looking for the point), but never really embraced Liberalism and so ideally isn't the one aligning superintelligence. It would be lousy if Russia or North Korea or Somalia was in that position and it would be fine if the UK or Denmark or Brazil or Ghana was, but none of that matters because none of them will be in that position. Only the US and China are playing the game.
If this speculated intelligence is so "super" why would it matter what its host country's commitments are? I would hope it would at least be intelligent enough to sort things out there? How can something be so potentially threatening, so "super," but also be like a baby, where we need to worry how its raised? Its super intelligent about everything except ideology? That doesn't really sound like (super)intelligence to me..
But ok, even granting that framing, if the issue is China's placement on the spectrum of "liberal", what would it take for them to be the good enough guys here?
I think it's more of a prestige discussion. Who leads, who follows, between countries. Not whose super intelligence is better. Who got their first, and how are they using it?
A supervillain is nevertheless super. Intelligence doesn't correlate with morality- most probably there doesn't exist a 'true ideology' that can be solved for in absolute terms. Do you imagine a superintelligence could calculate how beauty 'ought to' be traded off against truth?
When I say China is a bad choice because it's not as liberal as ~the west, I do imagine a reader in China thinking the opposite. I don't think they're dumb and I don't think they've been duped; they have a coherent ideology that fits their values. I just don't want it to stomp out mine.
Maybe I'm wrong and you can solve for morality or at least the meta-morality of Liberalism/pluralism where you permit various moralities to coexist. Hopefully so. Maybe the value system in China is closer to mine than I imagine and it's just operating under different constraints. But I don't want to gamble on that when winning is within reach and is a guarantee given alignment to any human values is achieved at all.
I spent more than half my day yesterday telling Claude to correct itself because it did things I explicitly told it not to do in my prompt.
“You’re right - I overstepped”
Is the new “You’re absolutely right”.
I don’t know if we can qualify something that actively goes against the explicit instructions you give it as “something great”. It just sounds like Dario is building snake oil and selling it too.
I have a script at work that writes out some config files and I'm having Claude run them after making changes. The script if it detects breaking changes will spit out a message saying what the breaking changes are, and not do anything, telling you to rerun it after validation with the override flag.
If I don't tell Claude about this behavior, it ignores the script output and lies about passing tests that validate if the config files were regenerated.
So I added to my prompt instructions to observe it, and if it sees that message, double check its work and then inform me and ask what to do before proceeding.
This has had the net result of Claude either running the script with the override flag from the get go (explicitly forbidden) or it seeing the message and convincing itself that the override is warranted and running it a second time with the override flag. It's never once stopped to ask me what to do like instructed.
This is one of a few reason I strongly prefer GPT and its codex variants. It seldom frustrates me, sure its not omnipotent in any way, but it just feels very "tuned in" when it comes to understanding intent and scope.
Imagine worker that did loop of "you're absolutely right -> same fuckup again" multiple days every week, wasting time of whoever told them to do the task
I do want to fire Claude at this point and switch to Codex. Unfortunately the guy with the purse strings is ride or die full Claude psychosis and our business can’t afford to just buy anything and everything for funsies.
That depends on the company. I worked at an S&P 500 company that muddled along like this. They still make critical software for local and state governments.
Something changed with Dario a year or so ago. I think he started out with good intentions, although really hard to tell. Maybe it was really all about power and control for him from day one. Certainly now he's a different person - appears totally corrupted by money and power.
Dario used to at least emphasize the potential positives of AI while being worried about the negatives, but unlike Hassabis/DeepMind he has done nothing to bring about the positive part and is now just accelerating the harmful part as fast as he can. Google is an AI company, bringing us things like AlphaFold, and Anthropic (also OpenAI) are just LLM companies.
It's just the worst version of capitalist game theory. If I don't do the bad thing and get rich, then someone else will do the bad thing and I won't get rich.
But this new tool is not a blacksmith’s tool in the traditional sense. It’s more like an automated blacksmith that works fast, for cheap, does mediocre work, but has this mediocre skill level in an exceptional broad range of tasks.
Why not? Blacksmithing and coding have a hell of a lot in common. In both disciplines toolmaking is extremely important. Often you have to make custom tools to accomplish a design--e.g. a twisting wrench or a form tool. Sometimes you have to make tools that get used once and thrown away, like a jig temporarily welded to a piece to hold it in place while you build its sibling assembly. Sound familiar? I do this kind of thing all the time in code.
Another similarity is the relative simplicity of the underlying structure of the system. You essentially have two hammers (one small one you swing with your hand and another big one that is planted on the ground), some material, and some heat. You build the rest.
Another similarity is the resistance to automation. A skilled blacksmith is a versatile worker. You can create assembly lines to automate any one thing they might produce. The end product will not have the same quality--it will not truly be wrought iron, each piece will not be unique, there will be nothing of the aesthetic taste of the artist in it, but if you're just some bean counter who doesn't care about those things you'll be able to sell it. But if you need the optionality to produce any of those things.. automation is not your friend. And some things just cannot be automated, at least not without extreme costs or very poor results--shoeing horses comes to mind.
We’ve also done true intelligence a disservice by using AI to name the current implementation of LLMs. It’s stretching ‘intelligence’ quite a bit. They can be super useful, but we’ve downplayed how phenomenal the human brain is.
I really like ‘Actual Intelligence’, that’s a clever one from Woz. People need to be reminded to use their brains, they’re a brilliant product of evolution (or your favourite god’s work).
No AI company is addressing the elephant in the room that you need someone experienced constantly monitoring any agentic workflows. This means that the cost savings of agents are a myth.
My company actually did an internal study of agent usage for coding and found it only improved productivity by 10-20%, basically on the same level as good code templates or an autocomplete.
I maintain a part of my team's CD process and I've observed a 30% increase in PR velocity since we started adopting agentic tools but it was a "one-off" increase (as-in, it hasn't continued to increase beyond that since about a half-year ago).
I'm guessing though that there are other improvements in code quality and feature velocity. I've noticed personally that AI is really good at catching smaller things that are easy to miss (e.g., if you ask it to rename fooTheBars it also updates all the relevant comments or enums that you might have missed).
> My company actually did an internal study of agent usage for coding and found it only improved productivity by 10-20%, basically on the same level as good code templates or an autocomplete.
That's still a pretty good outcome. 20% more output across a company is huge when you think about it. Definitely not going to change the world completely though.
> No AI company is addressing the elephant in the room that you need someone experienced constantly monitoring any agentic workflows. This means that the cost savings of agents are a myth.
I mean, it depends on the agentic workflow. Like for production code, definitely. For document and claim review, you probably need a targeted sample on a daily basis but you get massive gains.
I mean, what's the goal here? AI does everything and then... and then what? We all die? Most of us die except a select few who rule the world (which, by the way, would not include you or me)?
The reason we won't let AI run rampant is because it's anti-human. Even if AI can do everything, why would we allow it to? We don't have to do that, that's very much a choice.
We're at the point where we have humans advocating not only against other humans, but against themselves. If people are going to be out here arguing for self-destruction on the order of suicide, they might as well save us all the trouble and put a gun in their mouths. At least that way, humanity might have a few less pathetic losers who put computer programs before themselves.
The alternative is to let humans do everything. Think about what that entails.
I have seen enough human incompetence that AI incompetence is beginning to look quite favorable. AI performance, at least, goes up generation to generation. If you've ever faced minimum wage workers, you know that they are, at best, consistently the same level of awful.
Guess why. People did their damn best to replace human labor everywhere they could, as soon as the tech allowed for it. Because human labor sucked for everyone involved.
I don't believe there will ever be any artificial intelligence, not with Markov chains (next token prediction), not otherwise. Especially not now when the current ML hype is already winding down. And yes this is a matter of belief since I don't think any science precludes agi from existing nor is there any reason to be sure it could someday materialize. I honestly would rather believe societal collapse hits us before agi can even be theorized.
AI was here in the 1970's too for that matter, in the form of expert systems. "AI" is the label that perennially gets applied to whatever current technology does something that was previously considered similar to human intelligence, then later on gets removed and applied to something new.
You'll know were making progress towards AGI when LLMs start being called LLMs again, and something new starts being called AI.
Kind of, but I wouldn't exactly put it like that since AI has never meant anything more than automated intelligence/decision making of some sort. The bar isn't moving, just this almost meaningless label is just forever getting slapped on the latest shiny new thing.
You could legitimately call a thermostat "AI". Expert systems were previously called AI. Today it's Large Language Models. Tomorrow it'll be something else.
I think the issue is that technologists use the term AI to mean one thing, which is something we've had since the 60s with lisp, forth, and prolog. And the average person uses AI to mean something more like Data from Star Trek, which we don't have and may or may not have this century. We're all talking past each other.
There is no rigorous definition of intelligence, let alone artificial intelligence. What you're referring to is people simply not knowing what they're talking about.
More to the point, there has always been a cottage industry in predicting an amazing future, just around the corner. 'AGI' is just the latest incarnation.
Woz is a different kind of geek, appreciates the craft, and can sort out the cruft out of it.
AI will be there, but it'll transform. When I say I don't use AI (i.e. LLMs, chat interfaces, agents and "autocomplete") for coding, research and whatnot, people label me as a luddite. The fact is I know how to use them. I test them from time to time. Occasionally these tools help. More often they hinder.
"Resistance is futile, hand your brain over!" is a hype filled dystopian fatalism noting that future is inevitable. It's inevitable. You can use this correctly, and we don't got back to our senses to understand how to use this correctly and efficiently.
We are just cooking our planet right now, with heat, poisoned water and slop.
Auto-complete on steroids, is still my favorite analogy for AI. I don't mean that in a negative way either. Autocomplete is very good, but that never stopped me from learning English grammar and spelling.
Quite right. I'm worried about the impact that LLMs will have on the learning process, especially in programming, but also in writing. Programming and writing are both skills that seem simple, but take an absolutely staggering amount of practice to master.
Think about how much your own writing (and programming, if you were lucky enough to start early) evolved from, say, age 12 (when a lot of smart kids start to tackle 'real' books) to age 18 (when you supposedly have a good enough education for 50% of work in most countries) to age 25.
All of that evolution is a direct result of one thing: practice! But with a magic answer box available in everyone's pocket, it'll take truly Herculean effort from a learner to actually grind through the practice instead of just cheating for an answer. I really worry how much an LLM user will actually comprehend their own code or even prose; if you've scarcely written a line of code, how can you really understand what's going on in a debugger? If you haven't done the legwork of writing essays and constructing coherent arguments and comprehending grammar, how will you ever communicate effectively?
Maybe I'm just a dinosaur and these kids will sail a whole level of abstraction above my own understanding of writing and programming, much like how my own generation preferred Python to C, and how the previous generation evolved from assembly to C/BASIC/etc. But then I come back to those missing fundamentals, that empty mental model. It's not like my English or CS teachers had me grind through essays and implementing linked lists and Djikstra's Algorithm for pure busywork. They did it because practice is the only way to truly immerse a student in a practical subject. Maybe it'll work for programming, as long as LLMs get good enough that you can always ask them to fix low-level errors for you? But it seems unlikely to work in prose. And even those generational programming jumps I mentioned (assembly to C to Python) were lossy; most kids I went to school with would be absolutely useless writing C code, and even as a bit of a dinosaur I'm pretty awful at even debugging assembly.
Like you said: you still need to learn grammar and spelling. And I suspect a whole skill tree of other fundamentals!
One angle I'm exploring, as a non-dev who nonetheless works in tech, is using Claude as a professor. Make learning timelines for me for Leetcode, break it down in phases, start with theory, ask me questions, then give me a coding challenge. Save that to an html artifact I can export and read on my phone.
It still gets things wrong, I can tell as I get through problems.
But it was either that or that dreary 'Cracking the Coding Interview' book. At least I'm learning fundamentals by asking question after question and making it track the concepts I had trouble with.
That's one use. Will most people use it to learn? Probably not. But most people are ... most people.
Yup, I used to believe that people would all use the Internet to educate themselves, and we all know how that turned out (loads of people did, but the majority didn't).
The way I think of it has evolved a lot over the last 5 years. At this point I think human brains probably do something analogous to next token prediction when we think. For all the hype, I think LLMs are actually more, not less, intelligent than that average person realizes. I think it’s legit, actual intelligence, not just “artificial” intelligence. That may be a hot take but it’s just my perception.
> At this point I think human brains probably do something analogous to next token prediction when we think
That's reasonable, but it doesn't mean that LLMs are close to being brains.
For a start, when humans think/talk, we often think ABOUT something - whatever is swirling about in our mind, or what we are currently seeing/feeling/etc. An LLM generating tokens/words is doing so only based on it's weights and the word sequence it is currently generating ... the human parallel would be more like a rapper spitting out words based on prior words, essentially on auto-pilot, or when we get triggered into spitting out stock phrases like "have a nice day".
If you want to compare an LLM to a human brain, it's basically equivalent to our language cortex if you ripped out all the external connections and ripped out all the feedback paths that make it capable of learning.
Of course there is a lot more to our brain than just our language cortex, but that alone should make you realize there is no real comparison beyond the fact that our language generation is also going to be based on prediction, and partly auto-regressive.
Having shame would require the LLMs to actually be able to recognize mistakes they make.
People love to put a lot of meaning on what an LLM responds with when asked why it made a mistake, but it's critical to remember that the answer to that prompt is just another series of probabilistic tokens, and has no actual relation to how the error happened.
They "recognize" mistakes just fine because you explicitly tell them. They recognize them well enough to correct (...sometimes). The way in which mistakes don't register is "Oh shit, that bad result was a result of my inappropriate actions. I must pay attention to not doing that again or the user will think I'm an idiot. I should even think about it some more to avoid the whole class of mistakes". Think of emotions as an attention mechanism that LLMs lack.
And many times, after an error is pointed out and an LLM offers an "explanation" for what happened, the LLM then gives the exact same erroneous result.
It's language. Language itself is the thing that makes us smart in the unique way that we are among the other animals, and it weirdly turns out to be transferable to machines to at least some degree.
At least 50% of humans have no "inner voice" and are not thinking in the same way as you. Many animals like dolphins, dogs, rats, crows are also very intelligent yet appear to only have primitive language capabilities.
A lot of human intelligence is really societal rather than individual, based on knowledge transmitted down through generations by writing (the real enabler). If you take that away then what you are left with is something more like an isolated hunter-gather tribe.
I personally think that the "inner voice" is a non-falsifiable claim, and therefore more of a religious belief than something which can be part of any materialist theory. In this regard, I'm a strict empiricist and wouldn't be able to claim that I have one myself. In fact, I find that thinking "out loud" or "on paper" produces much better results in most instances, probably because I'm grounding my thinking in natural language, which is a fantastic medium for thought. If my "inner voice" were comparable in efficacy to actually speaking or writing, we wouldn't notice this effect, but I'm definitely not alone in this regard.
Your point about writing and social intelligence is, to me, more evidence for the "it's language that's smart, not us" hypothesis. We start off in small bands of hunter-gatherers that store their intelligence in an oral culture. Language then jumps to clay tablets, papyrus, codex books, etc. The printing press allows it to escape containment to a wider public than just a caste of priests and bureaucrats. As soon as we invent automatic calculators, we start networking them and using those to process language, albeit in a primitive way (email, the web, etc.). Recently we discovered some abstruse math that, with the assistance of a bunch of beefy video cards, can crunch centuries of human writing into a mathematical object that encodes at least some of the meaning of that writing into an even more "advanced" symbolic processing machine. There's a clear trajectory of language itself getting more and more free of the specific wetware it grew up on.
It's a falsifiable claim, in that if there is a way to train a useful LLM from scratch without any human authored input language to bootstrap it (something I've been on the lookout for but haven't seen, though admittedly I'm not an AI researcher, just some Linux nerd with a day job as an SRE), then we can disprove it.
For the religious angle, look no further than John 1:
"In the beginning was the Word, and the Word was with God, and the Word was God."
Well, humans developed language. Language is just a tool that let's us leverage our innate intelligence.
I'm sure that we will eventually build artificial brains, capable of bootstrapping communications and language for themelves (if run en-masse in a simulation where the benefit of communication would emerge). An LLM can't do this since it is by definition/construction something only capable of learning a pre-existing language.
An artificial brain, just like a wet jiggly one, is always going to be more intelligent than a one-trick pony like an LLM - a language processor, but it is notable how intelligent that one-trick pony nonetheless appears to be.
I think it's interesting that you think we could bootstrap an artificial brain with no inputs from human culture. I disagree, but am open to an existence proof of this kind. Such an artificial brain would be totally alien to us, of course. I wonder how differently it would perform versus something more grounded in "real" culture and writing?
I'd say that human babies and LLMs are both existence proof that prediction and prediction error feedback is all you need to learn. The artificial brain/baby would be designed to learn just like us by prediction, and should therefore be capable of learning language from scratch just as we do.
You could choose to lock it in a virtual or physical basement with printout of the Common Crawl dataset and raise it like an LLM that learns language with zero real world grounding, in which case it may feel a bit like an LLM (but smarter - able to learn, etc), or you could let it interact with the real world and learn everything, including language, that way and be a lot more grounded.
It's hard to guess how the grounded version would feel to talk to vs the CommonCrawl one - I think it would mostly come down to how far we wanted to go in making the artificial brain have all the moving parts of a human one. In an odd way the more human we tried to make it, the more alien it might feel, but not for the "uncanny valley" reason you might imagine...
The thing with an LLM that makes it feel so human is that they are designed to 100% copy humans - their output is 100% driven by the training goal of trying to exactly match the samples in the training set. As soon as we start to try to build something more brain-like then it's behavior is going to be a lot less predictable - not just auto-regressive "auto-pilot speech", but driven by it's own internal thoughts, emotions, innate traits, etc - depending on how much of our brain we tried to copy in the artificial one. I expect it would feel a lot more organic, less robotic, to talk to, but at the same time perhaps less human since unlike the LLM it's not built just to mimic human speech.
Yes, this to me is also a good sign for the "it's language that's smart, not us" argument. It's an emergent trait that has evolved several times, like flight or carcinization. There's something about language that attracts evolution toward it. One would expect such a trait to have a big survival value (disclaimer: IANA biologist, philosopher, theologian, mathematician, or linguist).
> I think it’s legit, actual intelligence, not just “artificial” intelligence. That may be a hot take but it’s just my perception.
You might be redefining words here; there isn't a form of intelligence that isn't actual intelligence. It is all actual intelligence. Artificial in this context means it is something we're creating in a lab. LLMs can't avoid being artificial intelligence. The meaning of "AI" is to artificially create actual intelligence.
average person is absolutely awful judge on anything you put in front of average person tho.
And if anything, average AI user is vastly overstating how good/useful it is. Papers about it pretty much always show huge gap between "productivity person thinks they are achieving" and "actual growth of productivity"
Yes! We need them to have hope, but hopefully there can be substance behind it, otherwise it's like when the Hitler Youth got those badges before Hitler killed himself. In the sense that we are awarding people medals when their future is bleak
Someone has to be in the news first, and then you learn about it. To that someone, the change you see in tomorrow's legal news is today, was yesterday. It's more a gamble than you think, you just don't feel it until it bites you.
FWIW, we're in complete agreement. I stand by mt original statement that these things are nearly impossible for a layperson to be competent in. It's hard enough for people who do it every day for a living. An equivalent would be non-techies saying "I'm not going to pay a software engineer. Just tell the computer what you want it to do!" And that might sound reasonable to other non-techies, while you and I roll our eyes and laugh. I'm sure CPAs and tax attorneys and the like think the same when they hear engineers talk about filing their own non-trivial taxes.
A Mister does a good job of recreating period appropriate load times and quirks. You can put it in whatever old computer case you're most nostalgic for, connect an old CRT monitor and most peripherals should have some USB converter if necessary.
If you preserve the l2 distance you preserve the inner product, that's somewhat tautological in an L2 space. Just that the degree you can preserve inner products can be misleading, main problem is that orthogonal vectors may only become near-orthogonal which is sometimes a big deal, though perfect correlations are preserved because the JL transform is linear. Both can be seen looking at: https://en.wikipedia.org/wiki/Polarization_identity
As far as I can recall JL holds for affine transformations too, in any case it's an existence result. Have to double check on the affine bit.
The popular proof does uses random linear transforms and they indeed will not change the origin, but that's just one class of transforms with the JL property.
The division of lemmas and theorems is really a bit artificial for these things. But yeah I think the spirit is that a theorem is an object that you aim to study, while a lemma is something you use to do that. Fermat's last theorem was a target, but the techniques including lemmas used and developed for it are the real prize for a working mathematician. Sculptures are kind of the point, but there's no question the tools used for sculpting are more useful and "worth" more in that sense.
reply