While this is true, for many customers who aren't technical (in a computer sense, since they may have significant highly technical expertise in another field, such as agriculture), "tech" (meaning computers) just means what we'd call anti-features since from their vantage point everything with a computer (or with "tech") isn't respecting their ownership rights. And, even among people who understand the distinction, there's a reasonable expectation that computers embedded in products that don't specifically market otherwise will have such anti-features.
So, even if computers in and of themselves are completely valid in such product categories saying "No Tech" (which means "no computers") is a great way to market to people who really just want to avoid anti-ownership anti-features.
Lastly, I find it mildly amusing that a tractor (which is very clearly a form of technology, in the traditional definition of the word where fire and printing presses are technology too) is now being marketed as having no technology.
Why isn't self hosting (even just renting a GPU server, not necessarily on premise) at large companies or hosting via something like together AI to run the open weight models not more common? I've tried the open weight models and the premium models like Opus and Gemini Pro, and I find that the latter are a little better, but not nearly to the degree to justify the extreme price difference, since the differences largely don't matter for what I've tried them for, and I expect that many other users likely have similar use cases.
If the premium models are just about 10% better - that could justify the price vs. self hosting a ~0.5-1T open weights model.
Remember that utilization of these huge racks will not be 24h/7, and these are usually not GPU intensive shops that would train models on the spare compute. With prices of 100-200k USD and north with ~2 years lifetime, that would be hard to justify financially.
Self hosting could easily amount to ~1000 USD a month amortized across many developers. In rush hours - there will be hard rate limits.
Would that 1500-1000=500$ monthly USD justify the 10% decrease in "AI Productivity" ? I guess not. In most cases.
For everyone that asks me around, I'd say that in short term, unless there's a really good reason to self host these coding assistant models, then the big 2/3 coding assistants providers are the better choice.
I just went through a similar discussion in my $WORK (traditional finance company on NYSE with average IT expertise) and I think the thought process is as such: it's one thing to just give your stellar dev/hacker a beefy GPU server and run whatever model they can run; it's another thing to maintain such platform for company wide. You would need human resource (likely way above normal software dev paygrade) to understand and maintain such models, maintain backend, availability etc. All these extra hassle make it just easier to pay a top tier external lab + slap a reasonable spending limit on everybody.
Why do you think it would be more common? The pooling of GPUs to serve multiple users and connecting to docs/datalakes while respecting security controls, as a start, is non-trivial. You'd end up paying a team to manage that.
There’s probably plenty of money to be made in LLMs as a service - but not enough time has passed for the commodification to occur. I’m with you in that when the dust settles I don’t think any of the frontier model providers will have a moat. Just like during the dotcom boom a catchy URL and a webpage that could accept payments wasn’t a moat, either.
You tried that on a personal machine for yourself once. It's completely different calculation when serving a model to 3000 employees with ever evolving hardware and software requirements. You'll need dedicated hardware in data centers and experts to run them. A company will need to figure out how to manage acquisition, assets and expenses plus 1000 other things, in addition to its actual business. Guess who has figured out all of that already? AWS/Azure/OpenAI etc.
For the same reasons companies are not building data centers for their "regular" hosting and storage needs but put things on AWS, Azure etc.
It costs money to maintain the hardware and hire experts to manage the services. For something as common as LLM models, there is absolutely no reason a company serves models on their own hardware unless they are maniac about sending bytes to AWS.
What's the purpose of using EC2 over something much cheaper, like OVH, digitalocean, or Hetnzer?
Usually the argument is for scalability, but a single VM for personal use doesn't need that, and even if you do want that, you'll need more than a bare repo.
Everything about matrix is cumbersome and glitchy. I have last tried to use it a few years ago and it seemed that Riot/Element had the only decent clients, and those were all Electron on desktop and also seemingly for profit. Signal has the electron problem, as well as many others (like the backup UI being abhorrent), but at least the core functionality works without fuss.
More practically, this means (in America) that you need a JD degree (4 year grad school), to pass an exam, and pass a(n oftrn horrifically thorough) character background check.
> I like it when different producers select a different subset of priorities for their offer. Competition at work. One of the reasons we witnessed such an awesome evolution in the smartphone market.
>
> I hate it when a bureaucrat dictates a set of demands with absolutely zero regard to the cost or the tradeoffs involved in product decisions and market competition.
I generally agree with that sentiment, except we don't have a vibrant market of many options with many different trade offs. Finding headphone jack, solid reparability, user swappable battery, easily replaceable USB port, and all the other things that one might want is basically impossible. The vast majority of phones are highly unrepairable, have no headphone jack, have everything soldered to a tiny number of internal boards, and are full of anti repair dark patterns.
Phones with removable batteries are rare because only a small fraction of people want phones with removable batteries. Phone manufacturers also dislike removable batteries because customers buy cheap 3rd party batteries and complain when these batteries perform poorly or malfunction, sometimes by exploding. And then the headline is, “Phone made by company X explodes.” not, “Cheap battery explodes.” Removable batteries also introduce new failure modes like contacts degrading, causing phones to power off unexpectedly when jostled or flexed in certain ways. That increases the risk of a recall and bad PR.
I and millions of others want a phone that is smaller than the current offerings. Heck, my 13 mini is too big for my tastes. But I don’t think that means the government should force phone manufacturers to make smaller phones. So too for features like removable batteries, physical keyboards, or headphone jacks.
What do you mean by "rare"? You just click "order". It's not like you have to go on the quest for the lost arc or anything like that. They are uncommon in the sense that people don't actually get them, but that's not because of a lack of availability. People do not want them.
They mean the models are rare, not the devices. The claim is if you want feature X + removable battery, it's unlikely that you will find it. People's willingness to forgo the battery for feature X therefore doesn't tell you if people care about removable batteries in an absolute sense, just that they care relatively less than they do about feature X.
You could argue that the market already reflects people's desires via, eg., Apple's market research. They could argue that democracy in the EU also reflects people's desires.
> I have no doubt - if it hasn’t already happened - that some apps will unironically embrace the most ridiculous option by shipping as electron apps that implement a TUI layer as their front-end.
Whoever ends up using these devices second hand will be in for a rude awakening, which is bad for that person (even if it means that it just ends up going to ewaste and they get nothing) and bad for the environment. It's also bad for anyone who orders one new and isn't aware of the changes, although I agree that that is less bad than with phones due to the fact that a pi largely mitigates it.
This attitude is widespread enough to hold the world back by forcing everyone who interacts with the public Internet to support ipv4 (some technology), "for free". So, either way, we're forcing one of them. So, we might as well lean towards supporting the one that isn't hard capped at 4 billion addresses in a world with at least 2x as many devices. Have you ever tried to deal with NAT punchthrough? That's way more difficult to fix than having to properly configure your server.
> Have you ever tried to deal with NAT punchthrough? That's way more difficult to fix than having to properly configure your server.
Yes I did, and I no longer support that either. For my setups it is local private ipv4 networks all the way now! How tailscale or other VPN deals with that is not my problem!
If two nodes are on different networks, they should not be allowed to talk to each other anyway. Seems like security risk! Clean design, simple rules!
So, even if computers in and of themselves are completely valid in such product categories saying "No Tech" (which means "no computers") is a great way to market to people who really just want to avoid anti-ownership anti-features.
Lastly, I find it mildly amusing that a tractor (which is very clearly a form of technology, in the traditional definition of the word where fire and printing presses are technology too) is now being marketed as having no technology.
reply