Thank you. All these people applauding Apple for not jumping on the bandwagon.

When in reality, they _wanted_ to but have become so dysfunctional organization wise, they weren't able to. Kind of funny how that worked out.

I still think they're really dropping the ball. They could have local models running on devices, interfacing with a big cloud partner (Google, OpenAI, etc.) Make Siri awesome. But no.

There is no use case for local models.

See Gemini Nano. It is available in custom apps, but the results are so bad; factual errors and hallucinations make it useless. I can see why Google did not roll it out to users.

Even if it was significantly better, inference is still slow. Adding a few milliseconds of network latency for contacting a server and getting a vastly superior result is going to be preferable in nearly all scenarios.

Arguments can be made for privacy or lack of connectivity, but it probably does not matter to most people.

I think the real case is a future technology. Similar to speculative decoding but done over servers.

Local model answers and reaches into the cloud for hard tokens.

I've been around here for over a decade. I'm telling you, this has been happening for longer than a year. I'd say the last ~4 years.

It's free on Cloudflare.

Wow, what error?

Seriously I think this is needed. The industry has become delusional.

Sounds a lot like X these days.

Nice work.

I've thought about doing something similar as well! It drives me nuts this ban, everywhere I look I see these blocked images. I thought about making a chrome extension that proxies.

This is the default view I use. But I'm kind of an OG HN user myself.

I agree. Or someone who combines a lot of tough to build tech right at the perfect time.