Self-hosted might be the way to go soon. I'm getting 2x Olares One boxes, each with an RTX 5090 GPU (NVIDIA 24GB VRAM), and a built-in ecosystem of AI apps, many of which should be useful, and Kubernetes + Docker will let me deploy whatever else I want. Presumably I will manage to host a good coding model and use Claude Code as the framework (or some other). There will be many good options out there soon.
As someone with 2x RTX Pro 6000 and a 512GB M3 Ultra, I have yet to find these machines usable for "agentic" tasks. Sure, they can be great chat bots, but agentic work involves huge context sent to the system. That already rules out the Mac Studio because it lacks tensor cores and it's painfully slow to process even relatively large CLAUDE.md files, let alone a big project.
The RTX setup is much faster but can only support models ≤192GB, which severely limits its capabilities as you're limited to low Q GLM 4.7, GLM 4.7 Flash/Air/ GPT OSS 120b, etc.
I've been using local LLMs since before chatgpt launched (gpt-j, gpt-neox for those that remember), and have tried all the promising models as they launch. While things are improving faster than I thought ~3 years ago, we're still not there in terms of 1-1 comparison with the SotA models. For "consumer" local at least.
The best you can get today with consumer hardware is something like devstral2-small(24B) or qwen-coder30b(underwhelming) or glm-4.7-flash (promising but buggy atm). And you'll still need beefy workstations ~5-10k.
If you want open-SotA you have to get hardware worth 80-100k to run the big boys (dsv3.2, glm4.7, minimax2.1, devstral2-123b, etc). It's ok for small office setups, but out of range for most local deployments (esp considering that the workstations need lots of power if you go 8x GPUs, even with something like 8x 6000pro @ 300w).
I think this is the future as well, running locally, controlling the entire pipeline. I built acf on github using Claude among others. You essentially configure everything as you want, models, profiles, agents and RAG. It's free. I also built a marketplace to sell or give away to the community these pipeline enhancements. It's a project I wanted to do for a while and Claude was nice to me allowing it to happen. It's a work in progress but you have 100% control, locally. There is also a website for those not as technical where you can buy credits or plugin Claude or OpenAI APIs. Read the manifesto. I need help now and contributors.
Capture might not be the aim. The coming decades will see anonymous effective asymmetric warfare with USA infrastructure and the USA political establishment as prime targets. That's the big concern.
Huh... they better build some readily-available hyper-powerful infrastructure, pronto, or that next election could hand power to folks that don't have the best interest of the country in mind:
The past that we have yet to subject to our subjection also is effectively future.
Supplication for unknown outcomes surely already determined in the objective past wrt the present time still makes sense. The Divine Successive Relaxation with physical laws as the substrate, and the choices of human free will, human petition and desire, and Divine Will And Intention as boundary conditions, will solidify objective reality into a coherent whole in Open Theism.
The goal is kinetics and beautiful fireworks for the folks back home. "Acting now" is much easier than methodical interdiction, investigation, and potential justice. The home audience won't tolerate anything smacking of evidence and proof, that's too bookish and takes time.
No, no one at the highest policy/government levels is doing anything about that. It would require treating drug addiction as a health care issue, with a plan to solve the underlying root causes, support and care for the affected and substantial funding on the ground level for support networks and other institutions that actually work in the space. Housing programs and drug decriminalization may also be worthwhile evaluating.
But that’s hard, unglamorous work, out of the limelight with people that are sick, addicted, grimy. No spectacular fireworks.
>> care for the affected and substantial funding on the ground level
Not sure what you would expect, but on a state level. My state has hundreds of millions in funding for free clinics, free treatment centers, free methadone clinics, free housing, and free welfare. If people really want to get clean, they can.
This isn't a problem of funding either on the state or federal level, its a bunch of NGO's getting rich making sure the situation continues by handing out free tents, clean needles, narcan and "safe zones" where people can do their drugs without interference.
When you make it easier to stay addicted and homeless? Those who are in that situation will continue to take the easy route. Its not easy getting clean and this is 100% on the individual to make the choice to get clean. When you encourage addiction and make it socially acceptable, is it any wonder this issue isn't getting any better and in fact, is well past a crisis point now.
> its a bunch of NGO's getting rich making sure the situation continues by handing out free tents, clean needles, narcan and "safe zones" where people can do their drugs without interference.
There will also be those who try to take advantage of government funding. But the gross handouts to the Pentagon, we can and do audit NGOs. I doubt fraud is the biggest issue here, but if you have studies, please share them.
> Those who are in that situation will continue to take the easy route
Are you saying it's a personal choice and not a mental health problem?
> When you encourage addiction and make it socially acceptable
I don't understand how spending money on housing and addiction treatment is doing that. If you want to encourage addiction and make it socially acceptable, you just don't spend money on housing or treatment.
Enter the Dominionists, gaining steam now. Not a regime I want to live under. Here's a forty year old article describing the inception of those religious figures close to the current USA administration ... https://banner.org.uk/res/kt1.html
Yeah, I'm trying to branch out to build things outside my comfort zone. AI already is helpful in my normal day-to-day (backend engineer, mostly Python w/ PyCharm/Datagrip working in monorepo style), but Claude Code helped me graft on a Typescript/Javascript UI ecosystem with an independent Webstorm IDE setup that will integrate well, along with all the presumably latest build tools and package management. The research and self-education I would have needed to do this without Claude would have been extensive, and would have been nowhere as complete. I don't see any point in going back to pre-AI. And I don't generally use AI results blindly, I go back and review and set some breakpoints in an IDE and watch it do its thing.
reply