People talk about AI getting things wrong all the time, why is it "so clearly irrational" to be doubtful of a recipe that might include ingredients that can make you sick?

Because I hope that someone who's hands were required to assemble the recipe didn't blindly add ingredients like "bleach" if the AI happened to hallucinate them.

A naive hope perhaps, but this ignores the risk of LLMs just creating a bad recipe based on the blind combination of various recipes in their training data.

As the parent comment said the people seemed to be enjoying the food otherwise so the LLM didn't create an unpalatable combination, and I can't think of any combination of edible and unharmful ingredients that might combine to something harmful (when consuming a reasonable amount)

This is exactly what makes it dangerous. Food can taste ok but actually cause you to get sick. Not all bacteria is going to taste off. I'm assuming you're not a chef because if you were then you'd know how absurd your statement is.

For a super simple example, if you don't properly handle or cook raw meat then you risk getting sick even though the food might not immediately taste bad. Maybe that's obvious to you but might not be to the person preparing the food. Another example: Rhubarb pie is supposed to be made with the leaves and not the stalk because the stalk is poisonous and can cause illness. Just kidding, it's actually the other way around but if you were just reading a ChatGPT recipe that made that mistake maybe you wouldn't have caught it.

If meat was involved, the cooking time may have been unsafe if other precautions weren't taken by the cook (like checking the internal temperature).

let's take a second to think about the threat vectors here. The two obvious ones I can think of are: "AI hallucinates and tells you to put non-food into the food" and "AI hallucinates and gives you unsafe prep instructions" (e.g. "heat the chicken to an internal temperature of 110 degrees"). For both of those, it's not clear why "random recipe from an internet blog" is safer than something the AI generates. At some level if someone is preparing your food you need to trust that they know how to prepare food, no matter where they're getting their instructions from.

People who do not understand or even use AI are not in a position to even begin "thinking about threat vectors". That isn't how they've come to their worldview, at all.

Yeah, it's ideological, like a religion as someone else mentioned, that's then supported ex post facto.

Yeah, but I would trust a human writing a blog not to suggest heating chicken to 110F because the human writing the blog understands that they are taking responsibility for that recipe... The AI LLM model doesn't have a clue about responsibility except to regurgitate feel-good snippets about responsibility.

Wild takes in this thread. Copy and blog writing industry is just random fiverrs or hires from countries with cheap labour to pump up the SEO rankings.

Everyone grew up with an understanding to “never trust the random internet content for 100%”, now we’re trying to say that AI has to be 100% reliable.

Okay, captain pedantic. Clearly I'm assuming a known food blogger with a reputation at stake employed by bon appetite / food network / etc in this scenario. Not some random SEO spam.

>because the human writing the blog understands

Bold assumption

Because it assumes the person actually making the food has no common sense?

This is an experiment they ran and were prepared to lose money on. It seems perfectly reasonable for an AI company to test their products in adversarial conditions to have a better understanding of its flaws and limitations.

Fantastic history I hadn't heard, april fools day included

https://www.pcgamer.com/software/ai/anthropic-tasked-an-ai-w...

My wife does it all the time, and it's actually decent.

That's just pure nonsense. My partner is very competent cook and she invents new recipes and experiments all the time. I don't see why she can't use LLM output as an inspiration to combine with her own expertise, sense of taste, and preferences to come up with an excellent dish.

That's quite an assertion.

People get things wrong all the time as well, so I wouldn't trust them either.

People get things wrong in a different, more observable/predictable way. Sure, we are easily tricked dummies and we can't know if a human is right or wrong, but our human-trust heuristics are highly developed. Our AI-trust heuristics don't exist.

I mean I had people serve me expired food and chicken that was half raw. The latter I could observe, the former I couldn't so easily. Both were things that could have made me sick.

For sure. I'm not defending human perfection, I'm defending human caution (Disclaimer: The format of the preceding sentence was chosen without AI assistance).

Someone once try to feed me dinner from a recipie they found on the internet. I punched their lights out and then called the cops.

Another comment [1] has a solution - a new tool called pit, it just throws the whole 10k loc PR in a pit and forgets about it :p

[1] - https://news.ycombinator.com/item?id=47713650

Is it unrelated though?

> Today, with Git, we're all teaching swarms of agents to use a tool built for sending patches over mailing lists. That's far from what is needed today.

Investor narrative pointing out a relationship is not the same as substantive technological overlap.

The open source ecosystem has come very far and proven to be resilient. And while trust will remain a crucial part of any ecosystem, we urgently need to improve our tools and practices when it comes to sandboxing 3rd party code.

Almost every time I bump into uv in project work, the touted benefit is that it makes it easier to run projects with different python versions and avoiding clashes of 3rd dependencies - basically pyenv + venv + speed.

That sends a cold shiver down my spine, because it tells me that people are running all these different tools on their host machine with zero sandboxing.

meh not always. I do use uv IN docker all the time, its quite handy

Honest question - what are the main benefits for you when you use it in docker?

ps. I feel like I've been doing python so long that my workflows have routed around a lot of legit problems :)

For us, the DX of uv for dependency management is much better than just using pip and requirements.txt.

To be clear though, we only use uv in the builder stage of our docker builds, there is no uv in the final image.

Main reason I now use uv is being able to specify a cool down period. pip allows it but it's with a timestamp so pretty much useless..

And that doesn't prevent me from running it into a sandbox or vm for an additional layer of security.

> pip allows it but it's with a timestamp

A PR to be able to use a relative timestamp in pip was merged just last week

https://github.com/pypa/pip/pull/13837/commits

Mainly the "project" system. I'm only developing python in my free time, not professionally so I'm not as well versed in its ecosystem as I would be in PHP. The fact that there's tons of way to have project-like stuff I don't want to deal with thoses. I used to do raw python containers + requirements.txt but the DX was absolutely not enjoyable. I'm just used to it now

The thing that has always stood out to me about WordPress, is that you can get a site up without any of the usual technical steps I associate with creating a site, but still have access to the innards of the site. Does it often go wrong, sure, but it is a lot more approachable for less technical users.

In contrast, typical web frameworks (even static sites) require a code change, build, deploy, etc to update many aspects of a site.

I view every single line of code as a liability, the best solution is if you can avoid writing any code. Does that put me into group 1 or group 2?

It's tough to not interpret this as "I don't care about my website". Do you not check the copy? Or what if AI one-shots something that will harm your reputation in the metadata?

Then I'll read the diffs after the fact and have fix AI it. ¯\_(ツ)_/¯

That sounds better. I assume the stakes are low enough that you are happy reviewing after the fact, but setting up a workflow to check the diffs before pushing to production shouldn't be too difficult

Of course. I could do a PR review process, but what's the point. It is just a static website.

You would think that sending HTTP requests is a basic capability, but I've had fun in many languages doing so. Long ago (2020, or not so long ago, depending on how you look at it) I was surprised that doing an HTTP request on node using no dependencies was a little awkward:

  const response = await new Promise( (resolve, reject) => {
    const req = https.request(url, {
    }, res => {
      let body = "";
      res.on("data", data => {
        body += data;
      });
      res.on('end', () => {
        resolve(body);
      });
    });
    req.end();
  });

These days node supports the fetch API, which is much simpler. (It wasn't there in 2020, it seems to have been added around 2022-2023.)

Yes, thankfully! It's amusing to read what they say about fetch on nodejs.org [1]:

> Undici is an HTTP client library that powers the fetch API in Node.js. It was written from scratch and does not rely on the built-in HTTP client in Node.js. It includes a number of features that make it a good choice for high-performance applications.

[1] - https://nodejs.org/en/learn/getting-started/fetch

Why is it amusing?

I say amusing because it points out that something I (and many other people) assume to be basic clearly has a lot more nuance to it.

Note that node-fetch will silently ignore any overrides to "forbidden" request headers like Host, since it's designed for parity with fetch behavior in the browser. This caused a minor debugging headache for me once.

Web standards have rich support for incremental/chunked payloads, the original node APIs are designed around it. From this lens the Node APIs make sense.

And you don't handle errors at all...

It's node. Why do errors if you can simply ignore them.

Left as an exercise for the reader... ;p

I suspect that my normal workflows might just have evolved to route around the pain that package management can be in python (or any other ecosystem really).

In what situations are uv most useful? Is it once you install machine learning packages and it pulls in more native stuff - ie is it more popular in some circles? Is there a killer feature that I'm missing?

If you have hundreds of different Python projects on your machine (as I do) the speed and developer experience improvements of uv make a big difference.

I love being able to cd into any folder and run "uv run pytest" without even having to think about virtual environments or package versions.

Do you run those projects on the host system as your normal user without any isolation?

Yes, which makes me very vulnerable to supply chain attacks.

Yikes! I had a scare once, and since then I only run sandboxed code or scripts I've written with minimal 3rd party deps.

I assume you have other mitigations in place?

Not really. I have good backups and I try to stick with dependencies I trust.

I do a lot of my development work using Claude Code for web which means stuff runs in containers on Anthropic's servers, but I run things on my laptop most days as well.

Imo, uv scripts with the dependencies in the header.

https://docs.astral.sh/uv/guides/scripts/#declaring-script-d...

I guess that could be useful. I don't have many standalone python scripts, and those that I do have are very basic. It would be really nice if that header could include sandboxing!

So much this! I've been bugging Astral about addressing the sandboxing challenge for a while, I wonder if that might take more priority now they're at OpenAI?

UV is most useful because it is so much faster than everything else. All the other features I could do without.

Yep, the speed is nice, I can't argue with that!

At least in space there is lots of space and no heat /s - I'd love for Next to exist in a vacuum