Thank you! I built it because I work in legaltech, and we often need to embed entire case laws, statutes, contracts, and long opinions for comparison, clustering, and similarity analysis etc. I can see it as a built-in feature with Qdrant.
Excellent write up. I’ve been thinking a lot about caching and agents so this was right ilup my alley.
Have you experimented with using semantic cache on the chain of thought(what we get back from the providers anyways) and sending that to a dumb model for similar queries to “simulate” thinking?
The problem with "Hanlon's Razor" is that everything can be explained by incompetence by making suitable assumptions. It outright denies the possibility of malice and pretends as if malice is rare. Basically, a call to always give the benefit of the doubt to every person or participant's moral character without any analysis whatsoever of their track record.
Robert Hanlon himself doesn't seem to be notable in any area of rationalist or scientific philosophy. The most I could find about him online is that he allegedly wrote a joke book related to Murphy's laws. Over time, it appears this obscure statement from that book was appended with Razor and it gained respectability as some kind of a rationalist axiom. Nowhere is it explained why this Razor needs to be an axiom. It doesn't encourage the need to reason, examine any evidence, or examine any probabilities. Bayesian reasoning? Priors? What the hell are those? Just say "Hanlon's Razor" and nothing more needs to be said. Nothing needs to be examined.
The FS blog also cops out on this lazy shortcut by saying this:
> The default is to assume no malice and forgive everything. But if malice is confirmed, be ruthless.
No conditions. No examination of data. Just an absolute assumption of no malice. How can malice ever be confirmed in most cases? Malicious people don't explain all their deeds so we can "be ruthless."
We live in a probabilistic world but this Razor blindly says always assume the probability of malice is zero, until using some magical leap of reasoning that must not involve assuming any malice whatsoever anywhere in the chain of reasoning (because Hanlon's Razor!), this probability of malice magically jumps to one, after which we must "become ruthless." I find it all quite silly.
> It doesn't encourage the need to reason, examine any evidence, or examine any probabilities
Parent isn't advocating for assuming malice, or assuming anything really, but to reason about the causes. Basically, that we'd have better discourse if no axiom was used in the first place.
I agree. It seems to be an all too common example of both:
1. lack of nuance in thought (i.e. either assume good intentions or assume malice, not some probability of either, or a scale of malice)
2. the overwhelming prevalence of bad faith arguments, most commonly picking the worst possible argument feasibly with someone's words.
In this case instead of a possibility of it being a small act of opportunity (like mentioned above of just dragging feet) not premeditated, alternatives are never mentioned but instead just assumed folks are talking about some higher up conspiracy and on top of that that must be what these people are always doing.
Anyway thank you for your point it is an interesting read :)
It doesn’t say don’t think about malice as a possibility, it says that if you aren’t going to think about it, you should ignore malice as a possibility.
Yep, "Hanlon's Razor" is pseudo-intellectual nonsense. It sets up a false dichotomy between two characteristics, neither of which is usually sufficient to explain a bad action.
IMHO you're taking it a bit too literally and seriously; I suggest interpreting it more loosely, ie "err on the side of assuming incompetence [given incompetence is rampant] and not malice [which is much rarer]." As a rule of thumb, it's a good one.
To me the more problematic part is anchoring the discussion into rejecting a specific extreme (malice) when there will be a lot of behavior either milder, or neither incompetence nor malice. For instance is greed, opportunism or apathy malice ?
That's because actual malice IS rare. Corporations are not filled with evil people, but people make perfectly rational, normal decisions based on their incentives that result in the emergent phenomenon of perceived malicious actions.
Even Hitler's actions can be traced through a perfectly understandable, although not morally condone-able, chain of events. I truly believe that he did not want to just kill people and commit evil, he truly wanted to better Germany and the human race, but on his journey he drove right off the road, so to speak. To quote CS Lewis, "Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience."
The "malice" part of the razor is bait. People typically act out of self-interest, not malice. That's why anyone who parrots Hanlon's Razor has already lost; they fell for the false dichotomy between malice and incompetence, when self-interest isn't even offered as an explanation.
> Corporations are not filled with evil people, but people make perfectly rational, normal decisions based on their incentives that result in the emergent phenomenon of perceived malicious actions.
This rationalization is cope. All US Corporations making "normal" decisions all the time isn't casually obvious. I would say that wherever there is an opportunity to exploit the customer, they usually do at different levels of sophistication. This may mistakenly seem like fair play to someone who thinks a good UI is a good trade for allocated advertisement space, when it's literally social engineering.
Corporations make decisions that more frequently benefit them at the cost of some customer resource. Pair that with decisions rarely being rolled back (without financial incentive), you get a least-fair optimization over time. This is not normal by any stretch, as people expect a somewhat fair value proposition. Corporations aren't geared for that.
Agreed that actual malice is relatively rare (at least, relative to incompetence!). But I feel your take on Hitler is questionable. The question of evil is a tricky one, but I don't think there's a good case to be made that he was only trying to do the right thing. He was completely insane. But leaving aside moral culpability or metaphysical notions of judgment, for any definition of "malice", he embodied it to an the absolute maximum degree.
> That's because actual malice IS rare. Corporations are not filled with evil people,
Corporations don't have to be filled with evil people for malice to be rampant. All it takes is for one person in a position of power or influence who is highly motivated to screw over other human beings to create a whole lot of malice. We can all think of examples of public officials or powerful individuals who have made it their business to spread misery to countless others. Give them a few like-minded deputies and the havoc they wreak can be incalculable.
As for Hitler, if we can't even agree that orchestrating and facilitating the death of millions of innocent people is malicious, then malice has no meaning.
C. S. Lewis has written a great many excellent things, but his quote there strikes me as self-satisfied sophistry. Ask people being carpet bombed or blockade and starved if they're grateful that at least their adversary isn't trying to help them.
> Even Hitler's actions can be traced through a perfectly understandable, although not morally condone-able, chain of events. I truly believe that he did not want to just kill people and commit evil, he truly wanted to better Germany and the human race, but on his journey he drove right off the road, so to speak.
Disgusting take. Don't simp for hitler. How am I having to type this in 2025?
I recently heard on a podcast where one of the guests recounted what his father used to say about the employees making cash-handling mistakes in the small store he owned. It was something like, "if it was merely incompetence, you'd think half of the errors would be in my favor."
It probably is a glitch in this case, but it's hard not to see the dark patterns once you've learned about them.
If you short charge a customer they will demand correct change if you overpay a customer won't complain. In the cases of customers giving back extra money it becomes neutral.
His father's theory didn't take into account this.
Hanlon's Razor is for a situation where good faith can be assumed, or the benefit of the doubt given.
When the actors involved have shown themselves to be self-interested, bad-faith, or otherwise undeserving of the benefit of the doubt, it can be abandoned, and malice assumed where it has been clearly present before.
my first manager told my as i started my first oncall "we dont think anybody actually cares about this thing, so if it breaks, dont fix it too quickly, so we can see who notices"
I’m amazed at the prevalence of conspiracy theories on HN in recent years. Even for simple topics like a website crashing under load we get claims that it’s actually a deliberate conspiracy, even though the crashes have turned this from a quiet event into a social media and news phenomenon, likely accelerating the number of cancellations.
No, what I said was the COVID years. People became dramatically more prone to conspiracy theories and significantly more polarized in the 2020-2025 period. A lot more happened to people than just exposure to COVID, which was of course part of it. I'm not talking about the people who died or the healthcare workers. There was a meaningful step change in the way we interact with each other and what is acceptable. There was a huge impact to the social fabric and cohesion of society.
Using my eyes, I looked back at the text on my screen next to your username.
Your comment was 7 words, one of which is literally "COVID". Then you said you're weren't actually talking about COVID, but you actually meant something about how you think others are now prone to dramatic conspiracy theories.
It seems like you're experiencing some of this yourself or are stuck in some sort of race condition where if someone else doesn't agree with you, it's clearly a them issue. They're the conspiricist.
While explaining that you intended me to get a whole different message from your initial 7 words, you go on to say that while discussing the "COVID years" that...
> I'm not talking about the people who died or the healthcare workers.
Why aren't you focusing on these things? It seems much more important than whatever you are spinning on about the social fabric and cohesion of society as you type into a webform to a stranger about how everyone has conspiracies now.
Yes, I guess? The edge cases are the interesting cases :)
Seriously, though, what’s the point of trying to teach kids Newton’s Laws if you don’t try to give the kids a decent understanding of what situations they apply to and what one might conclude from them? Otherwise you’re just teaching cocktail party trivia.
This is downvoted for some reason and yet it’s absolutely correct. There’s a reason telcos specifically are notorious for the shittiest customer service ever - because major telcos typically have a monopoly and/or long-term contracts the customer can’t easily leave. They could literally bill you and not provide any service, and most people would still pay at least a couple months while fighting to cancel or get it resolved to avoid getting their credit report dinged.
i'll try to assume this isn't a snarky response! but it kinda sounds like it.
unfortunately, there are no "good" ISPs in my area. xfinity and the other big ones killed most of the competition via undercutting, lobbying and the legal system (ie, tying up municipal fiber/wifi initiatives in bureaucratic hell).
since i no longer work from home, it's mostly just a convenience to have internet service for personal use. i actually use it very little, and pay for the cheapest option available, so it's not much money.
ah well, we're dealing with economies of scale far greater than individual subscribers. and trust me, i've weighed going offline-only at home (not including mobile data), but for what relatively-little i do pay, i'm not ready to unplug just yet.
> By forcing you to make a decision without context.
Not the OP, but what would be the point to that? In any practical scenario there is always context, isnt it? I guess I don't quite get what we are trying to measure here.
It is not primarily a matter of difficulty. The problem with choices like these is that they seem to be predicated on a rather simplistic, formulaic view of software development - a view that experienced developers will recognize as flawed.
In this case, one tacit assumption is that a given developer will typically adopt just one of these approaches. Another is that they can meaningfully and objectively be ordered along either of the axes purportedly being measured.
