Interesting idea - I like seeing a list of pet-peeves followed by a proposal for a straightforward way to have a set of 'alternative defaults' that remains backwards compatible. If you don't want to opt in, don't run the new PRAGMA edition = 2026.
Too often it's just a list of issues and a wish that everyone else will change.
In (mild) defense of SQLITE_BUSY - busy_timeout just tells sqlite to sleep and retry up to the timeout when it receives SQLITE_BUSY. It seems like a sensible default for a library to leave that up the calling code - which may have something else it could do while it waits. However, that logic often gets missed!
This isn't so much a list of pet peeves as it is the almost universal way people that work seriously with SQLite configure the database. It's reasonable to suggest that the alternative settings for each of these suggestions is probably the wrong default for 2026.
> It's reasonable to suggest that the alternative settings for each of these suggestions is probably the wrong default for 2026.
That's the key concept here. When tightening up the defaults, an "edition" mechanism is a good solution.
Now we need this for C/C++, which have much legacy stuff which ought to go away for new code. This is more feasible than it used to be, because "Convert this Edition 4 code to Edition 5" is something LLMs can do now.
I'd never seen all the rules for SQLite soft typing written out before. Those are more complicated than strong typing.
P1881 Epochs proposed to WG21 (the C++ standards committee) in 2019 by Vittorio Romeo
The committee found plenty of problems with this, and made it clear that if Vittorio did all the hard work to resolve those problems they would find more, P1881 was abandoned.
So, you're not the first person to notice that this is a good idea, P1881 was written after Rust's 2018 Edition, but before 2021 Edition with its even more significant improvements. I firmly believe Rust's Editions unlock not only technical possibilities (though it does certainly do that) it unlocks an appetite from users which is good for your ecosystem.
> Does it make the compiler and other tools larger and more complex for each new edition?
Marginally.
> Are the migration tools always fully automatic, quick to execute, and flawless? Is any manual work, or expertise, required for upgrading in the worst case?
The ambition is always automatic migration, but there are almost invariably weird edge cases. For one thing Rust has a macro which includes arbitrary text in your program, like the C pre-processor #include but much less frequently needed, so short of auto-fixing all text documents on your computer such a migration will always be somewhat limited.
> If I read a blog about Rust, and the blog forgot to mention which edition it uses for its code examples, can that cause any problems? Should I just skip the blog if it is old?
Depending on how old it is, like anything else you read on a blog it might now be obsolete regardless of editions. Blog posts written last week about how England will face France in a World Cup final are irrelevant, both play a runners-up game instead on Saturday.
> As an example, why are these popular projects still on Rust edition 2021?
They all look like they're pretty mature, so, probably nobody thought it was worth doing?
Reading between the lines, it is as if you consider there to be large problems with Rust editions, but you do not wish to make Rust, Rust editions, nor the programming language concept of editions, look bad.
Does upgrading a large project require intimate knowledge of that project in the worst case? How much work is it in the worst case?
How do Rust editions and macros interact? Does definition of macros or usage of macros make upgrading a Rust edition harder? If yes, how much harder in the worst case?
> Reading between the lines, it is as if you consider there to be large problems with Rust editions, but you do not wish to make Rust, Rust editions, nor the programming language concept of editions, look bad.
I would say that - again reading between the lines - you've made an account on HN specifically to suggest that editions are a bad idea - but you're unwilling to associate this claim with anything else about yourself and you now realise that hurts your credibility.
That they can't (yet) be used to influence name resolution, so that they can't be leveraged for std API evolution. Work is being done on that front.
Older documentation that people find on the internet might be out of date and claim things that don't work but do on later editions, or vice-versa when trying out something that exists in a new edition in an older one. The mitigation for this is that if something is accepted in one edition but not another it must be taken into account in diagnostics so that the divergence is explained.
> Does it make the compiler and other tools larger and more complex for each new edition?
With editions 2015, 2018, 2021 and 2024, there are ~120 branches in the compiler for changes in behavior, including for changing diagnostics to be more specific. For comparison, just rendering a type error for if else having different types in its arms checks for 4 different conditions. 120 checks is a drop in the bucket compared to the number of things the compiler already has to check for, and the growth rate is very manageable accruing in the double digits over the course of 4 years.
> Are the migration tools always fully automatic, quick to execute, and flawless? Is any manual work, or expertise, required for upgrading in the worst case?
> Fully automatic
For most code, it is. We spend the time creating migration lints that can change your code as part of your upgrade, and we test those using crater to detect the cases that people are using in public libraries. It is conceivable that a project in a private repo is using a pattern that wasn't detected through that, so an appropriate structured suggestion isn't emitted, but I haven't heard of such a situation.
> quick to execute
it is as fast as cargo check
> flawless
If the compiler produces a structured suggestion marked as machine applicable, and I believe all edition suggestions are, our level of confidence on them being the right thing to do to get the users' code to compile with the intended behavior is high. Bugs can happen, but don't recall seeing any "incorrect suggestion" in edition lints.
> If I read a blog about Rust, and the blog forgot to mention which edition it uses for its code examples, can that cause any problems? Should I just skip the blog if it is old?
Differences between editions are actually quite small. You should be able to pick up The Book from 2015 and still learn the language. There are just things that The Book says don't work that now do, or features that were introduced that it doesn't talk about (both impl Trait and + use<'_> were introduced years later).
> If I see an interesting repository that I would like to learn from, but it was written in an old edition that I do not want to learn the rules for, will I have to upgrade the codebase myself before learning from it, or should I just skip it?
Just use the old edition or upgrade, either thing will just work. You seem to think that the behavior divergences are bigger than they are, and anything that does diverge should be emitting a diagnostic telling you what code to write instead. These are things like "in a previous edition you had to borrow here, in the next edition you don't".
> Are there any Rust projects that stay in an old edition for whatever reasons? How large a proportion of Rust projects that are still downloaded and used, are not on edition 2024? As an example, why are these popular projects still on Rust edition 2021?
Yes, there are crates that stay in older editions so that they can be used in projects that are frozen in time, like those that are shipped in some distros. (Edit: "frozen in time" as in "they won't update their toolchain", an earlier edition will always work on later ones due to backwards compat guarantees, the other way around of course can't.) To get a sense for it, serde which is a dependency that's almost inescapable is on 2021 edition and targets Rust 1.56. If a project decides that the impact on the ecosystem is bigger than the benefit of using newer language features, then it makes perfect sense to remain in an older edition.
https://lib.rs/stats sadly doesn't include statistics about the set edition, but it does have the graph at the bottom showing you rustc versions that the ecosystem supports, both for all crates and for the most recently updated. For reference, looking at releases.rs to refresh my memory, 1.31 is the introduction of edition 2018, 1.56 2021 and 1.85 2024. Those corresponds with the jumps in the chart. Note that looking at the chart it only tells you the likelihood of a random crate working on an older toolchain.
> How much work is it to upgrade someone else's codebase to a newer edition?
Not any more than updating your own codebase (and if you're upgrading someone else's it is because you've taken ownership of that code), which is to say, not much.
> Does upgrading a large project require intimate knowledge of that project in the worst case?
I can't think of any feature that would require that.
> How do Rust editions and macros interact?
The compiler has the concept of `Span`s, which is the byte offset pointing at a piece of your code. Every node in the AST has a `Span`, and is subsequently used in every later stage. It's what is used to render the diagnostics. But they also carry "context" information: whether a token comes from a macro expansion. And they also carry edition information. This let's you have a macro defined in one edition and use it in another, and the behavior will be the correct one, regardless of how you mix them.
> Does definition of macros or usage of macros make upgrading a Rust edition harder? If yes, how much harder in the worst case? Is upgrading always automatic?
The only situation I can think of is if there is a change in the parsing of macro arguments. For example, if Rust edition 20XX started parsing `A | B` as an anonymous enum type, then the macro call for `foo!(A | B)` would likely be parsed differently between editions <20XX and edition 20XX. That would be the kind of work you'd have to do. This has happened before in edition 2024, with expressions: https://doc.rust-lang.org/reference/macros-by-example.html#m.... In 2024 `expr` started not matching on `_` and `const {..}`. What was done then was introducing an `expr_2021` matcher with the prior behavior. When you update your edition to 2024 and apply the suggestions, your macro definitions get updated to use `expr_2021` instead of `expr`.
From your other comment
> Reading between the lines, it is as if you consider there to be large problems with Rust editions, but you do not wish to make Rust, Rust editions, nor the programming language concept of editions, look bad.
Reading your lines, it is as if you already had the conclusion that editions are bad, and are looking for a confession.
> Does upgrading a large project require intimate knowledge of that project in the worst case? How much work is it in the worst case?
I can't think of a situation where intimate knowledge of the project is needed.
> How do Rust editions and macros interact? Does definition of macros or usage of macros make upgrading a Rust edition harder? If yes, how much harder in the worst case?
The key concept of an "edition" is that you can deprecate stuff. "gets()" should have been pulled from the standard library around 1990 or so, along with "strcat" and its frenemies.
I’d say these are reasonable settings for most uses. Though do you know of surveys that back this up? I don’t mean to nit pick too much, I’d just like to see common uses and the data.
SQLite is used in a lot of unconventional settings (for SQL databases) where these settings don't make as much sense. But that's what makes the "edition" useful; it captures the use case we all mean when we're thinking of the "database" lego in an application stack.
While that's true, editions are more about leaving legacy decisions behind while keeping the backward compatibility promise.
Even if you're in one of those unconventional settings (say, a bare-metal microcontroller or something), you'd probably still start from edition 2026 and mutate your settings accordingly, rather than using the defaults that are 26 years old.
Yeah, that's important. Rust's 2015 edition is worse, not just different from what you'd write today with 2024 edition. There's a clear direction of travel.
You are probably correct, but I imagine the SQLite team's dedication to backwards compatibility has things the way they are so that existing systems can user later versions a swap without worrying about changing the SQL using it.
The entire point of "SQLite should have editions" is so that projects can opt into a set of modern defaults for 2026 and not get all of those backwards compatible decisions from 20 years ago.
GPP's wording suggests that the defaults simply be changed to what is being discussed as more generally sensible in current times, rather than being opt-in.
Given how many projects are potentially out there effectively relying on the current settings, and SQLite's general attitude to backwards compatibility, that would likely not be considered a good idea. Opting in with an edition flag for new (or updating) projects does seem like a good solution to this to serve all of old, active, and new projects, but it would increase potential bug surface area and therefor testing requirements, and the existing setting do allow all that to be opted in/out to/from already (and it is only four settings we are talking about here).
That FKs being enforced is set per-connection rather than at the database level is something that surprised me a lot when I found out. A way of setting that at DB creation (or via ALTER DATABASE after) seems like quite an omission because if you have multiple potential routes that can update the same DB any one of them could cause serious the others will encounter.
Yep. The whole locking database thing is this persistent myth about SQLite. All databases lock on write, it’s a question of the granularity of the lock. Multiple writers simply take turns.
yes but no... most database allow for at least as many parallel and concurrent writes as there are tables at a minimum.
The "lock on write" problem is that in MySQL i could run a OLAP pipeline for a few hours and have a fully functioning database with degraded perfomance, on SQLite the same pipeline would lock the database for the full hour. (there are surely ways to solve this (eg using the main db as read-only and a secondary db for writes or splitting the writes in incremental transaction), but it is not a "myth".
The “myth” is you can’t use it for a website because if you have multiple requests that need to write they can’t do it concurrently. (They just take turns of course for the milliseconds of write time.)
If you run a transaction with writes for an hour on any database, the data you update will literally be locked. So your example only works if results are independent of the data other programs want to use.
Of course more granularity of locking is better and enables more designs that would not otherwise work. But somewhere you run into the same problem of writers taking turns.
busy_timeout is often sidestepped (ignored) when a transaction attempts to upgrade from a read to a write producing SQLITE_BUSY.
By default, SQLite transactions start in DEFERRED mode, acting as read transactions until an actual write operation occurs.
If another connection begins writing to the database while your transaction is in this read state, an immediate SQLITE_BUSY error is triggered regardless of what you set busy_timeout to
Exactly. In cases where I expect long-running parallel connections from separate processes to the same sqlite file, I make sure that all read transactions do `BEGIN DEFERRED` so `COMMIT` releases the read locks, and all write transactions do `BEGIN IMMEDIATE` so that `SQLITE_BUSY` timeout is not side-stepped.
There was one case where all transactions were implemented using nested `SAVEPOINT bla` so `BEGIN IMMEDIATE` could not be used without more hassle, so this ended all “I know I'm going to write” transactions to instantly update a single-row table so that their lock would not begin as DEFERRED and eventually switch to `IMMEDIATE`; this way almost all `SQLITE_BUSY` side-steppings disappeared. (timeout was set to 30 seconds but all read/write transactions were instrumented to have less than 5 seconds duration).
lol, I briefly thought of such a technique, but in the end, found it simpler to just use a synchronization primitive at the application level to serialize db access on transactions where I knew I was going to write. it amounts to about the same honor code.
After read, it hit me that because sqlite is a DB, "editions" as-is not work.
Because it not tied to the data but to the code.
Instead, what I think should be is that the PRAGMAs become "data" that is always checked in full with "if manually set" and then on next "open" THEY GET APPLIED.
That is.
(and in the command line when open interactively they show up).
RE: SQLITE_BUSY: I would replace "often" with "nearly always." On top of that, it's often not fixed even when pointed out. "This software only has one writer, so we don't need to handle SQLITE_BUSY" translates to me sending SIGSTOP to a process any time I want to run some queries against its database.
I don't think it's a knock on the original prediction to say it didn't turn out to be true. In fact, at the time it probably felt like a pretty conservative prediction. That being said, if you think back to the year 2000, we were pretty far from building such a machine at that time.
To me it's an interesting lesson in how hard it is to predict technology advances which still applies to our own predictions of the future.
Right - but the big difference is that the received wisdom for healthcare is that the higher cost in the US is an unfathomable mystery and/or due to "waste". If you said "writing software is more expensive in the US because software engineers there have higher salaries" everyone one would nod their head in agreement.
Without making any value judgement, there’s a very clear distinction between earning wages as compensation for labor (even very high wages) and increasing your net worth through ownership of a company. That’s a valid distinction no matter how hard the second person works.
I have no idea if this is what AOC meant, but it’s clearly not possible to earn $1B via wages in compensation for labor.
Depends on what you mean by wages. Floyd Mayweather Jr.‘s career earnings are over 1 billion. Celebrities can get extremely high compensation packages especially when you adjust for inflation without an ownership stake in anything. Finance can be similarly well compensated at the very top.
Often very high compensation packages happen to include shares, but that’s just the form of compensation not an inherent requirement. Of course all paths to a billion dollars are so unlikely it’s really not a reasonable target unless you have already passed most significant hurdles.
By legislation and policymaking - just like any other numbers we as a society end up picking (like the age to get a drivers license or buy alcohol or the income tax brackets).
Just like those cases, what other countries are doing would mostly be irrelevant - except, just like now, people may try to find arbitrage opportunities by getting creative about where they live.
Dollars are the way we denominate wealth - no one who understands this thinks that these numbers represent cash that they hold. But that's a far cry from it being imaginary.
This seems to come up on every thread like this. Owning 9% of a company that generates ~$80B in profits and employees 1.5m+ people is literally a massive amount of wealth and putting a dollar figure on that is both straightforward and accurate.
Anyone who owns a house can understand that liquidity and net worth are two different things. But shares of Amazon are far more liquid than a typical home.
In case you need a real example, Bezos personally funds Blue Origin by selling around $1B worth of Amazon stock each year. That's 11000 people earning their salaries + a huge amount of capital investment that are all funded from this so-call "imaginary" money. I can assure you that each time those people get a paycheck, it's just as real as yours.
Sure, but we also attach imaginary dollars to things that wouldn't and can't sell for those imaginary dollars, or even large fractions. And I expect older children to at least catch on to that fact, but a great many adults never seem to.
> and employees 1.5m+ people is l
So is that what the leftists hate? That he employs 1.5 million people? You want that to stop. That's the the part of the him being a billionaire that hurts the most?
>and putting a dollar figure on that is both straightforward and accurate.
If that were true, he could sell it for that valuation tomorrow. But as soon as he tried, the amount would drop, and the company might even be in peril. So it's neither accurate nor straightforward. It's convoluted and overestimated.
>In case you need a real example, Bezos personally funds Blue Origin
So that's the part of his wealth that you despise... that he employs people making spaceships? Those 11,000 people are the problem?
You're not even staying consistent in your own replies in this one comment. Let me boil it down: are the 11,000 people who earn their salary at Blue Origin getting real money or not?
My point is has nothing to do with despising blue origin - it's just a direct contradiction to your absurd belief that this wealth is imaginary. You can't fund that big a company on imagination!
>are the 11,000 people who earn their salary at Blue Origin getting real money or not?
Are they earning (collectively) $260 billion? Are they earning anything like a significant fraction of $260 billion? Is the amount they collectively earn, whatever the total, coming out of Jeff Bezos' wealth, subtracted from it, or are they paid out of several different funding streams such as the government contracts and commercial revenue?
And you think this is somehow some sort of gotcha question. "Look, I've proved that Jeff Bezos has $260 billion!" (or whatever the amount was supposed to be). You're unable to think clearly or correctly on the matter. It's scary how confused you are.
I'm actually making a very simple point. You said most of Jeff Bezos wealth is imaginary / not real. But he's proven that he can turn it into cash at a rate of at least $1B per year.
And again, he only owns 9% of Amazon. Of course if he dumped all of that stock at once the price would go down, but Bill Gates sold almost all of his Microsoft stock for cash and Microsoft stock has continued to go up. Jeff Bezos could certainly do the same and come out the other end with around $260B in cash - but he has zero reason to go do that..
Here's another way to look at this: do you have a retirement account? Is the money in there imaginary? If not, why not?
I wonder if it’s just the colorful metaphors and an opportunity to bring out examples of surprising behavior. Plus it’s a topic that can always stir up debates.
Yes, there is a data race there. The value of a volatile can be changed by something outside the current thread. That’s what volatile means and why it exists.
Edit: thread=thread of execution. I’m not making a point about thread safety within a program.
Not from the standard’s point of view. The traditional (in some circles) use of volatile for atomic variables was not sanctioned by the C11/C++11 thread model; if you want an atomic, write atomic, not volatile, or be aware of your dependency on a compiler (like MSVC) that explicitly amends the language definition so as to allow cross-thread access to volatile variables.
Can also represent a register that has an effect reading it. Reading a memory mapped register can have side effects. Like memory mapped io on a UART will fetch the next byte to be read.
Not sure why you're being downvoted. That's completely right. The example is silly. The code is obviously bad, doesn't matter if it's UB or not.
I'm also not convinced (yet) that the example really is UB: I agree reading a volatile is "a side effect" in some sense, and GP cited a paragraph that says just that. But GP doesn't clearly quote that it's a side effect on the object (or how a side effect on an object is defined). Reading an object doesn't mutate it after all.
But whatever language lawyer things, the code is obviously broken, with an obvious fix, so I'm not so interested in what its semantics should be. Here is the fix:
volatile int x;
// ...
int val = x; // volatile read
printf("%x %d\n", val, val);
The problem is that the function call as a whole is UB. Having the original example compile to the equivalent of
volatile int x;
int a = x;
int b = x;
printf("%x %d\n", a, b);
is equally valid as
volatile int x;
int a = x;
int b = x;
printf("%x %d\n", b, a);
, and neither needs to have the same output as your proposed fix.
C could've specified something like "arguments are evaluated left-to-right" or "if two arguments have the same expression, the expression is [only evaluated once]/[always evaluated twice]". But it didn't, so the developer is left gingerly navigating a minefield every time they use volatile.
Not only is "arguments are evaluated left-to-right" less easy to formalize than you think, it would also make all C code run slower, because the compiler would no longer be able to interleave computations for more efficient pipelining. The same goes for "expression is [only evaluated once]/[always evaluated twice]".
Of course the developer is navigating a minefield every time they use volatile, that's why it's called "volatile" - an English word otherwise only commonly used in chemistry, where it means "stuff that wants to go boom".
Your argument makes no sense since the developer is expected to perform manual sequencing. Correctly written UB free code cannot be interleaved either.
All you've achieved is that the standard C function call syntax can no longer be used as is.
the compiler can still interleave anything it shows is side-effect free; it’s hard to show that something would benefit from being reordered without analyzing it well enough to determine what side effects it has
I understand, that's why I said the code is obviously broken. The problem is not about order of evaluation. It's not about an UB arising from unsequenced volatile reads or whatever.
The problem is simply that the there are two volatile reads where only one was intended. It doesn't matter if there is UB or not. The code doesn't express the intention either way. All you need to know to understand that is that volatile might be modified concurrently (a little bit similar but not the same semantics as atomics).
Too often it's just a list of issues and a wish that everyone else will change.
In (mild) defense of SQLITE_BUSY - busy_timeout just tells sqlite to sleep and retry up to the timeout when it receives SQLITE_BUSY. It seems like a sensible default for a library to leave that up the calling code - which may have something else it could do while it waits. However, that logic often gets missed!
reply