Maybe I’m mis-interpreting what you mean, but without a notification when a message is sent, what would you correlate a message-received notification with?
Unfortunately a lot of investigations start out as speculation/vibes before they turn into an actual evaluation. And getting past speculation/vibes can take a lot of effort and political/social/professional capital before even starting.
This is definitely not a context problem. Very simple things like checking for running processes and killing the correct one is something that models like opus 4.5 can't do consistently correct... instead of recognizing that it needs to systematize that sort of thing -- one and done. Like, probably 50% of the time it kills the wrong thing. About 25% of the time after that it recognizes that it didn't kill the correct thing and then rewrites the ps or lsof from scratch and has the problem again. Then if I kill the process myself out of frustration it checks to see if the process is running, sees that it's not, then gets confused and sets its new task to rewrite the ps or lsof... again. It does the same thing with tests, where it decides to just, without any doubt in its rock brain, delete the test and replace it with a print statement.
Having done app support across many environments, um - yes, multiple microservices is usually pretty simple. Just look at the open file/network handles and go from there. It's absolutely maddening to watch these models flail in trying to do something basic as, "check if the port is open" or "check if the process is running... and don't kill firefox this time".
These aren't challenging things to do for an experienced human at all. But it's such a huge pain point for these models! It's hard for me to wrap my head around how these models can write surprisingly excellent code but fail down in these sorts of relatively simple troubleshooting paths.
I'm not sure that finding and killing the correct process is something I'd consider to be a "sysadmin task". That's something you learn in the first day of just about any linux course/primer and there are many examples of its use online.
It's more that the default is to overuse tools that cast too-wide nets like pgrep and pkill. And it doesn't know how to use the output well enough. Like, when these systems do ps, it identifies random processes in the list instead of identifying the most recent process that it, itself, started.
It's as if some SRE-type person decided to hard code pgrep and pkill because it's their personal preference.
Really glad they brought up Outer Wilds -- it's exactly the sort of game where the tiniest detail is a spoiler. Knowledge discovery's the game, so any piece of information about the game that doesn't need to be discovered is like cutting ahead to the next chapter in a game. Like playing on someone else's game file.
Wish someone would wipe my memories of that game so I can play it again.
> Wish someone would wipe my memories of that game so I can play it again.
Felt the same for years, now I am doing a new playthrough.
I figured, of course I know the solution to the puzzle, but I am hard pressed to remember all the details of how I uncovered that answer, and I know that you can uncover the clues in nearly any order so I know this playthrough will be new in its own way.
When talking to someone at-risk of deportation earlier in the year, they asked me, "Why should I do anything differently? Obama and Biden did the same exact shit."
And there's a lot of truth to that which a lot of people need to reconcile with.
The fact that we don't have DACA solidified into a path towards citizenship by now is just sad.
And I agree with you, but that's not what I'm questioning. Given the 10x larger scale of deportations during the Obama's term, why were there no protests?
During Obama's term the practice of warrentless entry into actual citizens homes wasn't widespread.
During Obama's term the leaders of DHS / ICE were not blatently lying about events captured on film and evading legitmate investigations into deaths at the hands of officers.
During Obamas term people with no criminal record were not being offshored to hell-hole prison camps with serious abuses of human rights.
Can you link to the tweet in which Obama defended the agents right to threaten a child with rape?
From your linked article:
If the abuses were this bad under Obama when the Border Patrol described itself as constrained, imagine how it must be now under Trump, who vowed to unleash the agents to do their jobs.
The core issue is the media. I worked at a large news company in New York during the Obama’s term. There was a training for our reporters: anything negative about Obama was strictly prohibited. Ad revenue.
As many others have pointed out, the deeper issue is the size of the boot, the disregard for citizens rights, the extremes of the offshore gulags, the fevor with which the upper levels embrace the brutality.
I am unable to assist further with your stated struggle for comprehension.
Not to add fuel to the fire, but a lot of what you're saying is hard to take seriously when Obama himself's been known to brag about how good at killing he is.
You're right that things are significantly worse now, but it's important to recognize that what came before was still bad and in many ways is the foundation for where we are.
Thanks for the response, I'm happy to engage, although I almost missed this as you're well over the fold in my comment history and I have no mechanism for alerting me to replies (nor, I might add, am I looking for one).
With the preamble that I'm not a US citizen, have never thought to apply to be one, have been in and out of the US and many other countries a number of times, and don't play favourites with POTUS(n) on the basis of their asserted party ticket; ...
The upstream question and context here concerns differences between administrations wrt home soil immigration policy, to which I've been focused.
As points of note:
* Allegations of POTUS(X) boasting behind doors are a difference of behaviour from that of POTUS(Y) coming right out and stating they can freely kill in Times Square and get away with while glorifying the deaths of citizens in public and promising perpertrators they'll get away with it and have immunity.
* I'm no fan of remote double tap kills. Full stop. That said;
* POTUS(X) authorising kills in an "inherited" known and ongoing "war zone" known to all is distinct from POTUS(Y) authorising double tap kills from unmarked airframes of civilians in international waters prior to any declaration of war (via Congress or not).
* Regardless, the offshore behaviour of any POTUS is distinct from their behaviour toward their own citizens within their country.
In the arc of all the shitty behaviour by post WW POTUS(n) candidates, the current incumbent has significantly levelled up to achieve Kissinger level disregard for human life on home soil for purely political gain .. and played that hand badly.
That aside, I'm not a Communist - but I do admire Ash Sarkar's shut down of idiotic Obama / Trump faux dichotomy posings by a pompous right wing media pundit - https://www.youtube.com/watch?v=JD7Ol0gz11k
I equally admire our PM's "off the cuff" (approximately 15 mins rough note prep time) strip down of an opposition one time PM attempting to pin a third parties bad behaviour on the sitting government on the basis of them making no comment until after a Court case had completed (as per the law here) - https://www.youtube.com/watch?v=fCNuPcf8L00
It's not relevant to immigration policy, but it is a good example of off the cuff professional level political debate in sitting government.
You won't believe how many hours we have lost troubleshooting SysV init and Upstart issues. systemd is so much better in every way, reliable parallel init with dependencies, proper handling of double forking, much easier to secure services (systemd-analyze security), proper timer handling (yay, no more cron), proper temporary file/directory handling, centralized logs, etc.
It improves on about every level compared to what came before. And no, nothing is perfect and you sometimes have to troubleshoot it.
About ten years ago I took a three day cross-country Amtrak trip where I wanted to work on some data analysis that used mysql for its backend. It was a great venue for that sort of work because the lack of train-internet was wonderful to keep me focused. The data I was working with was about 20GB of parking ticket data. The data took a while to process over SQL which gave me the chance to check out the world unfolding outside of the train.
At some point, mysql (well, mariadb) got into a weird state after an unclean shutdown that put itself into recovery mode where upon startup it had to do some disk-intensive cleanup. Thing is -- systemd has a default setting (that's not readily documented, nor sufficiently described in its logs when the behavior happens) that halts the service startup after 30 seconds to try again. On loop.
My troubleshooting attempts were unsuccessful. And since I deleted the original csv files to save disk space, I wasn't able to even poke at the CSV files through python or whatnot.
So instead of doing the analysis I wanted to do on the train, I had to wait until I got to the end of the line to fix it. Sure enough, it was some default 30s timeout that's not explicitly mentioned nor commented out like many services do.
So, saying that things are "much better in every way" really falls on deaf ears and is reminiscent of the systemd devs' dismissive/arrogant behavior that many folk are frustrated about.
I had a situation like that with an undocumented behavior and systemd-tmpfiles. I wanted it to clean up a directory in /var/tmp/ occasionally. The automation using that directory kept breaking, however, because instead of either finding a whole intact git repo to update or a deleted repo, it instead found only a scattering of files that were root-owned with read-only permissions. There was yet another undocumented feature in systemd-tmpfiles where it would ignore root-owned, read-only files regardless of explicit configuration telling it to clean up the contents of those directories. Eventually this feature was quietly removed:
That was far from the only time that the systemd developers decided to just break norms or do weird things because they felt like it, and then poorly communicate that change. Change itself is fine, it's how we progress. But part of that arrogance that you mentioned was always framing people who didn't like capricious or poorly communicated changes as being against progress, and that's always been the most annoying part of the whole thing.
Speaking of systemd-tmpfiles, wasn't there an issue where asking it to clean all temp files would also rm -rf /home and this was closed as wontfix, intended behavior?
How can I cancel a systemd startup task that blocks the login prompt? / how is forcing me to wait for dhcp on a network interface that isn't even plugged in a better experience?
Your distribution has configured your GDM or Getty to have some dependency on something that ultimately waits on dhcpcd/network-online.target.
It’s not really the fault of systemd; it just enables new possibilities that were previously difficult/impossible and now the usage of said possibilities is surfacing problems.
It is the fault of systemd that there's no interactive control.
On other inits, I can hit ctrl-C to break out of a poorly configured setup. Yes, it's more difficult when there's potentially parallelism. But systemd is not uniformly better than everything else when it lacks interactivity.
And it might not be better than everything else if common distributions set it up wrong because it's difficult to set it up right. If we're willing to discount problems related to one init system because the distribution is holding it wrong, then why don't we blame problems with other init systems on distributions or applications, too? There's no need to restart crashing applications if applications don't crash, etc.
There’s a reason why Devuan (a non systemd Debian) exists. Don’t want to get into a massive argument, but there are legitimate reasons for some to go in a different direction.
After over a decade of Debian, when I upgraded my PC, I tried every big systemd-based distro, including opensuse, which I wholly loathed. I finally decided on Void and feel at home as I did 20+ years ago when I began.
There are serious problems with the systemd paradigm, most of which I couldn't argue for or against. But at least in Void, I can remove network-manger altogether, use cron as I always have, and generally remain free to do as I please until eventually every package there is has systemd dependencies which seems frightfully plausible at this pace.
Void is as good as I could have wanted. If that ever goes, I guess it's either BSD or a cave somewhere.
I'm glad to see the terse questions here. They're well warranted.
Not stopping. Just clashing with that and a hundred other things that I never wanted managed by one guy. Systemd.timer, systemd.service, yes, trivial, but I don't catalog every thing that bothers me about systemd - I just stay away from it. There are plenty of better examples. So where ever I wrote 'stop', it should read hinder.
systemd parses your crontab and runs the jobs inside on its own terms
of course you can run Cron as well and run all your jobs twice in two different ways, but that's only pedantically possible as it's a completely useless way to do things.
> Void is as good as I could have wanted. If that ever goes, I guess it's either BSD or a cave somewhere.
If systemd-less Linux ever go, there are indeed still the BSDs. But I thought long and hard about this and already did some testing: I used to run Xen back in the early hardware-virt days and nowadays I run Proxmox (still, sadly, systemd-based).
An hypervisor with a VM and GPU passthrough to the VM is at least something too: it's going to be a long long while before people who want to take our ability to control our machines will be able to prevent us from running a minimal hypervisor and then the "real" OS in a VM controlled by the hypervisor.
I did GPU passthrough tests and everything works just fine: be it Linux guests (which I use) or Windows guests (which I don't use).
My "path" to dodge the cave you're talking about is going to involved an hypervisor (atm I'm looking at the FreeBSD's bhyve hypervisor) and then a VM running systemd-less Linux.
And seen that, today, we can run just about every old system under the sun in a VM, I take we'll all be long dead before evil people manage to prevent us from running the Linux we want, the way we want.
You're not alone. And we're not alone.
I simply cannot stand the insufferable arrogance of Agent Poettering. Especially not seen the kitchen sink that systemd is (systemd ain't exactly a homerun and many are realizing that fact now).
Gentoo doesn't "exist" because it is necessary to have an alternative to systemd. Gentoo is simply about choice and works with both openrc and systemd. It supported other inits to some degree as well im the past.
The problem is not systemd vs SysV et al, the problem is systemd spreading like a cancer throughout the entire operating system.
Also trying to use systemd with podman is frustrating as hell. You just cannot run a system service using podman as a non-root user and have it work correctly.
Quadlet actually solves this. It's the newer way to define containers for systemd and handles the rootless user case properly. I migrated my services to it recently and it's much more robust than the old generate scripts.
Could you give an example system-level quadlet that accepts connections on a low port, like 80, but runs the actual container as a non-root user (and plays nice with systemd, no force kill after timeout to stop, no reporting as failed for a successful stop)?
My understanding is quadlet does not solve this, and my options are calling "systemctl --user" or "--userns auto". I would love to be wrong here.
As an alternative solution to the sibling comment, I do run everything rootless in systemd --user so my services don't have access to privileged ports, and use firewall rules to redirect the external interface low ports, to the local high ports (that sounds annoying but in practice I only redirect a single port - 443 - to traefik and the use it to route to the right container service depending on domain)
I solved the port 80 issue by adding AmbientCapabilities=CAP_NET_BIND_SERVICE to the Service section of the unit file. That lets you bind privileged ports while still defining a User= line to run non-root. The lifecycle management seems solid in my experience, no force kills required.
Quadlet are great but running podman via systemd as a non root user worked perfectly well before quadlets and I have no idea what your parent is talking about (I'm currently in the process of converting my home services from rootless podman over systemd to quadlet)
Fair, it worked, but podman generate systemd is deprecated now. I found the generated unit files pretty brittle to maintain compared to just having a declarative config that handles the lifecycle.
I agree 100%, I was stuck without quadlet in previous Debian stable so I had to work with systemd generate, but quadlets are undoubtedly better, and I was looking forward to upgrade Debian just for that, and now that I did, I'm really happy to migrate. Especially custom container image management is so much smoother.
Here are a few examples of problems systemd has caused me:
System shutdown/reboot is now unreliable. Sometimes it will be just as quick as it was before systemd arrived, but other times, systemd will decide that something isn't to its liking, and block shutdown for somewhere between 30 seconds and 10 minutes, waiting for something that will never happen. The thing in question might be different from one session to the next, and from one systemd version to the next; I can spend hours or days tracking down the process/mount/service in question and finding a workaround, only to have systemd hang on something else the next day. It offers no manual skip option, so unless I happen to be working on a host with systemd's timeouts reconfigured to reduce this problem, I'm stuck with either forcing a power-off or having my time wasted.
Something about systemd's meddling with cgroups broke the lxc control commands a few years back. To work around the problem, I have to replace every such command I use with something like `systemd-run --quiet --user --scope --property=Delegate=yes <command>`. That's a PITA that I'm unlikely to ever remember (or want to type) so I effectively cannot manage containers interactively without helper scripts any more. It's also a new systemd dependency, so those helper scripts now also need checks for cgroup version and systemd presence, and a different code path depending on the result. Making matters worse, that systemd-run command occasionally fails even when I do everything "right". What was once simple and easy is now complex and unreliable.
At some point, Lennart unilaterally decided that all machines accessed over a network must have a domain name. Subsequently, every machine running a distro that had migrated to systemd-resolved was suddenly unable to resolve its hostname-only peers on the LAN, despite the DNS server handling them just fine. Finding the problem, figuring out the cause, and reconfiguring around it wasn't the end of the world, but it did waste more of my time. Repeating that experience once or twice more when systemd behavior changed again and again eventually drove me to a policy of ripping out systemd-resolved entirely on any new installation. (Which, of course, takes more time.) I think this behavior may have been rolled back by now, but sadly, I'll never get my time back.
There are more examples, but I'm tired of re-living them and don't really want to write a book. I hope these few are enough to convey my point:
Systemd has been a net negative in my experience. It has made my life markedly worse, without bringing anything I needed. Based on conversations, comments, and bug reports I've seen over the years, I get the impression that many others have had a similar experience, but don't bother speaking up about it any more, because they're tired of being dismissed, ignored, or shouted down, just as I am.
I would welcome a reliable, minimal, non-invasive, dependency-based init. Systemd is not it.
As many, many people have pointed out -- many people don't know that their drives are encrypted or know that these protections exist. You're also assuming that the FBI doesn't investigate just random people. "I'm not doing anything bad, why should I worry?"
You're making a lot of assumptions about how people use their computers, their understanding of their own devices, and the banality of building argumentation around what someone should have done or should not have done in the face of how reality works.
I am not assuming the FBI doesn't investigate random people. I am, however assuming that the FBI does not randomly seize computers and obtain court orders demanding encryption keys for them from Microsoft. Unless Microsoft is lying, that happens about 20 times a year.
One of the privacy protections is simply that it's a lot of work to go through that process. The FBI wouldn't have the resources to do it to everyone it's merely curious about even if it had the authority, which it doesn't because warrants require probable cause.
I believe that it's generally acceptable that when law enforcement has probable cause for a search warrant, third parties grant them what access they reasonably can. I also believe people who actually want to protect their privacy and security should learn fundamentals like whoever has the key can unlock it and if nobody has the key, it's gone forever. If I was building a consumer product, I'd have to care quite a bit about the fact that many people won't do that, but I'm not so I don't.
Heh, I subpoena'd Microsoft once in part of some FOIA litigation I did against the White House OMB back in 2017. They, in no unclear terms, denied it. We were seeking documentation.
I realize it's not a court order, but just want to add to the stack that there are examples of them being requested to provide something within the public's interest in a legal context (a FOIA lawsuit) where their counsel pushed back by saying no.
How did you sub poena Microsoft without a court order? Are you saying the court denied your application for an order to produce after Microsoft objected?
I might actually the details wrong. We requested informally at first whether Microsoft could provide information and they declined. Doesn't look like we ended up going down the subpoena route in the end so it didn't really matter.
This matches my experiences as a non-quant, but havin done support work for quite a few of them. You can feel the novelty slough off of them as they get burned down into realizing they're just fitting curves.
reply