I worked on a product where there was basically no automated testing, just a huge product surface to click around with a bunch of options. Because of technical debt some of the options would trigger different code paths, but it was up to the developer to memorize all the code paths and test accordingly.
After I shipped a bug the Director of Engineering told me I should "test better" (by clicking around the app). This was about 1 step away from "just don't write bugs" IMO.
Yep, my first job was at a company like that. Huge Windows desktop app built in Delphi. No automated testing of any kind. No testing scripts either. Just a lot of clicking around.
My first job was exactly that, selling windows app in Delphi. I joined the new team working on .net windows apps and we had an army of people clicking on UI all day long.
They maintained their "test plan" on a custom software where they could report failures.
TBH, that was well done for what it was but really called for automation and lacked unit-testing.
I am forced to use a custom kv store for my current t project. That pos has a custom dsl, which can only be imported through a swing ui, by clicking five buttons. Also, the ui is for 1024 screens, they are tiny in my 4k monitor
Contrarian approach: $8000 is not a lot in this context. What did the CEO think of this? Most of the time it is just a very small speed bump in the overall finances of the company.
Avoidable, unfortunate, but the cost of slowing down development progress e.g. 10% is much higher.
But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this.
I'm sure it cost a lot more than $8000. That was only the direct visible cost to them. There were likely users hit with costs for the additional downloads, who never even knew what was the issue. Users working on a mobile hotspot who had to pay for extra data etc etc.
I doubt there’s a CEO. Despite the use of “we”, pretty sure this is one guy building the app. All the copyright notices and social media go back to one person.
Imagine if that was Meta that had over 1B users with their messenger desktop app update functionality that did just that. The loss would be in the hundreds of millions.
> But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this.
The lesson here is much better use of automated tests (The app likely has no tests at all) and proper use of basic testing principles like TDD would prevent such junior-level embarrassing bugs creeping up in production paid software.
That is the difference between a $100 problem vs a $200M problem.
See the case of Knight Capital [0] who lost $460M, due to a horrific deploy.
How do you adjust your testing approach to catch cases like this? In my experience, timing related issues are hard to catch and can linger for years unnoticed.
I would mock/hook/monkey patch/whatever the functions to get the current time/elapsed time, simulate a period of time (a day/week/month/year/whatever), make sure the function to download the file is called the correct amount of times. That would probably catch this bug.
Although, after such a fuck up, I would be tempted to make a pre-release check that tests the compiled binary, not any unit test or whatever. Use LD_PRELOAD to hook the system timing functions(a quick google shows that libfaketime[0] exists, but I've never used it), launch the real program and speed up time to make sure it doesn't try to download more than once.
Similar to doix said, consider reading the time as IO and then rewrite the code in sans-IO style so you can inject the time.
Then it's a unit test that looks too obvious to exist until you read the ticket mentioned in the comment above it
No need for monkey patching or hooking or preload
But before that you add a couple checkmarks to the manual pre-release test list: "1 hour soak test" and "check network transfer meters before and after, expect under 50 MB used in 1 hour (see bug #6969)"
I don't think the author is wrong for saying that certain kinds of code should be written carefully. I object to the implication that other code shouldn't.
From TFA: "Write your auto-updater code very carefully. Actually, write any code that has the potential to generate costs carefully." So the focus is on code that "generate[s] costs". I think this is a common delusion programmers have; that some code is inherently unrelated to security (or cost), so they can get lazy with it. I see it like gun safety. You have to always treat a gun like it's loaded, not because it always is (although sometimes it may be loaded when you don't expect it), but because it teaches you to always be careful, so you don't absent-mindedly fall back into bad habits when you handle a loaded one.
Telling people to write code carefully sounds simplistic but I believe for some people it's genuinely the right advice.
Just amazed. Yea ‘write code carefully’ as if suggesting that’ll fix it is a rookie mistake.
So so frustrating when developers treat user machines like their test bed!