or, maybe, there is something hugely wrong with your code, review pipeline or tests if adding randomness to unit test values makes your tests flaky and this is a good way to find it
or, maybe, it signals insufficient thought about the boundary conditions that should or shouldn't trigger test failures.
doing random things to hopefully get a failure is fine if there's an actual purpose to it, but putting random values all over the place in the hopes it reveals a problem in your CI pipeline or something seems like a real weak reason to do it.
Only if it becomes obvious why it is flaky. If it's just sometimes broken but really hard to reproduce then it just gets piled on to the background level of flakiness and never gets fixed.
To get around this, I have it log the relevant inputs, so it can be reproduced.
The whole concept of allowing a flaky unit test to exist is wild and dangerous to me. It makes a culture of ignoring real failures in what, should be, deterministic code.
Well, if people can't reproduce the failures, people won't fix them.
So, yes, logging the inputs is extremely important. So is minimizing any IO dependency in your tests.
But then that runs against another important rule, that integration tests should test the entire system, IO included. So, your error handling must always log very clearly the cause of any IO error it finds.
I remember having a flaky test with random number generation a few years ago - it failed very rarely (like once every few weeks) and when I finally got to fixing it, it was an actual issue (an off by one error).
humans are very good at overlooking edge cases, off by one errors etc.
so if you generate test data randomly you have a higher chance of "accidentally" running into overlooked edge cases
you could say there is a "adding more random -> cost" ladder like
- no randomness, no cost, nothing gained
- a bit of randomness, very small cost, very rarely beneficial (<- doable in unit tests)
- (limited) prop testing, high cost (test runs multiple times with many random values), decent chance to find incorrect edge cases (<- can be barely doable in unit tests, if limited enough, often feature gates as too expensive)
- (full) prop testing/fuzzing, very very high cost, very high chance incorrect edge cases are found IFF the domain isn't too large (<- a full test run might need days to complete)
I've learnt that if a test only fails sometimes, it can take a long time for somebody to actually investigate the cause,in the meantime it's written off as just another flaky test. If there really is a bug, it will probably surface sooner in production than it gets fixed.
Flaky tests are a very strong signal of a bug, somewhere. Problem is it's not always easy to tell if the bug's in the test or in the code under test. The developer who would rather re-run the test to make it pass than investigate probably thinks it's the test which is buggy.
people often take flaky test way less serious then they should
I had multiple bigger production issues which had been caught by tests >1 month before they happened in production, but where written off as flaky tests (ironically this was also not related to any random test data but more load/race condition related things which failed when too many tests which created full separate tenants for isolation happened to run at the same time).
And in some CI environments flaky test are too painful, so using "actual" random data isn't viable and a fixed seed has to be used on CI (that is if you can, because too much libs/tools/etc. do not allow that). At least for "merge approval" runs. That many CI systems suck badly the moment you project and team size isn't around the size of a toy project doesn't help either.
Can't one get randomness and determinism at the same time? Randomly generate the data, but do so when building the test, not when running the test. This way something that fails will consistently fail, but you also have better chances of finding the missed edge cases that humans would overlook. Seeded randomness might also be great, as it is far cleaner to generate and expand/update/redo, but still deterministic when it comes time to debug an issue.
Most test frameworks I have seen that support non-determinism in some way print the random seed at the start of the run, and let you specify the seed when you run the tests yourself. It's a good practice for precisely the reasons you wrote.
Absolutely for things like (pseudo) random-number streams.
Some tests can be at the mercy of details that are hard to control, e.g. thread scheduling, thermal-based CPU throttling, or memory pressure from other activity on the system
- generating test data in a realistic way is often better then hard coding it (also makes it easier to add prop testing or similar)
- make the current time an input to you functions (i.e. the whole old prefer pure functions discussion). This isn't just making things more testable it also can matter to make sure: 1. one unit of logic sees the same time 2. avoid unneeded calls to `now()` (only rarely matters, but can matter)
Similarly, I like .NET's TimeProvider abstraction [1]. You pass a TimeProvider to your functions. At runtime you can provide the default TimeProvider.System. When testing FakeTimeProvider has a lot of handy tools to do deterministic testing.
One of the further benefits of .NET's TimeProvider is that it can also be provided to low level async methods like `await Task.Delay(time, timeProvider, cancellationToken)` which also increases the testability of general asynchronous code in a deterministic sandbox once you learn to pass TimeProvider to even low level calls that take an optional one.
and applying a sealant isn't per-see the problem either
iff
- it's generally commercially available
- and re-applicable after replacement with just generic tools
- and removing the battery doesn't risk breaking your phone due to physical strong binding glue being used as sealant etc.
As a dump example you can design the phone as a sealed unit with the battery department being "outside" the seal. Then have the battery also sealed and apply a bit of "sealant" (wax?, glue?) on the electrical contacts braking the seal on both sides. As the battery and battery compartment back have to only be waterproof and not "rigid" this probably fits "just fine" into most phones (except the most over the top slim ones).
Which is probably more the actual problem. Thinks like phone makers over-obsessing with making phones slimmer on a sub 1mm standard ... and then people anyway putting "thick" cases on the phone to protect it...
water resistance + easily battery exchange for repairs is very viable (AFIK always had been, too.)
like this law isn't about users causally replacing batteries like on very old phones
but about an repair shop easily and without risk of breaking your phone being able to replace it without only holding on your phone for idk. 10 minutes
So that you can just drop by (once they have the replacement parts) wait a moment and have a new battery.
This means in the worst case something like needing to a add a bit of additional seal/wax/glue or similar to improve sealing is very much fully viable (Id the sealing agent is generally buy able.)
It just is something you have to design in from the get to go. And it's easier to not do so at all. And maybe if you obsess if your phone is 1/10mm smaller or not that gets in your way too. And not doing so is more profitable as people will buy successor products more likely, even if just very slightly more likely.
But in general? That really isn't the problem.
Also even if it where the problem. What is better? Having a less waterproof phone, but not needing to buy a new one for another one or two years or having to buy one now?
also camera just not being satisfying enough anymore is a big deal
sure on highest end phones you have very good cameras since a long time by now, but even there they find improvements here and there (e.g. zoom, low light pictures, even better image stabilization)
but middle to lower end phones are still have major improvements in every generation of a certain brand/line/price category. And you might be satisfied with a "acceptable" quality camera, until everyone around you has way nicer photos, or you now have a reason to make photes you didn't had in the past, or you get older and your hands a bit unsteady etc.
It's not "no attorney-client privilege for AI chats" in general.
But a situation where the same would also apply if, instead of going to an chat bot, the person had gone to a random 3rd party non-attorney related person.
As in:
- the documents where not communication between the defendant and their attorney, but the defendant and the AI
- the AI is no attorney
- the attorney didn't instruct the defendant to use the AI / the court found the defendant did not communicate with the AI with the purpose of finding legal consule
- the communications with the AI (provider) where not confidential as a) it's a arbitrary 3rd party and b) they explicitly exclude usage for legal cases in their TOS
Still this isn't a nothing burger as some of the things the court pointed out can become highly problematic in other context. Like the insistence that attorney privilege is fundamentally build on a trusting human relationship, instead of a trusting relationship. Or that AI isn't just part of facilitating communication, like a spell checker, word program or voice mail box, legal book you look things up. All potentially 3rd parties all not by themself communication with a human but all part of facilitating the communication.
Ironically drop box and one drive folders I can still somewhat understand as they are "backuped" in other ways (but potentially not reliable so I also understand why people do not like that).
But .git? It does not mean you have it synced to GitHub or anything reliable?
If you do anything then only backup the .git folder and not the checkout.
But backing up the checkout and not the .git folder is crazy.
Microsoft makes no guarantees on onedrive, you are responsible for backing up that data. Of course they try hard to keep it safe, but contractually they give no promises
> You are using it to mean "maintaining full version history", I believe?
No, they are using it to mean “backed up”. Like, “if this data gets deleted or is in any way lost locally, it’s still backed remotely (even years later, when finally needed)”.
I’m astonished so many people here don’t know what a backup is! No wonder it’s easy for Backblaze to play them for fools.
definition of the term backup by most sources is one the line of:
> a copy of information held on a computer that is stored separately from the computer
there is nothing about _any_ versioning, or duration requirements or similar
To use your own words, I fear its you who doesn't know what a backup is and assume a lot other additional (often preferable(1)) things are part of that term.
Which is a common problem, not just for the term backup.
There is a reason lawyers define technical terms in a for this contract specific precise way when making contracts.
Or just requirements engineering. Failing there and you might end up having a backup of all your companies important data in a way susceptible to encrypting your files ransomware or similar.
---
(1): What often is preferable is also sometimes the think you really don't want. Like sometimes keeping data around too long is outright illegal. Sometimes that also applies to older versions only. And sometimes just some short term backups are more then enough for you use case. The point here is the term backup can't mean what you are imply it does because a lot of existing use cases are incompatible with it.
> To use your own words, I fear its you who doesn't know what a backup is
Feel free to use my reputation, instead: when I say a system is backed up, data cannot be lost by that system being destroyed, because an independent copy always exists. This satisfies those whom it concerns, who put their money where their mouth is, whereas your more generous but insufficient definition would absolutely not be good enough.
When you assure a client that a system is backed up, which definition do they expect from you?
> When you assure a client that a system is backed up, which definition do they expect from you?
the one in the contract (and the various EU laws)
that is not a satisfying answer, I know
e.g. in some past projects the customers explicitly did _not_ want year long backups and outright forbid them, redundant storage systems + daily backups kept for ~1-2 weeks (I don't remember) had been pretty close to the legal limit of what we are allowed to have for that project (1)
the point I'm making was never that a good general purpose backup solutions shouldn't have versioning and years of backups
it's that
1. the word backup just doesn't mean much, so you have to be very explicit about what is needed, and sometimes that is the opposite of the "generic best solution"
2. If data is explicitly handled by another backup solution, even if it's a very bad one, it's understandable that the default is not to handle it yourself. (Through only the default, you should always have an overwrite option, be warned if defaults change, etc.).
Insisting a word means something it doesn't in a way where most non-tech people tend to use it in the definition you say isn't right just isn't helpful at all. Telling them that this is a very bad form of backup which they probably shouldn't use is much more likely to be taken serious.
---
(1): Side note: It's because all data we had is backed up else where, by a different solution, and sometimes can be a bit sensitive. So the customers preferred data loss (on our side, not on theirs) over any data being kept longer then needed (and as such there being more data at any point of time if there is some hacker succeeding or similar). And from what I have heard that project is still around working the same way.
But ironically that is similar to the case here, the data is owned/handled by a different system and as such we should not handle the backup.
But isn't that exactly what Dropbox does? If I delete a file on my PC, I can go to Dropbox.com and restore it, to some period in the past (I think it depends on what you pay for). In fact, I can see every version that's changed during the retention period and choose which version to restore.
Maintaining version history out to a set retention period is a backup...no?
normally this folder are synced to dropbox and/or onedrive
both services have internal backups to reduce the chance they lose data
both services allow some limited form of "going back to older version" (like the article states itself).
Just because the article says "sync is not backup" doesn't mean that is true, I mean it literally is backup by definition as it: makes a copy in another location and even has versioning.
It's just not _good enough_ backup for their standards. Maybe even standards of most people on HN, but out there many people are happy with way worse backups, especially wrt. versioning for a lot of (mostly static) media the only reason you need version rollback is in case of a corrupted version being backed up. And a lot of people mostly backup personal photos/videos and important documents, all static by nature.
Through
1. it doesn't really fulfill the 3-2-1 rules it's only 2-1-1 places (local, one backup on ms/drop box cloud, one offsite). Before when it was also backed up to backblaze it was 3-2-1 (kinda). So them silently stopping still is a huge issue.
2. newer versions of the 3-2-1 rule also say treat 2 not just as 2 backups, but also 2 "vendors/access accounts" with the one-drive folder pretty much being onedrive controlled this is 1 vendor across local and all backups. Which is risky.
I have multiple drives that started out as their own os. Each of them has a Dropbox folder in the standard location. Each of them has a different set of files in them (I deduped at one point), with some overlap of different versions. I no longer use Dropbox, so none of these are synced anywhere.
They don't need to be in my case, I'm only using them now because of existing shortcuts and VM shares and programs configured to source information from them. That doesn't mean I don't want them backed up.
Same for OneDrive: Microsoft configured my account for OneDrive when I set it up. Then I immediately uninstalled it (I don't want it). But I didn't notice that my desktop and documents folders live there. I hate it. But by the time I noticed it, it was already being used as a location for multiple programs that would need to be reconfigured, and it was easier to get used to it than to fix it. Several things I've forgotten about would likely break in ways I wouldn't notice for weeks/months. Multiple self-hosted servers for connecting to my android devices would need to reindex (Plex, voidtools everything, several remote systems that mount via sftp and connected programs would decide all my files were brand new and had never been seen before)
reply