The truth of the matter is, there is no possible way to test ALL programs and all possible program interactions. It is impossible.
I say this as someone who, as one part of my job, writes end user test scripts for medical software upgrades on focused systems used by a hospital system.
The real wild card is human beings.
I can wring my mind to think of every way a user might use the software, in what order tasks are performed, and how those tasks are performed, but no matter how much I try, I find that human beings have an infinite capacity for finding unusual and obscure ways to do things, and often, software simply doesn’t take that into account.
It is often funny and frustrating, teeth numbingly boring and professionally exhilarating, all at the same time.
I have spent weeks or months writing a plan and executing it, only to find out after approving workflow changes, usage instructions, and training regimens, that the application reacts in unexpected ways when installed and trained users begin using it. Often, you don’t see the problems (which are often extremely subtle) until weeks or even months after users begin using the software. (I had one time where a few months after installation, we began having certain problems, and it turned out that one woman had been out on medical leave and came back to work, and began doing a task in a way nobody else had thought of doing...which caused the application to behave in unexpected ways.
Then the sometimes excruciating process of figuring out the “why” takes place, and when you have the “eureka” moment, the process begins to have a vendor patch software, rewrite workflows for end users, retrain and write policy and procedure changes.
It boggles my mind to think about how vendors like Microsoft or Apple must have to test their products, which are far, FAR more complex, subjected to hundreds of thousands of times more users, workflows, and aberrant human innovation.
I suppose that is why they write EULAs that are as long as Tolstoy’s “War and Peace”, and force you to sign off on them.
Back in the mid-80s, I designed and wrote a relational database for the Food Bank I founded to handle client intake and tracking, as well as financial donations, and reports. . . and I designed it so that someone unfamiliar with computers could handle the interview process, close out a day, generate daily, weekly reports, etc.. I thought I was quite thorough in figuring out the ways it could be screwed up by people. How wrong I was!
It took me TWO YEARS of actual contact with users to really idiot proof it. But, after those two years, it ran for over twenty years with minimal error correction required. The secret was to limit the ways the idiots could enter data. But it was absolutely amazing the ways they found to screw things up.
Nope. You're right. I was a hardware tester for Microsoft during the Win95 beta. I had a blast.
I'll never forget a few days before release we had our last "bug" meeting. The Product Manager said we were shipping with over 40,000 open bugs.
But, that was part of the process. You hope to go out the door with a manageable number of bugs that nobody will hit. No software is ever released with ZERO bugs. Well, unless someone is lying - LOL!