bkmk
Can’t wait to get comments on this from my software pro buddy.
To my mind the most important question is did Ferguson screw up, or was this deliberate?
If it’s the latter...
You mean that someone wrote code that is worse than H-1B product? That is saying something!
From Robert, in one of the comments below the article...
“Science is about discovering the truth, no matter how inconvenient or unwelcome it may be to particular interested parties. Politics is about accomplishing the goal of interested parties and hiding any truth that would tend to impede that goal. The problem is not that “government has being doing it wrong;” the problem is that government has been doing it.”
All software models used to determine public policy must be open source. problem solved. global warming bs also fails with this requirement.
Two things stand out in the article.
> the problem goes away if you run the model in single-threaded mode
In other words, multiple threads are running and each is accessing the same memory without synchronization, corrupting the numbers. When run as designed, NOTHING valid can come from this program.
> Reports of random results are dismissed with responses like “that’s not a problem, just run it a lot of times and take the average”,
Averaging results is NOT for dealing with bugs. It makes sense to use random sampling and average the results rather than run through every possibility - such as in gambling for example. They are corrupting the data in unpredictable ways and expecting it to somehow average out.
This is a multi-trillion dollar software problem.
Anyone who has ever seen "academic code" knows that the majority of it is nearly worthless. About the only thing you can do with it is pull some equations out of it and throw the rest away. It is typically the byproduct of a professor with an interest in some other field (other than software development) - he or she is merely using it as a tool, a means to an end, rather than a pursuit in and of itself. I'm not knocking that - hey we build tools for people to use. But I am saying that kind of focus results in poor practices and poor code quality. Academic code is also typically written and maintained by barely interested grad students just hoping to get by and keep the prof happy.
This is not a recipe for good code. The inability to reproduce results given the same inputs speaks volumes. A single 15,000 line long file is also a joke. That's indicative of the lack of professionalism and care or concern for the project. It is the path of least resistance. Well, perceived as such by people who don't know what they are doing. While good organization and file structure requires a little more work and thought to set up, you reap the rewards many times over down the road. Unless of course you're a grad student just banging out a change for the prof and you're not going to be here next semester and it'll be someone else's problem...
Why is this program even still under development? At a mere 15k lines it should have been tossed out and re-written long ago. 15k lines sounds like a lot - and it is in one file - but in the world of programming that's a couple months work for a small to modest sized team - to produce tested verified code. Heck, when I was a consultant I personally managed a codebase of about 120k lines as sole programmer. Our team right now is managing in the neighborhood of 3 million SLOC. If this project were given to me tomorrow the first thing I'd do is set it aside and never compile it again. Clean-sheet design for something that size should take a few days, maybe a week. I could have a team code it in a couple of months, or knock it out myself in several weeks of undisturbed coding bliss. ;-)
The author is right - this code is an embarrassment. Ever action taken based on the results of running it should be retracted and an apology issued.
This is also why climate scientists like Michael Mann will never realease their source code. Mann was involved in a lawsuit in Canada where the source code became part of the discovery. He refused to give the court the code. The court ruled against him and he lost the lawsuit.
For later
The author compares the Ferguson Model to Simcity. I think that is a good comparison. Simcity is a game and may be fun to play but it, like the Ferguson Model, is not reality. A model is just one tool in the toolbox that may be used to solve a problem, not an end-all.
I have had personal experience with this. Academic code becomes a tool used by politically minded professors to convince policy makers to approve of the policy they want.
It makes them look authoritative, but the fact is a computer simulation easily gives the person(s) writing and running it the results they want.
If it doesn't, then "something is wrong with the code" and it has to be tweaked to get the "right" result.
I saw a hundred million dollar copper zinc mine in Wisconsin be legislated into oblivion (never developed or opened) because the legislators taxed it to death based on bogus academic code, written by grad students for a Marxist professor.
When I read the code, years later, it was clear one character change was the major culprit. A cost was being treated as income. It made all the difference.
Computer models should never be relied on for political decisions.
i am skeptical that microsoft would be the optimal entity to fix any problems. imho microsoft has been unable to fix the windows os, and windows was created by microsoft decades ago.
ideally, a software simulation would require some kind of independent verification, especially if it is monte carlo. one possibility would be an independently developed software simulation.
software management is difficult in the best of situations.
it sounds as if the university research management suffers from tunnel vision. regardless, regression suites are standard practice. monolithic source code program files do sound very much like ancient practice and not something originally written in either C or C++.
i presume fauci and birx referenced ferguson in all of the covid19 task force deliberations and conclusions, and if so, this would seem to explain a great deal.
15,000 lines of spaghetti code in a single file. And the output varies between runs, with the exact same initial conditions. Without explanation.
We put the world under house arrest for this. And won’t admit it was an awful mistake. So most places are still under some form of house arrest.
Cloward-Piven worked. The economic depression is just getting started. Hang on.
I took a glance at the Github repository (and this is the version of the code that Microsoft engineers spent a month cleaning up).
When you see comments like “ Bugs found - code is stable and seems deterministic again. Will finish more tests and send pull request”, it doesn’t reassure....
L8r