All Those Warnings About Models Are True: Researchers Given Same Data Come To Huge Number Of Conflicting Findings

William M. Briggs ^ | October 25, 2022 | William M. Briggs

[HKMk23]

Seventy-some researcher groups were given identical data, and asked to investigate an identical question. The groups did not communicate. Details are in the paper “Observing Many Researchers Using the Same Data and Hypothesis Reveals a Hidden Universe of Uncertainty“, by some enormous number of authors.

As is the wont of sociologists, each group created several models, about 15 on average. There were 1,253 different models from the seventy groups Each was examined after the fact, and it was discovered no two models were the same.

[sauropod]

I’m trying to remember who it was that said, “All models are wrong. Some are useful.”

[Robert DeLong]

A gentleman never reveals the actual data points without permission, but they are all great. 🙂

[sauropod]

That was it.

[The Great RJ]

One of our local TV stations sometimes will before major weather events share the different models they use. I always laugh because for estimated snowfall the models often go from a trace to 8+” and everything in between

[Hillarys Gate Cult]

Whatever happened to GIGO? Garbage In, Garbage Out.

[Islander7]

I’m somewhat of a weather geek. Check out https://www.tropicaltidbits.com/analysis/models/ . Do a screenshot of the forecast 9 days out. Then check the same time period every 12 hours or so as it draws nearer to present. The changes are dramatic. These models are pretty accurate 36 hours out, but all over the map further in the future.

[rottndog]

Gentleman? What century are you from???😎😎😎

[Robert DeLong]

What, you think no gentlemen still exist? 🤣

[HKMk23]

I’ll SEE your model, and — uh — see her some more... 8-)

[HKMk23]

https://freerepublic.com/focus/bloggers/4103393/posts?page=19#19

Elsewhere attributed to George E. P. Box.

[Robert DeLong]

Yeah, not all models are useless. 🙂

I like the ones into cars:

[HKMk23]

Thanks. That’s a pretty cool site.

[HKMk23]

I like the ones into MY car, specifically.

[Robert DeLong]

True dat. 🙂