Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

Five Ways to Fix Statistics
Nature ^ | 28 NOVEMBER 2017 | Jeff Leek

Posted on 11/28/2017 1:27:41 PM PST by nickcarraway

As debate rumbles on about how and how much poor statistics is to blame for poor reproducibility, Nature asked influential statisticians to recommend one change to improve science. The common theme? The problem is not our maths, but ourselves.

To use statistics well, researchers must study how scientists analyse and interpret data and then apply that information to prevent cognitive mistakes.

In the past couple of decades, many fields have shifted from data sets with a dozen measurements to data sets with millions. Methods that were developed for a world with sparse and hard-to-collect information have been jury-rigged to handle bigger, more-diverse and more-complex data sets. No wonder the literature is now full of papers that use outdated statistics, misapply statistical tests and misinterpret results. The application of P values to determine whether an analysis is interesting is just one of the most visible of many shortcomings.

It’s not enough to blame a surfeit of data and a lack of training in analysis1. It’s also impractical to say that statistical metrics such as P values should not be used to make decisions. Sometimes a decision (editorial or funding, say) must be made, and clear guidelines are useful.

The root problem is that we know very little about how people analyse and process information. An illustrative exception is graphs. Experiments show that people struggle to compare angles in pie charts yet breeze through comparative lengths and heights in bar charts2. The move from pies to bars has brought better understanding.

We need to appreciate that data analysis is not purely computational and algorithmic — it is a human behaviour. In this case, the behaviour is made worse by training that was developed for a data-poor era. This framing will enable us to address practical problems. For instance, how

(Excerpt) Read more at nature.com ...


TOPICS: Business/Economy; Computers/Internet; Science
KEYWORDS: mathscience; statisics
Navigation: use the links below to view more comments.
first 1-2021-25 next last

1 posted on 11/28/2017 1:27:41 PM PST by nickcarraway
[ Post Reply | Private Reply | View Replies]

To: nickcarraway

This book has been in continuous print since 1954.....................

2 posted on 11/28/2017 1:35:08 PM PST by Red Badger (Road Rage lasts 5 minutes. Road Rash lasts 5 months!.....................)
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

86.3% of statistics are made up on the spur of the moment.

That’s my claim and I;m sticking with it!


3 posted on 11/28/2017 1:36:32 PM PST by budj (Damn the torpedoes, full speed ahead!)
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

“There are three kinds of lies: lies, damned lies, and statistics.”....Mark Twain............


4 posted on 11/28/2017 1:36:53 PM PST by Red Badger (Road Rage lasts 5 minutes. Road Rash lasts 5 months!.....................)
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

Find your favorite fallacy. How about, hard statistical evidence followed by poorly supported assertions that have nothing to do with the evidence. That’s bullying, IMHO.


5 posted on 11/28/2017 1:39:49 PM PST by aspasia
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

Yep - it’s all about how you present the data. I never see confidence intervals, p-values, etc in any news reporting. Mainly because the only thing the idiots really understand is gross percentages...which can be fudged by using the other values.


6 posted on 11/28/2017 1:41:10 PM PST by reed13k
[ Post Reply | Private Reply | To 1 | View Replies]

To: budj
86.3% of statistics are made up on the spur of the moment.

Actually, that figure should be 86.3245769856%, which raises another separate issue. :=)

7 posted on 11/28/2017 1:44:49 PM PST by Bob (Damn, the democrats haven't been this upset since Republicans freed their slaves.)
[ Post Reply | Private Reply | To 3 | View Replies]

To: nickcarraway

Firat step, get a box.


8 posted on 11/28/2017 1:49:41 PM PST by TheErnFormerlyKnownAsBig (Repeal & replace Obamacare, tax reform, fix infrastructure, fixin military, Israel, kill enemies)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Red Badger

To date this book is probably the best book ever written on stats. Always keep a copy handy to lend to my friends (especially libs).


9 posted on 11/28/2017 1:55:08 PM PST by Thalean
[ Post Reply | Private Reply | To 2 | View Replies]

To: nickcarraway
OK Way #1, via virtuous confession: The root problem is that we know very little about how people analyse and process information.

(Ignorance is bliss!) Solution: More epidemiology on how people collect, manipulate, analyse, communicate and consume data.

As Shakespeare said, "For all the rest, they'll take suggestion as a cat laps milk. They'll tell the clock to any business that we say befits the hour."

10 posted on 11/28/2017 1:55:22 PM PST by aspasia
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway
Methods that were developed for a world with sparse and hard-to-collect information...

You mean that four out of five dentists DIDN'T really recommend Dentyne?

-PJ

11 posted on 11/28/2017 1:55:39 PM PST by Political Junkie Too (The 1st Amendment gives the People the right to a free press, not CNN the right to the 1st question.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Thalean

I have an original, printing from 1950’s.................


12 posted on 11/28/2017 1:57:22 PM PST by Red Badger (Road Rage lasts 5 minutes. Road Rash lasts 5 months!.....................)
[ Post Reply | Private Reply | To 9 | View Replies]

To: nickcarraway

Reality is too complex to model with numbers.

A famous example from WW2 was American statistician realized the German industry was continuing to grow despite massive bombing raids. This caused the military to switch from bombing factories to bombing people which strengthened German’s resolve to fight on. The reality was that weapons being made in the new German industry were generally crap and what the bombing raids were doing was reducing the quality of German arms which in turn reduced German fighting effectiveness. But this couldn’t be modeled statistically so it was ignored.

The most effective part of the US bombing campaign came in the last 6-8 months of the war where we targeted Germany’s transportation grid instead of its factories or people. The results of attacks that couldn’t be modeled by statistics. These attacks caused the Germany industry to grind to a halt as rare materials and food we were no longer able to get from point A to point B. If we’d listened to the air force generals urging attacks on the transport grid instead of the bean counters the war would have been over in late 1944.

Moral of the story: Never trust a bean counter.


13 posted on 11/28/2017 2:05:34 PM PST by JohnyBoy (The GOP Senate is intentionally trying to lose the majority.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway
The problem is not our maths, but ourselves. However, don't forget that other epidemiological principle: the distinction between normal and abnormal will not apply. Likewise distinctions between moral or immoral and between right or wrong fall below any prespecified threshold. If you feel a bit lost, don't worry, for that, too, is just a social construct.
14 posted on 11/28/2017 2:16:54 PM PST by aspasia
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

My youngest Daughter double Majored ... she Interned at Ford and at that time was a Theoretical Math Major, they told her she needed a salable Degree and recommended Statistics so she made that her Primary but she Loves Theoretical Math and just does Statistics to pay the way cause Dad told her “NO LOANS” and Dad is 3 outta 3 (3 kids w Degrees and 3 kids without Student Loans)


15 posted on 11/28/2017 3:24:02 PM PST by TexasTransplant (High quality, Low price, Speedy executionÂ…pick any two)
[ Post Reply | Private Reply | To 1 | View Replies]

To: nickcarraway

One of my favorite sayings:
“If you torture numbers they will confess to anything.”


16 posted on 11/28/2017 3:26:54 PM PST by TalonDJ
[ Post Reply | Private Reply | To 1 | View Replies]

To: JohnyBoy

*** “Never trust a bean counter” ***
Where to start ...
May start with GIGO (Garbage In Garbage Out)
Bean Counters don’t do Tactics so if Air Force Generals were taking Orders from the inept it was either Politicians, other Generals, poor Intel on the ground or most likely Politicians, Bankers and Royals.
The beauty of the US Forces in Europe WW2 was the lack of communications perceived or fabricated allowed those on the ground to do what they needed to do to get the job done with Great Leaders like Patton.


17 posted on 11/28/2017 3:35:58 PM PST by TexasTransplant (High quality, Low price, Speedy executionÂ…pick any two)
[ Post Reply | Private Reply | To 13 | View Replies]

To: TalonDJ

Stealing that!


18 posted on 11/28/2017 3:39:22 PM PST by TexasTransplant (High quality, Low price, Speedy executionÂ…pick any two)
[ Post Reply | Private Reply | To 16 | View Replies]

To: aspasia
I thought the fault was in our stars.

So no more asterisks!

19 posted on 11/28/2017 3:40:47 PM PST by who_would_fardels_bear
[ Post Reply | Private Reply | To 14 | View Replies]

To: TexasTransplant

She sounds like a smart young lady who takes after her dad. But I am surprised that Theoretical Math won’t pay the bills.


20 posted on 11/28/2017 4:18:03 PM PST by nickcarraway
[ Post Reply | Private Reply | To 15 | View Replies]


Navigation: use the links below to view more comments.
first 1-2021-25 next last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson