Input | Output |
---|---|
Link | YouTube |
Published | 2020/04/25 |
Theme | |
Status | article incomplete |
Beau says:
Beau outlines his method for selecting trustworthy studies, stressing the importance of peer-review, sample size, methodology, and common-sense reasoning in evaluating research findings.
Researchers, students, academics
The full transcript provides detailed insights into Beau's thought process when evaluating studies, offering practical guidelines for discerning trustworthy research findings.
#Research #Study #Evaluation #PeerReviewed #Methodology #CommonSense
Well howdy there internet people, it's Bo again.
So tonight we're gonna talk about studies
and how to pick the right one to put your faith in.
Because I had somebody ask me,
I've been following you a long time,
I've never seen you like buy into a study
and then it get debunked.
How do you do that?
And I had never thought about it until the person asked.
Um, there's a number of steps that I go through,
apparently subconsciously until now.
The one is the obvious, you know,
one of them is very obvious.
Is it peer reviewed?
You know, have other researchers looked at it?
Have they tried to poke holes in it yet?
What was the sample size?
This is all the normal stuff you ask.
A larger sample size is almost always better.
Ten people is not representative of ten million.
How did they recruit their participants?
Like if you wanted to do a study on
how the average American felt about Trump,
you wouldn't want to conduct the poll
in these comment sections.
You'd get a skewed result.
So what was the methodology?
So that's all one.
The second part is understanding that
studies aren't really there normally
to fully answer a question.
They're not there to draw a conclusion.
They're there to provide more information
so you can combine it with other stuff
to draw a conclusion.
It's very rare to find a study that truly
and fully answers something.
The third thing to look for
is what are they actually measuring?
Because this is how people skew things a lot.
I lived in a community that moved the time
the bars closed from 4 a.m. to 2 a.m.
A year later, they released a study saying,
you know, from 4 to 5 a.m., we had less wrecks.
Just like we said we were going to, we saved lives.
Except the number of wrecks didn't go down.
They just occurred between 2 a.m. and 3 a.m.
instead of 4 a.m. to 5 a.m.
That is a very common way of skewing a study
to produce the results you want.
Be certain you know what they're measuring.
But the first thing I do,
and I think the most important thing,
is I ask myself if there's another reason
for the apparent conclusion.
A common sense reason.
Good example, in the United States,
women who own horses live longer.
Is there a causal relationship there?
Maybe, you know, maybe horses cause you to exercise
and therefore you live longer.
Maybe because horses are funny, they make you laugh
and that makes you live longer.
Or, more likely, most women and men in the United States
who own a horse, well, they're not cowboys and cowgirls.
They own the horse for leisure, it's not for work.
Owning a horse is not cheap.
If you don't have a work reason for the horse, it's a hobby
and it's not a cheap hobby.
When you're talking about food, even if you have pasture,
you have to count the cost of the land, the barn, the vets,
and eventually you've got to get a horse.
It's expensive.
So, if you can afford a horse for leisure,
you can probably afford health insurance.
That will probably make you live longer.
No huge mystery there.
And that one's funny.
There's one that's a little more relevant
because it's going around right now.
And I don't have a lot of faith in the study
and you'll see why here in a second.
In Europe, 80% of the people who succumb, who didn't make it,
from what's going around right now,
they lived in areas with high air pollution,
specifically NO2, which comes from certain types
of automobiles.
Almost 80%, 78% to be exact.
That seems significant.
80% is high.
That is high.
It's worth paying attention to.
However, where does most air pollution occur?
Does it occur out here in the country?
Does it occur in urban areas?
Occurs in urban areas.
What percent of Europe's population lives in urban areas?
Almost 80%.
So it's actually, it's the baseline.
It's 74%, exactly.
But so you only have a 4% variance
from what you would expect.
And to me, it's surprising that it's not actually higher
because in urban areas, you're more tightly packed together.
Seems like it would be easier to transmit.
And you also have better health care in urban areas
because you have bigger hospitals.
They're more immediate.
They're closer than if you live in a rural area.
So I don't put a lot of faith in that study.
I haven't even sat down and read the whole thing
because the statistics are what you would expect.
So I would need to see something involving rates as far as per
capita rural versus urban and per capita high air
pollution versus low air pollution.
Because right now, 80% of the people who didn't make it
live where 80% of people live.
That's not really informative.
That doesn't suggest a causal link to me.
There could be more research on that, though,
because it makes sense.
It's a lung thing, and you breathe in pollution.
But that study alone doesn't do it for me.
That's not one I would put faith in.
So that's my process.
The last thing I covered, that's the first question I ask
before I even really get into a study.
Unless there's a really huge variance,
unless there's something that's really surprising,
yeah, I look for another reason for whatever
the apparent conclusion is first.
And if there is another reason, a common sense reason,
the study has to be spectacular for me
to put any stock in it at all.
Anyway, it's just a thought.
Y'all have a good night.
{{Shirt}}
{{EasterEgg}}