Input | Output |
---|---|
Link | Youtube |
Published | 2023/02/27 |
Theme | |
Status | article incomplete |
Beau says:
Beau explains the significance of confidence levels in intelligence reports, cautioning against using low confidence information to form opinions.
Policy analysts, researchers, journalists
The full transcript provides a detailed breakdown of how confidence levels in intelligence reports impact decision-making and opinion formation.
Well, howdy there, internet people.
It's Bo again.
So today, we are going to talk about confidence, and reports,
and intelligence, and where things came from.
And we're going to learn about another set of categories
that exist on reports that are really important.
And we will do something for the first time on this channel,
I think.
pretty sure this will be the first time we do something. So if you have no idea
why we're going to talk about this, over the weekend a report was kind of made
public, got a lot of attention, and it dealt with the origin of our public
health issue. And it says it came from a lab. It's prompted some questions. Here's
Here's one.
You've never really taken a firm position on where you think our, quote, public health
issue came from, but you were always visually and audibly skeptical, I like that, of lab
theories.
I'm wondering how the new report has altered your opinion.
OK, so when reports are made, there are findings,
there are estimates, there are assessments.
For the purposes of this conversation,
they are products, intelligence products.
When intelligence products are made,
they are also given a confidence level.
And this is really important.
It became the standard to apply a confidence level in 2004.
Think about what happened in the previous two years that
might have led to that category being applied.
So there are three categories when
it comes to confidence levels.
One is high.
High means this is solid information.
It's a good read on good information.
It's not just rating the finding, the end assessment.
It's also rating the information that that assessment
was based on.
A high confidence level, that is what most people would picture
when they think of an intelligence report.
Somebody gets some information.
It is solid, and they make a report about it.
This is the type of stuff that ends up in the PDB,
the presidential daily brief.
That is what most people probably picture
as all intelligence products.
It's not the case.
To use channel terminology to describe this category,
this is stuff I would say is probable or likely, highly
likely that terminology goes with this category.
The next level is moderate.
Moderate means that's our best guess.
This is probably right.
It's based on decent information.
There may be some gaps.
But this is a good read based on the information
we have at hand.
and channel terminology.
It's plausible.
Yeah, this tracks.
It could be right.
And then you have low.
Low, if I'm not mistaken, the actual definition
includes the phrase implausible information.
And it's fragmented.
They don't have good information.
Therefore, the final intelligence product,
it's not going to be any good either.
But they're doing the best they can with bad information
or giant holes in their information.
This is what I would say, well, it's possible.
I mean, it could be true.
But it doesn't necessarily mean that it is.
Given the fact that I'm going over all of this,
you can probably guess what confidence level
this report had, it was low.
So it's a low confidence report.
They have low confidence in the idea
that that's where it came from.
Now, this is the first time something has happened.
To my knowledge, thinking back, I
have never discussed an intelligence product
on this channel that had a low degree of confidence.
Because frankly, I treat them with about as much faith as I
treat a meme on Facebook.
Don't listen to my father, kids.
He gets his news from Facebook.
That's what the shirt says.
They don't really mean a lot.
Low confidence doesn't mean much of anything.
It's worth noting that this system came into being, I
actually think it's law.
But it came into being around 2004.
The information that led us into Iraq
probably would have been engaged and moderate by this system.
And you know how that turned out.
I don't put a lot of faith in it.
Now, at the same time, you have to understand why these exist.
Because policymakers, the consumer of the intelligence
product, they can ask for products on anything.
Make no mistake about it, if President Biden called up the DCI and was like, hey, I really
want to know if Ted Cruz is a lizard, they would put something together.
Now hopefully, it would say he's not with a high degree of confidence, or at least moderate.
But it might go the other way.
They might say they have a low degree of confidence that he is.
Low is bad.
It's not something you can base a decision on.
So it hasn't altered my opinion.
Another thing that's worth noting is that to my knowledge, and I've read most of them,
There is not a product with a high degree of confidence backing any theory.
None of the origin theories, to my knowledge, have a product that says, yes, we have good
information and we have a good read and this is what it is.
I've yet to see one like that.
I've seen some moderates backing pretty much all of the theories that are out there.
And that, again, goes to show you that moderate probably isn't something you should base a
decision on either.
And that's kind of one of the things that I find weird about this is because I'm pretty
sure that there's actually a product from, I want to say the FBI, from a few years ago
had a moderate degree of confidence backing that theory.
So I don't really understand why people latched on to the low confidence one.
But it hasn't changed my opinion, I don't know.
And I am very skeptical of that theory in particular, mainly because it coming out right
now, the way it has, that's a little convenient. I mean, that is a little
convenient. It's coming out at a point in time when U.S.-Chinese relations are not
great and there's a lot of pressure being exerted and people are trying to
find their ways on the foreign policy scene. Something like this coming out
right now. It's surfacing at this moment. It seems very convenient to me,
especially given the fact that it's low confidence. You normally do not get
reporting about intelligence products that are low confidence because most
people know what it means. Most people who cover this kind of stuff, they know
that, I mean you could have a low confidence report saying that Kennedy
was involved in a car accident and that's how it happened. I don't put a
lot of faith in them and it's important to understand that these categories
exist for a reason, because when they didn't, reports could be requested, they
could be put together, and then they could be framed politically, and in ways
that went really bad. Those categories exist for a reason, and they should
definitely be at least acknowledged, and people should understand that unless
it's high, it shouldn't be something to really sway your opinion. Take the
information, look at it, but at the end of the day, the people creating it, when
it's low, they're kind of saying, yeah, I mean, this is possible, but I wouldn't
stake my reputation on it. So just bear that in mind. I would imagine that this
This is going to get a lot of coverage this week.
But it is important to acknowledge it's a low confidence product.
And it's also important to acknowledge that, again, to my knowledge, we don't have a high
confidence product backing any theory.
Anyway, it's just a thought.
Have a good day.
{{Shirt}}
{{EasterEgg}}