The role of statistical evidence in politics and policy
See "stop and frisk" thread for background/source of inspiration.
Do you believe statistical reasoning and analysis has a legitimate role in debates about--and decision making in--policy and politics? In legal proceedings? To what extent or in which specific ways? What are the limitations that you know of, and can the problems they pose be mitigated? Do you know of examples of good and bad use (objectively and scientifically speaking) of statistics in current political and legal matters?
"One day, we shall die. All the other days, we shall live."
Well, the banking and financial industry (specifically trading exchanges and HFT) uses quite a lot of computerized algorithms and quantum math to "reason" for the human investor. But computers that operate on speed can't filter out extraneous "noise" of social media or news headlines. Then we see volatility and market fluctuations based on "fat finger" mistakes with one too many zeroes, or erroneous reports like "White House bombed, Obama injured".
That's another way of saying that "statistical validity" depends on data collection and computer programming---which is still heavily influenced and dominated by human actions/inputs/controls.
As far as I can tell, there's still no sure way to divorce statistics from lies, or damn lies.
Last edited by GGT; 08-28-2013 at 08:29 AM.
The question is what the alternative is? The alternatives are usually far worse than the use of statistical evidence (usually some combination of wishful thinking, bias, selective use of evidence, and the use of underlying theories and assumptions that are neither voiced nor tested). I would be hesitant to use this kind of evidence too strongly in criminal trials though. What does it mean that the probability of something happening given the observed value is under 1%? If you use that evidentiary standard 100 times, you'll probably convict at least one innocent person.
Hope is the denial of reality
I question the use of statistical evidence in legal proceedings, at least in the US with our extensive procedural rules and adversarial trial model and particularly as evidence in jury trials. You can maybe slide it in as the basis behind the claims in expert testimony but beyond that. . .
the problem is there's no good way for juries (or typically for judges either) to really assess the reliability and accuracy of the information. They're given the information by obviously biased parties with a profound personal interest, they get no access to the data behind it and probably couldn't use it even if they had it, and they're actually forbidden from looking around for any third party or independent source which might be able to corroborate or confirm the data.
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
If their use is questionable and/or unreliable in LEGAL proceedings, they should be just as suspect in crafting police policy. Police don't have the luxury of processing and/or deliberating over information like prosecutors/judges/juries do. But they are trained to act and react, according to policy and procedure, almost like second nature.
Regarding Stop-and-Frisk policy (in NYC at least) when police find illegal guns in fewer than 2% of the five million frisks conducted....they've been physically harassing a ton of innocent people, and should change their policy.What does it mean that the probability of something happening given the observed value is under 1%? If you use that evidentiary standard 100 times, you'll probably convict at least one innocent person.
We're talking about two vastly different things.
Hope is the denial of reality
How so? Mayoral policy translates to city legislation, police SOP, and legal proceedings. If the initial policy is based on invalid or unproven statistical probabilities, that skews everything down the chain, and impacts "innocent people" long before they enter the judicial process.Originally Posted by Loki
Because you're conflating a million different things. My problem was using tests of significance as the primary basis for determining guilt. Your first point was about doing something that rarely has the desired effect. Your second point was about judging the merits of policies. I'm not quite sure what an unproven statistical probability is either.
Hope is the denial of reality
*Checks thread title and OP* Yeah, pretty sure I'm not conflating a million different things. If your problem is using "statistics" in determining guilt in a court room, shouldn't the same problem apply to police policy using "statistics" to determine pre-guilt on public streets?
Especially since police arrests are 'feeders' to the court system?
My problem is not with using statistics, it's with statistical tests of significance for instances where a false positive means locking an innocent person away in jail.
Hope is the denial of reality
That has been used here in a controversial case here, as evidence of a nurse killing patients. Subsequently it turned out this analysis was flawed and ultimately lead to the conviction being overturned.
http://en.m.wikipedia.org/wiki/Lucia_de_Berk
I remember that. I wonder if it would be possible to get these kind of statistical tests peer-reviewed, much the way journal articles are. That would get rid of most potential errors (or intentionally misleading statistics). Since the burden of proof is on the prosecution, it would be their role to prove that their stats meet disciplinary standards.
Hope is the denial of reality
You payed absolutely no attention to why I think they're questionable in legal proceedings. What makes those different from crafting policy, hmm?
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
You said bias from adverse party interest, without the ability of juries to confirm using independent or third party sources.
IMO, policy can also be adversarial, with party interest and bias, and citing 'independent sources' that may have a favorable political agenda. Think Tanks, lobbyists, PACs, and law makers do that statistical dance all the time.
Yeah but unlike the juries, they have the potential to meaningfully evaluate them. The fact that policy-making can sometimes have an adversarial element is meaningless. It matters with juries because court rules actually require them to rely solely on those biased sources of presentation. That taken togther with their own limits on evaluating and interpreting the information makes the use statistical evidence questionable. Two different factors combined make it questionable, not outright bad or unworkable. Furthermore, unlike legal proceedings, policy-making absolutely relies and requires statistical evidence. I realize you far prefer feelings over data but that's because you purposefully and deliberately refuse to let what may actually be the case interfere with your perception of it. Not everyone is in your position, some of them actually have to try and prove what they claim.
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
Right backatcha, Fuzzy. Your perceptions and feelings (about me) interferes with your ability to read (my) posts objectively.
Who provides policy-makers with statistical evidence, how it's been collected, and how it's used, is a legitimate issue. It's been debated for years if those meaningful evaluations are actually taking place in political circles, or if they cherry-pick 'evidence' that favors their agenda and work backward. I don't mean just groups like ALEC, Freedom Works, or economic or policy 'institutes' with Right-or-Left leaning tendencies...but also those refuting/challenging data from agencies like BLS, OBM, CBO, CDC, NIH, even US Census.
The adversarial 'elements' in policy/politics is not meaningless. It matters when science-deniers or flat-earthers are members of congressional committees dealing with science, technology, agriculture, environment, etc. It's meaningful when legislators want to create laws with a religious or 'biblical' basis, and manipulate or distort statistical data to support their claims (particularly where sexual behavior, birth control, and marriage are concerned). It's important when policy-makers debate math models used to determine inflation and CPI, or accounting methods used to calculate taxes....and both sides rely on statistical evidence to support their policy position.
My opinion is based on these facts, not feelings.
It's interesting this distinction between the making of policy and the application of laws, seeing as how the thread was inspired by the intersection of the two. Would your opinions be different when it came to ascertaining the legality of policies after the fact?
"One day, we shall die. All the other days, we shall live."
Our judicial branches decide "legality", ie constitutionality. Those positions are either elected directly, or appointed indirectly, by voters. Since we're a Representative democracy....that means a political party with State majority power (Governor, House, Senate) can change voting districts (aka gerry-mandering) AND voting laws.
*Voters don't just decide our judges, prosecutors, or public defenders.....but also our jury pools.*
South Carolina is the latest and worst example.
Last edited by GGT; 08-31-2013 at 09:12 PM.
Except you want to rely less on statistics and more on other things (what ever those might be)...Right backatcha, Fuzzy. Your perceptions and feelings (about me) interferes with your ability to read (my) posts objectively.
Who provides policy-makers with statistical evidence, how it's been collected, and how it's used, is a legitimate issue. It's been debated for years if those meaningful evaluations are actually taking place in political circles, or if they cherry-pick 'evidence' that favors their agenda and work backward. I don't mean just groups like ALEC, Freedom Works, or economic or policy 'institutes' with Right-or-Left leaning tendencies...but also those refuting/challenging data from agencies like BLS, OBM, CBO, CDC, NIH, even US Census.
The adversarial 'elements' in policy/politics is not meaningless. It matters when science-deniers or flat-earthers are members of congressional committees dealing with science, technology, agriculture, environment, etc. It's meaningful when legislators want to create laws with a religious or 'biblical' basis, and manipulate or distort statistical data to support their claims (particularly where sexual behavior, birth control, and marriage are concerned). It's important when policy-makers debate math models used to determine inflation and CPI, or accounting methods used to calculate taxes....and both sides rely on statistical evidence to support their policy position.
My opinion is based on these facts, not feelings.
Hope is the denial of reality
True, I don't base my opinions on just numbers or statistics, devoid of context. I doubt many people do, including you.
Do you have a purely empirical, statistical, risk/benefit analysis, regarding military intervention in Syria?
If you don't trust statistics, on what basis would you decide if policy X is better than policy Y? On which feels better?
Hope is the denial of reality
I'm not certain what you mean by this. Legality is determined by what is enacted by duly constituted legislative or administrative authorities.
No they don't. Constitutionality and legality are two different things. If the US Congress voted to ban any criticism of itself by the media and the President signed it, it would be a legal action and the NYT criticizing Congress would be illegal. Congress acting in such a way would be unconstitutional and the courts would strike down the action using the judicial review power they possess but until that happened, criticism would be illegal.
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
An example would be the sort of statistical evidence used in the case against NYC's stop-and-frisk policy (as it was being implemented in practice), specifically wrt violation of the right to equal protection.
"One day, we shall die. All the other days, we shall live."
When I think statistics, I think hypothesis testing, not summary statistics.
Hope is the denial of reality
"It's interesting this distinction between the making of policy and the application of laws, seeing as how the thread was inspired by the intersection of the two. Would your opinions be different when it came to ascertaining the legality of policies after the fact?"
I answered minx's question as it was framed -- judiciary decides legality of policies after the fact (ie constitutionality) -- that's the intersection between politics/policy, and legislation/law. You questioned using statistical evidence in legal proceedings, not just because of adversarial rules, but because jurors are limited to courtroom evidence....and I agree!
I think there are similar limits to *evaluating* policy and legislation, since party politicking propaganda abuses/misuses "statistical evidence". It's almost a goal in campaigns and stump speeches, and leads to statements like "47% don't pay taxes".Sure, the voting public can search and research "independent sources" to confirm or refute....but "statistics" show most don't, but rely on headline news or trust their political party to be truthful. And I'm saying is that affects our election and voting rules, who makes up legislatures and jury pools, and ultimately the judiciary itself.
What exactly is the connection between evaluating policy and stump speeches?
Hope is the denial of reality
Am I writing in Greek?What exactly is the connection between evaluating policy and stump speeches?
Except policy isn't evaluated based on crap politicians say/promise on the stump. Why not oppose the English language given that that's misused on the stump as well?
Hope is the denial of reality
I'm still not clear. Statistical evidence can demonstrate the accuracy of facts used to justify a law, or to determine whether it is successful in its purported function but neither has anything to do with its legality.
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
No it's not. Constitutionality and legality are two DIFFERENT things. The legality of policies is determined by acts of legislature and administration, it is not decided by the judiciary (there is a notable semi-exception to this but it does not come into play here).
Look, if the courts decided the legality of things then they could make something illegal, right? But they can't do that. Something that was illegal can effectively be made legal (because the law making it illegal could be struck down) but making something illegal requires some sort of positive enactment. And the courts have no such powers. They can only strike down, not enact. Constitutionality and legality are not interchangeable terms.
I explicitly spelled it out for you that I questioned it because of the COMBINATION of the two. YOU DON'T READ!You questioned using statistical evidence in legal proceedings, not just because of adversarial rules, but because jurors are limited to courtroom evidence....and I agree!
You're also weaseling. Last time you tried to emphasize that adversarial conditions applied to both and argued from there.
The voting public does have some limits in evaluating, viewed in aggregate. They're nowhere near as strong as the ones in the jury room or judge's chambers but they're there. The voting public doesn't make policy though, that's why this is a representative democracy. But those limitations are one of the reasons I'm not a fan of California's overreaching initiative and public-democracy system.I think there are similar limits to *evaluating* policy and legislation, since party politicking propaganda abuses/misuses "statistical evidence". It's almost a goal in campaigns and stump speeches, and leads to statements like "47% don't pay taxes".
Last night as I lay in bed, looking up at the stars, I thought, “Where the hell is my ceiling?"
Posting Permissions
Reply With Quote