Skip to content

Avoiding decision-based evidence-seeking

Evidence-based decision-making is all the rage nowadays. Everyone, of every political stripe and every cause, claims to be advocating that decisions should reflect the “evidence.
Charles Cirtwill is President and CEO of Northern Policy Institute, an independent social and economic think-tank based here in Northern Ontario.

Evidence-based decision-making is all the rage nowadays. Everyone, of every political stripe and every cause, claims to be advocating that decisions should reflect the “evidence.”

Of course, they tend to be a little picky about what they choose to cite as evidence and even more particular about how the evidence is presented.

While governments at all levels are to be applauded for at least stating their intent to be driven by data and evidence, the proof, as they say, is in the pudding.

As an example, the federal government just announced, via twitter of all places, that small passenger jets would not be allowed to land at Billy Bishop airport in Toronto.

This little tidbit matters to Northern Ontario for three reasons: 1) it impacts Bombardier, a big employer in the Northwest; 2) it affects air service into Northern Ontario (jets carry people faster and cheaper allowing more direct routes); and 3) evidence-based decision-making is important; if they don’t use it here, they will be free to ignore it elsewhere. Say, in the Ring of Fire, for example, or where the investments to support a digital economy go.

According to Terrance Corcoran in the Financial Post, the decision about the jets was made sometime between 6 p.m. and 9 p.m. on a Thursday following a cabinet meeting. At 6 p.m., the new minister was “examining all the factors.”

By 9 p.m., he was tweeting that the “GofC position is the same as LPC commitment,” i.e. no jets. The problem here is twofold. One, it is not clear that this in fact IS the LPC official position. Second, and more importantly, as Corcoran points out: “detailed environmental and traffic studies are underway. Data is being collected, evidence produced.” The decision was made before the evidence was fully before cabinet.

That is not evidence-based decision-making; that is decision-based evidence-seeking. Decision-makers go looking for evidence that supports a decision already made, and ignore, or downplay, evidence that does not support their pre-existing position.

Decision-based evidence-seeking is largely what passes for evidence-based decisions in many government circles. There is a simple test for those of us on the outside to decide which route government (or lobby and advocacy groups for that matter) have engaged in. Which comes first, the decision or the evidence?

In the case of the jets it is pretty clear, the decision came first. A similar example was raised by Ross McKitrick in the National Post in relation to recent federal discussions about climate science and climate change. He points out that there are at least nine major global data sets for temperature.

One of them shows a particularly steep warming trend. The math behind that spike has apparently caused heated discussion and debate. Yet it is he one with the “best” spike that was used to set the tone for a national briefing on climate.

This is far subtler decision-based evidence-seeking, but that is what it is.

Now, McKitrick himself has a dog in this fight being what some call a “skeptic” and others a “denier.” But the point he raises is a valid one. If the evidence is nuanced, then you should base decisions on the nuance. Unless you have already made up your mind. If a “climate change denier” selected only the data showing a significant drop in temperature we would be quick to pounce, yet when government does the opposite it is, in this case, applauded.

This applause goes to demonstrate that government is not the only culprit that commits decision-based evidence-seeking. We, in the general populace, do it all the time, too. If the evidence agrees with our preconceived notions, we promote it. If it doesn’t, we dismiss it. Rare, indeed, is the fair assessment of where our friends went astray or our opponents got it right. We should be unsurprised when advocacy groups, public servants and politicians follow our lead.

So, when government commits itself to being evidence-driven, you need to help them out. You need to hold their feet to the fire, not for decisions you like, but for decisions demonstrably based on analysis of the evidence. Nothing is ever “settled”; every decision blends evidence and politics, and measurement of results, good or bad, is key.

Evidence-driven decision-making means changing your mind once in a while, as the government did on the timing of the Syrian refugee resettlement, for example. We need, and have needed for a very long time, more of that and less of the hard-nosed sticking to the party “commitments,” real or imagined.