Aspiration and inspiration: good for goals, bad for measurements

By Rebecca Oas, Ph.D. | April 23, 2016

There’s no doubt that the UN’s shiny new 2030 Agenda is ambitious: the total elimination of global poverty is only the first of its seventeen(!) goals. Political commitments like this are meant to inspire and unite activism, effort, and funding and function as a starting point for partnerships. Then, that high-flying aspirational goal is anchored to reality by practical measurements to track progress, monitor spending, and help maintain accountability and transparency.

But when that aspirational thinking starts to invade the way we count things, the whole agenda becomes unmoored from reality and leads to misdirected funds, wasted efforts, and a lot of confusion.

Let’s consider some examples:

I’ve written extensively about the purported “unmet need” for family planning (for instance, here), and why it’s a poor indicator of progress toward women’s health or empowerment, because:

  • it is routinely misconstrued as a measure of lack of access.
  • the vast majority of women labeled as having this “need” have no stated desire or intent to use contraceptives.
  • it dramatically oversimplifies women’s reproductive intentions as measured on household surveys.

But perhaps the biggest problem with the “unmet need” concept is that it gets incorporated, flaws and all, into a wide variety of predictions and projections, many of which begin with the seemingly-simple starting premise “if we just satisfy the unmet need for family planning…” and end with utopian visions of peace, prosperity, equality, good health, and lower carbon emissions.

While the relationship between any of those things and global fertility levels is the subject of much debate, it is necessary to first step back and consider a few facts: 1) “unmet need” is not the same as lack of access to family planning, 2) access to family planning is not the same as use, and 3) turning all the women with “unmet need” into users would require changing their religious beliefs, overriding their concerns about health risks, and turning ambivalence about having a(nother) child into strong motivation to avoid it. In other words, it would mean eroding away the much-repeated caveat that family planning use must be voluntary…by refusing to take no for an answer.

In summary, “unmet need” fails as an indicator because it contains more aspirational thinking than factual reality. Rather than anchoring the state of global family planning to the bedrock of established facts, it serves as a blueprint for policy structures built on sand.

One of the latest reiterations of this kind of magical thinking comes from the World Bank, which recently released the third edition of its Disease Control Priorities (Third Edition). In the volume on “Reproductive, Maternal, Newborn, and Child Health,” the authors predict the outcomes of scaling up a full package of health interventions for women and children.

From a summary published in The Lancet:

“Scaling up all interventions in these packages from coverage in 2015 to hypothetically immediately achieve 90% coverage would avert 149 000 maternal deaths, 849 000 stillbirths, 1 498 000 neonatal deaths, and 1 515 000 additional child deaths.”

But wait, there’s more:

“In alternative calculations that consider only the effects of reducing the number of pregnancies by provision of contraceptive services as part of a Reproductive Health package, meeting 90% of the unmet need for contraception would reduce global births by almost 28 million and consequently avert deaths that could have occurred at 2015 rates of fertility and mortality. Thus, 67000 maternal deaths, 440 000 neonatal deaths, 473 000 child deaths, and 564 000 stillbirths could be averted from avoided pregnancies.”

Lest we forget, less than 10% of “unmet need” results from cost or lack of access to family planning methods, yet this projection starts with the highly questionable assumption that a full 90% of women with “unmet need” could somehow be converted into users.

Also included in those projections is another indicator in need of scrutiny: “deaths averted.” This is, by definition, the measurement of events that do not take place, but would have happened in the absence of a specific intervention. From the standpoint of computer modeling, it could be a useful output value, provided the inputs are good (unlike “meeting x% of “unmet need.”) But as a policy indicator, there are too many unknowns: it’s generally not a good idea not to rely too heavily on measurements that require access to both an alternate universe and a time machine.

None of this is to say that good indicators are perfect. No one measurement can do everything, and even the most apparently simple measurements can become difficult when you factor in poverty-stricken regions, rural areas, and minority communities that could be easily overlooked.

During the Millennium Development Goals era, disgruntled advocates for the controversial notion of “sexual and reproductive health and rights (SRHR)” were strongly critical of the use of maternal mortality ratios (or MMRs, measured as the number of maternal deaths per 100,000 live births) as the main indicator under the maternal health goal. Some scholars criticized MMRs for being measured on the basis of unreliable country-level data; others expressed annoyance at the fact that they didn’t account for the different causes of maternal mortality by country and region. Still others were frustrated by the fact that MMRs offered little assistance to activists seeking to pressure countries to legalize abortion. As Harvard professors Alicia Yamin and Kathryn Falb write (disapprovingly):

“The indicator has come to drive a certain technical agenda regarding the relatively depoliticised domain of maternal —often connected with child— health.”

For all their real weaknesses, MMRs do have a some important strengths. Unlike “deaths averted,” a measurement far better suited to prospective computational models, MMRs are grounded in actual events, not hypothetical ones. If MMRs are unreliable, it’s because some countries—particularly the poorest ones, which often have the highest levels of maternal death—lack the capacity to accurately measure vital statistics. Even the most concrete life events, such as births and deaths, can fall through the cracks. In particular, when they occur close together as in the cases of stillbirths and the deaths of newborns, the short lives of those children may never be recorded in any official capacity.

But rather than seeing this as an argument against the use of MMRs, it can be seen as a strong argument in favor of increasing poor countries’ capacity to maintain robust national statistics, which will benefit everyone, including mothers and babies. Difficult, perhaps, but certainly achievable. Yamin and Falb rightly point out that a measure of access to emergency obstetric care for pregnant women would be a useful additional measurement to include. But making birth safe for mother and child is not their ultimate endgame:

“Maternal mortality is notoriously difficult to measure. SRHR, which is not simply the absence of maternal mortality, is even more difficult to measure. However, while maternal mortality is difficult to measure for statistical and practical reasons, attempting to measure the enjoyment of SRHR implicates an attendant host of conceptual and normative complexities.”

Needless to say, those “complexities” include the notion of abortion as a human right. It’s a good thing SRHR didn’t make it into Agenda 2030…we’ve got enough on our plate with finally eradicating poverty, and the other 16 items on our global to-do list. But if we are to meet the lofty demands of this new framework, we need to avoid counting our progress by measurements based on wishful thinking.

Share on Facebook
Share on Twitter