A proposed hierarchy of evidence for Defence

Not all evidence is created equal. Throughout any given year, Defence will develop, review and implement numerous policies that will have a direct impact on its personnel. Many are broad reaching and will affect every serving member, whereas others will only have an impact on a small section of the workforce.

Regardless, it would be reasonable to assume that every new or revised policy has a strong and robust foundation in evidence. Unfortunately, this is not always the case and whether it is through the unavailability of evidence or the specific nature of the decisions that need to be made, the ADF is regularly at risk of formulating policy based on evidence that is less than ideal.

Distinguishing between good and bad—or relevant versus irrelevant—evidence is not always straightforward. For example, glossy and well‐produced reports can mask poor research. In contrast, good research can be compromised by poor presentation, complex language or a lack of accessibility. Superficial barriers such as these are only the beginning of the problems that await policymakers as they attempt to sort through a spectrum of research documents related to their policy area. Although rarely acknowledged, sorting through the available evidence is a core task of policymakers, a skill that is not widely developed or even instilled outside our academic institutions.

In this article, some guidelines are proposed to help personnel policymakers distinguish between evidence that is robust and able to support policy decisions and that which might be useful but should be treated with a healthy level of scepticism.

Author