ACE-ÉCO Home > Vol. 2, No. 2 > Art. 14

Copyright © 2007 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf version of this article

The following is the established format for referencing this article:
Armstrong, D. P., and M. A. McCarthy 2007. Big decisions and sparse data: adapting scientific publishing to the needs of practical conservation. Avian Conservation and Ecology - Écologie et conservation des oiseaux 2(2): 14. [online] URL: http://www.ace-eco.org/vol2/iss2/art14/

Big Decisions and Sparse Data: Adapting Scientific Publishing to the Needs of Practical Conservation
 
Grandes décisions et données éparses: adapter le processus de publication scientifique aux aspects concrets de la conservation

Doug P. Armstrong ¹ and Michael A. McCarthy ²

• Abstract
• Responses to this Article
• Acknowledgments
• Literature Cited

The biggest challenge in conservation biology is breaking down the gap between research and practical management. A major obstacle is the fact that many researchers are unwilling to tackle projects likely to produce sparse or messy data because the results would be difficult to publish in refereed journals. The obvious solution to sparse data is to build up results from multiple studies. Consequently, we suggest that there needs to be greater emphasis in conservation biology on publishing papers that can be built on by subsequent research rather than on papers that produce clear results individually. This building approach requires: (1) a stronger theoretical framework, in which researchers attempt to anticipate models that will be relevant in future studies and incorporate expected differences among studies into those models; (2) use of modern methods for model selection and multi-model inference, and publication of parameter estimates under a range of plausible models; (3) explicit incorporation of prior information into each case study; and (4) planning management treatments in an adaptive framework that considers treatments applied in other studies. We encourage journals to publish papers that promote this building approach rather than expecting papers to conform to traditional standards of rigor as stand-alone papers, and believe that this shift in publishing philosophy would better encourage researchers to tackle the most urgent conservation problems.

Le plus grand défi de la biologie de la conservation est de fermer l’écart qui existe entre la recherche et l’aménagement. Pour ce faire, un obstacle majeur réside dans le fait que plusieurs chercheurs sont peu enclins à entreprendre des projets susceptibles de générer des données peu nombreuses ou complexes, car les résultats risquent d’être difficiles à publier dans des revues avec comité de lecture. La solution évidente aux données éparses est de combiner les résultats de plusieurs études. En conséquence, nous suggérons qu’en biologie de la conservation, une plus grande attention soit portée à la publication d’articles sur lesquels des travaux futurs peuvent se baser plutôt qu’à la publication d’articles présentant eux-mêmes des résultats concluants. Cette approche « constructive » requiert : (1) un cadre de référence théorique plus solide, par lequel les chercheurs tentent d’anticiper les modèles qui seront pertinents pour les études futures et qui incorporent les différences attendues entre études; (2) l’utilisation de méthodes modernes de sélection des modèles et d’inférence multi-modèles et la publication d’estimés de paramètres pour un ensemble de modèles plausibles; (3) l’incorporation explicite d’information préalable dans chaque étude de cas et (4) la planification de traitements d’aménagement dans un contexte adaptatif qui considère les traitements appliqués dans d’autres études. Nous encourageons les revues à publier des articles qui font la promotion de cette approche « constructive » plutôt que d’attendre des articles qui se conforment aux standards traditionnels de rigueur à titre d’articles indépendants. Nous pensons que ce changement dans la philosophie de publication encouragerait les chercheurs à s’attaquer aux plus urgents problèmes de conservation.

Key words: adaptive management; Bayesian methods; hypothesis testing; scientific publishing; strong inference.

The biggest challenge in conservation biology is breaking down the gap between research and practical management. Murphy (1990:203) asked: “Why has the convergence of science and policy proven so confusing to individuals working on the interface? And why are so many conservation biologists doubtful that their field can contribute anything of value?” These questions remain equally relevant today, and the gap between research and management has recently been documented in two different ways. First, Pullin et al. (2004) illustrated that conservation managers make little use of scientific evidence when making management decisions. Second, Fazey et al. (2005) documented that only a small proportion of the papers published in conservation biology journals assess the effectiveness of conservation actions.

The modes of inference used in practical management are typically quite different from those used in published papers, creating a barrier between research and management. Managers tend to make decisions based on sparse data that has been collected over short time periods, often using intuitive expert opinion and/or consensus at meetings. However, research submitted for publication is largely evaluated based on its scientific rigor, i.e., on whether sample sizes, replicates, and controls are adequate to make reliable inferences. The degree of rigor expected in journal articles has clearly increased over the 30 yr since the Society for Conservation Biology was formed, and the rising rejection rates of most journals mean that empirical studies with weak inference are unlikely to be published.

Although all conservation biologists are concerned about loss of biodiversity, many, if not most, of us are evaluated on the basis of our ability to produce refereed publications rather than on our ability to recover species or restore ecosystems. Researchers, therefore, tend to be reluctant to take on problems with small sample sizes and confounding effects, and particularly discourage their postgraduate students from taking on thesis topics when good data are not guaranteed. On the other hand, managers are aware that most of what they do is unpublishable, and are reluctant to take the time to attempt publication or read the literature. The outcome is that managers tend to work in isolation, both from the research world and from each other, and this is the worst possible scenario when trying to squeeze inferences from limited data.

Although it is easy for researchers to criticize managers for being ignorant of the literature and making decisions based on weak inference, we suggest that at least part of the problem lies in the nature of scientific research and publishing. Murphy (1990:203) defined conservation biology as “the application of classical scientific methodology to the conservation of biological diversity,” and that conservation biologists should religiously follow the procedures termed strong inference by Platt (1964), i.e., devise explicit hypotheses and test them experimentally. Romesburg (1981) made a similar plea in regard to wildlife management. We agree with most of what Platt (1964), Romesburg (1981), and Murphy (1990) have to say. In particular, we agree that it is important to construct explicit theoretical frameworks when alternative hypotheses are considered, then carefully design data collection to assess this theory, and we agree that experimentation is a powerful mode of inference when possible. However, there are aspects of Platt’s (1964) philosophy that we suggest are counter productive for conservation biology in the 21st century.

Platt (1964:351) emphasized the importance of crucial experiments that could eliminate particular hypotheses, and argued that the key attribute of successful researchers was the ability to envision the experiments that allowed such breakthroughs. He was contemptuous of researchers who believed that their small studies would “add another brick to the temple of science”, suggesting that “most such bricks just lie around the brickyard.” He also suggested that most of the “mathematicizing” in physics and chemistry was irrelevant if not misleading, and that the emphasis should be on qualitative exclusion. Ernest Rutherford reflected similar sentiments in his famous statement that “If your experiment needs statistics, you ought to have done a better experiment.” Given that another of Rutherford’s famous quotations was “In science there is only physics; all the rest is stamp collecting,” we suspect that Rutherford, and maybe Platt, would be unimpressed by the messiness of conservation biology. On the other hand, Rutherford also stated at one point that “We haven’t got the money, so we’ve got to think.” With unlimited funding, time, and no constraints, it should be easy to perform the crucial experiments promoted by Platt and to obtain the clear results preferred by Rutherford. It is when data are sparse and confounding effects are numerous that really clever thinking is needed.

In fact, a lot of clever thinking has taken place since Platt’s (1964) paper was published, and some clever technology has been developed. Fifty years ago, networks of researchers were relatively small, calculations were largely done on paper, communication was by mail, and results were published on paper by a small number of journals. Given these constraints, it made sense to concentrate on experiments that did not require complex analysis, and to only publish studies that gave clear results. It is also not surprising that the disciplines admired by Platt for their rapid progress, i.e., high-energy physics and molecular biology, lent themselves to relatively simple reductionist experiments. Today we have vastly improved analytical methods, desktop computers with number-crunching capabilities undreamed of 50 yr ago, the ability to communicate with people around the world at the touch of a button, and the ability to publish a huge amount of information electronically. These developments give us much greater capacity to deal with the constraints and complexities of conservation biology. However, taking full advantage of this capacity will require integrating data from multiple sources, and few individual projects are likely to produce breakthroughs that allow researchers to shout “Eureka!” Continuing to expect individual studies to convincingly eliminate hypotheses will mean that researchers continue to avoid conservation problems when data are likely to be sparse and messy, that potentially useful data will never be published, and that the gap between research and practical management will never be closed.

We suggest that research and publishing in conservation biology should be deliberately modified to facilitate the progressive building of research results rather than placing an emphasis on stand-alone studies, i.e., that a deliberate “brick-laying” philosophy is embraced. However, it is important to consider Platt’s (1964:351) point that most “bricks” actually just lie around the brickyard. We in no way advocate the publishing of data collected from ad hoc monitoring in the hope that it will of use to somebody in the future, and suggest that at least three things are needed for a building approach.

First, contrary to Platt’s (1964) implication that proponents of the brick-laying approach are dullards that are unable to envision the crucial experiments, we suggest that a true building approach requires a stronger theoretical framework than for studies considered in isolation. A building approach requires that differences between studies be considered explicitly, either through random effects or by modeling the effects of factors likely to account for differences. In contrast, results from conservation biology projects currently tend to be considered to be completely case specific or generally applicable, and neither of these viewpoints is sensible. For researchers to claim that their results can be built upon, they need to show the necessary theoretical framework that can be used with explicit models that allow for differences among studies.

Second, the building approach requires largely abandoning traditional statistical hypothesis testing and adopting modern methods of model selection and multi-model inference, a point that has already been made emphatically by Johnson (1999), Burnham and Anderson (2002), Hobbs and Hilborn (2006) and others. Traditional hypothesis testing is effective when a simple, but nontrivial null hypothesis can be confronted with excellent data, allowing it to be rejected convincingly or accepted with a high level of power. However, the method works poorly for considering multiple hypotheses simultaneously, is difficult to interpret when results are indeterminate, and is generally ineffective for comparing studies. Although people often attempt to compare multiple studies based on hypothesis tests, the inferences that can be made are limited, both because it is usually unclear how much the acceptance or rejection of null hypotheses is attributable to power and because there is no capacity to compare strength of effects between studies, i.e., the factor of interest is simply considered significant or insignificant. The more advanced approach is to weigh a range of possible candidate models based on information-theoretic or Bayesian criteria, and to report estimates of the parameters of these models taking both model uncertainty and sampling error into account (Hobbs and Hilborn 2006). Such information can then be built on by subsequent research, and uncertainty progressively reduced, rather than requiring estimates to meet set standards of precision before they are considered useful.

Third, the building approach requires conservation biologists to make explicit use of the available prior information rather than treating the slate as clean at the start of each study. Bayesian inference is particularly applicable for incorporating prior information, and the numerical methods are now available for fitting a wide range of models (McCarthy 2007). An example of a situation in which prior information can be extremely useful is the estimation of the annual survival rate of a species under good conditions. Population viability projections are often highly sensitive to survival estimates, so can be highly uncertain until large amounts of survival data have been collected (McCarthy et al. 1996). As a result, population viability models are not routinely used for management decisions that cannot wait for extended periods of data collection. However, McCarthy and Masters (2005) showed that a prior estimate of survival derived from an allometric relationship could potentially be used to improve precision if few data were available for the species of concern. When measured in terms of precision, prior information derived from allometry was initially worth approximately 5 yr of mark-recapture data in the case study chosen (McCarthy and Masters 2005). Bayesian methods can also combine expert opinion with data (Martin et al. 2005), then expert opinion can be evaluated (McCarthy and Masters 2005) and differences of opinion compared (Crome et al. 1996).

Fourth, the building approach can be facilitated by an adaptive management framework whereby management treatments are applied to complement treatments applied in other studies. Adaptive management provides many of the benefits of traditional experiments (Lee 1999) but allows greater flexibility, less risk, and gradual accumulation of data if sample sizes are small. Adaptive management can be applied to individual case studies. For example, Armstrong et al. (2007) used adaptive management over an 8-yr period to gradually improve estimates of population growth rates of a small population of the endangered hihi (Notimystis cincta) under different possible management regimes. However, Williams et al. (2002) noted that they knew of few cases of true adaptive management being practised, and this may reflect the fact that it is often difficult to vary management over a meaningful scale for individual projects. In Australasia, and perhaps other parts of the world, there is currently a lot of informal adaptive management taking place in terms of different management regimes being tried for different conservation projects. The challenge is to make full use of the data through analysis with state-of-the-art quantitative tools, as envisioned by Walters (1986), and to ensure that the information is published so others can build on it.

If this building approach is to be widely adopted by researchers, it needs to be encouraged by journals. Consciously or not, most journals advocate the traditional scientific method by insisting that each research paper should be able to stand alone and be formatted in the introduction/methods/results/discussion format designed for reporting crucial experiments. Surely it would be better to allow more flexible formats conducive to reporting adaptive management, and to be able to publish tentative, but valuable, results that others could build on. In a world of electronic publishing, the costs of publishing research are not much more than the costs of rejecting research. In comparison, what are the costs of limiting information flow, and discouraging researchers from tackling conservation problems in which “good data” are not guaranteed?

We do not advocate that journals become less discerning in what they publish, as journals play a valuable role in organizing information flow, ensuring that the inferences stated in papers are justified based on the data, and improving the analysis and presentation of material submitted. What we suggest is that journals put greater emphasis on publishing papers that take a building approach to conservation biology, following the requirements outlined above, and place less emphasis on publishing stand-alone papers that conform to traditional standards of scientific rigor. The journal Avian Conservation and Ecology is ideally positioned to take on this role, given that it is a solely web-based journal and focuses on a taxon in which large numbers of people can potentially can contribute data. Will it lead the way?

We thank Jay Gedir and Yvan Richard for comments on the manuscript.

Address of Correspondent: Doug P. Armstrong Wildlife Ecology Group, Massey University, Private Bag 11222, Palmerston, North, New Zealand D.P.Armstrong@massey.ac.nz