Model transferability and implications for woodland management: a case study of Pinyon Jay nesting habitat

Understanding the transferability of ecological models from one area to another is important for the effective conservation of species of management concern, particularly when the ability to sample across multiple areas is limited. Two measures of transferability, area under the receiver operating curve (AUC) and threshold-specific classification accuracy (sensitivity and specificity), are often employed in assessing models of bird occurrence or resource selection. Although informative, these measures may have limited practical utility in guiding on-the-ground habitat management like forest thinning, which often relies on simple metrics such as tree diameter. We addressed this challenge in studying the occurrence of Pinyon Jay (Gymnorhinus cyanocephalus) nest locations in four piñon-juniper woodland sites in New Mexico, USA. Using generalized linear mixed models, we employed covariates describing woodland structure at several scales to discriminate nest from non-nest plots. We found fair transferability of model predictions between sites via AUC (mean = 0.71), fair threshold-specific specificity (mean = 67%), and poor threshold-specific sensitivity (mean = 42%). Under a hypothetical scenario of forest thinning, we employed a covariate predictive in models of each site, nest (or non-nest plot center) tree root crown diameter (RCD), to assess a management-practical measure of transferability. Using critical RCD values at which 75% of nest trees were retained, we found fair transferability between sites (MEAN = 70%), though we observed retention of nest trees as low as 21%. Average retention increased to 75% (minimum = 60%) when information from multiple sites was combined. This example illustrates that the application of information from one area to the management of another may be effective but may also result in loss of important habitat or other resources. We found that management of Pinyon Jay nesting habitat should proceed on a single-site basis when local nesting information is available. Caution should be exercised when prescribing management in locations where information on nesting is lacking, and information from as many nearby sites as possible should be employed. Transférabilité de modèles et implications dans l'aménagement de secteurs boisés : étude de cas de l'habitat de nidification du Geai des pinèdes RÉSUMÉ. La compréhension de la transférabilité de modèles écologiques d'un secteur à un autre est importante si on veut conserver efficacement des espèces préoccupantes, particulièrement lorsqu'il est difficile d'échantillonner plusieurs endroits. Deux mesures de transférabilité, la surface sous la courbe (AUC) et la précision de classification spécifique au seuil (sensibilité et spécificité), sont souvent employées pour évaluer des modèles d'occurrence d'oiseaux ou de sélection des ressources. Bien qu'informatives, ces mesures peuvent s'avérer d'une utilité pratique limitée pour guider des activités concrètes d'aménagement d'habitat comme l'éclaircie forestière, laquelle repose souvent sur des paramètres simples tel le diamètre des arbres. Nous nous sommes penchés sur ce problème en étudiant l'occurrence de nids de Geai des pinèdes (Gymnorhinus cyanocephalus) dans quatre sites localisés dans des secteurs boisés de pins-genévriers au Nouveau-Mexique, États-Unis. Au moyen de modèles linéaires généralisés à effets mixtes, nous avons utilisé des covariables décrivant la structure des secteurs boisés à différentes échelles afin de discriminer les parcelles avec nids de celles sans nid. Nous avons trouvé que les prédictions des modèles pour les divers sites présentaient une bonne transférabilité selon l'AUC (moyenne = 0,71) et la spécificité spécifique au seuil (moyenne = 67 %), mais une mauvaise transférabilité selon la sensibilité spécifique au seuil (moyenne = 42 %). Sous un scénario hypothétique d'éclaircie forestière, nous avons utilisé une covariable explicative dans les modèles pour chaque site, soit le diamètre à la base du tronc (DBT) d'un arbre avec nid (ou d'un arbre au centre d'une parcelle sans nid), pour évaluer la transférabilité d'une mesure pratique d'aménagement. Lorsque nous avons utilisé les valeurs critiques de DBT auxquelles 75 % des arbres avec nid étaient retenus, nous avons trouvé une bonne transférabilité entre les sites (moyenne = 70 %), quoique nous ayons observé la rétention d'arbres avec nid à un taux aussi faible que 21 %. La rétention moyenne a augmenté à 75 % (minimum = 60 %) lorsque l'information tirée de plusieurs sites était combinée. Cet exemple illustre que l'application d'information provenant d'un endroit pour l'aménagement d'un autre peut être efficace, mais peut résulter aussi en une perte d'habitat important ou une perte d'autres ressources. Nos résultats indiquent que l'aménagement de l'habitat de nidification du Geai des pinèdes devrait se fonder sur la base d'un seul site lorsque l'information locale de nidification est disponible. Des précautions devraient être prises dans le cas de recommandations d'aménagement aux endroits sans information sur la nidification, et l'information provenant du maximum de sites environnants possible devrait alors être prise en compte.


INTRODUCTION
Model transferability, in which the accuracy of an ecological model developed in one location is evaluated when applied to another location, is increasingly important to ecologists, particularly when models provide the basis for land management (Moon et al. 2017). When model transferability is good, managers can make better-informed conservation decisions about new, unsampled areas (van Reusel et al. 2007). Among studies of birds, transferability is typically assessed for models of resource selection, habitat suitability, or species distribution. Most studies assessing model transferability employ covariates from remotely sensed or GIS datasets such as land cover (Gray et al. 2009, Street et al. 2015. Less commonly, the transferability of these models has been assessed at smaller scales, such as the area immediately around nests of breeding birds (Latif et al. 2016). Transferred models generally perform well within the same environment or climate (van Reusel et al. 2007, Street et al. 2015. However, they may transfer poorly despite apparent ecological similarities between locations (Leftwich et al. 1997) or when projected across geographic or climatic spaces (Randin et al. 2006, Torres et al. 2015, Latif et al. 2016). Thus, model transferability is an important consideration (Wogan 2016).
The transferability of models from areas of model development to areas of application is often measured via rank metrics such as the area under the receiver operating curve (AUC; Fielding and Bell 1997). AUC offers the advantage of being thresholdindependent and therefore robust to imbalanced classes. This is desirable in assessing the transferability of models that may employ varying proportions of zeros because of rarity or nonstandardized data collection, or to increase statistical power (Franklin et al. 2009). However, AUC and other thresholdindependent metrics are limited when assessing the classification of occurrences (sensitivity) or nonoccurrences (specificity; Lobo et al. 2008). This information may be critical to management, which may place higher conservation value on occurrence locations such as a forest patch than nonoccurrence locations (Latif et al. 2016).
Threshold-dependent metrics such as sensitivity and specificity may adequately identify model transferability under complex management needs (Latif et al. 2016). A requirement for this approach is the availability of the same measures at the application location as those employed at the development location. Unfortunately, remotely sensed or other habitat data may not be readily available for a new area, or model covariates may occur at resolutions that cannot be measured via satellite or other remote platforms, such as the diameter of individual trees (Latif et al. 2016). In addition, even when finer scale data are available for modeling, the consequences of on-the-ground management based on model-predicted thresholds are not readily apparent.
More pragmatic approaches to management have based recommendations on the numerical distribution of one or more finer scale covariates that (1) discriminate locations of animal occurrence from those of nonoccurrence and (2) are typically employed in management, e.g., tree diameter (Bunnell et al. 2002, Marcot et al. 2010. Although most studies using these approaches have not addressed transferability of this information among areas, they have been interested in applying known information on habitat use to guide on-the-ground management in new areas. The impact of transferring critical values such as median nest tree diameter from one area to prescribe management at another is likely to vary, e.g., in percent nest trees removed, depending on the characteristics of the areas involved. Few if any studies have extended model transferability to evaluate the management consequences of transferring covariate critical values across areas. The Pinyon Jay (Gymnorhinus cyanocephalus) is a medium-sized jay of the corvid family (Corvidae) inhabiting roughly 350,000,000 ha across the western USA, primarily in piñonjuniper (Pinus edulis, P. monophylla, Juniperus spp.) woodlands. Pinyon Jays have a mutualistic relationship with piñon pines whereby piñon trees produce highly nutritional seeds in large crops at irregular intervals (mast crops; Forcella 1981, Parmenter et al. 2018 and Pinyon Jays serve as the trees' main long-distance seed disperser (Ligon 1978, Lanner 1996. Abundant piñon seeds sustain Pinyon Jays throughout the winter, support successful nesting, and significantly influence population viability (Marzluff and Balda 1992). In turn, Pinyon Jays can transport millions of piñon seeds to caching areas up to several miles from the source woodland and cache them in favorable microhabitats for seed germination (Ligon 1978). Pinyon Jays are social, flocking in winter and nesting colonially, sometimes cooperatively, on traditional nesting grounds (Marzluff and Balda 1992). They nest primarily in piñon and juniper trees (Johnson et al. , 2017. Flock home ranges can cover up to 4000 ha in the breeding season and 6000 ha year round (Johnson et al. 2016). Winter flocks can number in the hundreds (Balda 2002) and may move outside the home range when food is scarce (Balda 2002).
North American Breeding Bird Survey data indicate that Pinyon Jay populations declined by an estimated 3.69% per year during 1967-2015 (Sauer et al. 2017). The Pinyon Jay was listed as Vulnerable on the Red List of Threatened Species by the International Union for Conservation of Nature (BirdLife International 2017). Potential impacts to Pinyon Jays include climate change (Johnson et al. 2017) and woodland thinning ), a management practice employed worldwide to reduce potential wildfire risk (Huffman et al. 2009), increase timber productivity (Taki et al. 2010), or improve ecological condition, e.g., thinning to restore ecosystem structure and processes (Brown et al. 2019). Because of long-term population declines (Sauer et al. 2017, Boone et al. 2018) and recent and predicted impacts of climate change on their piñonjuniper habitats (Williams et al. 2010, McDowell et al. 2016, regulatory agencies and land managers need guidance on habitat management to benefit Pinyon Jays. We address the transferability of habitat models in the context of woodland management at fine scales for the Pinyon Jay. Although Pinyon Jay behavior is well-studied (e.g., Johnson 1988a, b, Marzluff and Balda 1988, 1992, Bednekoff and Balda 1996, few studies have investigated variation in patterns of occurrence or habitat use, and none has evaluated transferability of models describing these patterns or the management consequences of transferring critical covariate values from one area to another. In addition to the conservation-related need for information on the generality of habitat models, Pinyon Jays are an appropriate model for a transferability study because, although most nesting Table 1. Acronyms, descriptions, and other details of Pinyon Jay (Gymnorhinus cyanocephalus) nesting colony locations included in this study. The climate of all locations was continental, characterized by cool, dry winters and warm, dry summers. Average monthly temperatures across piñon-juniper habitats in the study areas ranged from approximately 0 °C in January to nearly 30 °C in July. Total annual precipitation mainly occurred during July and August in the form of short, intense thunderstorms. Nest column is the total number of nests measured in each site over the study. Colonies column is the number of nonoverlapping (95% convex hull) nesting colonies at each site. Years column indicates the range of years (inclusive) in which sites were monitored. We use the occurrence of Pinyon Jay nests as the ecological response at several local breeding areas (Johnson et al. , 2015, where varying habitat conditions may confound one-sizefits-all habitat management prescriptions. Our objectives were the following: (1) to fit nest-scale models of Pinyon Jay nest occurrence at four areas in New Mexico, (2) to quantify model transferability among the four areas, and (3) using a management scenario of tree thinning for fuels reduction, assess the implications of managing one site based on the use of nest trees at another site.

Field methods
We studied the occurrence of Pinyon Jay nests relative to habitat characteristics and transferability of occurrence models at four nesting areas ( Table 1). We use "site" hereafter to refer to the four nesting areas, "colony location" to mean the location of a nesting colony within sites, and "plot" to refer to a nest or non-nest plot within a colony.
We located Pinyon Jay nesting colonies by watching flock movements during roadside and walking searches in April and May (protocol described in detail in Petersen et al. 2014). We performed all colony searches in the morning hours. We conducted driving surveys where Pinyon Jays had been reported or in areas of suitable habitat. Pinyon Jays are very vocal and can be heard over road and engine noise if driving slowly. Because only birds close to the road can be detected visually and by calls, we stopped and exited the vehicle approximately every 0.8 km to listen for vocalizations for approximately 10 min. Avian Conservation and Ecology 14(2): 17 http://www.ace-eco.org/vol14/iss2/art17/ When we detected Pinyon Jays showing breeding behaviors (Petersen et al. 2014), we remained in the area for at least one hour or until we confirmed nesting, monitoring the birds from a distance to avoid disturbing them. We found nests by watching for nesting behaviors such as nest construction, incubation, and nest attendance. We recorded active nest locations with a GPS waypoint while standing at least 20 m from the nest and recorded GPS points at inactive and old nests. After all birds had finished nesting, we searched thoroughly within and around the area delineated by nests and recorded GPS coordinates of all additional nests.
We collected vegetation data following a modified BBIRD protocol (Martin et al. 1997). Circular plots were centered on a nest tree, and we generated a sample of non-nest plots in each colony location by centering plots on a tree 100 m in a randomly selected direction from each nest tree without evidence of previous nesting. We measured vegetation within radii of 5 m (0.01 ha) and 11.3 m (0.04 ha) of each nest tree and center tree on non-nest plots. We chose covariates that significantly discriminated nests from non-nests at one or more sites in previous analyses (Johnson et al. , 2015, after eliminating those highly correlated (|r| > 0.7) with other covariates, which left nine covariates at three scales for inclusion in models. At the nest tree or center tree on non-nest plots, we measured root crown diameter (RCD, the diameter of the trunk at the soil level, used in lieu of diameter at breast height for tree species having multiple trunks), height, and canopy cover (mean of canopy presence or absence at the plot center and 1-5 m in each cardinal direction). At the 5-m scale, we measured canopy cover and the stem density, mean RCD, and mean height of all trees over 1 m tall. We assigned each tree on the 11.3-m plot to a size class. To estimate RCD from these classes we used the mean RCD values within these classes (small < 7 cm; medium = 7-21 cm; large > 21 cm) across the smaller plot radius to estimate RCD at the larger plot size. These mean values were 5 cm (small), 13 cm (medium), and 35 cm (large). We calculated stem density and RCD at this scale.

Statistical modeling
We modeled the occurrence of nests within colonies using generalized linear mixed effects models with binomial error (lme4 package, Bates et al. 2017), in which year and colony were included as crossed random effects. These random effects were intercepts assumed to be independent and identically distributed. To assess model transferability, we repeated modeling steps using observations of nests and non-nests at (1) single development sites and (2) combinations of three development sites.
We transferred model sets from single development sites to the other single application sites. We applied model sets from all combinations of three development sites, with a model selection process for each site combination (see below), to single application sites. Transferring model sets combining multiple sites reflects scenarios in which knowledge from several known areas is applied to an unknown area.
Another approach to assessing transferability is k-fold crossvalidation (Wenger and Olden 2012). In the k-fold process, models originally fit to observations across all development sites are iteratively refit using the same covariate combination, by withholding each site and predicting the probability of occurrence at locations in this withheld site. We did not adopt this approach here, because using a model-selection process fit to a data set of all sites combined is not reflective of a realistic management scenario in which the focal site is unstudied.
Modeling for each site or site combination involved several steps. Because of the tendency for breeding birds to select nesting locations with characteristics having intermediate values relative to their availability (Götmark et al. 1995), we first assessed whether quadratic terms for each covariate were potentially predictive. We made this assessment in mixed effects models as described above, using data from single and combined sites by comparing models with (1) only the linear form of the covariate vs.
(2) both a linear and quadratic form of the covariate. If (1) the Akaike information criterion for small samples (AICc) was lower in the quadratic model and (2) the quadratic term was informative (P < 0.15; Arnold 2010), we allowed its inclusion during model selection.
We undertook model selection by first fitting a global model of the nine covariates described above (plus predictive quadratic forms) for each site or combination of three sites. We then performed an all-subsets model selection process (Burnham and Anderson 2004) by identifying subset covariate combinations using the MuMin package (Bartoń 2018). We limited model subsets by removing those models exhibiting high variance inflation factors (> 2; Zuur et al. 2010) or containing uninformative parameters (Arnold 2010). The former reduced the potential effects of multicollinearity while the latter avoided overparameterization by preventing the inclusion of parameters with weak effects. This resulted in candidate sets of 10-194 models. Finally, within these candidate sets, we considered competitive those models within four AICc units of the lowest-AICc model (ΔAICc < 4) in each model set (Burnham and Anderson 2002). Models in which three sites were combined into a single data set employed weighted observations such that each site received the same weight in models and weights summed to the total number of observations. This total varied across combinations of three sites, and was 304 when all sites were combined.

Predicted probabilities
To avoid problems associated with model-averaged parameter estimates (Cade 2015), we used competitive model sets to generate predicted probabilities of nest occurrence.

Transferring models
The process of transferring models involved the same steps as described for internal predictions. However, instead of using linear combinations of parameter estimates (betas) and a matrix of observed covariate values used in model fitting (i.e., data from the development site), we used the matrix of observed covariate values from application sites not used in model development.
After generating transferred predictions (via weighted averages if necessary) we employed these predictions to evaluate the ability of models to discriminate nests from non-nests when transferred to other sites. We (1) used AUC as a general, thresholdindependent measure of discriminatory accuracy, and (2) predicted probability thresholds at which development-site model sensitivity reached 75%. We expected transferability using the first approach (AUC) to range from ≤ 0.5, indicating transferability no better than random, to 1, indicating perfect transferability. Transferability using the second approach (via thresholds) corresponded to an acceptable loss of 25% of nest sites at model development sites. Under this management scenario, trees with predicted probabilities below thresholds could be removed during woodland management. We expected variation in transferability of these thresholds ranging from a combination of high sensitivity and low specificity (all nests retained, few non-nests removed) to a combination of low sensitivity and high specificity (few nests retained, all non-nests removed).

Transferring critical covariate values
In addition to assessing transferability using model predictions (AUC and probability thresholds), we assessed the transferability of site-specific information about habitat use by nesting Pinyon Jays. This process can be useful in meeting management objectives of conserving the majority of used habitat while allowing alteration of unused habitat. We used a covariate present in all competitive model sets, plot center tree RCD, which is commonly used by foresters to identify trees for removal (Lee and Irwin 2005). We used critical values corresponding to the lower quartile (25%) of RCD values within each site or across combined sites. This is akin to probability thresholds used in determining 75% sensitivity across development locations (Latif et al. 2016). In our scenario of prescribed forest thinning for fuels reduction, critical RCD values from one site are used to guide thinning at another site. We transferred these prescriptions to the other three application sites or withheld site (when critical RCD was calculated for combined sites) and determined percentages of nest plots retained and non-nest plots removed under this thinning scenario. All analyses were conducted in R (R Development Core Team 2019).

RESULTS
Vegetation varied among the four sites and between nest and nonnest plots within sites (Table 2). Although center tree RCD and height varied across sites, nest trees within sites were generally larger diameter, taller, and had higher canopy cover than nonnest center trees. Mean canopy cover and number of trees were higher on the 5-m nest plots than on non-nest plots.
The complexity and number of supported models varied between development sites (Table 3), with 3-15 models supported and 1-7 covariates included per model set. All covariates examined were present in supported models for one or more sites (Table 3) and eight covariates had 95% confidence intervals not bounding zero in at least one single-site model set. Of these eight covariates, three (nest tree or non-nest center tree RCD, nest tree or non-nest center tree height, canopy cover within 5 m) had 95% CIs not bounding zero and appeared in model sets of more than one site. Most covariates had overall positive effects on the probability of nest occurrence. The size of these effects varied between sites; e.g., nest  Table 2) with low variance inflation factors (VIF < 2) and without uninformative parameters (85% confidence intervals include zero). Competitive models are those within 4 AICc units of the lowest-AICc model. Null model ΔAICc indicates the range of differences in AICc values of competitive models and a model that only included random effects. Covariates are listed along with overall direction of effects. Asterisks indicate the presence of a quadratic effect. Further information on sites can be found in Tables 1 and 2 or non-nest center tree root crown diameter had an effect at CROW nearly 10 times that at KIRT. Most effects were linear and trending positive, indicating selection for larger trees, taller trees, higher canopy cover, and denser woodlands. Exceptions in models from FARM included quadratic effects of center tree RCD, indicating selection for intermediate-sized nest trees, and negative or quadratic relationships of several covariates at 5-m or 11.3-m plot scales, indicating selection for nesting sites near smaller, somewhat shorter trees.
The internal accuracy of model predictions as measured by AUC was fair (defined as 0.7-0.8) to good (defined as 0.8-0.9; Table 4), with a mean AUC across single-site models of 0.81 (range = 0.77-0.89; Table 4). Model transferability was overall fair between individual sites (mean = 0.71; range = 0.51-0.86) and fair from combined sites to individual sites (mean = 0.71; range = 0.52-0.88). Development sites varied in how well they transferred to other sites (range of means = 0.69-0.73), with CROW transferring most poorly, and KIRT and WSMR transferring most accurately. Likewise, application sites varied in how well they were predicted (range of means = 0.57-0.82), with KIRT and CROW being lowest and highest, respectively.
Our hypothetical scenario of forest thinning, in which tree removal would result in a retention of at least 75% of nests (nest tree RCD > 25th percentile of nest tree RCD in each site; Table  4), would allow the removal of a mean 53% (range = 25-86%) of non-nest trees within development sites. Transferring critical RCD values from individual development to application sites would result in both retention of nest trees above and below the nominal 75% (mean = 73%, range = 21-100%) and allow the removal of an average 57% (range = 25-100%) of non-nest plot center trees. Variation in transferability of RCD values in this scenario would be reduced and the average number of correctly classified nests increased if critical RCD values were calculated from multiple (combined) development sites and transferred to application sites (retained nests: mean = 75%, range = 60-93%; removed non-nest trees: mean = 55%, range = 35-93%). Transferring critical RCD values from FARM would result in the lowest average nest retention (mean = 40%, range = 21-53%) and allow the highest non-nest tree removal (mean = 75%, range = 56-100%), while values from KIRT would retain both a high proportion of nests (mean = 90%, range = 79-100%) and allow for adequate removal of non-nest trees (mean = 49%, range = 25-79%). Neither transferred nest retention nor non-nest tree removal was correlated with AUC (|r s | < 0.5). Although transferred nest retention was not correlated with either sensitivity or specificity (|r s | < 0.6), transferred non-nest tree removal was correlated with both (|r s | > 0.7, P < 0.01). No measure of transferability was correlated with the distance between development and application sites (|r s | < 0.2).

DISCUSSION
Consistent with previous findings (Johnson et al. , 2015, Pinyon Jays appeared to select larger diameter and taller trees in areas with higher canopy cover than available areas within colonies. Despite these similarities, characteristics discriminating nests from non-nests differed across sites, as evidenced by their effects in models and the transferability of both prediction thresholds and critical values of RCD. As measured by AUC, the internal accuracy of single-site models was on average good. Transferability from single development sites to application sites varied from unsuccessful (AUC < 0.6) to good but was overall fair. The same cannot be stated about transferring model-predicted thresholds between sites; given a target sensitivity of 75%, the mean transferred sensitivity of 42% represents an unacceptable degree of misclassification, particularly if these thresholds are to be employed in targeting areas for conservation. Low agreement between transferred AUC and sensitivity reiterates concerns that while classifiers such as AUC offer the advantage of threshold-independence, they may have limited utility in real-world scenarios (Lobo et al. 2008). Although these patterns are clear, predictions used in calculating AUC and thresholds were derived from a single analysis framework (generalized mixed-effects models); additional approaches, e.g., random forest, may have resulted in better transferability (Wenger and Olden 2012).
Utilizing critical RCD values corresponding to 75% sensitivity resulted in overall good between-site transferability under a treethinning scenario. There were some notable exceptions, the most extreme of which would have resulted in a loss of nearly 80% of nests in one site. The variability of critical-value transferability was greatly reduced and average transferability increased when critical values were calculated from combinations of three sites.
Reduced accuracy of models predicting occurrence has been associated with a variety of species traits (McPherson andJetz 2007, Wogan 2016). That Pinyon Jays share few of these traits, e.g., large body size, nonmigratory rather than migratory, woodland-rather than wetland-associated, may contribute to the generally good accuracy of the single-site models. However, the scale of our study was much smaller than those reviewed in the above studies. In addition, the close association of Pinyon Jay with piñon-juniper woodlands in the southwestern USA, and their reliance on piñon trees for both nesting and food, is a fundamental feature of their biology operating across their New Mexico range. This accounts for the inclusion of nest/center tree measures in all single-site nest models.
The differences among the single-site models may be influenced by necessary trade-offs, e.g., predator avoidance vs. thermal considerations, which likely vary spatially and temporally. The main causes of Pinyon Jay nest failure are avian predators and cold and snowy spring weather (Marzluff 1988;K. Johnson, unpublished data). Pinyon Jays made adjustments in nest placement depending on the success of previous nests, moving nests to warmer locations after failure caused by cold weather and to more concealed locations after failure due to predation, but choosing similar locations after nesting successfully (Marzluff 1988). This suggests that choice of a nest location is influenced by multiple selective factors that interact with habitat variability across the Pinyon Jay's range.
The question motivating this study came from habitat managers who repeatedly requested specific values of Pinyon Jay habitat covariates, e.g., "typical" canopy cover or tree diameter, to guide management of Pinyon Jay nesting habitat. These requests assumed that habitat use is fixed, i.e., important covariates and their values are similar across sites. Alternatively, habitat choice could be relative, i.e., habitat use depends on what is available at a site.
The differences in effects within covariates across single-site models indicate that Pinyon Jays in this study did not employ fixed habitat use. Likewise, the varying inclusion of covariates in model sets indicates birds were not operating on the same "model" of selection for nesting locations. Our results indicate that choice of nesting habitat is relative to the available woodland structure at a site. It is also sometimes complex, with multiple influential habitat features that differ among sites. We do not mean to imply local adaptation, in which natural selection has differentiated populations by habitat preference. Pinyon Jays may disperse far beyond their home ranges (Balda 2002). The distances between sites in our study, while large, are certainly within known dispersal distances of this species. Our results suggesting adjustable habitat selection are consistent with the intelligence and behavioral flexibility of this bird (Balda 2002, Bond et al. 2007. Given the fairly wide geographical separation and variation in vegetation structure of our study sites, transferability between sites was surprisingly high when the sites shared similar covariates and covariate parameter estimates in their competitive model sets. For example, good transferability of models from FARM and WSMR to CROW was likely due to the positive effect of nest or non-nest center tree height in models for each site. In contrast, models for FARM and CROW failed to transfer to KIRT (AUC < 0.6), likely because of strong positive effects of nest or non-nest center tree height in the former sites and its absence from competitive models for the latter site.
Transferable models should "describe biologically meaningful environmental relationships that generally determine species distributions while avoiding location specific relationships that are not generalizable" (Latif et al. 2016:778). Studies of model transferability in woodland birds at the nest scale are rare, but one study of two woodpecker species nesting in burned habitats (Latif et al. 2016) features some points of comparison. In that study, models discriminated nests from non-nests at development locations fairly well, but performance and transferability at application locations were variable, and largely poor. As here, that study concluded that single-area models are unlikely to be generally applicable across the entire range of their study species and recommended integration of data from multiple development locations. However, that study did not include a management scenario as presented here.

Management implications
In our scenario, employing critical values of RCD from one site to dictate the extent of thinning at another site resulted in highly variable rates of nest tree retention and non-nest tree removal. For example, although the transfer of critical RCD values from FARM to other sites would allow managers to thin most nonnest trees (mean = 75%, range = 56-100%), this would also result in unacceptably low retention of nest trees (mean = 40%, range = 21-53%). In contrast, transferring critical RCD values from KIRT to other sites would result in high retention of nest trees (mean = 90%, range = 79-100%) and high allowable thinning of non-nest trees (mean = 51%, range = 21-75%).
RCD appears informative in Pinyon Jay habitat management scenarios because it (1) is used by foresters to identify trees for removal (Lee and Irwin 2005) and (2) was a predictive measure in competitive models across all sites. Nonetheless, it is important to consider the shortcomings of focusing on a single covariate when management planning. First, competitive model sets of nest occurrence for each site included up to seven covariates; one or two measures did not adequately describe patterns of nest occurrence in most sites and are therefore likely to exhibit lower accuracy (AUC, sensitivity, and specificity) in discriminating nests from non-nests in unknown areas. Second, removing trees based on RCD would affect stem density, which was present in model sets of two individual sites and model sets of all combinations of sites. Finally, if site-specific models do not share covariates, model transferability will likely be low, and a management scenario like our RCD example will likely be unproductive.
Based on these results, we recommend the best approach to managing habitat for Pinyon Jays and other woodland-dependent species is to understand the local combination of factors discriminating locations of occurrence from those of nonoccurrence. With this information, management for retention of occurrence locations, e.g., nests, to desired levels can be achieved by identifying critical values of discriminating factors, e.g., RCD, tree density, or canopy cover.
If occurrence is unknown in an area or if habitat measurement and modeling of occurrence vs. nonoccurrence are not possible, we recommend conservative management based on an approach similar to that employed in our scenario of thinning based on RCD. Assuming habitat measurements are available from more than one occupied area, we recommend the use of critical values of relevant woodland covariates combined among these areas, then transferring these to other areas with unknown occurrence. This is akin to recommendations made by Latif et al. (2016) in transferring multiarea thresholds for improved management and reduces the potentially biasing influence of areas with habitat characteristics at more extreme ends of their distribution. Additional testing with more areas and multiple influential covariates would improve confidence in this approach. We suspect this process will assist managers in identifying more optimal critical values of one or more woodland covariates necessary to achieve target retention levels at unknown occurrence locations or nonoccurrence locations.