Using eBird data to model population change of migratory bird species

Citizen science projects provide a vast amount of biological data that can be used to model population trends of species. Robust statistical modeling techniques are necessary to account for multiple sources of bias inherent to the data. One such citizen science project, eBird, is an online database of avian checklist data entered by birdwatchers from discrete locations and visits. The eBird dataset may be large enough to fill information gaps left by other monitoring programs if biases in the data are modeled appropriately and if the models can be validated against reliable survey data. We compared eBird and North American Breeding Bird Survey (BBS) data from southern Ontario to determine if patterns in annual indices and long-term trends were similar for 22 species that reach the northern limit of their range in that region. Mixed-effects models were used to address varying observer skill and uneven geographic coverage in eBird, and the number of species per checklist was used as a covariate to represent effort and to accommodate historic lists lacking effort information. The average Pearson’s correlation coefficient between eBird and BBS annual indices across species was 0.35, and the correlation between trends estimated from the annual indices was 0.72. eBird data generally agreed with BBS data with the exception of two common species that showed opposite trends, several species with low detection rates, and for two species with little long-term change in occurrence based on BBS data. Our results suggest that eBird data can be used to generate long-term trends that could complement data from traditional surveys, yet more work is needed to understand circumstances that lead to disagreement between eBird and other surveys. Utilisation des données eBird pour modéliser les tendances de populations d'espèces d'oiseaux migrateurs RÉSUMÉ. La participation citoyenne à des projets scientifiques concourt à amasser une grande quantité de données biologiques pouvant être utilisées pour modéliser les tendances de populations d'espèces. Des techniques de modélisation statistiquement robustes sont nécessaires afin de tenir compte des diverses sources de biais inhérents aux données. Projet scientifique à participation citoyenne, eBird est une base de données en ligne colligeant des listes d'oiseaux remplies par les ornithologues amateurs pour un site particulier et une visite précise. Le jeu de données eBird peut permettre de combler le manque d'information de certains programmes de surveillance si les biais associés aux données sont modélisés correctement et si les modèles sont validés avec des données d'inventaires fiables. Nous avons comparé des données provenant d'eBird et du Relevé des oiseaux nicheurs d'Amérique du Nord (BBS) pour le sud de l'Ontario afin de déterminer si les indices annuels et les tendances sur une longue période étaient similaires pour 22 espèces à la limite nord de leur aire de répartition. Nous avons utilisé des modèles à effets mixtes dans le but de corriger l'expérience variable des observateurs et la couverture géographique inégale des données d'eBird; le nombre d'espèces par liste a été utilisé comme covariable indicatrice de l'effort et pour pallier les listes sans mention d'effort. Le coefficient de corrélation de Pearson moyen entre les indices annuels d'eBird et du BBS pour l'ensemble des espèces était de 0,35, et la corrélation entre les estimations de tendances obtenues à partir des indices annuels était de 0,72. Les données d'eBird concordaient généralement avec celles du BBS sauf dans les cas suivants : deux espèces communes montrant des tendances opposées, plusieurs espèces au faible taux de détection et deux espèces affichant peu de changement de l'occurrence sur une longue période selon les données BBS. Nos résultats indiquent que les données d'eBird peuvent être utilisées pour produire des tendances couvrant une longue période et compléter ainsi les données d'inventaires traditionnels; davantage de travaux doivent toutefois être entrepris pour comprendre les circonstances qui mènent à des divergences entre les données d'eBird et celles d'autres relevés.


INTRODUCTION
The proliferation of data generated by citizen science projects provides a new and valuable resource for biological monitoring efforts (Dickinson et al. 2012, Theobold et al. 2015).Citizen science projects are capable of covering geographic and temporal scales that may exceed what is financially or logistically possible for structured monitoring regimes, and thus have the potential to fill important information gaps (Conrad andHilchey 2011, Tulloch et al. 2013).However, because citizen science data are typically collected in a less rigorous manner than formal biological surveys, there are additional sources of variability and bias that must be accounted for.These include: observer skill, geographic coverage, effort (in time and space), detectability (in time and space), and bias in reporting rates between rare and common species (Isaac et al. 2014, Kamp et al. 2016).Various statistical modeling approaches can be used to help account for the variability and bias inherent in citizen science data to extract meaningful biological signals.The benefits and limitations of some of these methods are described and compared in Hochachka et al. 2012, Bird et al. 2014, and Isaac et al. 2014.One such citizen science project, eBird, is a global online bird database that compiles bird checklists from discrete locations and dates made by observers of any skill level, ranging between beginner and professional ornithologist (Sullivan et al. 2009(Sullivan et al. , 2014)).The eBird program uses several basic checklist protocols (e.g.traveling, stationary, incidental) to categorize the manner in which most birdwatching checklists are generated, which enables the aggregation of checklists ranging from simple bird lists to highly structured surveys in the same format.The database has been growing exponentially since the project was launched in 2002, and there are currently over 18.8 million checklists from across the world, with the bulk (17 million+) from North America.Data are publicly displayed on the eBird web site (http://www.ebird.org)and available for download for scientific purposes.Data from eBird have been used extensively for species distribution modeling and many other contemporary studies (e.g., Fink et al. 2010, Sullivan et al. 2014, Kelling et al. 2015), but to our knowledge, have not yet been used to model long-term trends in species abundance.The checklist protocol types offered by eBird provide a means by which historical checklists can be entered and archived, provided that each checklist includes, at a minimum, a date and specific location.A large proportion of eBird checklists (~85%) are from the past decade, which may limit the utility of the dataset for modeling historical population trends, but there exist enough historical checklists from North America to justify exploratory modeling of long-term population trends using eBird data.
The North American Breeding Bird Survey (hereafter BBS), was initiated in 1966 and is the primary source of long-term population trend information for most species of birds breeding in North America (Sauer et al. 2013).Although the BBS protocol is more structured than that of eBird, the BBS is volunteer based and subject to some of the same sources of bias and variability as eBird data, e.g., observer skill, uneven geographic coverage, and detectability.There are some limitations to the BBS data because of its volunteer-based, roadside protocol that include: under-and over-representation of some habitat types along roads, uneven rates of land cover change between roadsides and interior areas, lack of coverage in boreal and arctic habitats, and low detection rates for nocturnal, cryptic, rare, disturbance-prone, and water-bird species (e.g., Betts et al. 2007, Harris and Haskell 2007, North American Bird Conservation Initiative Canada 2012).The eBird dataset could potentially fill some of the information gaps left by the BBS by providing long-term population change estimates for species not well covered by that protocol.For example, eBird data might be used to assess population change in boreal and arctic breeding species that are encountered primarily during migration, as is done with the Canadian Migration Monitoring Network data (Crewe et al. 2008).
A logical first step to assess the utility of the eBird data set to monitor long-term population change in species is to compare eBird data to BBS data for species with reliable trend estimates and constrained (geographically) breeding distributions.Several other studies have compared population trends derived from citizen science data to those derived from more formal surveys, with mixed results.For example, strong correlations in reporting rates of bird species were found between informal area searches in locations selected by volunteers, and similar searches conducted using a formal stratified-random design in Australia (Szabo et al. 2012).In Sweden, a weak correlation was found between presenceonly citizen science based bird data and a national monitoring program, and for some groups of bird species, the relationship was negative (Snäll et al. 2011).In the Netherlands, trends in dragonfly and butterfly site occupancy matched well between a citizen science dataset and formal monitoring programs (van Strien et al. 2013).In Denmark, data from an online citizen science bird database (DOFbasen) were compared to a formal bird monitoring program, and the citizen science data failed to detect declining trends for over half the species for which the monitoring program showed significant declines (Kamp et al. 2016).In both the Swedish and Danish studies, bias due to selective reporting of species was a potential explanation for lack of concordance in trends from citizen science versus formal monitoring programs (Snäll et al. 2011, Kamp et al. 2016).
We compare annual indices and trends generated from a subset of the North American eBird data set to those generated from the BBS data set.Our objective was to validate a methodology for modeling population change using eBird data that could then be used to estimate population change and trends for species with unreliable monitoring data.The general approach is comparable to that employed by Francis and Hussell (1998) for comparing trends derived from migration monitoring data to BBS.We focused the analysis on the geographic area of southern Ontario for the years from 1970 through 2015.We chose southern Ontario because it is a region with both good BBS and eBird coverage for the breeding season, and because it contains several well-known migrant traps that concentrate birds and birders and so also has reasonable historical coverage.We selected species that were migratory but near the northern edge of their breeding range so that eBird data from spring migration would reflect the local breeding population monitored by BBS routes within the study area, rather than populations of migrants that might breed outside the study area.Strong correlations between annual indices from eBird data and BBS data would indicate that we can have confidence in applying the methodology to other species of unknown population status.

Study area, dates, and species
We selected southern Ontario, Canada (south of 44.75° latitude and west of -79.00° longitude) as the geographic region of interest.Ontario has very good eBird coverage, with more checklists entered (> 730,000) than any other province in Canada and all but three states in the U.S. (California, New York, and Texas).The region selected in southern Ontario lies entirely within the Lower Great Lakes/St.Lawrence Plain Bird Conservation Region, where many southern bird species reach the northern limits of their breeding ranges at the edge of the Canadian Shield (Environment Canada 2014).The region was historically dominated by hardwood and mixed forest, but current land cover mostly comprises agricultural land (59%), forest (17%), wetlands (13%), and urban/suburban areas (8%; Environment Canada 2014).The conversion to agriculture historically led to a rise in grassland bird species and decline in forest species; however, many farms were abandoned during the second half of the 20th century leading to a modest recovery of forested land and an increase in forest bird species.Grassland bird species have declined in the region largely because of increasing forest cover and changes in agricultural practices, e.g., early season haying (Perlut et al. 2008).
A major migratory stopover site and birdwatching destination, Point Pelee National Park (hereafter Pelee), is situated within the study region and was selected as a site for comparing eBird data from a single location during migration to BBS data from southern Ontario.Pelee is primarily known as a spring migration hotspot, and birders from across North America visit in April and May leading to a wealth of eBird checklists, both contemporary and historical.The number of eBird checklists during fall migration from both Pelee and southern Ontario as a whole were far fewer than during spring (eBird participation rates peak across North America in May), so only spring migration and breeding season data were used in this study.
For our comparisons to BBS data, we used eBird data from June to mid-July from southern Ontario to represent the breeding season.To increase sample sizes of historical eBird checklists in southern Ontario, we also compared eBird data from the migration and breeding seasons combined (April-mid-July) to BBS data.Additionally, to determine if eBird data from a single popular migratory stopover site were reflective of the regional breeding population, we compared data from Pelee in April and May to BBS data for southern Ontario.The years included in the analysis were 1970-2015 because the BBS was initiated in this region in 1970.

Data filtering and statistical models eBird
We downloaded the eBird basic dataset from the eBird web site, including all checklists entered before November 2015 in Ontario (eBird Basic Dataset 2015).The eBird basic dataset includes all validated observations from checklists entered into eBird, and all covariates entered into the checklists regarding location and effort (Sullivan et al. 2014).The resulting file from eBird was very large, so it was converted to an SQLite database with indices for year, month, day, and species to enable queries on the dataset using R package RSQLite (Wickham et al. 2014).We filtered the data to include checklists from within southern Ontario (south of 44.75°l atitude and west of -79.00° longitude), and for Pelee we included all checklists from within 5 km of a central location (41.952°l atitude, -82.515° longitude), to encompass the numerous public and personal eBird checklist locations from within the park.
A pivotal decision in the direction of the modeling approach was whether to include only checklists with effort, e.g., start time, time elapsed, or distance travelled, or to be more inclusive in order to maximize the number of historical checklists.Most broad scale studies using eBird data have opted to restrict data to effort based checklists, typically those of short distances and durations from the early morning hours (e.g., Fink et al. 2010, Kelling et al. 2015).Large percentages of checklists from years in the 1970s and 1980s (mean 28%, range 17 to 65%) were from protocol types with incomplete effort information, i.e., Casual Observation and Historical, so we elected to include checklists from any protocol type.Similarly, a high proportion of checklists lacked count data for some or all species; for years prior to 2012, the mean proportion of observations lacking count data across our target species ranged from 25 to 60% (mean = 38%).As a consequence, we elected to use occurrence as a response variable for all models.Some additional filtering and manipulation of the eBird data was performed prior to fitting statistical models.We included only checklists labeled as complete lists of species observed in our models to reduce selective reporting bias.The frequency of complete checklists reporting just a single species was abnormally high suggesting that some observers did not accurately label their checklists as complete or incomplete; we therefore excluded checklists with only a single observation from the analysis.
Data entered as shared checklists from multiperson birding parties were collapsed into single checklists including all the species seen by the party, and assigned to the first observer associated with the list.Specialized checklist protocol types were reclassified to correspond to one of the five primary protocol types (Traveling Count, Stationary Count, Exhaustive Area Count, Casual Observation, or Historical) based on the effort information included.Rusty Blackbird Blitz, IBA Canada, and Random Location Count (with distance) protocols were reclassified as Travelling Counts, while Standardized Yard Count, My Yard Count, Random Location Counts (without distance), and Pri Mig Banding protocols were reclassified as Stationary Counts.Pelagic and Nocturnal Flight Call protocols were excluded from these analyses because we were not interested in pelagic species or detection rates of nocturnal migrants.The resulting number of eBird checklists per year following the data filtering are included in Appendix 1.
We fit models relating the occurrence of a species on an eBird checklist to a set of predictor variables using binomial generalized linear mixed-effects models.We fit all models using package lme4 (version 1. [1][2][3][4][5][6][7][8][9][10][11] in the R (version 3.2.5)statistical programming language (Bates et al. 2015, R Core Team 2016).We fit models to three separate data sets to assess how the choice of data influenced the relationship with BBS data.One data set included only data from the major hotspot in the region (Pelee) during spring migration (defined as the months of April and May).A second data set included all checklists from southern Ontario, but only in the breeding season, and the third data set included all checklists from southern Ontario but including both the spring (April and May) and the breeding season (June-July).
Models were fit for each species including fixed effects for date (with a second order polynomial to accommodate an expected curvilinear pattern of detection with date), the number of species on the checklist (as a surrogate for effort; log-transformed where that improved model fit), the protocol type, the interaction between the number of species on a checklist and the protocol type, and year (modeled as a factor), and random intercept terms were included for observer and locality.Using the number of species per checklist as a surrogate for effort has been validated by several studies (e.g., Roberts et al. 2007, Szabo et al. 2010) though may have some limitations if a large proportion of species have declined (or increased) dramatically (Isaac et al. 2014).The interaction between the number of species per checklist and the protocol type allowed for differences in effort by protocol type that may have changed through time, which would otherwise introduce bias in the use of the number of species as a measure of effort.Year was fit as a factor to generate a separate annual index for each year instead of an overall trend (e.g., as in Crewe et al. 2008, Sauer et al. 2013, and Smith et al. 2014).This allowed us to compare annual estimates derived from eBird and BBS data, which more readily allows comparisons when there are cyclical, interannual, and other nonlinear patterns of occurrence through time.The random effect terms are a simple way to account for the substantial effects of variability in observer skill and familiarity with target species, and variation between sites due to habitat suitability on the detection of species.For locality, sites were aggregated by assigning personal locations to the nearest eBird hotspot (when entering a checklist, birders may select from a personal location or a hotspot, where hotspots are public locations at which multiple observers may submit checklists for the same site).To derive an annual index for each year, we calculated predicted values from the models for each year using the average value for random effects terms, the peak date of occurrence for each species, travelling count protocol, and the overall mean of the mean number of species on eBird checklists per decade.

Breeding Bird Survey
Raw BBS data were downloaded from the BBS web site for Bird Conservation Region (BCR) 13, the Lower Great Lakes/St.Lawrence Plain (Pardieck et al. 2016).The BBS data were filtered similarly to the eBird data to only include routes within southern Ontario (south of 44.75° latitude and west of -79.00° longitude), yielding data from 42 routes.To compare indices for year between eBird and the BBS, we used only the presence or absence of each species at each stop on a route as a response variable.We fit binomial generalized linear mixed effects models using package lme4 (Bates et al. 2015) using route and observer as random effects in the BBS.
To ensure that reducing the BBS data to presence/absence data did not distort patterns in annual occurrence or abundance, the count data from the same BBS routes were analyzed using models with identical terms but using a Poisson distribution.Annual indices from BBS count data were strongly correlated with the derived presence/absence data for all species, and the mean Pearson's correlation coefficient across species was 0.97.The relationship between the number of BBS stops at which a species was detected and the number of individuals detected on a route was linear for all species examined, and except for the three most common species shared a similar coefficient of 1.13 ± 0.07 birds per stop, based on a generalized linear model.This indicated that models of occurrence data were representative as indices for population size, and trends or annual indices evaluated in terms of percent change would be directly comparable.
We examined the similarity between BBS and eBird data for assessing trends by computing weighted Pearson's correlation coefficients between annual indices derived from the BBS occurrence models and eBird models, using the inverse of the sum of the variance around BBS and eBird annual parameter estimates as the weights.The weighting assigned less influence to years with greater uncertainty in the annual index.We also assessed fit by plotting annual indices computed from the two datasets beside each other, along with loess smoothers and trend estimates using package ggplot2 in R, and by plotting annual indices from the two datasets against each other (Wickham 2009).Trends were calculated for eBird from southern Ontario (Apr-Jul) and BBS data by fitting binomial generalized linear models to the annual indices of occurrence produced by the mixed models using annual sample sizes as the weights for each, and extracting the slope coefficients for year.Pearson's correlation coefficient was computed between the slope coefficients from the eBird and BBS models across species.To aid interpretation, we used trend estimates from the BBS dataset to group species based on whether they were declining, increasing, or stable.

RESULTS
The weighted Pearson's correlation coefficients between annual indices predicted by the eBird models and those predicted by the BBS models are shown in Table 1.The models using eBird data from southern Ontario between April and mid-July had the highest mean correlation coefficient across species (0.35), followed by Pelee in April and May (0.26) and southern Ontario between June and mid-July (0.21).We therefore focused on assessing the models using data from all of southern Ontario from April to mid-July.Frequency of occurrence of each species in the eBird datasets and for BBS stops is included in Table 2. Plots of annual indices from the eBird models and BBS models for each species are included in Appendix 2. Plots of predicted annual indices from the BBS against those from eBird in southern Ontario (Apr-Jul) are included for each species in Appendix 3 to further examine the strength of the relationship between the models.
There was a strong correlation between trends estimated using eBird data from southern Ontario (Apr-Jul) and BBS data across species (r 20 = 0.72, P = 0.0002), when trends were calculated as the slopes of binomial generalized linear models fitted to the predicted annual indices of occurrence from the mixed effects models (Fig. 1).The estimated slopes for year with 95% confidence intervals for eBird data from southern Ontario (Apr-Jul) and BBS data are presented in Table 3.There were 14 species for which BBS data indicated declining populations (group A), six species showing increasing populations (group B), and two species that showed no long-term change in occurrence (group C).For the 14 species in group A, there was good agreement between the annual indices from eBird and those from the BBS, and significant positive correlations between the datasets for 10 species (71%; Table 1; e.g., Brown Thrasher in Fig. 2) Trend estimates agreed on direction and significance for 11 species (79%; Table 3).Of the four species that did not show agreement between datasets, two species Black-billed and Yellow-billed Cuckoos were detected very infrequently in both eBird and the BBS and there were few detections in the historical data (Table 2).However, correlations between the datasets were positive and significant for the two cuckoo species when eBird data were limited to the breeding season (Table 1).For the other two species Baltimore Oriole and Brown-headed Cowbird correlations were negative (-0.20 and -0.40, respectively) and annual indices from the two data sets clearly showed opposite trends (Fig. 2).In both cases, trends derived from eBird data showed gradual increases where trends from the BBS data showed sharp declines.These were the two most frequently reported species in the eBird data set of the 22 species analyzed (Table 2).
Of the six species in Group B, four had significant positive correlations between annual indices from the two datasets (67%) and three agreed on direction and significance of trend estimates (50%).However, for two of the species that disagreed Blue-gray Gnatcatcher and Blue-winged Warbler the discrepancy appears to be driven by lack of detections in the BBS rather than in eBird, and overall patterns in the annual indices matched well between datasets based on the plots in Appendix 2 (Figs.A2.19-A2.20,e.g., Bluewinged Warbler in Fig. 2).For House Wren, there was a positive correlation between annual indices between datasets, but the confidence interval of the eBird trend estimate overlapped with zero while the trend in BBS did not.The eBird data for House Wren appear to be affected by three outlying years in the 1970s and one in the 1980s, but otherwise show a similar pattern to BBS data (Fig. A2.18)Neither of the two species in group C showed statistical agreement between the two datasets, however plots of the annual indices showed some similarities (Figs.A2.21 and A2.22).For Great Crested Flycatcher, neither dataset indicated that there had been a large change in frequency, and both showed a gradual declining trend (though not significant in BBS data) and substantial year to year variation compared to the overall trend (Fig. A2.22,Table 3).For Wood Thrush, the relationship was affected by several large outliers in the 1970s in the eBird dataset, but otherwise both datasets showed high interannual variation relative to the overall trend and little long-term change in frequency (Fig. A2.21).
Table 3. Slope coefficients and their 95% confidence intervals (scaled by annual percent change) from binomial general linear models fit to annual indices generated using eBird data from southern Ontario (Apr-Jul) and Breeding Bird Survey (BBS) data, 1970-2015.Group A includes declining species, group B includes increasing species, and group C includes species with no significant long-term trend (based on trends calculated using BBS data from southern Ontario).See Table 1 for scientific species names.

DISCUSSION
Correlations between the annual indices from models using eBird data and those using BBS data suggest that using eBird data to estimate population change is a promising avenue for future work.
Modeling probability of occurrence on a checklist using the number of species per checklist as a measure of effort avoided removing a high proportion of historical checklists that lacked effort, and was effective in estimating trends that were similar to those estimated from BBS for 15 of 22 (68%) species analyzed.The models used were effective at reproducing both decreasing and increasing population trajectories evident in the BBS data, which indicates that some of the bias due to changes in eBird usage through time had been accounted for.
Those species with large absolute trends in the BBS data had positive correlations between annual indices derived from the two data sets, except for several species for which frequency of detection was very low in either eBird or BBS data, and two common species (Baltimore Oriole and Brown-headed Cowbird).
Correlations between annual indices for those species with small or no BBS trends were weaker, which was expected because those species exhibit smaller signal to noise ratios in the data.Annual indices for those species with small or no underlying trends were also more susceptible to the effects of outliers from older years in the eBird dataset with limited numbers of checklists.However, patterns in overall change in occurrence were generally similar for those species, suggesting that the eBird data were sufficient for estimating whether or not there had been a large long-term change in occurrence.The results also suggest that monitoring long-term trends of some locally rare species may in fact be more readily accomplished using eBird data than BBS.For example, Blue-gray Gnatcatchers and Blue-winged Warblers were not detected along BBS routes for most of the 1970s and 1980s, yet eBird data indicated both species were present in the region at that time.However, other rare species such as Upland Sandpiper and Grasshopper Sparrow, were not well documented in the eBird data set and highlight a possible bias of observers toward parks and forested areas and away from grassland habitats.
The large differences in eBird and BBS estimates for Baltimore Orioles and Brown-headed Cowbirds could arise from their high frequencies of occurrence on eBird checklists.If a species occurs on most checklists and the average number of species per checklist is high, then the approach we have used means that ability to detect a change in that species is limited.This problem would be rectified by using count data, but as indicated in our methods, that would seriously diminish the numbers of historical checklists available.In the case of Baltimore Orioles, Canadian Migration Monitoring Network data from Long Point Bird Observatory in southern Ontario also show an increasing long-term trend similar to that produced by the eBird data and contrary to BBS data for the area (Crewe et al. 2008).There were sufficient detections and captures of Brown-headed Cowbirds at Long Point Bird Observatory for analysis, but unfortunately, annual indices are not generated from those data (Crewe et al. 2008).Other explanations for the discrepancy in the annual indices generated by the two methods include changes in breeding phenology, changes in northern limit of breeding range, feeder bias, or observer bias in eBird data toward productive birdwatching sites and habitats.To expand on these explanations, if breeding phenology has advanced and peaks in detectability are earlier in the season, then the BBS would show a decline because survey dates are essentially fixed, while eBird would still capture the peak in detectability if data from spring and early summer are combined.If the extent of the breeding range of the focal species has expanded to the north of the study area, the eBird data from the spring and summer combined could include the birds breeding north of the study area as they pass through in migration, while BBS data would only include the portion of the population breeding within the study area.Feeder bias could be present in the eBird dataset because it is well known that many eBird checklists come from yards.Feeders may artificially increase detectability of some species, which could introduce bias if the proportion of checklists from feeders has changed over time, or if occurrence of the target species at feeders does not reflect changes in occurrence across the landscape.Note that of the 22 species analyzed, Baltimore Orioles, Brown-headed Cowbirds, and Eastern Towhees are the only species that visit feeders regularly.Similarly, patterns of occurrence at parks and protected areas frequented by birdwatchers may change at different rates than those across the landscape, and may not reflect changes in occurrence at a regional scale for some species.
Sample sizes of eBird checklists per year appeared to be adequate for most species in question in southern Ontario when the migration and breeding season were combined, except for those species that were reported on fewer than 3% of checklists.However, it was evident that the eBird data from the breeding season in southern Ontario (June through mid-July) were insufficient for modeling annual indices at that geographic scale, given the lower correlation coefficients with the BBS.The minimum number of checklists required for these analyses depends on their quality, the locations sampled, and the number of species on each list.In our results, years that appear to be outliers in most models had fewer than approximately 100 checklists per year.That meaningful results are obtained with so few checklists suggests that encouraging birders to submit historical checklists to eBird with or without effort data, entering checklists from notebooks of deceased birders, or incorporating checklists from other sources such as Breeding Bird Atlases or bird banding stations could make a huge difference in assessing population trends of species over historical periods.The advantage of doing this through eBird is that it provides a common mechanism for birders and ornithologists to undertake this endeavor, provides ready public access to the data, and perhaps most importantly, provides a simple means of achieving long-term (archival) storage for historically valuable data.
There were enough historical checklists at Pelee to model changes in occurrence at that scale, which was surely assisted by the high average number of species per checklist at that site.Overall, patterns of annual occurrence at Pelee were very similar to those in southern Ontario in the eBird data, and correlations with BBS annual indices were nearly as strong as those between southern Ontario eBird data and BBS data.Thus, data from a single migratory stopover area and popular birdwatching destination appear to be representative of breeding migratory species in that region, provided that the species in question occurred at that site regularly.Using a subset of eBird data from popular migrant hotspots across North America could be a valid method for estimating long term trends because numbers of historical checklists should be high for these sites, and site specific differences in relative frequency between species could be modeled accurately.Such an effort could complement data from migration monitoring stations that are used partially for the same purpose.
Despite uneven sampling intensity over time, eBird data capture the long-term population trends for species with large changes in abundance through time, as well as those that are relatively stable.
The eBird dataset should thus be considered as a useful addition to the suite of surveys used to monitor populations of North American bird species, and in select cases, should be used to estimate annual indices for species not monitored well by the BBS.Further work should be undertaken to understand the circumstances under which models using eBird data produce results contrary to the BBS (and why) as well as the extent to which improved or more sophisticated statistical models might improve estimates (or confidence in them).Our study focused on a region with dense eBird coverage, and although there are many regions with comparable eBird participation that could be investigated in this manner, there are also entire U.S. states and Canadian provinces for which historical eBird data are insufficient.We strongly encourage birders and researchers to submit any historical checklists regardless of effort content, because the information content of each list is increasingly valuable with age.Group B: species that show increasing trends in BBS data.Group C: species that show no significant trend in BBS data.

Fig. 1 .
Fig. 1.Slopes of binomial regressions (log odds scale) fit to annual indices from eBird models against those from Breeding Bird Survey (BBS) models for the 22 species of birds analyzed in southern Ontario, using eBird data from Apr-Jul.The diagonal line has a slope of 1.

Fig. 2 .
Fig. 2. Example plots of annual indices for (top to bottom) Brown Thrasher (Toxostoma rufum), Baltimore Oriole (Icterus galbula), Brown-headed Cowbird (Molothrus ater), and Bluewinged Warbler(Vermivora cyanoptera), as predicted by models of eBird data from southern Ontario Apr-Jul (left), and BBS data (right).The solid line is a LOESS smooth with a span of 2, and the dashed line is from a binomial generalized linear model fit to the annual indices.Both are weighted by annual sample size.See appendix 2 for a complete set of plots for all species and datasets analyzed.

Fig. A2. 10 .
Fig. A2.10.Predicted annual indices from eBird and BBS models for Upland Sandpiper.The y-axis was adjusted for S. Ontario (Apr-Jul) so that a large outlier would not affect interpretation of the plot.The outlier not depicted was for 1974 with a predicted frequency of 0.028.

Table 1 .
Weighted Pearson's correlation coefficients between annual indices calculated for eBird data and the Breeding Bird Survey in southern Ontario, 1970-2015.Group A includes declining species, group B includes increasing species, and group C includes species with no significant long-term trend (based on trends calculated using BBS data from southern Ontario).

Table 2 .
Frequency of observation of the 22 species for which eBird and Breeding Bird Survey (BBS) data were compared, 1970-2015.See Table1for scientific species names.
Funding was provided under the Contributions to Support Biodiversity -Wildlife and Habitat Program from Environment and Climate Change Canada.We thank J. Brzustowski for technical support working with SQLite databases.We thank C. Francis, M. Cadman, and A. Smith for review of the manuscript and critique of the modeling approach, and an anonymous reviewer for comments that improved the manuscript.We would also like to thank the volunteers who conducted Breeding Bird Survey routes, contributors to eBird including the observers, project team, and data reviewers, and the United States Geological Survey and Cornell Lab of Ornithology who have provided open access to their datasets.Bates, D., M. Mächler, B. Bolker, and S. Walker.2015.Fitting linear mixed-effects models using lme4.Journal of Statistical Software 67(1):1-48.http://dx.doi.org/10.18637/jss.v067.i01Betts, M. G., D. Mitchell, A. W. Diamond, and J. Bêty.2007.Uneven rates of landscape change as a source of bias in roadside wildlife surveys.Journal of Wildlife Management 71 (7):2266-2273.http://dx.doi.org/10.2193/2006-004 Responses to this article can be read online at: http://www.ace-eco.org/issues/responses.php/960Acknowledgments:Bird,T. J., A. E. Bates, J. S. Lefcheck, N. A. Hill, R. J. Thomson, G. J. Edgar, R. D. Stuart-Smith, S. Wotherspoon, M. Krkosek, J. F. Stuart-Smith, G. T. Pecl, N. Barrett, and S. Frusher.2014.Statistical solutions for error and bias in global citizen science