Building Project BirdMap

Last time, I shared how my growing interest in birdwatching led me to eBird, an incredible scientific community that’s collecting hundreds of millions of bird sightings and being used to map migration patterns, population trends, and much more. eBird has lots of great reports, but I couldn’t find a way to answer the question I had over the Christmas holidays: where could I go next year to see lots of woodpeckers?

Getting the data

Every living woodpecker species has sightings in eBird with location data, and even range maps; what I couldn’t find was a way to cross-reference these. To answer my question, I’d need to compile a list of every woodpecker species (from some other source, since eBird has no way to browse by family), pull up every range map, note the areas where each bird is most common, and look for overlaps.

I’m the kind of person who, if told it will take 8 hours to do something by hand, and 9 hours to build a tool to do it for you, will build the tool every time. If I could access all that eBird data, I knew I could find a way to create the report I wanted to see.

I started by researching the eBird APIs, but those were mostly useful for recent sightings, and I wanted to search years of records. Fortunately, eBird publishes bulk data exports to a repository called the eBird Observation Dataset (EOD). I was able to browse and filter that data and download a CSV file with all the woodpecker sightings in the last 10 years (more detail on this in the next post). This file contained more than 26 million individual observations, each with species and location info including latitude and longitude. I threw all this into a database, knowing I’d need to search it later.

Making the map

My plan was to take all 26 million observations, aggregate them, crunch some numbers, and then put them on a map. I was familiar with an open-source mapping library called Mapbox GL JS from an earlier project, so it didn’t take long to get a simple map up and running.

eBird’s range maps use a sampling grid, where the world is divided into a grid of squares. Squares where a species appears occasionally are light purple; squares where a species is common are dark purple, and ones where the species has never been seen aren’t colored at all. I wanted to do something similar for my map, but instead of counting occurrences, I wanted to count the number of species reported in that grid square.

To keep my grid simple, I decided to work in degrees of latitude and longitude. I wrote a query to search for all observations in each latitude and longitude box, then count up to the total number of birds, and how many species had been seen there. The output looked something like this:

{
	"lngMin": -82,
	"lngMax": -81,
	"latMin": 29,
	"latMax": 30,
	"numSpecies": 3,
	"observedSpecies": [
		{
			"verbatimScientificName": "Melanerpes carolinus",
			"commonName": "Red-bellied Woodpecker",
			"speciesCode": "rebwoo",
			"observationCount": 6099
		},
		{
			"verbatimScientificName": "Dryobates pubescens",
			"commonName": "Downy Woodpecker",
			"speciesCode": "dowwoo",
			"observationCount": 3822
		},
		{
			"verbatimScientificName": "Dryocopus pileatus",
			"commonName": "Pileated Woodpecker",
			"speciesCode": "pilwoo",
			"observationCount": 3318
		}
	]
}

Mapbox can load data layers over the base map in GeoJSON format so I wrapped my sample grid data in GeoJSON polygons and loaded them up. I set up rules to color-code the grid (a technique called a chloropleth): the more species in a square, the darker the orange. After some tweaking, the result looked like this:

Finally, some answers!

Here’s the finished woodpecker map: https://ohiodave.com/birdmaps/woodpeckers.html

Using this map, I could finally answer my original question. If I want to see lots of different woodpeckers on a single trip, inland California (especially the area east of Bakersfield), the Arizona / New Mexico border region, southwestern Utah, and central and eastern Oregon all look like great places to start! Each of these regions boasts at least 12 different woodpecker species.

Zooming out to look at the rest of the world, I found that northwestern South America and Southeast Asia are the world’s woodpecker hotspots. Now I know that someday I’d love to visit Kaeng Krachan National Park in Thailand to see the beautiful Greater Flameback and the world’s largest woodpecker, the Great Slaty.

Once I had the basic map in place, I started adding bells and whistles. First, I added a detail bar on the right to show information about the total number of sightings and the sightings for each species within each grid square. Since I had the eBird species code, I could also link to eBird’s species description page for each, and even show a list of recent sightings in that area.

Next, I added was the ability to filter the map to show one species at a time. Technically eBird already has this (and since their data isn’t filtered to the last 10 years they show more sightings), but it was neat to be able to quickly flip between species and see what parts of the world they call home.

The last thing I added was the ability to show range overlaps. In the screenshot above, you can see the parts of Asia where you might find both a Greater Flameback and a Great Slaty Woodpecker (including Kaeng Krachan National Park). This will come in handy in the future if I’m looking to plan a trip to see specific species, instead of just the larger number of species possible.

Taking it further

By the time I had the woodpecker map mostly done, I was already thinking about mapping different kinds of birds. After woodpeckers, my next favorite family is the corvids (crows, ravens, magpies, and jays), so I set out to repeat the process.

After I’d done it once, setting up a new family map turned out to be super easy. The worst part was waiting for gigabytes of data to import into my database and waiting for my sampling script to re-run. In the couple of weeks since I started Project BirdMap, I’ve created versions a total of 7 bird families:

And I’ve already got plans for more! In Part 3, I’ll dive into the technical details behind Project BirdMap and some of the issues I solved along the way.