When a Picture is Worth a Thousand Animals

Transcript

We’re on the arid, desert plains of Northern Kenya. It’s a sunny January weekend, and a team of hundreds of people have gathered to take part in a global operation. They come from all over. Data scientists, local government officials, wildlife biologists, tourists and even school kids — all with a common goal. To photograph and count zebras.

Small teams of three or four pile into Land Rovers, cameras at the ready. For three days, they spread out over the 15,000 square miles of wilderness, doing their best to photograph as many zebras as they can. It’s called the Great Grevy’s Rally.

The Grevy’s Zebra is a tall species of zebra with big ears and beautiful, narrow stripes. It looks like a living work of art. And it is perilously close to extinction.

Rosemary Warungu
Grevy zebras are endangered, and the reasons why they are endangered is because of the foliage and water competition with the livestock. The other reason is overhunting for their beautiful skin. And also hunting for meat.

That’s Rosemary Warungu. She’s a zebra project manager at the Mpala research center in Kenya. She’s explaining how local farmers see the zebras as pests, competition for their livestock’s food and water.

Faced with threats to its habitat from poaching and farming, the population of Grevy’s Zebras has declined to somewhere around 2,000, from a high of 15,000 just a few decades ago. But that’s just a guess.

How many are there, really? Without a clear number, it’s difficult to know where to concentrate the efforts to save them, and whether those efforts are even working at all.

That’s where the rally comes in. It’s part of a remarkable story of how teams with wildly different specialties and needs can work together, using a shared tool – a technology platform called Wildbook – all in service of one vitally important goal: to stop the sixth great extinction currently threatening the more than 40,000 endangered species around the world, of which Grevy’s Zebra is just one example.

I’m Gabriela Cowperthwaite and this is Teamistry — an original podcast from Atlassian. This show is all about the chemistry of teams – proving that when teams work together, and teams of teams work together, they can achieve more than they ever thought possible.

The story of Wildbook begins with a research expedition off the coast of Mexico in 2002. There are no zebras in those crystal blue waters, of course, but there is another beautiful endangered species: the gentle whale shark. The largest fish in the world, they’re threatened by overfishing, poaching, and climate change.

Jason Holmberg
We didn’t see a single whale shark but what it gave me was the chance to sit next to a biologist for an entire week and chat.

This is Jason Holmberg. He’s the co-founder of WildMe, the organization that created and runs Wildbook. Sitting in that boat, he listens to the biologist describe how they track individual whale sharks.

The method is crude, to say the least. They literally spear each shark with a plastic tag with an identifying number on it. Every time they see that tag again, they record it, and from that, they try to roughly figure out the population.

Jason Holmberg
I ask the biologists, what percentage at the time do you re-sight that tag that you’re trying to deploy. He said, “Well, probably less than 1% of the time.” Well, my background is in chemical engineering, and engineers are problem solvers, so my first thought was somewhat humorously, well, if you’re 1% efficient, I can guarantee I can get you to two, but clearly a re-sight rate on a tag that is 1% efficient, that required all of us to sit for a week, bobbing in a boat…is really almost wasted effort.

There has to be a better way. Holmberg has an epiphany—what if he uses the relatively new technology of digital photography and takes advantage of the fact that the markings on the side of each whale shark are unique?

Jason Holmberg:
I sat down and said, “Okay, why don’t we modernize how we tag whale sharks,” and I began working on spot pattern recognition. Can I take tourism photographs, like my own, and map the spots on the whale sharks and think of those like a fingerprint and do the matching?

It’s, of course, not as simple as that. Holmberg teams up with Zaven Arzoumanian, a NASA astronomer who leads Holmberg to, of all places, the Hubble telescope. Because as it turns out, identifying spots on the sides of a whale and groups of stars in the sky isn’t that different a task.

They publish a paper, but soon realize that the real challenge isn’t identifying the spot patterns on photos of whale sharks. The challenge is getting the photos at all. How to bring together teams of researchers and conservation agents from around the world to work on a common goal? The answer is to centralize it all, and, importantly, to bake the complicated algorithms into the process so that anybody could use them.

Jason Holmberg:
90% of our work was simply data management. How do we get all these photo catalogs out of different potentially competing researchers, get them all in a common online place where they can be compared, and then make spot pattern recognition, something that’s as easy to run as the click of a mouse button in a browser. That became what is now whaleshark.org and voila, we have, as of about 2005, an online platform in which researchers can pool their whale shark data from all over the world and actually compare and match whale sharks in the browser, running very advanced computer vision behind the scenes just by clicking a button and waiting for the results to pop up.

The results completely change our understanding of the whale shark population.

Tanya Berger-Wolf:
The global population was estimated to be around 103,000 with a standard error between 27,000 and 180,000.

This is Tanya Berger-Wolf, the co-founder of Wildbook. She’s explaining what the whale shark numbers looked like before their project. They were all over the place.

Tanya Berger-Wolf:
So it is a very scientific way of saying we have no clue. Since Wildbook has been in place, we currently have almost 12,000 identified individual whale sharks known to Wildbook that come from more than 75,000 reported sightings, contributed by 8,700 citizen scientists, 212 researchers and conservation organizations, and one very clever and very intelligent AI bot. That’s the network, right? And that’s the impact of the network.

It’s not that the species is doing worse. It’s just that now we know exactly how they are doing.

You can think of Wildbook as a series of social media profiles for endangered species. So there’s a Wildbook for whale sharks, one for giraffes, one for jaguars and so on. Every known member of a species has its own profile on the platform. When new photos are added, by scientists, volunteers, or just shutterbug tourists, the AI powering it identifies the individual zebra or turtle in the photo and tags it. The result is a searchable, up-to-date database that lets researchers track both the size of endangered populations and their movements.

This is also being done with an animal close to my heart, orcas. When I made the documentary “Blackfish”, I learned that Seaworld had captured not just whales but breeding females, which played a part in threatening the population of wild Southern Killer Whales. But by having a system to count and understand these animals individually, people can deduce what is hurting and decimating these populations. And hopefully, how to stop that from continuing.

The power of this centralized platform to bring researchers together can be seen in a paper on whale shark biology published in 2017. It lists 36 authors, most of whom met on the Wildbook platform. By building software accessible to researchers around the world and letting it live publicly on whaleshark.org, the Wildbook team is able to harness their power as a collective, benefiting them all.

Here’s where the zebras come in. At the same time that the whale shark project is collecting all this incredible data, Tanya Berger-Wolf and her team at Ohio State university are working on a similar pattern-recognition algorithm for Grevy’s Zebras.

But even the best algorithm in the world isn’t going to be much use without an infrastructure to support it. It needs data, in the form of photographs—and lots of them. It needs a centralized platform where these photos can live, and where the data it analyzes can be shared.

And so… Wildbook is born.

Meeting with Holmberg and his team, the two decide to join forces.

Tanya Berger-Wolf:
When we put together the need and the understanding of biologists of why we need this information, the computational solutions from data science and computer vision, with a data management architecture, that’s when Wildbook was really born. And we went from whale sharks and zebras to now more than 30 species spanning from marine and to terrestrial all over the globe.

But putting together software engineers and wildlife biologists isn’t as simple as putting them in a room together and letting them hash everything out. In many ways, they’re speaking completely different languages.

This is especially difficult when it comes to the subject of artificial intelligence and machine learning, or “ML,” that Wildbook is based on.

These teams can’t work together and even accelerate each other’s work until they can talk to and understand each other. One of the WildMe team’s smartest decisions was to approach that as a problem of translation. Here’s Jason Holmberg again.

Jason Holmberg:
We serve as language translators, being able to talk to biologists in a very focused language that they understand and being able to turn around and talk to the ML engineer and then getting those two to be able to talk to each other so that there’s common assumptions.

One way to make sure this highly technical and complex information can be translated across disciplines is by making sure the teams themselves are interdisciplinary, that they aren’t divided by their own particular specializations. Not only does that ease communication, it also increases their commitment to their mission and gives them autonomy as they move forward in their roles.

Jason Holmberg:
A deliberate decision I made as a leader is that engineers need to understand the wildlife biologists they work with. And so, in many cases, what I’ve done is partnered with an engineer, specifically with a set of wildlife biologists. I’ve allowed each of the engineers to put a personal stake in the game with a specific community, and I think by and large, that has been an excellent decision, because I see the passion.

Integrating the teams so profoundly means that motivation is never a problem.

Jason Holmberg:
I don’t have to give motivational speeches. I don’t need to try to pretend that something is important when it isn’t. What we do is important. We are trying to stop the sixth mass extinction and even if my staff are not motivated by that, we work with really cool wildlife that quite frankly, we don’t understand very well. Just that simple curiosity is enough to motivate most engineers to work a little bit extra at night and work a little bit on the weekend, even if you don’t ask them to.

It’s an example of what’s been shown again and again in studies: that being personally invested in a project is how to get the best out of people. So Wildbook isn’t only connecting the right people to each other, it’s cultivating that relationship.

Now, you might ask yourself why, exactly is counting the precise number of animals so important? Let’s say, the Grevy’s Zebra for example? How does distinguishing one individual zebra from another help us to help them survive?

I’ll let Daniel Rubenstein explain. He’s a behavioral ecologist at Princeton specializing in zebras, and he’s worked on the ground in Kenya for over 30 years. He’s seen how the population has dropped dramatically. Poachers are an ever-present problem. And many locals consider Grevy’s zebras to be nuisances — vermin, even — and will shoo them away from the watering holes where their livestock also drink.

Daniel Rubenstein:
When I started studying the Grevy zebra in the late 1970s, early 1980s, there were about 15,000 ranging in Kenya and a few other thousand in Ethiopia… The numbers have plummeted somewhere down around 2,000, but the uncertainty around that number was very, very high. And so no policy maker is going to invest in trying to conserve a species where they don’t know how many there really are there. In other words, if they make big changes in behavior that costs lots of money, if you don’t have faith in the numbers now, how are you going to have faith in the numbers after you do this big investment?

Now, they did count zebras before WildBook came along. But it was a much, much longer and laborious process. Rubenstein used to actually go into the field and draw the stripes. Later, photography became part of the equation, but the images still had to be gone over by hand.

Here’s zebra project manager Rosemary Warungu, who we heard at the beginning of the episode, talking about how time-consuming and laborious the process used to be.

Rosemary Warungu:
We used to come back with even 400 to 700 photos in one day. So, you have to sit down, go through them one by one, counting, feeding those lines in the database, so that the name of that individual may come up and you will be able to know who that male was or who that female was. It was tedious and we used a lot of time.

So when Wildbook creates its Grevy’s Zebra algorithm, it’s obviously of interest to Warungu and Rubenstein and their work. Even though it’s very early days for the zebra project, Rubenstein pushes the team to create a minimum viable product that automates all of that laborious work, and get it out in the field. Here’s Tanya Berger-Wolf.

Tanya Berger-Wolf:
So at the beginning, when we created, a little bit skewed algorithm, we can recognize animals from photographs. It was absolutely not usable by biologists. And to Dan Rubinstein’s credit… he very quickly from the moment we had the very sort of prototype duct tape version of the algorithm, he started pushing us to use it. And he committed us as a team to the great zebra and giraffe count. He said, “Let’s have people just drive around for a day, take pictures of zebras and giraffes, and we’ll run it through a program. And that will tell us how many zebras and giraffes are in Nairobi National Park. And we’ll do it in two days.” It’s a cold sweat inducing kind of statement for an engineer. But I think that partnership between biologists and computer scientists would push the partnership between computational thinking and conservation thinking from the onset. And it became very inclusive partnership.

With this powerful, accessible tool, suddenly, this isn’t just a product for computer scientists. It’s not even just a product for computer scientists and biologists. It’s for everyone with a camera.

By moving quickly and turning the technology into something that everyone can use, tailored to their needs, Wildbook creates tremendous buy-in. This also helps the project get bigger, able to include even more teams, who can help each other. Now it’s not just the scientists and the biologists but the public as well—the exact people needed to add more data thanks to their thousands of photographs.

Tanya Berger-Wolf
Our youngest photographer was three years old. He came up to us and very proudly, looked at his pictures and wanted to know exactly which animals he saw right there on the spot. And the spark that we saw in that kid’s eye, that’s something that clear was going to last and then through these events and that’s a great example of sort of thinking differently and the impact that it has.

Anybody — yes, even a three-year-old, can make a meaningful contribution to the project. And that’s because of the unique machine learning approach WildBook brings to the table. It doesn’t matter if the photos aren’t perfectly composed, or if they’re taken on old mobile phones. All data is good data as far as the algorithm is concerned. The most important thing is that they get a lot of it. And while tens of thousands of photos would have taken an impossible number of work hours to go through before Wildbook, this is exactly the kind of job that AI is built for.

Daniel Rubenstein

Daniel Rubenstein (06:52):
Machine learning through artificial intelligence becomes very, very important in tuning or tweaking, or turking many, many images so that we can teach the machine that this individual is the same as that individual’s image in our archival database, even though the pictures are slightly different. and we can then start to put together databases of where the zebras were on a landscape, how often we see it, and who it associates with at different seasons under different ecological conditions and therefore how it navigates that landscape.

This data is incredibly valuable. It helps guide policy makers and conservationists to focus their efforts on how to save the Grevy’s, and let them know how well they’re working. If they can show that, for instance, protecting the zebras’ water and food resources and slowing down human development in the region actually has an affect on the zebras’ population, it gives those efforts more weight.

Those efforts are centered around the Great Grevy’s Rally, the first of which takes place in 2016. 350 people show up to take part in the event, which covers a staggering 15,000 square kilometers. It’s a large-scale act of teamwork that only works with plenty of careful planning.

Daniel Rubenstein:
We gather at a common place… We know roughly where they’re staying, so we give them a map of the vicinity where they’re going to be, whether they’re camping or staying at a lodge. And we remind them that, in fact, we want them to re-drive the routes over and over again. We don’t want them to avoid a particular area because they did it yesterday. The whole point of our estimates is seeing an animal once, seeing an animal twice. That effort will then tell us what fraction of animals have not been seen and that gives us the power of estimating how many animals there actually are. And so we encourage people to go out. Lots of stories about flat tires and all the mishaps that they have, but everybody comes back and tells us the stories with a smile on their face because they know they’ve done some environmental good.

The team is always careful to include the local population, even if they don’t fit the mold of a traditional team member. In fact, they consider all contributors team members. And bringing them into the team has benefits beyond just increased people power.

Here’s Rosemary Warungu, zebra project manager again.

Rosemary Warungu:
We involve even lower school kids. And also, by involving kids and school children, and also colleges, we are making them understand the importance of Grevy zebras, and they will grow up knowing that it’s good to conserve because they saw us conserving the endangered Grevy zebras. So, when they go home, they can’t allow their parents to just bring in their livestock and move away the Grevy zebras, because now they have that they are part and parcel of conserving them.

That’s another side benefit of creating that sense of buy-in. The Great Grevy’s rallies aren’t just generating valuable data, they’re helping build a sense of conservation, a sense of personal investment in the mission, among the stakeholders on the ground: the locals whose actions can most directly affect the population. Farmers no longer just see the zebras as competition for their own livestock’s resources.

Daniel Rubenstein:
We had one mom in one of our rallies who said zebras were always wallpaper. “I drove by them. They’re pretty, never gave them a second thought. Now, when I look through a view finder, they start doing incredible things. In order to get good picture, I have to watch them and learn about them. They’re not wallpaper. They’re wonderful animals.” And so people become much more aware of the wildlife, in particular zebras, on their landscape. And in doing so, it democratizes science because they become the champions and the ambassadors of the species, which to them it was just an endangered species that they were somehow going to help.

Back when I started working on my documentary Blackfish, I remember the first time I looked into an orca’s eyes and realized that it was more than just an animal. I remember doing an interview with a former Seaworld trainer, John Jett, and I remember him saying, “When you look into their eyes, you know somebody’s home.” And I think for all of us, when you hit that moment, and you realize that this is a living, feeling creature, with sentience and personality and a sense of humour, Wildbook is helping create that feeling on the ground in Kenya.

The rallies provide a tremendous amount of photographic data for WildBook’s algorithms to sort through. It provides an in-depth count of the Grevy’s population on a level that had never been attempted before. And its impact is immediate.

Tanya Berger-Wolf:
So when we did … the first Great Grevy’s Rally in 2016, we ended up with 40,000 photographs of zebras by the end of the two days. We identified all the zebras in them and provided the most accurate population census to date, to Kenya Wildlife Service. So the estimate was 2300 plus minus 92. That is the entirety of the species. When we did this in 2018, with more than 1,000 people, the estimate was 2,800 plus minus 150. These are the types of confidence bounce that the Kenya Wildlife Service has never, ever seen. Few organizations that monitor animal populations have ever seen these kinds of bounce.

That’s more than a 20% increase in just a couple of years—proof that the efforts to save the Grevy’s are working.

By not just including the local population but making them a crucial part of their efforts, the WildBook team is able to both obtain a much larger data set than they ever could have done on their own, and create a powerful buy-in on the ground. The data wasn’t just good, it also has a real, immediate impact on their partners’ attitudes, helping further their mission. Suddenly the conservation advocates and the stakeholders on the ground are speaking the same language.

Daniel Rubinstein:
We suspect maybe the Grevy zebra is now back over 3,000 in Kenya, which means by working with people, making them partners in the process of getting data and understanding the dynamics of the so-called vermin so that they’re not as awful as they were perceived to be, all of a sudden tolerance and sharing increase and species have in their toolkit the ability to take care of themselves if given the opportunity to do so. And by partnering with people, that opportunity becomes real.

Tanya Berger-Wolf:
The wise warden of Kenya Wildlife Service, Simon Gitau, after the second rally that we did in 2018, said, “Data is power. Data is wealth.” And in this case, it is Kenyans driving conservation.
It is the citizens of the country. It is the ordinary people who are producing the data that is then leading to decisions. And the decisions were made based on those data.

Wildbook’s efforts just keep growing and growing. It’s not just big events like the Great Grevy’s Rally, either. While those can attract hundreds of participants, there’s another source of data that dwarfs even the best-attended rally: the internet.

Their new software scours social media and YouTube for keywords like “whale shark” and searches the posts for images or videos to feed into the algorithm. Once it identifies a unique animal, the intelligent agent will automatically post a comment saying something like “hey, that’s whale shark MX700 in your video!” plus a link to Wildbook.

Tanya Berger-Wolf:
People who had no idea about the entirety of research and conservation body around whale sharks suddenly found out that their vacation video from Mexico, Philippines or Madagascar contributed to this. That this whale shark has been seen over the course of the last 20 years, that it has a history, that we know it’s nickname. You can actually adopt it if you would like to. People respond to these comments at a very high rate. And the most common response that we have is, “Whoa, this is amazing, how can I help?”

It all sounds good on paper. But if you want to know the real impact of what WildMe and its team of teams are doing, you just have to ask Rosemary Warungu about her favourite photograph. It shows two beautiful zebra foals left alone in the wild. Because of land degradation, their mother had to go further and further away from them to find water. And they’re just standing there, unprotected.

Rosemary Warungu:
If you look at that photo, they are like… scared. What will happen to that foal if, for instance, a lion comes in and eat them? So, I’m like, how can I take care of these foals, and I can’t carry them and I can’t call their mothers to come back quickly? How can we be able to have the sources of water near the areas where the Grevy’s are grazing, so that the females don’t have to go and leave the foals behind? Because, this one now, it’s the one of the reasons why we are resulting to high foal mortality. And it is one of the major threats to the survival of the species.

All this from a single photograph. Imagine the power of hundreds of thousands of them.

Wildbook and WildMe still have an awful lot of work to do. But they can be proud of what they have accomplished so far. In Kenya, for example, as farmers start to live in better harmony with Grevy’s zebras, conservationists can see, as numbers rise, the impact that is having. Wildlife biologists around the world can work better and more effectively on conservation strategies to protect endangered animals because they now have evidence-based data on numbers and migration. Even collecting that data is no longer eating up huge amounts of their valuable time.

All these very different groups of people can work together now that, thanks to WildBook, they can all access the data and information they need. Thanks to one common technology platform, disparate teams are also able to contribute different types of value. They don’t all use the platform for the same reasons, but they are able to feel personally connected to and invested in the mission, in a way that makes sense from their relationship with wildlife.

We are staring down the barrel of the sixth great extinction. Some scientists believe that over half the species on earth will be extinct by the end of this century. Obtaining accurate information on threatened species is a crucial part of fighting for their survival. And the creativity, passion and teamwork of Wildbook’s creators has given hope to anyone fighting that threat to our planet.

For more on the lessons of teamwork from this episode, and also to see the photograph of the zebra foals that Rosemary talks about, check out Atlassian dot com slash Teamistry.

Wow, it’s just crazy looking through the wildbook website, how many endangered animals they are cataloguing. Look at this, there’s manta rays, Iberian lynx, right whales, Saimaa Ringed Seals, polar bears, sea turtles, Flapper Skates, sperm whales, jaguars, giant sea bass…