We built a measurement of when the sea ice leaves each of Antarctica's emperor penguin colonies, then handed it two published catastrophes it knew nothing about and asked whether it could find them. It found the first one at ranks 1, 3, 4, 6 and 11 out of 62. It missed the second one completely, at rank 13 of 54.
The miss is the more valuable of the two results, and this post is mostly about why. A test that only ever passes tells you nothing about where your instrument stops working.
The measurement
For each of 67 emperor colonies we take the ocean cells within 100 km, read the daily satellite sea ice record from 1979 to 2023, and find one date per ice year: the day the ice around that colony fell below its own long term normal and stayed below for the rest of the year. Not how much ice, when it went. An ice year runs from one February minimum to the next, following the standard definition used in the sea ice literature.
A date is a sharper instrument than an average. An average over a season blurs a fortnight of drama into a number that looks like every other year. A crossing date does not.
Test one: the Bellingshausen Sea, 2022
In 2022 the sea ice in the Bellingshausen Sea failed early and comprehensively, and five emperor colonies along that coast were published as catastrophic breeding failures. We ranked all 62 colonies with a 2022 date by how early that year's retreat was against their own prior record, in standard deviations, with no tuning and no knowledge of the answer.
| colony | rank of 62 |
|---|---|
| Rothschild Island | 1 |
| Smyley | 3 |
| Verdi Inlet | 4 |
| Pfrogner Point | 6 |
| Bryan Coast | 11 |
Four of the five in the top six. Rothschild Island is first at 3.56 standard deviations early: its sea ice opened on 28 September in 2022 against a fitted norm around mid-December. Second of all 62, uninvited, is Emperor Island, the only emperor colony known to have been lost, which we wrote about yesterday.
A second, entirely separate metric on the same archive, the mean amount of sea ice through the breeding season rather than the date it goes, puts the same five colonies at 1st, 2nd, 4th, 6th and 7th of 67. Two measurements built for different questions, agreeing on which coast broke.
Test two: Halley Bay, 2016
Halley Bay in the Weddell Sea held one of the largest known emperor colonies until it failed from 2016 onward, published by Fretwell and Trathan in 2019. It is the most famous emperor penguin failure on record.
Our metric ranks Halley 13th of 54 in 2016, at 1.19 standard deviations early. Early, unremarkable, the kind of year a dozen colonies have without anything happening. If you had come to this dataset asking which colony was in trouble in 2016, you would not have said Halley.
Why one and not the other
The two events are different physical things wearing the same words.
The 2022 Bellingshausen event was a regional sea ice collapse. Hundreds of kilometres of pack ice went early across a whole sector. A 25 km satellite cell sees that without effort, because the thing that changed is far larger than the thing doing the measuring.
The 2016 Halley event was a fast ice break-out inside one bay. Fast ice is the sea ice held fast to the coast, and it is what emperors actually stand on, lay on and raise chicks on. It is often a fringe a few kilometres wide. A 25 km cell that straddles that fringe is mostly reading the open sea beyond it. The colony's platform disappeared and the number barely moved, because the number was never measuring the platform.
That is the boundary, and it is now written into this site as a rule: this archive measures the sea around a colony, it detects regional collapses, and it is blind to a bay. Every page that publishes one of these numbers says so.
Why we published the miss
Because a caveat you wrote in advance is a guess, and a caveat with a failed test attached is a measurement. Before running these checks we already had a sentence in the method notes saying the metric could not resolve fast ice. It was true and nobody could have told you how much it mattered. Rank 13 of 54 on the most famous emperor penguin failure on record tells you exactly how much.
It would have been easy not to run the second test. The first one had come back at ranks 1, 3, 4, 6 and 11 and there was every temptation to call the method validated and move on. Validation against the case you expect to pass is not validation. This project has a standing rule about it, arrived at the hard way: a sensitivity test that passes on the one series you ran it on is not a passed test. Run every check on every series, and treat the disagreement between them as the finding rather than as a problem to tidy away.
What the boundary permits
Plenty, as it turns out. Regional sea ice collapse is exactly the thing that has been happening in the Bellingshausen and Amundsen sectors, and this metric sees it three separate ways. Fifteen of 61 colonies carry a significant trend in when their sea ice leaves, nine earlier and six later, and the split is geographic rather than scattered: the earlier ones cluster on the Amundsen and Bellingshausen coasts and the Peninsula, the later ones in Victoria Land. Bear Peninsula has moved 90 days, from the sea opening around 20 January at the start of the record to around 22 October now, midsummer to mid-spring.
What it does not permit is any sentence of the form "the satellite record shows this colony lost its breeding platform". It does not show that. It shows the sea around the colony, and the day that sea opened, and an early opening is a plausible companion to a breeding failure rather than a cause of one. There are no penguin counts anywhere on this site, and there will not be until a colony detector exists that survives a check against ground counts. Ours, so far, does not.
The dates are on the break-out page, the amounts on the colony ice map, and every colony has its own page with its full 46 year record and the gates each of its trends passed or failed. The underlying data is the free NOAA and NSIDC passive microwave climate data record, 46 files, and the whole 16,802 day record reads in under a minute.