1 Million Years of Stream Flow Data

The 9,322 gauges in the GAGES II database are picked for having over 20 years of reliable streamflow data from the USGS archives. Combined, these gauges represent over 400,000 years of data.
They offer a detailed sketch of water availability over the past century. But they miss the opportunity to describe a even fuller portrait.

In the AWASH model, we focus on not only gauged points within the river network and other water infrastructure like reservoirs and canals, but also on the interconnections between these nodes. When we connect gauge nodes into a network, we can infer something about the streamflows between them. In total, our US river network contains 22,619 nodes, most of which are ungauged.

We can use the models and the structure of the network to infer missing years, and flows for ungauged junctions. To do so, we create empirical models of the streamflows for any guages for which we have a complete set of gauged of upstream parents. The details of that, and the alternative models that we use for reservoirs, can be details for another post. For the other nodes, we look for structures like these:

Structures for which we can infer missing month values, where hollow nodes are ungauged and solid nodes are gauged.

If all upstream values are known, we can impute the downstream; if the downstream value is known and all but one upstream values are known, we can impute the remaining one; if upstream or downstream values can be imputed according to these rules, they may allow other values to be imputed using that new knowledge. Using these methods, we can impute an average of 44 years for ungauged flows, and an average 20 additional years for gauged flows. The result is 1,064,000 years of gauged or inferred streamflow data.

We have made this data available as a Zenodo dataset for wider use.

A new year, new resolutions

I love new years resolutions, and the constant inventiveness that goes with learning how to change one’s behaviors. It’s a time to think about who I want to become, and how to rebalance the weights on my path. This year’s crop has a couple things for socialization, a couple things for exercise, and a big one for building out my online presence:

  • Learn the names of everyone at GRI by the end of January.
  • Take a dance class with Flame.
  • Try out a “tracking your steps” app (suggestions welcome!).
  • If I don’t exercise before an opportunity to have dessert, I must refuse.
  • If I haven’t contributed to a social media by the end of the day, I have to do it before sleep.

The last one is going to be distributed mostly between this blog, my professional blog, and Twitter, and will range from simple reposts to longer contributions to personal messages. When I reunited with my coin system, I’ll drive it that way.

The “if-then” form of the last two is a perversion of a good idea recently suggested somewhere (but I can’t find where): rather than set up a behavior change as a continuous demand, identify a situation where the problem occurs and decide on a new course of action when presented with it.

J² Honeymoon Trip: The Good Parts Version

November 7 – November 22, 2017

Johanna and I just returned from two weeks in South Africa and Tanzania: two weeks of early morning flights, of summer temperatures, of relaxation and ignoring emails. We packed in a little of everything: city walking, beach lazing, exquisite meals, museums and galleries, a cooking class, a snorkeling trip, and a day-long safari. Here are a couple of the highlights.

South Africa, Part 1

South Africa is bursting with Brooklyns. In Jo’burg, there was Maboneng, with design studios and bookstore-studios; to Melville, with its rows of crafty bars. In Cape Town, we found Woodstock, with a steampunk clothing store, a factory turned into lifestyle shops, and the Streetwires Artists Collective, half the space populated with the artists loudly work-socializing.

Our first museum attempts were stymied. The Museum of African Design was closed and the Johannesburg Art Gallery was being reappointed (but it didn’t stop us from getting into its empty halls). We had much better luck with the galleries of Rosebank (SMAC, CIRCA, and Everard Read) with impressively museumesque collections and staging.

Our cooking class was in little Prince Albert, in the middle of the desert 4 hours east of Cape Town. We arrived at 2am, meeting our host at an abandoned gas station, and following him down empty dirt roads. The next morning, we got a personal lesson from an innovative chef, making focaccia with candied figs and olives; stovetop smoked eggplant; upside-down beat pastry; and beat and chocolate cake.

What we expected would be a two hour drive turned out to be a 5 hour one, with an misplaced mountain range blocking the road. But that furnished us with our first encounters with tiny antelopes, baboon families, and guinea hens. The biome shifted from desert to savanna to forest to coastal brush. And we passed the oldest post office in South Africa, a tree first used for letter-leaving in 1501.

We spend two nights at a beach spa hotel in Arniston, a town at the site of its namesake disaster, the second worst boat wreck after the Titanic. James clambered over rocks and through a tunnel to reach a huge sea cavern. The entrance to the ocean was half-filled with each wave, causing the cave to darken and then boom as the wave spread out. Then we hurried to the southern-most point on the African continent, marked by a plaque that read, “<- Indian Ocean | Atlantic Ocean ->”.

On the way back to Cape Town, we ate at the exquise La Colombe, on a mountaintop overlooking vine-covered hills and groves. The most memorable of the dozen courses were the enchanted forest and the taste test. The enchanted forest was a contrived picnic, hosted by the Mad Hatter himself, regaling us with the most recent events of the land. One of the three deserts was a tray of five confections, accompanied by a card to match up which was salty, sweet, sour, bitter, and umami.

Cape Town also had the best museum we explored, the Zeitz MOCAA. The Zeitz is a bold contemporary art museum, with a disturbing number of artists younger than us.

Tanzania Excursion

We were able to carve out 5 days in Tanzania, grabbing some cheap tickets from Cape Town to Dar es Salaam, and then back to Jo’burg.

We arrived in Dar late, a crowded around counters to give away our passports and crisp $100 bills, in hopes of a visa stamp. It seemed to be a case of first push, first serve, but every passport was collected before anyone could leave. Over an hour after we arrived, a woman exited the enclosed office with a small pile of passports, calling names. We weren’t in the first pile, and another 20 minutes passed, and the prayer calls and roosters started before we could fall asleep.

The next morning, we woke up at 4:30am, thanks to James’s internal alarm which realized that his external one was set for 15 minutes after our flight was to depart. We taxied to the airport, over a craggy road better suited for the donkey that passed us, past women tending open flames in sheds with neither roof nor walls. Our plane to Zanzibar had 8 seats, including the captain’s, and an aisle 6 inches wide (and yet, two people passed me while I had my leg in it).

We stayed on Chumbe, an eco-reserve island made of fossilized coral. There are four lofted cabins for staying the night, each with a hammock and a mosquito-netted bed. One rope tied to the wall can be used to let more light in… by lowering away the wall of the cabin. The chalkboard of activities included snorkeling through the island-wide stretch of (mostly dead) coral, a nature walk featuring strangler figs and rhino rocks, and a night-excursion to see the pillow-sized coconut crabs when they emerge.

The next morning, we retraced our steps to the car, and ended up on a personal spice tour from a friend of the driver. The best part was the tour guide’s knife-wielding friend, who would continuously grab leaves and twigs and widdle on them. At the end of the tour, he presented us with crowns, bracelets, 2 necklaces, a tie, and two bags, from his craft.

One night, we went out to House of Spices, an excellent rooftop restaurant. Alas, we forgot to bring enough cash, and they had no way to take cards. Leaving Johanna as collateral, I went out looking for a reputedly closeby ATM. It was a dark night, but the streets were alive with vendors with roasting stands. I asked 6 people, and each said, “An ATM? Oh, huh, I guess we probably have one of those around here somewhere.”

Johannesburg, Part 2

Perhaps the most awesome experience was our all-day safari, when we returned to Jo’burg and drove to the eastern border of the country. The safari was coordinated by our hotel, Bushwise Lodge on a wildlife sanctuary. We drove into Kruger National Park, and no sooner signed releases than were surrounded by impalas, the main feed-species. Close behind were lions, hyenas, crocodiles, wild dogs, a leopard, and fellow prey like wildebeests, buffalos, zebras, giraffes, and elephants.

We weren’t allowed to exit the jeep, except in special double-electrified basecamps. That didn’t stop the monkeys from stealing apples right from our table, or a baboon from swooping away a bag of leftovers. The jeep didn’t seem like so much projection, the two times that nearby elephants sized us up for an playful charge. We watched as a lioness waited for her friend, and then the two of them stalked away together. We saw a leopard leave some uneaten, bloody remains in a tree, and later hyenas come for their share. We watched as a swarm of a dozen crocs drilled into a bloated water buffalo, spinning to rip out its entrails.

One of our last stops was the Cradle of Humankind, a series of caves with some of the best Australopithecus excavations. The poor creatures fell in by chance or were dropped in by predators, waiting until mining was finally invented to set them free. Dripping stalactites lined the walls and ceilings that arched high above our heads. The water table, 60 m down, led to an underwater river with maggots and the ghosts of its first explorers. Johanna was visited by many biting ants and was ‘over it’ before the tour even started.

On our last full day, we went to Pretoria, seat of Apartheid. We skipped those monuments, in favor of Freedom Park, a winding path upwards to a series of spaces for contemplation. It had an excellent-looking museum, reminiscent of the Native American Smithsonian, but we raced through it to get to the gardens before they closed. The next day, though, we explored the Apartheid Museum, filled with video clips and wire-cage rooms. The most disconcerting part was the disney-like amusement park directly oppose it.

Our last night, we hit up a jazz bar called the Orbit, on their open-mic-like night. On stage was something between a band and a “guy and friends”: the pianist, drummer, and bassist stayed up, while the MC/band-leader orchestrated a series of improvisations. The first few improvizations started with one instrument or singer, but the last was the highlight. He brought up a singer without a song, who waited as the music converged a little, and then added some lyrics, made up on the spot. Five minutes in, he brought up a second singer, while the first was still going, and the two started playing off each other. The audience was on the edges of its seats for over half-hour, as the group of them played oscillated from beautiful chaos to islands of coherence. At a certain point, tension appeared between the singers, with the first (a black woman) starting to sing-chant “I love black.” The other (an Italian man), responded with a liturgy about how there were so many different kinds of people. Eventually they made up, with the woman chanting, “Zion,” and the man joining in.

From there, it was back to Boston, to Springfield for a first Thanksgiving, to Lowell for a second one, to the visa office to finally give away our passports, and then to Cape Cod to finish off the best vacation of 2017. See the rest of our photos here!

An allegory for life

It’s said that the universe follows natural law, but it seems to me that the real force behind the big events in life is allegory.

A few years ago, I was bemoaning my poor memory while wandering around Nice, France. This came to a head as I treated myself to a prix fixe menu, and resolved to practice committing more to memory. I looked at the awning above me to start with the name of the restaurant I was in. As it turned out, the restaurant was called “Memoire”.

Allegory has a way of creeping up on you. Reading books slowly over several weeks is a good way to realize the allegorical nature of life. You recognize yourself in the book, start seeing new developments of the book reflected in your own life, and realize that you are walking in lock-step, hoping the book ends well for you.

Periods in which much is changing are prime targets for allegorical forces. Flame and I are going through just such a period now, preparing for our trans-Atlantic move, a new city and a new job for me, and moving back in together.

Two months ago, my parents sold the last house I lived in with them. They were trying to downsize, with their emptier nest, but accidentally got a bigger house. Now they live 20 minutes away from that house, in a different state (Ah, New England), and are happy to be out of the city.

Two weeks ago, my childhood neighborhood in California was burnt to the ground. Thankfully, my grandmother and uncle, despite getting mandatory evacuation orders (which the uncle ignored), are safe along with their homes.

I am convinced that the universe is trying to tell me something. Something about leaving home, as I am preparing to do.

Travel has always been a good way for me to uncover the allegorical conspiracies around me, so I suspect that this is all just a prelude. Tuesday, we start out on a two-week honeymoon to South Africa and Tanzania. A little beach time, a little safari, a little good food, a little exploration we haven’t planned yet. A lot of leaving one temporary home and acclimating to another. I just hope it tempts the gods to spill their secrets, because I am slowly reading some works of military history, and I don’t think those will end nearly as well.

Following my ancestor

For the last couple months, I’ve been following the journey of my Y-chromosomal ancestor, also named James Rising, as he found his way to the New World in 1635. He turns out to be a fascinating character (at least to me). From a print-only article in The New England Historical and Genealogical Register:

James Rising was born in 1618 in Beccles, Suffolk; he and his twin brother were the youngest children of an illiterate cordwainer. James had just turned 17 when he sailed from London for Bermuda on a ship filled with very young laborers for the tobacco fields. For twelve years he remained in Bermuda, a country with few prospects for new arrivals. In 1647 he was among the seventy settlers who sailed from Bermuda for Eleuthera, an island a thousand miles away in the Bahamas, with the intention of establishing a Puritan colony. Near its destination, the ship floundered and their provisions were lost, but with help from Puritans in New England, the passengers survived. The colony struggled for ten years, but gave up in 1657, when the last of the settlers sailed back to Bermuda– all except James Rising, who sailed, instead, on a ship bound for New England.

I’ve been using a few different sources to follow his progress. John Ogilby’s 1675 road maps are incredible, with descriptions of the towns, road-side markers, and individual buildings some of which are still there today. I’ve been using one site to find the plates and a different one to get high-resolution images. Here’s the path I followed, from Beccles to Ipswich to Cambridge to Oxfordshire to London:

One of the most interesting source I’ve found for a sense of walking through these cities is Through England on a Side-Saddle in the Time of William and Mary, a 17th century woman’s diaries of her prolific journeys.

On September 30, 1635, James Rising boarded the Dorset in London, and I’m using current winds to try to get a sense of how it got from there to Bermuda. Here is where he is “now”, on October 23, finally having caught a decent wind:

I just hope he isn’t caught by a hurricane before he ever has children. My other records of his journey are on Twitter #rtnewworld.

Eclipse 2017: You can’t fake that

Totality is different! I’m back from eclipse-viewing in Southern Illinois, at a secluded wildlife refuge just 3 hours from the point of maximum eclipse. This isn’t my first eclipse, but it’s my first total eclipse, and I’m still in awe.

I trekked through a little wood to a languid river, with fish jumping at gnats, butterflies and dragonflies, a white crane and a raccoon that came up to check me out. I got out my picnic and book and enjoyed the perfect blue sky.

About a half-hour after the partial eclipse started, I noticed the air cool. The sky looked a shade darker, and contrasts softer. Even at the height of the partial eclipse, all I could notice without the glasses was like a minor dimming of the sun.

And then, totality. I was looking through the filter glasses as the crescent dwindled away, one moment appearing not to move at all, and gone the next. Through my glasses, the sky was completely black. I took them off to a touch of beauty: a blazing one-ring, hanging in the sky, like the sun had been plucked out and just a hole left behind:

Total eclipse

(That picture is doctored, combining two real pictures I took, but it’s closer than either to what it was really like.)

It was suddenly twilight. The empty circle in the sky was sharp, perfect, and delicate. I was transfixed, until I noticed beautiful colors on the horizon, like the pink of sunset. I edged to the water to get a better view, not realizing what the colors portended: as I found my footing, it was suddenly bright again, and the eclipse was over:

Eclipseset

But after a day of listening to podcasts in the car of a changing world and the ever-growing potential of fake news, the eclipse gave me hope. Our grasp of reality can seem so tenuous, but we understand that the sun doesn’t just dim. You can fake a lot, but you can’t fake this.

Economic Damages from Climate Change

When I tell people I study climate change, sooner or later they usually ask me a simple question: “Is it too late?” That is, are we doomed, by our climate inaction? Or, less commonly, they ask, “But what do we really know?”

With our new paper, Estimating Economic Damage from Climate Change in the United States, I finally have an answer to both of these questions; one that is robust and nuanced and shines light on what we know and still need to understand.

The climate change that we have already committed is going to cost us trillions of dollars: at least 1% of GDP every year until we take it back out of the atmosphere. That is equivalent to three times Trump’s proposed cuts across all of the federal programs he cuts.

If we do not act quickly, that number will rise to 3 – 10% by the end of the century. That includes the cost of deaths from climate change, lost labor productivity, increased energy demands, costal property damage. The list of sectors it does not include– because the science still needs to be done– is much greater: migration, water availability, ecosystems, and the continued potential for catastrophic climate tipping points.

But many of you will be insulated from these effects, by having the financial resources to adapt or move, or just by living in cooler areas of the United States which will be impacted less. The worst impacts will fall on the poor, who in the Untied States are more likely to live in hotter regions in the South and are less able to respond.

Economic damages by income deciles

One of the most striking results from our paper is the extreme impact that climate change will have on inequality in the United States. The poorest 10% of the US live in areas that lose 7 – 17% of their income, on average by the end of the century, while the richest 10% live where in areas that will lose only 0 – 4%. Climate change is like a subsidy being paid by the poor to the rich.

That is not to say that more northern states will not feel the impacts of climate change. By the end of the century, all by 9 states will have summers that are more hot and humid than Louisiana. It just so happens that milder winters will save more lives in many states in the far north than heat waves will kill. If you want to dig in deeper, our data is all available, in a variety of forms, on the open-data portal Zenodo. I would particularly point people to the summary tables by state.

Economic damages by county

What excites me is what we can do with these results. First, with this paper we have produced the first empirically grounded damage functions that are driven by causation rather than correlation. Damage functions are the heart of an “Integrated Assessment Model”, the models that are used by the EPA to make cost-and-benefit decisions around climate change. No longer do these models need to use out-dated numbers to inform our decisions, and our numbers are 2-100 times as large as they are currently using.

Second, this is just the beginning of a new collaboration between scientists and policy-makers, as the scientific community continues to improve these estimates. We have built a system, the Distributed Meta-Analysis System, that can assimilate new results as they come out, and with each new result provide a clearer and more complete picture of our future costs.

Finally, there is a lot that we as a society can do to respond to these projected damages. Our analysis suggests that an ounce of protection is better than a pound of treatment: it is far more effective (and cheaper) to pay now to reduce emissions than to try to help people adapt. But we now know who will need that help in the United States: the poor communities, particularly in the South and Southeast.

We also know what needs to be done, because the biggest brunt of these impacts by far comes from pre-mature deaths. By the end of the century, there are likely to be about as many deaths from climate change as there are currently car crashes (about 9 deaths per 100,000 people per year). That can be stemmed by more air-conditioning, more real-time information and awareness, and ways to cool down the temperature like green spaces and white roofs.

Our results cover the United States, but some of the harshest impacts will fall on poorer countries. At the same time, we hope the economies of those countries will continue to grow and evolve, and the challenges of estimating their impacts need to take this into account. That is exactly what we are now doing, as a community of researchers at UC Berkeley, the University of Chicago, and Rutgers University called the Climate Impacts Lab. Look for more exciting news as our science evolves.

Sustainability, Engineering, and Philosophy