When very meets extreme

In the database world, this is a time of conferences.

Two weeks ago, there was VLDB 2013: the 39th International Conference on Very Large Data Bases.

This week, it's XLDB 2013: the 7th Extremely Large Databases Conference.

VLDB, of course, is the grand old conference in this part of the computing world, and the technical program reflects that. The list of speakers, and the program committee, represent pretty much a Who's Who of the database research community world wide. It's hard to find an area of database research, or a database research team, that isn't represented at VLDB, and the always-excellent papers are a treat to pore over.

XLDB is a younger upstart, but has quickly become a very exciting conference. As their home page outlines,

XLDB attempts to tackle challenges related to extreme scale data sets. Main activities include identifying trends, commonalities and roadblocks related to managing and analyzing extreme scale data sets, and facilitating development and growth of appropriate technologies including (but not limited to) databases.

XLDB is rather unabashedly commercial, and this year's conference was no exception, with talks by teams at Google, Facebook, eBay, Oracle, Target, Chevron, and Netflix, among others.

I think it's great to see both conferences doing well, and I'm sure I'm going to be studying the various research that was announced at these events for months.

Read More

Rediscovering Watership Down

As a child, I was a precocious and voracious reader. In my early teens, ravenous and impatient, I raced through Richard Adams's Watership Down. Now, forty years later, I felt a strong desire to return to it, and so I did.

But this time, with less time remaining to me, I took more time, and enjoyed a summer of rediscovering Watership Down.

It turned out that I remembered almost nothing. Of course, I remembered brave Hazel, Prince of the Rabbits. And I remembered that I had identified strongly with Fiver, the introverted, moody, unathletic young rabbit who had visions of the future.

There are many more rabbits in Watership Down, of course: Bigwig, who is powerful and fearless; Dandelion, with his marvelous stories; Blackberry, fast and clever; Silver, the faithful lieutenant; Pipkin, the tiny rabbit with an enormous heart; Speedwell, the adventurous scout; and Holly, the wise old veteran who shares his wisdom with the youngsters.

This seems so cliched when summarized thus, and it's fair to attribute to Watership Down a certain amount of "me too". Following in the footsteps of such ground-breaking work as The Wind in the Willows or A. A. Milne's tales of the Hundred Acre Wood, Adams clearly was climbing on the shoulders of the giants that went before him.

But Adams brings a fresh spirit and a very graceful style to his epic tale, and his love of his characters carries him past cliche into a work that truly deserves to be considered in the same class.

Surely what most readers remember about Watership Down is the way that it immerses you in the world of the rabbit. "The holes and tunnels of an old warren become smooth, reassuring and comfortable with use," writes Adams, describing what it's like to be a rabbit away from home on a wet morning, sheltering in a dank hole instead:

Bigwig, with all his usual brisk energy, set to work. Hazel, however, returned and sat pensive at the lip of the hole, looking out at the silent, rippling veils of rain that drifted across and across the little valley between the two copses. Closer, before his nose, every blade of grass, every bracken frond was bent, dripping and glistening. The smell of last year's oak leaves filled the air. It had turned chilly. Across the field the bloom of the cherry tree under which they had sat that morning hung sodden and spoiled. While Hazel gazed, the wind slowly veered around into the west, as Cowslip had said it would, and brought the rain driving into the mouth of the hole. He backed down and rejoined the others. The patterning and whispering of the rain sounded softly but distinctly outside. The fields and woods were shut in under it, emptied and subdued. The insect life of the leaves and grass was stilled. The thrush should have been singing, but Hazel could hear no thrush. He and his companions were a muddy handful of scratchers, crouching in a narrow, drafty pit in lonely country.

Others may remember the clever and fascinating fables and legends re-told by the rabbits as their oral history, tales of the great rabbit prince El-ahrairah and his sometime companion Rabscuttle. Just reading the titles of the stories will give you a hint at the delights they hold:

• The Story of the Blessing of El-ahrairah

  "All the world will be your enemy, Prince with a Thousand Enemies, and whenever they catch you, they will kill you. But first they must catch you, digger, listener, runner, prince with the swift warning. Be cunning and full of tricks and your people shall never be destroyed."

• The Story of the King's Lettuce
• The Story of the Trial of El-ahrairah
• The Story of El-ahrairah and the Black Rabbit of Inle
• The Story of Rowsby Woof and the Fairy Wogdog

Still others will prefer to read Watership Down as being a story with a greater purpose. Adams goes out of his way, in his introduction, to state that

I want to emphasize that Watership Down was never intended to be some sort of allegory or parable. It is simply the story about rabbits made up and told in the car.

But whether this is false modesty, or whether it is more a description of what he "intended" than what actually became, it is clear that Watership Down is, in fact, far more than "simply the story about rabbits".

Firstly, there is the topic of gender identity. Nearly every character in the book is a rabbit, and nearly ever rabbit is male; there are female rabbits in the story, but rarely are they even given names, and no female rabbit plays any significant role until very late in the book, when several of the male rabbits choose mates and settle down to raise families. Moreover, in some sort of bizarre throwback, the otherwise modern-thinking rabbits think nothing of organizing and executing raiding parties to kidnap females from other tribes for their own purposes. Worse, at times the roles of the females are described in the most demeaning sort of way:

Buck rabbits on their own seldom or never go in for serious digging. This is the natural job of a doe making a home for her litter

Trying bringing this sort of attitude up in any second grade classroom nowadays and you'll be hooted out of school.

Secondly, there are of course the social issues raised by the book. In the large, Watership Down is a story of the clashes of various sorts of social organization, from democratic to authoritarian to socialist to libertarian. Almost every animal in the book has an opinion about the right way to organize things, and the central crisis of the book involves an epic battle to overthrow the cruel dictator who has imprisoned his rabbits with both physical bullying and psychological manipulation. Adams skillfully describes the subtle and effective techniques that the tyrant uses to cement his power:

Woundwort was no mere bully. He knew how to encourage other rabbits and to fill them with a spirit of emulation. It was not long before his officers were asking to be allowed to lead patrols. Woundwort would give them tasks -- to search for hlessil in a certain direction or to find out whether a particular ditch or barn contained rats which could later be attacked in force and driven out.

...

The patrols were the training grounds of cunning trackers, swift runners and fierce fighters, and the casualties -- although there might be as many as five or six in a bad month -- suited Woundwort's purpose, for numbers needed keeping down and there were always fresh vacancies in the Owsla, which the younger bucks did their best to be good enough to fill. To feel that rabbits were competing to risk their lives at his orders gratified Woundwort, although he believed -- and so did his Council and his Owsla -- that he was giving the warren peace and security at a price which was modest enough.

A more true description of the rise of a warlord I have rarely read.

And there are religious discussions, too, as for example when Fiver and Hazel get into a discussion about whether there is life after death:

"Well, there's another place -- another country, isn't there? We go there when we sleep; at other times, too; and when we die. El-ahrairah comes and goes between the two as he wants, I suppose, but I could never quite make that out, from the tales. Some rabbits will tell you it's all easy there, compared with the waking dangers that they understand. But I think that only shows they don't know much about it. It's a wild place, and very unsafe. And where are we really -- there or here?"

"Our bodies stay here -- that's good enough for me. You'd better go and talk to that Silverweed fellow -- he might know more."

Still, in the end, I think that Watership Down remains for me the same book that it was forty years ago when I first spend time between its covers: a stirring and action-packed tale of derring-do and adventure, of exploration and escapades and thrills and chills, and of great battles to sing of around the hearth:

Along the western horizon the lower clouds formed a single purple mass, against which distant trees stood out minute and sharp. The upper edges rose into the light, a far land of wild mountains. Copper-colored, weightless and motionless, they suggested a glassy fragility like that of frost. Surely, when the thunder struck them again they would vibrate, tremble and shatter, till warm shards, sharp as icicles, fell flashing down from the ruins. Racing through the ocher light, Bigwig was impelled by a frenzy of tension and energy. He did not feel the wound in his shoulder. The storm was his own. The storm would defeat Efrafa.

What's not to like about that? The world can still use some fine books about youngsters who set out to make a better world and, along with a few scrapes, mis-steps, and false starts, persevere and succeed in the end.

I'm glad I returned to Watership Down. I know not if I shall return to it again; I suppose only time will tell.

Read More

Big Bro

Your must-read article of the month is on the Pro Publica website: Revealed: The NSA’s Secret Campaign to Crack, Undermine Internet Security

the N.S.A. spends more than $250 million a year on its Sigint Enabling Project, which “actively engages the U.S. and foreign IT industries to covertly influence and/or overtly leverage their commercial products’ designs” to make them “exploitable.”

And be sure to read their Editor's Note:

There are those who, in good faith, believe that we should leave the balance between civil liberty and security entirely to our elected leaders, and to those they place in positions of executive responsibility. Again, we do not agree. The American system, as we understand it, is premised on the idea -- championed by such men as Thomas Jefferson and James Madison -- that government run amok poses the greatest potential threat to the people’s liberty, and that an informed citizenry is the necessary check on this threat.

Well done, Pro Publica.

Read More

Starting today, the games count

In honor of the occasion:

The Autumn Wind is a pirate,
Blustering in from sea,
With a rollocking song, he sweeps along,
Swaggering boisterously.


His face is weather beaten.
He wears a hooded sash,
With a silver hat about his head,
And a bristling black mustache.


He growls as he storms the country,
A villain big and bold.
And the trees all shake and quiver and quake,
As he robs them of their gold.


The Autumn Wind is a raider,
Pillaging just for fun.
He'll knock you 'round and upside down,
And laugh when he's conquered and won.

Or, if you prefer, watch it! (and don't forget to turn those speakers up LOUD).

Read More

Palette Choice in Inciweb Rim Fire mapping

The mapping addicts at Wired have been doing some great work with the Rim Fire mapping data from Inciweb, trying to understand how to make it most understandable.

• Last week, in Yosemite Fire’s Destruction Mapped in Beautiful, Frightening Color, the team highlighted one of the Inciweb-provided maps and how it used color to help explain the movement of the fire from day to day:

  The progression of the Rim fire into Yosemite National Park has been strong, steady, and scary, fueled by extra-arid conditions after an exceptionally dry winter in California. This map shows the growth particularly well, with each color representing the area burned each day.

• This week, the team returns to the analysis, studying how the choice of color palette affects the experience of the map: The Color of Fire: How Palette Choice Impacts Maps of Yosemite’s Rim Fire

  The Inciweb map grew incrementally and is being used to follow the fire on specific days, so using vastly different bright colors that easily stand out from each other was probably a conscious decision. But Simmon’s map gives people like us who are not on the front lines a much better grasp of the life of this fire.

• Meanwhile, "since I've got you on the phone," don't miss this superb reporting over at the Santa Barbara Independent: On the Safety of Firefighters: Real Time Danger During the Jesusita Blaze and Human Lives vs. Expensive Homes.

  Over the past several years, there has been a noticeable shift in attitude regarding the dangers in which we place wildland firefighters. In 2008, the Forest Service Employees for Environmental Ethics (fseee.org) initiated a lawsuit designed to push for wildland firefighting reform. “Firefighters should not be asked to defend a home that is indefensible,” said former FSEEE field director Bob Dale. “No home is worth a firefighter’s life.”

  As is the case in Santa Barbara, the primary concern is not the backcountry wildland fires like the 240,000-acre Zaca Fire in 2007 that burns for two months but consumes no houses. It is the homes that have been built in the past 20-30 years along the base of the Santa Ynez Mountains that worry firefighters. This is the point at which the wildland meets an encroaching urban community. What is different here is that you are asking Forest Service employees trained to fight wildland fires to respond to the urban interface and you are asking city firefighters trained in defending structures to do so in a wildland environment.

The Rim Fire effort appears to have gone quite well over the last week. Once the controversial decision to conduct "burnout operations" in America's premier National Park was approved and acted upon, the last week has gone mostly as the incident team predicted it would, and so far as I know has been essentially casualty free, so kudos to the team for that great result.

Read More

Reactions

The bridge is dead, long live the bridge!

• New Bay Bridge span opens

  The first "recognized" driver, guided into the number one lane by Caltrans, was Christen Gray, of San Leandro. As she tried to explain her excitement over being among the first over the new span, her passenger, Anthony Thomas, blurted out, "It's shiny!"

• New Bay Bridge Opening Completes Decades-Long Dream

  “The story is ending well but the road to get here was far too long and far too winding,” said Metropolitan Transportation Commission executive director Steve Heminger, who has been involved in the $6.4 billion project to build the new span almost from the beginning because he was its project manager.

• S.F. Bay Bridge re-opens with new 'quakeproof' span

  Lt. Gov. Gavin Newsom, the former mayor of San Francisco, cut a chain with a blow torch to mark the opening Monday afternoon after leading those who had gathered around the bridge's toll plaza in a countdown to the reopening. The old eastern span will eventually be demolished.

  "I hope this is more than just connecting two land masses," he said. "I hope that the progress that's being represented at this moment is for a generation to dream big dreams and to do big things."

• Spirit of victory prevails at opening of new Bay Bridge

  "We have overcome what at times seemed insurmountable," said CEO Michael Flowers of American Bridge Company, the lead contractor on the project, at the opening event at the Bridge Yard building, formerly a Caltrans warehouse. "The finish line is in sight."

  The new structure replaces an original span that cost $77 million and was built with the world's deepest marine foundation in just 3 1/2 years. Both were built during tough economic times: the Great Depression and the Great Recession.

• Bay Bridge opens as drivers compete for 'first' trips across new eastern span

  State Sen. Mark Leno (D-San Francisco) said legislators "will review and learn from our mistakes" to ensure that similar delays and cost overruns do not happen on other large infrastructure projects in the state.

• New Bay Bridge Opens Ahead of Schedule

  "It was incredible, man," Nhua Rodriguez told NBC Bay Area. "Nice, clear, no traffic. Very good, man."

  Omar Hassan also liked his ride: "I thought it was excellent. Spectacular. I was blown away."

• New Bay Bridge, same old traffic

  Traffic across the bridge is typical of a weekday, the California Highway Patrol reported Tuesday morning. Cars were bumper to bumper at the toll plaza early and the speed across the bridge was about 40 mph, the CHP said.

Who said "History is written by the victors"? It's interesting to see the different slants that people give: some see the decades of failures and cost overruns; some see the triumph that the new bridge was built at all.

Some want to remember, some want to forget.

For now, there is a new bridge, and: "It's shiny!"

Read More

Rim Fire updates

I've been spending a lot of time at the superb Inciweb site recently. Why aren't all government web sites like this? Inciweb is just a superb site, with detailed, accurate, exact information, clear descriptions, no fluff.

Of course, the reason I've been on Inciweb is due to the Rim Fire.

I'm sure this isn't the first time you've heard of the Rim Fire, though if you're not from this part of the world you've probably heard it called the "Yosemite Fire". If you're just getting caught up on the Rim Fire, let's start with a few highlights from today's Fact Sheet:

• Day 16
• Acreage: 222,777 Square miles: 348.1
• Largest wildfire in the United States to date in 2013
• Acreage in Yosemite National Park: 60,214
• Proportion of the fire burning in Yosemite National Park: 27 percent
• Proportion of Yosemite National Park within the fire perimeter: 7.9 percent

The commitment to fighting this fire has been mammoth; at this point, I believe that every aerial asset that CalFire possesses has been involved in the effort.

So far, the major human-related loss to the fire has been the legendary Berkeley Family Camp, which was a complete loss:

Very bad news tonight for the thousands of East Bay families who are fans of Berkeley Tuolumne Family Camp. It burned down today, according to John Miller, spokesman for the US Forest Service. Owned and operated by the City of Berkeley, the beautiful and popular camp along the Tuolumne River near Yosemite National Park has been around since 1922. But today, it fell victim to the massive Rim Fire

My colleague in the cubicle to the left of me has been taking his family to Berkeley Family Camp for 7 years.

My colleague in the cubicle to the right of me has been taking her family to Berkeley Family Camp for 11 years.

They're still wearing their Family Camp tie-dye T-shirts to work on Fridays, but it's with a much different emotion now.

Thank goodness that that just-as-legendary Camp Mather was saved:

According to Camp Mather officials, the San Francisco Fire Department, National Park Service, U.S. Forest Service and Recreation and Park Department have all had a hand in keeping the core of Camp Mather safe from the fire.

Even though the containment is now 60%, as this morning's status update reminds us, the situation remains critical:

Fire activity continues to be active in the south and southeast with moderate rates of spread and torching. Today winds will be coming from the south southwest with up to 20mph gusts. Fire activity has been slow and moderate in the north end of the fire. Today’s fire weather is extreme. Very active fire and extensive spotting continues to hamper suppression efforts and pose risks to firefighters.

One reason that this fire has a very personal interest to me is that, just exactly a year ago, I was backpacking right in the middle of this fire. It is very emotional to look at my pictures from last year and imagine how much that part of the world has changed in just the last 16 days.

The best map of the fire, I think, is this one. The black line represents the control line. See if you can find Hetch Hetchy Reservoir. Note that the south shore of the reservoir is listed as a containment line. Even though the fire is so active that it's leapt over 300 feet at a time to spot and initiate new outbreaks, there's no way that it can go directly across the reservoir.

That is why the burnouts from Hetch Hetchy south to Harden Lake were so critical, and why that was the line that the firefighters fought like crazy to secure. (It's the straight vertical north-south line at the rightmost edge of the fire.) Without that line, the fire would just race up the Tuolumne canyon (known as "the Grand Canyon of the Tuolumne"), and they probably couldn't stop it again until it reached Tuolumne Meadows, at which point most of the park would have burned.

The moment of decision came on Thursday night, August 29th, when, after days of waiting for a break in the almost non-stop winds, there came an afternoon of calm, and a forecast of a cool and (relatively) humid evening. That night, as Inciweb dryly puts it:

Firefighters began burning operations south of Hetch Hetchy and along Old Yosemite Road. ... Night crews will continue with burning operations as long as weather conditions allow

So when you go back and read the hourly reports of the fire on Inciweb (or elsewhere) and wonder why there is so much discussion of tiny little Harden lake, well, it wasn't really about Harden Lake, it was about trying to save Yosemite National Park from experiencing what happened to Yellowstone in 1988.

in the summer of 1988, Yellowstone caught fire. The fires, which began in June, continued to burn until November, when winter snows extinguished the last blazes. Over the course of that summer and fall, more than 25,000 firefighters were brought in from around the country.

In the end, the flames scorched about 1.2 million acres across the greater Yellowstone area.

Now, the Rim Fire is still nothing like 1988; as Bill Gabbert reminds us:

On the worst single day, “Black Saturday” on August 20, 1988, tremendous winds pushed fire across more than 150,000 acres.

That's nearly the entire Rim Fire acreage, on a single day.

But if you're trying to understand the Rim Fire, and struggling with the dry and terse information on Inciweb, accurate and exact though it may be, may I recommend that you spend some time with the marvelous document prepared five years ago by the Yellowstone staff for the 20th anniversary of the fires: The Yellowstone Fires of 1988

Since 1988, fires have continued to burn in Yellowstone—more than 85,000 acres as of 2007. According to historic records, that’s to be expected. However, the average number of lightning started fires has been increasing each year since the 1990s. The majority of scientists believe this increase is due, in part, to climate change. They say that, generally, the western United States will experience increasingly intense fires—fires similar to those of 1988.

Humans have a complicated arrangement with our world. What we do matters, and it's often hard to detangle cause from effect.

But Yosemite National Park is perhaps the most beautiful spot on our beautiful planet, and I wish those five thousand hard-working firefighters the very best of luck (and safety) over the next several weeks.

And thank you for cutting that line down to Harden Lake.

Read More