Saturday, April 23, 2011

Failure is the new Success

This week was a real eye opener in the IT world. Amazon's EC2 cluster suffered a massive meltdown (just Google Amazon EC2 I wont bother with the links). The media was all over it about how half the internet went down and major sites were DOA for 24-48 hours ... Massive Collapse. Like the Japanese nuclear reactors (except the part where the internet being down didn't actually hurt anyone except financial ... or did it .. but thats another story).

But like the Japanese disaster, there is a hidden success story. The world didn't come to an end. The Japanese reactors didn't actually explode in an apocalyptic meltdown and kill everyone on earth turning it into the green glowing dawn of the living dead. The system was actually contained and it was mainly the news media which focused on the drama while ignoring the real tragedy of 12,000+ dead due to a tsunami. I guess thousands dead due to a natural disaster isn't as exciting as no one actually dead in a nuclear reactor that didn't actually explode. "But it could have! " ... well it didn't. The hidden success in the Japan Disaster is that in fact the reactors stood up to 9+ quake AND a tsunami.

Similar with the Amazon story. The media wants us to think the cloud suffered a meltdown taking down the internet and we cant trust it anymore. The reality is that Amazon worked exactly as advertised ... well maybe off by a few 9's but whats statistics. The reality is that a single geographical region suffered a 'network meltdown' taking down the whole region for a while, then within 12 hours only a single 'availability zone' As of now (2011-04-23 19:39:00 EST) Amazon is mostly but not entirely up. But it actually worked ! Other zones were unaffected. A single geographic zone went dark for 48+ hours but the rest kept chugging on. And in fact it seems that all data is restored, nothing lost.

So who suffered ? those vast number 'mainstream internet sites taken down' ? Well the ones that suffered were the ones that didn't plan for failure. Netflix is based on Amazon but kept on chugging because it was designed for the conditions Amazon advertises. That is a single geographic region might actually die so don't put all your chickens in one basket. The people that didn't understand that were hurt. Those that embraced failure rode the wave. Even those that didn't embrace the failure were just down for a while and amazon restored their data and servers from redundant storage ... it just took a while.

So whats to learn ? The fact is there is no totally perfect system. I don't think any company could build a data-center better then Amazon ... Google and Microsoft might equal it but no ones perfect. Failures are going to happen. Period.
Amazon was designed for geographic redundancy and exposes the necessary API's to take advantage of it. If you don't, its your fault.
well maybe ...

Maybe not. The whole Cloud Computing concept is based on redundancy and trust in large scale distributed systems. Its the outsourcing of the IT department. Why should individual developers who subscribe to a cloud system be required to manage this ? While I agree that Amazon performed exactly as it advertised, and that if you planned for it and took advantage of its capabilities you'd have rode out the storm just fine ... it points to a weakness in the system. I argue that cloud computing should hide this from you. Its an artificial artifact that a "virtual machine" actually resides somewhere physically and that you have to care. The next generation of cloud computing should hide this from the users much like the Amazon S3 storage (which hides its physical location), the EC2 and EBS system should be able to migrate to different geographic locations without the programmer having to architect dynamic load balancing, fault torrence and hot swap failover. Isn't that the whole point of "the cloud" ? To let the 'big boys' figure that out and leave us to writing apps ?

Maybe thats the real point of clouds like Google Cloud Computing ? Its time Amazon wake up and accept its awesome but still needs to go the extra mile. If the biggest web sites crashed because they failed to make use of the advanced features of cloud computing, maybe its time to make those features less "advanced".

Tuesday, March 22, 2011


What is Patience ? I've noticed a pattern when I show off my new hobby, model trains (
Almost universally the comment is "Wow, that must take a lot of Patience, I could never do that". At first I thought I understood, as certainly it takes a lot of *time* and *effort*. But reflecting on this, I cant understand the term "Patience".

This takes no "Patience" whatsoever. To contemplate, how can a hobby or an art or work or similar thing take "Patience" ?. More so then say Jogging, or watching TV or reading. Raising children, now THAT takes patience ! But doing something which every second is enjoyable and at the end builds something. That takes no patience. It doesn't matter how long it takes. The longer the better. I actually worry that I'll complete it and have no more to do.

Watching Television, 40 years later and you've accomplished nothing. That doesn't take patience ? but building something does ? I guess I don't understand what people are thinking. Perhaps its as ancient and almost cliché as eastern philosophy about the journey not the destination.

I look back at my life at the things which I have a problem with Patience. I consider myself *very impatient*. I cant stand sitting around at "parties" while people invoke "small talk". Give me a gun I could shoot myself. Waiting for a delayed plane. Waiting for my life and career to progress to the point I could afford a house. Driving in traffic. Waiting for a shipment of a new train part! Trying to persuade someone who doesn't agree with you. All those things take "Patience". But actually doing something ? Immeshed completely in the act of doing something that takes enough of your senses to distract and absorb the ongoing ramble of an overactive brain ? Thats not patience.

Balisage - Now on Advisory board !

I'm honored to announce I've been accepted to the Balisage Conference Advisory Board.

This conference is by no comparison, *the* markup conference which I've been lucky to attend every and honored to present at many during the last 5 years.

I hope that I can give back to the community in some small portion of what I've received by attending this conference series.

Wednesday, February 23, 2011

Curaçao Dive Trip

My friend and I just came back from a great scuba dive trip to Curaça, part of the "Netherlands Antilles" ABC islands (Aruba, Bonaire , Curaçao )

Lets get it out of the way. Its pronounced "Kur-A-Sow" (not "kur-aahh-koo").
And yes that funny ç is a "C with Cedill"

Before last year I had never heard of this country. Ignorant Americans. Now I'm convinced its the Hidden Jewel of the Caribbean. (full history)

Settled by Spain in the 1500s then captured by the Dutch and used as a cornerstone for the slave trade until 1863. Then oil was discovered and then tourism. Now it is a marvelous little island, well developed, quite clean, modern, and friendly.

We went primarily for Scuba Diving (at the suggestion of our local dive shop), and for convenience and price stayed at Breezes which hosts a full dive shop, all incusive all-you-can-eat-or-drink and a reasonable tropical paradise look & fell almost (but not quite) equaling a Disney resort. Breezes, is well, a Breezes. A great value for the money, but not particularly high end. But since we went for diving not entertaining it worked. The rooms were spacious and clean, most everything worked and nothing to complain about except the usual ( long lines at the bars, "mini" beers served in 6 oz cups, but you could get as many as you want, a TV that went into "demo mode" every 5 minutes, a pretentious "pretending to be fancy" specialty restaurant which would rather I wear dirty tennis shoes then classy strapped sandals, hard bed, only 1 US plug in the room, crowds, cheesy entertainment, rooms often not cleaned until 6pm and kids everywhere.) But hey, its a budget family resort and truly a great value for the money.

The flight there and back was miserable. The planes were fine (5 hours total) but from Jasper our best flights were through Miami, with an 7 hour departing, and 9 hour returning layover. But that got us a $550 flight instead of a $1200 flight. Talking to others, there are better flights from different places, including a direct flight from New Jersey (4.5 hours).

On arrival, customs and transit were flawless. 10 minutes to get through customs and we found the Breezes representative waiting for us with our names. 10 minutes later we were on the air conditioned bus and 20 minutes later at the resort.
Check in took about 10 minutes (although I heard later some people at peek times took over an hour). The drive from airport to hotel was interesting. Of first note was the "shanty" part of town near the airport. A step up from Jamaica or Dominican Republic, clean, but definitely poor. But by the time we went through the city (which was immaculate and modern) and then to the hotel, things picked up. Clean modern homes, reasonably kept roads. A lot of new construction all fairly modern.

Diving was both excellent and unimpressive. Depending on what you expected. The dives were very easy, which I enjoyed. The boat picked us up at the resort dive shop (about a 2 minute walk from the room), sometimes picked up a few from other local hotels (10 minutes max) then at most a 10 minute ride to the dive site. Except for the first day at the tail end of a storm, the sea was calm with absolutely no currents. Clarity was fair (40' - 60') and improved with depth. A huge amount of wildlife, but mostly small fish, few large critters. That was the unimpressive part. If you were expecting sharks, rays, 10 foot barracudas, lobsters, giant crabs, well, you'd be disappointed. But if you liked millions of small colorful fish, amazing variety of coral and plant life, warm water (78deg/C +) and a long pleasant easy dive it was heaven. We did see some interesting ones, a few Huge Morey eels (1 foot across, 8+ ft long !) a turtle, a few big trumpet fish, but for the most part fish in the 1 inch to 2 foot length and a lot of "empty apartments" (coral nooks and crannies totally empty of fish). 2 lobster but small and hidden, and one turtle.

We dived 5 days (2 tanks/day) in the mornings, back by 12:30 for lunch drinks, and naps. We did a group checkout dive on sunday, then later in the week did a solo (just us 2) dive from the shore (unlimited free shore dives).
I used Nitrox/32 (no thats not Nitrous) for all the boat dives. I'd been Nitrox certified last year and this was my first try with the magic go-juice. I'm sold. I felt better after each dive then I started. I came up feeling wide awake, relaxed, clear headed and just fantastic. But then maybe I would have anyway, who knows. Great relaxing diving - 12 dives total. Could have done more but boy are those afternoon naps after a few beers are great.

The weather was great. about 75/C at night and up to 88/C in the day. The constant breezes kept the air cool and humidity was reasonable ! Our scuba gear actually mostly dried outside overnight (unlike Cozumel where they would get *wetter* if left outside), and I didn't sweat through my tee-shirts and create a sopping mess in an hour (again un-like Cozumel).

On Saturday we took the free shuttle to town. Wonderful and totally surprising. This was Willemstad, (previously called "New Amsterdam"). A beautiful town with a definite dutch look. Very pretty buildings and architecture. Clean, and a marvelous mix of cultures. All the people speak at least 4 languages (Papiamentu, English, Spanish, Dutch). The town was a mixture of tropical/island and modern. Little fishing boats docked selling fresh fish and fruit, along with chain stores, a McDonnalds, some dutch pubs, jewlery stores, trinket stores. Hard to describe. Very friendly and safe feeling, clean and reasonably modern with an old-world flair. Most of the tourists seemed to be European (probably dutch), with maybe 20% american and a mix of south american (its near Venezuela).

So why do I call this "The Hidden Jewel of the Caribbean" ... because it seems noone knows it exists yet is the nicest most beautiful island or tropical area I've been to that is both modern and native. I have suspicians that like the old Greenland/vs/Iceland trick long ago, the Dutch are not advertising Curaçao much and instead send foreigners to Aruba and keep Curaçao to themselves as their hidden jewel.

I'd definitely go back and recommend it as a good value for a vacation for singles, couples, or families. Plenty to do and see, wonderful weather, and just plain pleasant.