CouchDB Catchup

The big trip

So last week I flew to San Jose for some meetings and to give 2 talks about CouchDB. I unfortunately missed my connection in Las Vegas and had to be booked on a later flight, which meant it wasn't until 2:30 am PT that I got to my room. Which for me, wasn't until 5:30 am. Then when I finally got to bed, I couldn't fall asleep. My sleep cycle had reset and I was wide awake for at least another hour and slept like crap. But when I woke up, I was so pumping with adrenaline it hardly mattered.

I started the day at IBM's Silicon Valley Labs (SVL), and though it's technically in Silicon Valley, it's out in the middle of what looks to be orchards and farms and undeveloped land. It rained nonstop the whole trip, but the scenery there was still beautiful. It must look incredible in the spring.

I met in-person for the first time David Fallside, my manager and a very interesting guy. I learned he'd been involved in a zillion different standards, like XML and SOAP. And despite my XML-hatin', he still digs CouchDB.

My talk was later at Almaden Research Center (ARC), which is set in the middle of a nature preserve in the hills that border San Jose. And I thought the SVL views were amazing, ARC is one of the most beautiful office facilities I've ever been too, inside and out. I'm told it was built in the 80's when IBM was crazy rich.

The talk was well attended, there was a handful of external guests and everyone else was from IBM. There were lots of DB2 developers and researchers there, and including PHDs, Distinguished Engineers and Fellows. Cool, except for... well, lets just say, my first ever CouchDB talk didn't go as well I would have liked. It makes for a funny story though. Ask me sometime.

Then I had a meeting with the JAQL and JSON team that went much better (though I think I was losing coherency towards the end). JAQL is a new JSON query language they are developing and open sourcing, and is currently being integrated with HBase and Hadoop. The nice thing about JAQL is the ability to do straight forward "joins", which CouchDB doesn't have a simple answer for yet. They also were working on some really cool inverted indexes for semi-structured data that would allows for some really interesting dynamic queries. Of everything I saw there, the stuff they showed me got me most excited. This is a huge benefit to working with the people that I am at IBM, access to some really cool technology and brilliant people.

I'll be writing some more about this in the future.

The next day I had more meetings, and gave a talk to the DB2 XML guys. Despite being 30 minutes late (I was sent to the wrong IBM facility, doh!) and having to rush my still unpolished talk and dissin' XML, they seemed pretty receptive to CouchDB. And again, in talking with them I saw they had some deep insights into semi-structured data that was very applicable to CouchDB. These people get it.

One thing many IBMers were very interested in is Erlang, which I discovered I completely suck at explaining. But people seemed genuinely interested in learning about it. My advice is just start playing with it. That's how I finally "got it".

IBM Training

I just spent 2 days at the IBM Training for all new permanent hires. In Armonk. In January. So you know I was totally looking forward to it.

Fortunately the hotel and facilities were first rate, and we had a great instructor whose been with the company for 20 years and her enthusiasm was infectious. So the whole thing was better than I thought it would be, even if a little propagandistic. But one thing that has always impressed me is IBM really does have a long history as a progressive company and a commitment to diversity and integrity, and they make a big deal about that, that it's taken seriously and you can be fired for being a shithead. My previous experiences have shown they do take it seriously and I think its a big part of why it's still a thriving company after nearly 100 years. I think that's pretty cool.

Big Company Blues

Something in my HR records got seriously messed up when I joined, and somehow all our HR stuff, including live paychecks, was getting sent to our previous address from a year ago. In the IBM system, everything I had access to had the correct address, but when we'd call for support they'd have our old address. And we'd ask them to change it, but they said they didn't know how, that we'd have to change it on the web. The one that had the correct address already.

And it's not an isolated case, a woman at the IBM Training was experiencing the EXACT same thing. The address records she had access on the web were correct, but mail was getting sent to a previous address, and support reps couldn't help her. Another guy said his pay and everyone is his new division was wrong by hundreds of dollars.

Anyway my wife spent a ton of time on the phone about this issue and she just now got it fixed today, after 2 weeks of calling up various support people. We think it's fixed anyway. Yay big companies!

To the start-up in the Boston area who needs an Erlang Expert

I have been contacted by exactly 6 different headhunters for this job. Stop calling. I know it's the same job because they all say top salary, working from home, dual 30" Apple Displays and an Aeron chair, or variation thereof. It sounds sweet and I'm flattered, really. But you really can't pay me enough to stop working on CouchDB. Sorry.

BTW, if any Erlang expert wants to be referred for a pretty sweet sounding job, just let me know. Maybe I can get a big fat commission.

Scale Baby!

Here is someone who actually did some scalability testing on CouchDB. I'm not sure of what to make of the data, but he hit the limit of the testing tool at 20k concurrent requests. That's pretty cool.

On a related topic, Brian Aker wrote recently about the connection thread pooling work I implemented for MySQL 6.0. The improvements in client scalability look really impressive. I hope it scales like that on Solaris :)

Eating Dogfood

Christopher Lenz is now running his live blog off of CouchDB. Awesome. Please backup frequently Chris!

The CouchDB Components

Two drawings of the same thing.

I like the hand drawn version, but it actually takes more effort than using Apple Keynote.

Apache

Sam Ruby and David Fallside have made a ton of progress with the Apache work. Sam has created an Apache Incubator Proposal and we are presenting to the Apache Review Board next week. Things are moving ahead quite nicely.

Talk Slides

Here is a PDF of the slides for the talk I gave, with some minor changes and additions.

The Amazon Elastic Compute Cloud

While it sounds like a bad 70's Disney movie, it's actually a fast and cheap way to create a big computing cluster using Amazon.com's resources. And someone has a created CouchDB Amazon Machine Image, calling it ElasticDB. Nice.

Posted January 31, 2008 10:59 PM