Spring 1997 - A Different "Cube"

My time with the ValueRx Reporting team had been going well, though having moved the team here, is was becoming apparent that this team would be pretty straightforward and... well... dull. Writing reports isn't really the most glamourous job. But one day, Mike and Mark had a wild new idea.

Now, Mike and Mark shared the cube next to mine and Cobol Guy's, and they peeked their heads over frequently. And one day they peeked over and DEMANDED I join them in the conference room immediately. So I went... Mike had discovered something called "Data Warehousing" and wanted to tell me about it. He tried to explain:

"So you have a CUBE. And we're talking about CARS. So down one side, it's colors - Red, Blue, Silver, Black, White. And across the top it's 2-door, 4-door, wagon, van. And along the back side it's brands - Ford, Chevy, Dodge, Mercedes. And inside this cube is NUMBERS. So you go 'Red, 2 Door, Dodge = 5' and 'Silver, Wagon, Mercedes = 3' oh and these are sales figures."

Much as you probably are in reading this, I sat completely confused while he drew and re-drew the cube, finally scribbling in the middle "ANYTHING CAN BE IN HERE - IT'S INCREDIBLE". I furrowed my brow and nodded meaningfully, muttering, "yes, I think I see... there's really some potential there..."

Turns out there was a Data Warehouse project already under way, but it had stalled out under the watchful eye of a Price Waterhouse billing farm (2 persistent resources, then 3-5 greenies rotating through every 2 months, generating almost nothing). My job was to wrest control from Price Waterhouse and get the project back on track. Which wasn't easy because I had no fricking idea what this cube thing was.

Fortunately, Mike was a big fan of learning on the job: We ordered "the bible" of data warehousing by Ralph Kimball: We liked his writing style better that Bill Inmon and Claudia Imnoff's take on the subject - they were much more dry. Not that we actually read much of the book - both Mike and I are learn-by-doing people, so after the preface and the first 2 chapters, the spine wasn't cracked on the book.

We also booked travel to a conference of "The Data Warehouse Institute" in San Diego... and we learned a lot. Some things we learned: Conferences are almost useless, expense reports are rarely scrutinized, and martinis are delicious. We were very bad boys. But we actually learned at the conference that whatever data warehousing really was, what we were already doing wasn't terribly far FROM it. Thus energized and vitalized, we returned to really tear up the DW, and I'm pleased to say we actually did a good job.

Ok - I'll back up and actually define this now: Data Warehousing is taking data FROM your core transactional systems and moving it TO a new system that has been specifically optimized for reporting... but not boring invoicing reporting: Analytical reports: Show me Sales by region by salesperson by product line... now resort that so that product line is the primary sort, roll it up to region, now show me only the bottom 10% of our product line.... A well designed warehouse allows your team to have truly interactive access to your company's information, "turning data into knowledge".

Our warehouse was far from that - it was basically a huge dump of claims data, growing and growing. The reports you wrote against this were not fundamentally different than the ones you'd write against the invoicing system... but at least they didn't slow down the main system anymore. But we wanted things to be better.

Another Martini-fueled evening, this time at the Monte Carlo in downtown Minneapolis, and Mike, Mark, and I came up with our new strategy: Data Marts: You take your data warehouse and create even smaller subsets of data, at pre-calculated summary levels: You ditch the lowest level of data and just store the numbers (the facts) and the dimensions (colors, brands, styles): This was the cube that Mike was ranting about, but in truth, none of us had truly grokked it yet. The concept had been there from the beginning but all at once, we truly UNDERSTOOD it (after a few drinks and steak!). We drew it out on a napkin and brought it into work the next day and started planning...

The night seemed magical because our new CEO (who we DID like) Kevin wandered through the restaurant: We imagined ourselves to be insanely cool, to have come up with this great strategy and to have been hobnobbing in the same restaurant as our CEO.

By Spring 1997, we had moved to a new location - corporate headquarters built just for us, and on day one, we were already out of space. I wanted to punch the facility planner (who oddly was a cousin-in-law of mine). Mike and I had wrested control of the warehouse entirely away from Price Waterhouse and they were in the final throes of the handover. I was the lead architect, and Mike had a new title: Director of Data Warehousing. Whoah. He was a Director! He was actual MANAGEMENT. And he was 32! And with my promotion to architect, my friends at Safenet wrangled a rate increase. And two weeks after that, Mike TOLD about the rate increase, and I went right to my buddy Jay, and caught him trying to tell me there wasn't an increase. Dude tried to pocket the increase flat out. Which was extra-dirty pool. Caught, he increased my rate, but not before grumbling that the rate increase was really HIS because he had negotiated it, so I shouldn't have cared as long as I was being paid an acceptable rate.

Yeah. So I soon severed my relationship with Safenet... in a pretty awesome move.