Data warehousing for the busy DBA

Part 2: What a data warehouse isn’t

Many data requirements can be fulfilled without going to the trouble of a data warehouse.  In some cases the requirements are in conflict with the main requirements of a data warehouse system.

Data warehouses are not real-time systems

Real-time data warehousing is almost never a genuine requirement.  Real-time analytic and aggregate reporting requirements are actually quite rare with very few exceptions.  There are two very good reasons not to implement a real-time system unless you really need it: more »

Data Warehousing for the Busy DBA

Part 1: What is a data warehouse?

So, one of the enterprise architects has convinced management that we need a Data Warehouse.  You’re the DBA, you know about databases, right – we need you to go and design it.  Can we have an estimate by next Thursday for the board meeting?

This series of articles walks through key data warehouse concepts.  It is aimed at a database professional – a DBA or SQL developer – who is now getting involved in a data warehouse project.  Therefore the articles assume some background in database development or administration.

The term ‘data warehouse’ gets used quite loosely at times.  To start, here is a definition of what a data warehouse is (and is not) and the reasoning behind that definition.  We’ll also look at some alternatives to a data warehouse if your requirements aren’t congruent with an actual data warehouse project.  A data warehouse is more than just a reporting database.  A full-fledged data warehouse will have most or all of these requirements: more »

After having answered some 250 questions here on dba.SE, and over 1,500 questions on StackOverflow, I’ve read some good questions, and I’ve read some bad questions. I’m no Jon Skeet, but I think I can offer some perspective on how to ask effective database-related questions on this site and get solutions to your problems. So what follows is some advice about things you should be sure to think about before, during and after posting your question. Not all points will be relevant for all types of questions, but some are universal. I know this may seem like a big laundry list of rules, but please, bear with me – at the very least, read all the section titles (and the last section about reading over your question).

more »

I usually see newcomers not use these to their benefit, tags. If you did not notice when you ask a new question there is a box that requires you to put at least one tag before you can even submit the question. You are allowed a max of 5 tags, so choose wisely but please choose more than one. I don’t mean always pick 5 tags on every question you ask, but at least pick 2 or 3 that generalize what the question is regarding. With questions on DBA.SE, generally speaking, you should at least have a tag for the RDBMS you are working with (I did say generally speaking). more »

This question gets asked a LOT on our meta, on meta.se, and it gets asked even more frequently in our chat by new users once they really learn our scope. Here’s some examples, just for fun:

• We need to formalize a site title/name, does Database Administrators still ring true?
• “Database Administrators” needs to die
• Migrations to DBA appear arbitrary, and include questions that are outside the regular DBA’s job description
• Migration of [data-warehouse] and [database-design] questions from SO to Database Administrators?
• Doesn’t this question belong on Stack Overflow?

And by rights, the title of “DBA” (database administrator) is not one that should be bestowed upon me. My pedigree is web-programmer, but I’ve been described by friends as a renaissance man, since I can do the entire field of web programming, from server configuration to database setup and configuration to web programming. It’s true that I know how to best utilize JSON or XML, that I know what binary transfers look like, I can decode compressed streams on the fly for debugging, and I’ve been playing with this stuff since before Apache had a version 1.0 (ok, that’s hyperbole, but I was there long before the 2.0 debut). But it’s also true that I can’t be as specialized as the three dbas that sit across the hall from me during the day.  So I make up for that by keeping my ears open, and sharp, and paying attention to what they do.

And therein likes the crux. more »

In keeping with our interested parties list we’ve launched the blog with about 10 contributors, more or less. And since we veterans know that the StackExchange Network runs on MSSQL, I thought this was a catchy little title. Don’t let that make you think that we have to only write about MSSQL content. Trust me, we’re all about data storage and retrieval [1], so if you have Oracle, Microsoft SQL Server, MySQL, Postgresql, couchdb, mongodb, memcached, bigtable, hadoop, or any other similar related technology, you’re welcome to bring us questions on it. We hope to devote some of this blog space to defining some of the common problems people face, giving some walkthroughs, and giving history lessons (you don’t know where you’re going till you know where you’ve been). We’re also going to talk about Business Intelligence and all those other fancy buzzwords, as our expertise covers the lot of data, analysis and retrieval (but we’re not statisticians, they have their own site at Cross Validated). We also generally turn down questions that only ask how to do front-end programming, for that you’ll want to visit Stack Overflow.

more »