Groups | Blog | Home
all groups > sql server data warehouse > july 2005 >

sql server data warehouse : Relational to DW.?


Luis Esteban Valencia
7/29/2005 10:46:33 AM
Can sombeody point me in the right direction. How can I convert a relational
database to a datawarehouse?

Thanks

Peter Nolan
8/1/2005 1:37:48 AM
Luis,
interesting question......I assume your 'relational database' is for a
transaction based system?

So, you will not want to much to convert the database as to create a
second database as the DW as it is not advisable to try to run a
transaction system on a DW model.

On my home web site there are lots of papers and presentations
discussing what a DW is and the various modelling techniques that are
commonly employed. The modeling technique you might use depends on what
you want the DW for....if you need to answer any question you ever
might want to ever ask from any data you ever had without havining any
idea about waht questions might be asked you need an archival data
store of some type.....

If you have a reasonable idea of what questions the DW needs to answer
and do not need to store every value of every change in every field
forever you can live without the archival data store and use all
dimensional models.

There are a lot of books/materials available about dimensional models
now....I have pointers to the ones I recommend on my beginners page.

If you finish up going with building a DW database you will need to
answer some of the FAQs I have on my FAQ page....what database? what
ETL tool, what query tool etc. So I suggest you browse thru the FAQ
page as well....

Since you are asking here, MSFT offer SQL Server, Analysis Server, DTS
and Report Services as their main BI offerings. Of course MS Office can
also be used to present data from the DW. These all work pretty well in
their space......well enough that I am actually building a BI product
on top of it all as I think MSFTs BI for the masses message is going to
stimulate demand for the product we are building (we hope!).

There are well documented/defined techniques for transforming
transaction based data to dimensional data. (I've published free code
to do so on my site.)

What is not well documented/published are all the ideas of how to
integrate data from many different places and combine them
effectively...butit sounds like you are not doing that yet....

Ralph Kimballs last book on ETL toolkit is also worth reading...I have
just finished it but have not put the pointer to it on my web site
yet....

Anyway, happy reading....I hope the materials I have published are
helpful.....if you feel there are things missing please let me know. I
am building my site as a place where 'newbies' can go and find lots of
useful information as well as useful links.....my little effort at
putting something back into DWing.. :-)

Best Regards

Peter Nolan
www.peternolan.com
Akinja
8/4/2005 6:57:42 PM
This is very interesting timing. I am trying to find more information about
Analysis Services, Data Warehousing, and Data Mining and your website is a
great place to start. I have been interesting in Data Warehousing and
Analysis Services for some time. However, here is my problem (or rather
opportunity).

I need to convince my boss and his bosses that this is viable, needed, and
would benefit us without breaking us (cost wise). We are moving to new
servers and I have convinced them to move away from transactional and read
data on the same server. Now using replication and DTS packages, I am
moving data that is imported to one server that will do the initial process
to another server that our Web services and application will access. Sounds
like a perfect candidate for DW. This is especially true because our data
is now at 300+ GB and will most likely double in the coming year.

So what will DW buy me and my company? How much recoding of applications
will all this require? I have read a lot about DW or AS along with learning
MDX and this seems to be a daunting task. (I just love the smell of a fresh
challenge in the morning:) )

Any help here to where to point my nose next is greatly appreciated.

[quoted text, click to view]

Akinja
8/5/2005 12:00:00 AM
Thanks for the input. I will take a look at it.

Akinja


[quoted text, click to view]

Peter Nolan
8/5/2005 3:03:46 AM
Hi Akinja,
I have now bundled my old 'newsletters' on my downloads page.......it
is just a zip file that says newsletters before 2004 or something...

Newsletter number 1 is 'Cost Benefit Anlaysis and Business Benefits
Realisation'.......cost benefit analysis used to be the number 1
question in the early 90s......when DWing was considered something of a
mystical art form and there were very few public examples of
success........

That paper lists a whole lot of successes...some of which I was
involved in, some of which I was not...

The benefit of BI?

Any senior managers that cannot see the benefit of BI today are not
worth the title 'senior manager'. I have not bothered to sell the
'Benefits of BI' for about 6 years now. I simply do not bother with any
account that is not totally committed to building the EDW. There's no
point. If the senior managers have not figured this out yet there are
way too many battles to fight to move the project forward and it will
probably fail anyway.....so why bother?

Unfortunately IT people have made something of a hash of BI and DW/BI
has 'got a bad name'. Not because there is no return, but because the
people building them have not had the slightest idea of how to get a
return.....every IT person (and his dog) wants to have BI on their
resume and they feel that if they read a book or two and build
something that passes for a DW they can then go out and sell themselves
as BI consultants.......this has been unfortunate as it has added to
the failure rate of projects.

You do not get any return just by building a DW. You only get a return
when you use it to drive the business forward. Most notably, you get
great returns when you make the DW the basis for all large decisions
made by the company.

Every survery in IT says that 'Understanding of the business and
communicating the business needs to IT' is the number one problem...and
it is more so in BI.

The fastest payback for a project I have done? We got our money back
for the entire project even before we finished the project. The project
went on to be the foundation of a 10x per annum return on investment.
The company was called 'the jewel in the corporate crown' and the
Managing Director of the company was promoted to CEO of the entire
group of companies....it does not get any more successful than that.

No other IT type projects have this capability. Few business projects
can return 10x on investment and few can return all the cash for the
project before the project goes live.....the second fastest I have
heard of is 2 weeks......which is also excellent....

Done properly the first iteration of a BI project should return the
full cost of the project in 3-6 months....I would consider 6 months
very slow......but sometimes the project is large and it takes a while
to get that money back.

As I said...just building the DW does not make the money...exploiting
it does....and I see very little understanding of this around the
world....I started a 'Business' focused DW discussion group and it has
gone no-where in 2 years....

So, by all means browse through that first newsletter, and the others.
But if your senior managers do not already know BI is one of THE palces
where HUGE value can be achieved I wouldn't bother trying to persuade
them if I were you....

Best Regards

Peter Nolan
www.peternolan.com
AddThis Social Bookmark Button