Groups | Blog | Home
all groups > sql server data warehouse > february 2007 >

sql server data warehouse : Snapshot


Joe
2/5/2007 2:43:46 PM
One of the first steps in creating an ETL for a data mart - is moving =
data from the source system into a staging area.

A current system we have now - copies all data, all tables and values, =
and replaces the staging area each time it is ran - even tables and =
values that aren't being used by the data mart or in any tranforms. It =
is ran once a month right now.

From what I see - SSIS will allow me to just bring over into staging =
what needs to be staged or used in the ETL to result in the DM. This =
seems like a much better approach and less strain on the systems. =20

Are there any other reasons I need to consider to support having all =
AlterEgo
2/5/2007 5:09:24 PM
Joe,

The only reason I can think of is the staging process might use =
supplemental tables to validate, verify or transform the data it is =
loading.

-- Bill
[quoted text, click to view]
One of the first steps in creating an ETL for a data mart - is moving =
data from the source system into a staging area.

A current system we have now - copies all data, all tables and values, =
and replaces the staging area each time it is ran - even tables and =
values that aren't being used by the data mart or in any tranforms. It =
is ran once a month right now.

From what I see - SSIS will allow me to just bring over into staging =
what needs to be staged or used in the ETL to result in the DM. This =
seems like a much better approach and less strain on the systems. =20

Are there any other reasons I need to consider to support having all =
AlterEgo
2/5/2007 5:09:24 PM
Joe,

The only reason I can think of is the staging process might use =
supplemental tables to validate, verify or transform the data it is =
loading.

-- Bill
[quoted text, click to view]
One of the first steps in creating an ETL for a data mart - is moving =
data from the source system into a staging area.

A current system we have now - copies all data, all tables and values, =
and replaces the staging area each time it is ran - even tables and =
values that aren't being used by the data mart or in any tranforms. It =
is ran once a month right now.

From what I see - SSIS will allow me to just bring over into staging =
what needs to be staged or used in the ETL to result in the DM. This =
seems like a much better approach and less strain on the systems. =20

Are there any other reasons I need to consider to support having all =
Joe
2/6/2007 9:02:10 AM
I almost see no reason to even bother with moving any data to a staging =
area - just let the ETL handle what it needs to use for SCD and other =
updates - right?=20
[quoted text, click to view]
Joe,

The only reason I can think of is the staging process might use =
supplemental tables to validate, verify or transform the data it is =
loading.

-- Bill
[quoted text, click to view]
One of the first steps in creating an ETL for a data mart - is =
moving data from the source system into a staging area.

A current system we have now - copies all data, all tables and =
values, and replaces the staging area each time it is ran - even tables =
and values that aren't being used by the data mart or in any tranforms. =
It is ran once a month right now.

From what I see - SSIS will allow me to just bring over into staging =
what needs to be staged or used in the ETL to result in the DM. This =
seems like a much better approach and less strain on the systems. =20

Are there any other reasons I need to consider to support having all =
AlterEgo
2/6/2007 11:08:01 AM
Joe,

One of the primary reasons for the staging area is to prepare the data =
(validation, transformations, etc.) for loading. I would keep the =
staging area. As your application grows, so will the need for a staging =
area. There may be other valid opinions on this.

-- Bill
[quoted text, click to view]
I almost see no reason to even bother with moving any data to a =
staging area - just let the ETL handle what it needs to use for SCD and =
other updates - right?=20
[quoted text, click to view]
Joe,

The only reason I can think of is the staging process might use =
supplemental tables to validate, verify or transform the data it is =
loading.

-- Bill
[quoted text, click to view]
One of the first steps in creating an ETL for a data mart - is =
moving data from the source system into a staging area.

A current system we have now - copies all data, all tables and =
values, and replaces the staging area each time it is ran - even tables =
and values that aren't being used by the data mart or in any tranforms. =
It is ran once a month right now.

From what I see - SSIS will allow me to just bring over into =
staging what needs to be staged or used in the ETL to result in the DM. =
This seems like a much better approach and less strain on the systems. =20

Are there any other reasons I need to consider to support having =
AddThis Social Bookmark Button