Ragu (ragudba@sbcglobal.net) writes:
[quoted text, click to view] > I joined a project where 100,000 rows were added everyday. Now due to
> additional customers the expectation is 2 million reocrds/day ie 10 GB
> worth of textfiles. We have to estimate the hard disk, memory, # of CPUs
> etc.We will have one yearworth of data in the db. Rest will be in tapes
> etc.
> ...
> 1. Tohandle every day load, I thought that we will have a table for each
> day (pre created in the database )and have a view with union all
> selecting fromall these 365 tables. (This is the only way to partition
> in MSSQL Server right?).
Yes, you set up a partitioned view over all this tables. Be careful
to apply your check constraints accordingly.
However, I'm in doubt that having one table per day is really a good
thing. Yes, loads will be fast, if the table is empty and clean
each day. But with a clustered index on date, you would append the
new rows at the end. One table per month seems more manageable....
[quoted text, click to view] > 2. The requirement is to populate datawarehouse tables with all the
> data. However there will be only inserts mostly but there can be updates
> too which happenned in the past 12 days.Hence we have to use the data
> from the last 12 days and massage it etc and populate into datawarehouse
> tables.
>
> How can I do this so that I will have the datawarehouse tables with n-12
> days of data and I will alwys add the last 12 days data to it.
Too little information to say anything substantial, but a common technique
is to first load into a staging table, and then take it from there. Note
that for the previous days, you would need to do both and UPDATE and an
INSERT, presuming that new rows may appear too.
--
Erland Sommarskog, SQL Server MVP, sommar@algonet.se
Books Online for SQL Server SP3 at