|
|
You're in the
sql server data warehouse
group:partitioning a table question...
sql server data warehouse:
I have to plan a datawarehouse structure where the primary reporting usage is to list detailed information about customers. And also, we have OLAP cube for analysis purpose. I'll have 100 000 customers and a fact table containing between 35 to 45millions of rows by year. (we also have 2 other fact table with 5 to 6millions of rows by year) Generally, this fact table is filtered for a particular organizational unit and for data in 1 year. (for report generation using report server) So, I plan to partition my fact table by organizational unit and by year. But I have 180 units. Does the partitioning will works fine with 180 * 5 years = 900 tables? on this article: http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm there is a note that the maximum number of tables is 256. But the performance gain can be very high! The estimated size of the DW is 10gb / year (maybe more with additional indexes) My DTS package is ready to support partioning table loading. (the package automatically create the new table, indexes and update the view for each detected partition in the staging source table) My users access my reportserver interactively, scheduling anything is not an option. For higher analysis, my olap cubes are ready. thanks for your feedback. Jerome.
Hi Jerome:
I have some kind of "rule". Just make a partition for each 1 GB of data. Or doing it for a year, would be great. If you need more granularity, give it a try to month partitioning. Tell me if you need some help on this Sincerely -- Alejandro Leguizamo MVP SQL Server Colombia [quoted text, click to view] "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message news:uwuxtzxzEHA.3336@TK2MSFTNGP11.phx.gbl... > Hi, > > I have to plan a datawarehouse structure where the primary reporting usage > is to list detailed information about customers. > And also, we have OLAP cube for analysis purpose. > > I'll have 100 000 customers and a fact table containing between 35 to > 45millions of rows by year. (we also have 2 other fact table with 5 to > 6millions of rows by year) > > Generally, this fact table is filtered for a particular organizational > unit and for data in 1 year. (for report generation using report server) > > So, I plan to partition my fact table by organizational unit and by year. > But I have 180 units. > > Does the partitioning will works fine with 180 * 5 years = 900 tables? > > on this article: > http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm > there is a note that the maximum number of tables is 256. > But the performance gain can be very high! > > The estimated size of the DW is 10gb / year (maybe more with additional > indexes) > My DTS package is ready to support partioning table loading. (the package > automatically create the new table, indexes and update the view for each > detected partition in the staging source table) > > My users access my reportserver interactively, scheduling anything is not > an option. > For higher analysis, my olap cubes are ready. > > thanks for your feedback. > > Jerome. > > >
but does SQL Server support 500 tables in a partitioned view?
I don't need help about this, I just want to plan correctly. and I don't want month partition but only year + organizational unit Because I don't do sum or count aggregation in SQL (or just a little sum compared of what my cube provide). Generally my reports contains SQL statement like this: * Last kown value at a specific date (subquery required) (by customer) * events which start before and end after a specific date (between statement) (by customer) Its the reason of not using monthly partitions, because I never know when these dates appear in the database. "Alejo Leguizamo (MVP SQL)" <SQL@sql.sql> a écrit dans le message de news: %23thX4VC0EHA.1204@TK2MSFTNGP10.phx.gbl... [quoted text, click to view] > Hi Jerome: > > I have some kind of "rule". Just make a partition for each 1 GB of data. > > Or doing it for a year, would be great. If you need more granularity, give > it a try to month partitioning. > > Tell me if you need some help on this > > Sincerely > > > -- > Alejandro Leguizamo > MVP SQL Server > Colombia > > "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message > news:uwuxtzxzEHA.3336@TK2MSFTNGP11.phx.gbl... >> Hi, >> >> I have to plan a datawarehouse structure where the primary reporting >> usage is to list detailed information about customers. >> And also, we have OLAP cube for analysis purpose. >> >> I'll have 100 000 customers and a fact table containing between 35 to >> 45millions of rows by year. (we also have 2 other fact table with 5 to >> 6millions of rows by year) >> >> Generally, this fact table is filtered for a particular organizational >> unit and for data in 1 year. (for report generation using report server) >> >> So, I plan to partition my fact table by organizational unit and by year. >> But I have 180 units. >> >> Does the partitioning will works fine with 180 * 5 years = 900 tables? >> >> on this article: >> http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm >> there is a note that the maximum number of tables is 256. >> But the performance gain can be very high! >> >> The estimated size of the DW is 10gb / year (maybe more with additional >> indexes) >> My DTS package is ready to support partioning table loading. (the package >> automatically create the new table, indexes and update the view for each >> detected partition in the staging source table) >> >> My users access my reportserver interactively, scheduling anything is not >> an option. >> For higher analysis, my olap cubes are ready. >> >> thanks for your feedback. >> >> Jerome. >> >> >> > >
Jerome,
Views are based on select statements. A select statement can have a max of 256 tables in the From clause. So the answer to your specific question is no. Danny [quoted text, click to view] "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message news:e1ghnJD0EHA.2624@TK2MSFTNGP11.phx.gbl... > but does SQL Server support 500 tables in a partitioned view? > I don't need help about this, I just want to plan correctly. > and I don't want month partition but only year + organizational unit > Because I don't do sum or count aggregation in SQL (or just a little sum > compared of what my cube provide). > Generally my reports contains SQL statement like this: > * Last kown value at a specific date (subquery required) (by customer) > * events which start before and end after a specific date (between > statement) (by customer) > > Its the reason of not using monthly partitions, because I never know when > these dates appear in the database. > > "Alejo Leguizamo (MVP SQL)" <SQL@sql.sql> a écrit dans le message de news: > %23thX4VC0EHA.1204@TK2MSFTNGP10.phx.gbl... >> Hi Jerome: >> >> I have some kind of "rule". Just make a partition for each 1 GB of data. >> >> Or doing it for a year, would be great. If you need more granularity, >> give it a try to month partitioning. >> >> Tell me if you need some help on this >> >> Sincerely >> >> >> -- >> Alejandro Leguizamo >> MVP SQL Server >> Colombia >> >> "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message >> news:uwuxtzxzEHA.3336@TK2MSFTNGP11.phx.gbl... >>> Hi, >>> >>> I have to plan a datawarehouse structure where the primary reporting >>> usage is to list detailed information about customers. >>> And also, we have OLAP cube for analysis purpose. >>> >>> I'll have 100 000 customers and a fact table containing between 35 to >>> 45millions of rows by year. (we also have 2 other fact table with 5 to >>> 6millions of rows by year) >>> >>> Generally, this fact table is filtered for a particular organizational >>> unit and for data in 1 year. (for report generation using report server) >>> >>> So, I plan to partition my fact table by organizational unit and by >>> year. >>> But I have 180 units. >>> >>> Does the partitioning will works fine with 180 * 5 years = 900 tables? >>> >>> on this article: >>> http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm >>> there is a note that the maximum number of tables is 256. >>> But the performance gain can be very high! >>> >>> The estimated size of the DW is 10gb / year (maybe more with additional >>> indexes) >>> My DTS package is ready to support partioning table loading. (the >>> package automatically create the new table, indexes and update the view >>> for each detected partition in the staging source table) >>> >>> My users access my reportserver interactively, scheduling anything is >>> not an option. >>> For higher analysis, my olap cubes are ready. >>> >>> thanks for your feedback. >>> >>> Jerome. >>> >>> >>> >> >> > >
:(
sniff I think I'll do this: partition by organization for the current year and by year for the history... does this works in a partitioned view? "Danny" <istdrs@flash.net> a écrit dans le message de news: 7Lkod.23022$Rf1.15232@newssvr19.news.prodigy.com... [quoted text, click to view] > Jerome, > > Views are based on select statements. A select statement can have a max > of 256 tables in the From clause. So the answer to your specific question > is no. > > Danny > > "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message > news:e1ghnJD0EHA.2624@TK2MSFTNGP11.phx.gbl... >> but does SQL Server support 500 tables in a partitioned view? >> I don't need help about this, I just want to plan correctly. >> and I don't want month partition but only year + organizational unit >> Because I don't do sum or count aggregation in SQL (or just a little sum >> compared of what my cube provide). >> Generally my reports contains SQL statement like this: >> * Last kown value at a specific date (subquery required) (by customer) >> * events which start before and end after a specific date (between >> statement) (by customer) >> >> Its the reason of not using monthly partitions, because I never know when >> these dates appear in the database. >> >> "Alejo Leguizamo (MVP SQL)" <SQL@sql.sql> a écrit dans le message de >> news: %23thX4VC0EHA.1204@TK2MSFTNGP10.phx.gbl... >>> Hi Jerome: >>> >>> I have some kind of "rule". Just make a partition for each 1 GB of data. >>> >>> Or doing it for a year, would be great. If you need more granularity, >>> give it a try to month partitioning. >>> >>> Tell me if you need some help on this >>> >>> Sincerely >>> >>> >>> -- >>> Alejandro Leguizamo >>> MVP SQL Server >>> Colombia >>> >>> "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message >>> news:uwuxtzxzEHA.3336@TK2MSFTNGP11.phx.gbl... >>>> Hi, >>>> >>>> I have to plan a datawarehouse structure where the primary reporting >>>> usage is to list detailed information about customers. >>>> And also, we have OLAP cube for analysis purpose. >>>> >>>> I'll have 100 000 customers and a fact table containing between 35 to >>>> 45millions of rows by year. (we also have 2 other fact table with 5 to >>>> 6millions of rows by year) >>>> >>>> Generally, this fact table is filtered for a particular organizational >>>> unit and for data in 1 year. (for report generation using report >>>> server) >>>> >>>> So, I plan to partition my fact table by organizational unit and by >>>> year. >>>> But I have 180 units. >>>> >>>> Does the partitioning will works fine with 180 * 5 years = 900 tables? >>>> >>>> on this article: >>>> http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm >>>> there is a note that the maximum number of tables is 256. >>>> But the performance gain can be very high! >>>> >>>> The estimated size of the DW is 10gb / year (maybe more with additional >>>> indexes) >>>> My DTS package is ready to support partioning table loading. (the >>>> package automatically create the new table, indexes and update the view >>>> for each detected partition in the staging source table) >>>> >>>> My users access my reportserver interactively, scheduling anything is >>>> not an option. >>>> For higher analysis, my olap cubes are ready. >>>> >>>> thanks for your feedback. >>>> >>>> Jerome. >>>> >>>> >>>> >>> >>> >> >> > >
The primary key of the underlying tables in the view must be the same. For
this to work, you will have to query current and history separately with each haing its own view. Danny [quoted text, click to view] "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message news:uKoQZzP0EHA.1932@TK2MSFTNGP09.phx.gbl... > :( > sniff > > I think I'll do this: partition by organization for the current year and > by year for the history... does this works in a partitioned view? > > "Danny" <istdrs@flash.net> a écrit dans le message de news: > 7Lkod.23022$Rf1.15232@newssvr19.news.prodigy.com... >> Jerome, >> >> Views are based on select statements. A select statement can have a max >> of 256 tables in the From clause. So the answer to your specific >> question is no. >> >> Danny >> >> "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message >> news:e1ghnJD0EHA.2624@TK2MSFTNGP11.phx.gbl... >>> but does SQL Server support 500 tables in a partitioned view? >>> I don't need help about this, I just want to plan correctly. >>> and I don't want month partition but only year + organizational unit >>> Because I don't do sum or count aggregation in SQL (or just a little sum >>> compared of what my cube provide). >>> Generally my reports contains SQL statement like this: >>> * Last kown value at a specific date (subquery required) (by customer) >>> * events which start before and end after a specific date (between >>> statement) (by customer) >>> >>> Its the reason of not using monthly partitions, because I never know >>> when these dates appear in the database. >>> >>> "Alejo Leguizamo (MVP SQL)" <SQL@sql.sql> a écrit dans le message de >>> news: %23thX4VC0EHA.1204@TK2MSFTNGP10.phx.gbl... >>>> Hi Jerome: >>>> >>>> I have some kind of "rule". Just make a partition for each 1 GB of >>>> data. >>>> >>>> Or doing it for a year, would be great. If you need more granularity, >>>> give it a try to month partitioning. >>>> >>>> Tell me if you need some help on this >>>> >>>> Sincerely >>>> >>>> >>>> -- >>>> Alejandro Leguizamo >>>> MVP SQL Server >>>> Colombia >>>> >>>> "Jéjé" <willgart@BBBhotmailAAA.com> wrote in message >>>> news:uwuxtzxzEHA.3336@TK2MSFTNGP11.phx.gbl... >>>>> Hi, >>>>> >>>>> I have to plan a datawarehouse structure where the primary reporting >>>>> usage is to list detailed information about customers. >>>>> And also, we have OLAP cube for analysis purpose. >>>>> >>>>> I'll have 100 000 customers and a fact table containing between 35 to >>>>> 45millions of rows by year. (we also have 2 other fact table with 5 to >>>>> 6millions of rows by year) >>>>> >>>>> Generally, this fact table is filtered for a particular organizational >>>>> unit and for data in 1 year. (for report generation using report >>>>> server) >>>>> >>>>> So, I plan to partition my fact table by organizational unit and by >>>>> year. >>>>> But I have 180 units. >>>>> >>>>> Does the partitioning will works fine with 180 * 5 years = 900 tables? >>>>> >>>>> on this article: >>>>> http://msdn.microsoft.com/library/default.asp?url=/library/techart/partitionsindw.htm >>>>> there is a note that the maximum number of tables is 256. >>>>> But the performance gain can be very high! >>>>> >>>>> The estimated size of the DW is 10gb / year (maybe more with >>>>> additional indexes) >>>>> My DTS package is ready to support partioning table loading. (the >>>>> package automatically create the new table, indexes and update the >>>>> view for each detected partition in the staging source table) >>>>> >>>>> My users access my reportserver interactively, scheduling anything is >>>>> not an option. >>>>> For higher analysis, my olap cubes are ready. >>>>> >>>>> thanks for your feedback. >>>>> >>>>> Jerome. >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >> >> > >
Don't see what you're looking for? Try a search.
|
January 2003
March 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
| home · blog · groups | Questions? Comments? Contact the dn |