|
|
You're in the
sql server programming
group:dts a lot of data between 2 tables
sql server programming:
Id like to dts a lot of data( around 100GB) from one table to another.. How
can I do so without blowing the Tlog ? Basically I guess I need to commit more frequently and dont know how to do so Or maybe I could use an insert select statement to do this instead of DTS But how can I commit after a few records are inserted Basically I want to insert all data to this new table from a table based on a date column So i want to select all data from a table that has date < 7/1/05 and populate the new table and insert maybe 50000 rows at a time and commit so the log does not blow up.. Can someone help ?
Assuming this statement SELECT <column list> FROM Table2 WHERE col >=
'20050701' AND col <'20050702' has say 5000 rows and I want to insert 1000 at a time and commit, then how can I do so. I understand the rowcount 1000 and doing a while loop, but how do i keep track of the 1000 rows that are inserted so as to not avoid inserting them again ? [quoted text, click to view] "Uri Dimant" <urid@iscar.co.il> wrote in message news:uNvlY52hFHA.2072@TK2MSFTNGP14.phx.gbl... > Hassan > INSERT INTO Table1 (column list) > SELECT <column list> FROM Table2 WHERE col >= '20050701' AND col <'20050702' > > This script is written by Vays > See if it helps you as well > > SET ROWCOUNT 1000 > WHILE 1 = 1 > BEGIN > UPDATE Basket > SET Inactive = 1 > WHERE ItemAddedDate < DATEADD(dd, -30, CURRENT_TIMESTAMP) AND Inactive = 0 > > IF @@ROWCOUNT = 0 > BEGIN > BREAK > END > ELSE > BEGIN > ------Or doing T-LOG Backup > CHECKPOINT > END > END > > SET ROWCOUNT 0 > > > > "Hassan" <fatima_ja@hotmail.com> wrote in message > news:OB87uZ2hFHA.500@TK2MSFTNGP09.phx.gbl... > > Id like to dts a lot of data( around 100GB) from one table to another.. > How > > can I do so without blowing the Tlog ? Basically I guess I need to commit > > more frequently and dont know how to do so > > > > Or maybe I could use an insert select statement to do this instead of DTS > > > > But how can I commit after a few records are inserted > > > > Basically I want to insert all data to this new table from a table based > on > > a date column > > > > So i want to select all data from a table that has date < 7/1/05 and > > populate the new table and insert maybe 50000 rows at a time and commit so > > the log does not blow up.. > > > > Can someone help ? > > > > > >
I guess Im not clear... :-(
[quoted text, click to view] "Uri Dimant" <urid@iscar.co.il> wrote in message news:OAUzCa3hFHA.708@TK2MSFTNGP09.phx.gbl... > Hassan > SELECT TOP 1000 clause.......... > > "Hassan" <fatima_ja@hotmail.com> wrote in message > news:OH2CR92hFHA.3300@TK2MSFTNGP15.phx.gbl... > > Assuming this statement SELECT <column list> FROM Table2 WHERE col >= > > '20050701' AND col <'20050702' has say 5000 rows and I want to insert 1000 > > at a time and commit, then how can I do so. > > > > I understand the rowcount 1000 and doing a while loop, but how do i keep > > track of the 1000 rows that are inserted so as to not avoid inserting them > > again ? > > > > > > "Uri Dimant" <urid@iscar.co.il> wrote in message > > news:uNvlY52hFHA.2072@TK2MSFTNGP14.phx.gbl... > > > Hassan > > > INSERT INTO Table1 (column list) > > > SELECT <column list> FROM Table2 WHERE col >= '20050701' AND col > > <'20050702' > > > > > > This script is written by Vays > > > See if it helps you as well > > > > > > SET ROWCOUNT 1000 > > > WHILE 1 = 1 > > > BEGIN > > > UPDATE Basket > > > SET Inactive = 1 > > > WHERE ItemAddedDate < DATEADD(dd, -30, CURRENT_TIMESTAMP) AND Inactive > = > > 0 > > > > > > IF @@ROWCOUNT = 0 > > > BEGIN > > > BREAK > > > END > > > ELSE > > > BEGIN > > > ------Or doing T-LOG Backup > > > CHECKPOINT > > > END > > > END > > > > > > SET ROWCOUNT 0 > > > > > > > > > > > > "Hassan" <fatima_ja@hotmail.com> wrote in message > > > news:OB87uZ2hFHA.500@TK2MSFTNGP09.phx.gbl... > > > > Id like to dts a lot of data( around 100GB) from one table to > another.. > > > How > > > > can I do so without blowing the Tlog ? Basically I guess I need to > > commit > > > > more frequently and dont know how to do so > > > > > > > > Or maybe I could use an insert select statement to do this instead of > > DTS > > > > > > > > But how can I commit after a few records are inserted > > > > > > > > Basically I want to insert all data to this new table from a table > based > > > on > > > > a date column > > > > > > > > So i want to select all data from a table that has date < 7/1/05 and > > > > populate the new table and insert maybe 50000 rows at a time and > commit > > so > > > > the log does not blow up.. > > > > > > > > Can someone help ? > > > > > > > > > > > > > > > > > > > >
Hassan
INSERT INTO Table1 (column list) SELECT <column list> FROM Table2 WHERE col >= '20050701' AND col <'20050702' This script is written by Vays See if it helps you as well SET ROWCOUNT 1000 WHILE 1 = 1 BEGIN UPDATE Basket SET Inactive = 1 WHERE ItemAddedDate < DATEADD(dd, -30, CURRENT_TIMESTAMP) AND Inactive = 0 IF @@ROWCOUNT = 0 BEGIN BREAK END ELSE BEGIN ------Or doing T-LOG Backup CHECKPOINT END END SET ROWCOUNT 0 [quoted text, click to view] "Hassan" <fatima_ja@hotmail.com> wrote in message news:OB87uZ2hFHA.500@TK2MSFTNGP09.phx.gbl... > Id like to dts a lot of data( around 100GB) from one table to another.. How > can I do so without blowing the Tlog ? Basically I guess I need to commit > more frequently and dont know how to do so > > Or maybe I could use an insert select statement to do this instead of DTS > > But how can I commit after a few records are inserted > > Basically I want to insert all data to this new table from a table based on > a date column > > So i want to select all data from a table that has date < 7/1/05 and > populate the new table and insert maybe 50000 rows at a time and commit so > the log does not blow up.. > > Can someone help ? > >
Hassan
SELECT TOP 1000 clause.......... [quoted text, click to view] "Hassan" <fatima_ja@hotmail.com> wrote in message news:OH2CR92hFHA.3300@TK2MSFTNGP15.phx.gbl... > Assuming this statement SELECT <column list> FROM Table2 WHERE col >= > '20050701' AND col <'20050702' has say 5000 rows and I want to insert 1000 > at a time and commit, then how can I do so. > > I understand the rowcount 1000 and doing a while loop, but how do i keep > track of the 1000 rows that are inserted so as to not avoid inserting them > again ? > > > "Uri Dimant" <urid@iscar.co.il> wrote in message > news:uNvlY52hFHA.2072@TK2MSFTNGP14.phx.gbl... > > Hassan > > INSERT INTO Table1 (column list) > > SELECT <column list> FROM Table2 WHERE col >= '20050701' AND col > <'20050702' > > > > This script is written by Vays > > See if it helps you as well > > > > SET ROWCOUNT 1000 > > WHILE 1 = 1 > > BEGIN > > UPDATE Basket > > SET Inactive = 1 > > WHERE ItemAddedDate < DATEADD(dd, -30, CURRENT_TIMESTAMP) AND Inactive = > 0 > > > > IF @@ROWCOUNT = 0 > > BEGIN > > BREAK > > END > > ELSE > > BEGIN > > ------Or doing T-LOG Backup > > CHECKPOINT > > END > > END > > > > SET ROWCOUNT 0 > > > > > > > > "Hassan" <fatima_ja@hotmail.com> wrote in message > > news:OB87uZ2hFHA.500@TK2MSFTNGP09.phx.gbl... > > > Id like to dts a lot of data( around 100GB) from one table to another.. > > How > > > can I do so without blowing the Tlog ? Basically I guess I need to > commit > > > more frequently and dont know how to do so > > > > > > Or maybe I could use an insert select statement to do this instead of > DTS > > > > > > But how can I commit after a few records are inserted > > > > > > Basically I want to insert all data to this new table from a table based > > on > > > a date column > > > > > > So i want to select all data from a table that has date < 7/1/05 and > > > populate the new table and insert maybe 50000 rows at a time and commit > so > > > the log does not blow up.. > > > > > > Can someone help ? > > > > > > > > > > > >
Its OK:-)
[quoted text, click to view] "Hassan" <fatima_ja@hotmail.com> wrote in message news:OxRh6c3hFHA.2852@TK2MSFTNGP15.phx.gbl... > I guess Im not clear... :-( > > "Uri Dimant" <urid@iscar.co.il> wrote in message > news:OAUzCa3hFHA.708@TK2MSFTNGP09.phx.gbl... > > Hassan > > SELECT TOP 1000 clause.......... > > > > "Hassan" <fatima_ja@hotmail.com> wrote in message > > news:OH2CR92hFHA.3300@TK2MSFTNGP15.phx.gbl... > > > Assuming this statement SELECT <column list> FROM Table2 WHERE col >= > > > '20050701' AND col <'20050702' has say 5000 rows and I want to insert > 1000 > > > at a time and commit, then how can I do so. > > > > > > I understand the rowcount 1000 and doing a while loop, but how do i keep > > > track of the 1000 rows that are inserted so as to not avoid inserting > them > > > again ? > > > > > > > > > "Uri Dimant" <urid@iscar.co.il> wrote in message > > > news:uNvlY52hFHA.2072@TK2MSFTNGP14.phx.gbl... > > > > Hassan > > > > INSERT INTO Table1 (column list) > > > > SELECT <column list> FROM Table2 WHERE col >= '20050701' AND col > > > <'20050702' > > > > > > > > This script is written by Vays > > > > See if it helps you as well > > > > > > > > SET ROWCOUNT 1000 > > > > WHILE 1 = 1 > > > > BEGIN > > > > UPDATE Basket > > > > SET Inactive = 1 > > > > WHERE ItemAddedDate < DATEADD(dd, -30, CURRENT_TIMESTAMP) AND > Inactive > > = > > > 0 > > > > > > > > IF @@ROWCOUNT = 0 > > > > BEGIN > > > > BREAK > > > > END > > > > ELSE > > > > BEGIN > > > > ------Or doing T-LOG Backup > > > > CHECKPOINT > > > > END > > > > END > > > > > > > > SET ROWCOUNT 0 > > > > > > > > > > > > > > > > "Hassan" <fatima_ja@hotmail.com> wrote in message > > > > news:OB87uZ2hFHA.500@TK2MSFTNGP09.phx.gbl... > > > > > Id like to dts a lot of data( around 100GB) from one table to > > another.. > > > > How > > > > > can I do so without blowing the Tlog ? Basically I guess I need to > > > commit > > > > > more frequently and dont know how to do so > > > > > > > > > > Or maybe I could use an insert select statement to do this instead > of > > > DTS > > > > > > > > > > But how can I commit after a few records are inserted > > > > > > > > > > Basically I want to insert all data to this new table from a table > > based > > > > on > > > > > a date column > > > > > > > > > > So i want to select all data from a table that has date < 7/1/05 and > > > > > populate the new table and insert maybe 50000 rows at a time and > > commit > > > so > > > > > the log does not blow up.. > > > > > > > > > > Can someone help ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
use select into to create the table with the data in one statement. That way
you'll get speed and very little in the transaction log. Think of the transaction log as a recording of all the SQL Server has to do to roll back a transaction. The less granular the statement, the less that goes into the transaction log. With select into all that is required to rollback would be drop table. Insert statement require loads of logging to guareentee a safe rollback facility. Once you done the select into you can create all your constraints and indexes which is again very fast due to minimal transaction logging. regards, Mark Baekdal http://www.dbghost.com http://www.innovartis.co.uk +44 (0)208 241 1762 Build, Comparison and Synchronization from Source Control = Database change management for SQL Server [quoted text, click to view] "Hassan" wrote:
> Id like to dts a lot of data( around 100GB) from one table to another.. How > can I do so without blowing the Tlog ? Basically I guess I need to commit > more frequently and dont know how to do so > > Or maybe I could use an insert select statement to do this instead of DTS > > But how can I commit after a few records are inserted > > Basically I want to insert all data to this new table from a table based on > a date column > > So i want to select all data from a table that has date < 7/1/05 and > populate the new table and insert maybe 50000 rows at a time and commit so > the log does not blow up.. > > Can someone help ? > >
Don't see what you're looking for? Try a search.
|
January 2003
March 2003
April 2003
May 2003
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
April 2008
| home · blog · groups | Questions? Comments? Contact the dn |