Groups | Blog | Home
all groups > sql server dts > july 2007 >

sql server dts : SSIS: import only missing (appended) records



Bob Yang
7/2/2007 3:57:13 PM
Hi,

does anyone know whether there is an option in SSIS to insert only
difference? For example, I have an appended txt file to insert. I have
inserted the first part of data a while ago so I only want to insert
the new records this time.

thank you!
Allan Mitchell
7/3/2007 12:00:00 AM
Hello Bob,

You need to handle this yourself and the usual way to do this would be to
either use a Lookup transform or a Merge Join transform. You can still choose
to handle this the ELT way and stage the rows from the flat file and then
do a SET based INSERT afterwards but the transforms are the cuter way to go.

--

Allan Mitchell
http://wiki.sqlis.com | http://www.sqlis.com | http://www.sqldts.com |
http://www.konesans.com

[quoted text, click to view]

Bob Yang
7/3/2007 4:53:27 PM
Allan,


thank you!

here is what I got from your message, correct me if i am wrong. thank
you!

1. use Lookup and save those new records (cannot find records) to a
flat file. (There is no PKey in my flat file, I will test if SSIS can
lookup for all rows/cells)
2. insert the data in the flat file to the SQL database. (as a SET)

do you know there is a better way than this? I just wonder if the
records keep on increasing, then it will be slower and slower.


thank you!


[quoted text, click to view]

Allan Mitchell
7/4/2007 7:07:30 AM
Hello Bob,


For the rows you have not seen before you would not stage. You would use
an OLE DB Destination adapter stright into SQL. You shouldn't really stage
to text file either. The recommended way would be to use a Raw File to do
this as it is quicker.



--

Allan Mitchell
http://wiki.sqlis.com | http://www.sqlis.com | http://www.sqldts.com |
http://www.konesans.com

[quoted text, click to view]

AddThis Social Bookmark Button