Groups | Blog | Home
all groups > dotnet performance > june 2005 >

dotnet performance : inserting rows into a DB


Kenny M.
6/8/2005 6:31:14 AM
I have heard that the most time consuming task into a DB is the when we use
the Insert Statatement,

Is that true?

Ok I have an application that sends from 10 to 500 rows to the DB, those
rows are compared again a fixed value and then they are inserted to the DB if
the comparison is ok.

I would like to do all that inside the DB (a SP) but I don’t know how, so I
created a WebService to do that, my point is that I’m not inserting row by
row instead I hold the rows in a Dataset (memory) and later I make one big
insert to the DB.

My question is should I leave that code like this or should go to the DB in
order to insert row by row to gain some performance?

Can you think a better way to do that, because that is the heart of my app
and I want to minimize the time it takes.

Thnks.

--
Robbe Morris [C# MVP]
6/9/2005 10:07:45 PM
The DataSet still sends these one row at a time by
creating the appropriate InsertCommand,UpdateCommand, or
DeleteCommand.

The DataSet not being run against a stored procedure is
probably a little slower. Would it drastically impact
your app? Depends on how many columns you
are updating and various other items.

You may want to consider retrieving the comparison
values first, then deleting rows from the DataSet
that don't comply, then .Update your DataSet.



--
2004 and 2005 Microsoft MVP C#
Robbe Morris
http://www.masterado.net

Earn $$$ money answering .NET Framework
messageboard posts at EggHeadCafe.com.
http://www.eggheadcafe.com/forums/merit.asp



[quoted text, click to view]

Kenny M.
6/10/2005 5:37:08 AM
Well I first make the comparison and if that is ok I copy the row into the
Data Set

My question is:

Should I send the row to the SP to insert at this time? I mean one y One?

The row has 6 column to be saved, and I 'm looking the faster performance,
millseconds are important to me.

thks
--
Kenny M.


[quoted text, click to view]
Robbe Morris [C# MVP]
6/12/2005 5:46:50 PM
I suspect that you may be better off not using a DataSet at all.
Perhaps an ArrayList of your own data classes. An array
if you know the exact amount of records.

Theoretically, the one row at a time calling the stored
procedure should be faster. The DataSet is eventually
going send each row one at a time anyway.

--
2004 and 2005 Microsoft MVP C#
Robbe Morris
http://www.masterado.net

Earn $$$ money answering .NET Framework
messageboard posts at EggHeadCafe.com.
http://www.eggheadcafe.com/forums/merit.asp



[quoted text, click to view]

Rafal Gwizdala
6/13/2005 10:33:05 AM
There's quite a nice technique you can use: build a batch of insert commands
and execute them all at once. If the number of rows is small (hundred or
two) you will make only one roundtrip to SQL server and will probably get
the best performance.

[quoted text, click to view]

Travis
6/17/2005 1:43:02 PM
I've used XML to handle tasks such as this (and parent/child operations)

In your .NET code, build up an XML string:
"<items><item><name>Bob</name></item><item><name>Sam</name></item></items>"

(you can use the string builder class OR the xml classes)

In your stored proc, pass in the XML

CREATE PROCEDURE spTest
@xml ntext
AS

DECLARE @hdoc int;
DECLARE @table TABLE (firstName nvarchar(50));

exec sp_xml_preparedocument @hdoc OUTPUT, @xml

INSERT INTO @table
SELECT [name] FROM OPENXML(@hdoc, '/items/item', 2) WITH ([name] nvarchar(50))

SELECT * FROM @table;

exec sp_xml_removedocument @hdoc


Not sure how this would perform against multiple calls to the DB, but would
be worth a test...

Mike Swaim
7/13/2005 2:04:59 PM
[quoted text, click to view]

Often yes.
If you are using an Oracle database, and the Oracle .net client, it
supports array arguments for SQL statements. So you can issue 1 insert
statement with arrays as the argument. This can be a fairly big win.
Another option, which someone else mentioned, is to concatenate SQL
strings. This'll cut down on the network traffic, with a possible
increase in time spent executing the commands.
Another option might be to spin the SQL into another thread. Most of
the time spent executing queries is spent waiting, so if you could be
doing something else while the SQL's executing, that could be a decent
solution. I've gotten pretty good speedups on code with 5+ threads
doing parallel inserts with separate DB connections. (The server was
pretty hefty, though. YMMV.)


--
Mike Swaim swaim@hal-pc.org at home | Quote: "Boingie"^4 Y,W & D
MD Anderson Dept. of Biostatistics & Applied Mathematics
mpswaim@mdanderson.org or mswaim@odin.mdacc.tmc.edu at work
AddThis Social Bookmark Button