Groups | Blog | Home
all groups > dotnet windows forms databinding > july 2005 >

dotnet windows forms databinding : CSV Import


Kerr
7/22/2005 2:30:47 AM

Hi all,
Using VB.NET i have a collection of 12 fairly large (largest file is
100MB) CSV files. I have a SQL Server data structure that mimics the
structure of these csv files. I need a method in code of getting the
data from these CSV files into my SQL Server.

Slight addition to this problem is that these files are not using pure
comma delimiters. They have a custom delimiter so I need code in which I
can specify a delimiter.

Any help on this would be appreciated.

Cheers

Kerr


shriop
7/22/2005 10:35:24 AM
I need some more info. The quickest way to get the data into SQL server
is using a DTS package, which you can create ahead of time, but then
execute from VB.Net code. You can also use a CSV parser like the one I
sell, http://www.csvreader.com , to loop over each line in the CSV
files and do whatever with them, import them, convert them, etc. Both
of these methods might have issues with the custom delimiter depending
on how custom it is. Neither methods handle crazy delimiters I've seen
like <!> or !#! type stuff, so in that case, depending on your data,
you may have to run a raw conversion run over the file first before
attempting to import.

[quoted text, click to view]
Kerr
7/23/2005 6:20:45 AM
Thanks for the reply. I have tried using a DTS package but the import
process takes too long which is why I wnat to do it in code.
Unfortunately I don't have a budget for the application I am building so
purchasing your parser is not an option.
All I am looking for is a method of getting the data from these CSV
files into SQL Server.

I am hoping that once i import the CSV files into a .net component i can
then merge into SQL through code and improve my performance issues.

any articles!!

Isaac
7/24/2005 8:19:47 PM
[quoted text, click to view]

I would guess that doing it via .net and then into SQL would be even
slower than using DTS - baring in mind that DTS is designed to do
exactly this sort of thing...

Are you doing any complex processing on the DTS task (e.g. ActiveX
scripts) or is it just straight column-copying?

You could always try kicking off the DTS task from your .net app on a
AddThis Social Bookmark Button