| Groups | Blog | Home |
dotnet xml : Flat File To XML Roundtrip
OK I'm on a steep learning curve with XML et.al. and need some advice.
I'm writing a B2B front door for a new application. I have multiple data suppliers all sending various formats of flat files. Most of the inbound files are CSV (but a couple are tab-delimitted and one is an Excel spreadsheet). The goal is to read/process/validate each file and to "transform" the data into a common, internal, file format which will by sent on to a downstream application for final processing and consumption. I'm using VB.Net 2003 as my development platform. My novice approach to do this will be to: 1) Convert each inbound file to XML (hopfully using some template-driven parser). I'd like to maintain x number of file-descriptor templates but use common code to convert the flat files. 2) Use XSLT to transform the new inbound XML file into the output target file format 3) Load the XSLT-transformed file (now having the desired outbound XML format) into a DataSet (for validation and processing) 4) Write/Convert the validated contents of the DataSet to the output flat file. My Question is... Does this sound like a good approach? and are there any good how-to examples which address these steps? I need to get the job done but I'm also wanting to use this as a good training exercise in .Net and XML.
In there user samples on www.gotdotnet.com you will find an XmlCsvReader
which may be of some help to you for step (1). Converting the DataSet back to XML is easy, then from XML back to CSV can be done with an XSL transform, which I think is included in teh XmlCsvReader download. So yes, this approach makes sense. A lot of folks are standardizing on XML for this kind of processing. Hopefully eventually, you'll be able to convince your business partner to just send you XML at some point in the future, and then you're processing will get a bit more streamlined. [quoted text, click to view] "Glenn Owens" <gowens@nixonpeabody.com> wrote in message news:bc4291aa.0406180532.749a23d2@posting.google.com... > OK I'm on a steep learning curve with XML et.al. and need some advice. > > I'm writing a B2B front door for a new application. I have multiple > data suppliers all sending various formats of flat files. Most of the > inbound files are CSV (but a couple are tab-delimitted and one is an > Excel spreadsheet). The goal is to read/process/validate each file and > to "transform" the data into a common, internal, file format which > will by sent on to a downstream application for final processing and > consumption. I'm using VB.Net 2003 as my development platform. > > My novice approach to do this will be to: > > 1) Convert each inbound file to XML (hopfully using some > template-driven parser). I'd like to maintain x number of > file-descriptor templates but use common code to convert the flat > files. > > 2) Use XSLT to transform the new inbound XML file into the output > target file format > > 3) Load the XSLT-transformed file (now having the desired outbound XML > format) into a DataSet (for validation and processing) > > 4) Write/Convert the validated contents of the DataSet to the output > flat file. > > My Question is... > > Does this sound like a good approach? and are there any good how-to > examples which address these steps? I need to get the job done but I'm > also wanting to use this as a good training exercise in .Net and XML. > > Thanks in advance for your input!
[quoted text, click to view]
"Chris Lovett" <chris@!nospam!.net> wrote in message news:<10d7s2o3f8pvtba@corp.supernews.com>... > In there user samples on www.gotdotnet.com you will find an XmlCsvReader > which may be of some help to you for step (1). > > Converting the DataSet back to XML is easy, then from XML back to CSV can be > done with an XSL transform, which I think is included in teh XmlCsvReader > download. > > So yes, this approach makes sense. A lot of folks are standardizing on XML > for this kind of processing. Hopefully eventually, you'll be able to > convince your business partner to just send you XML at some point in the > future, and then you're processing will get a bit more streamlined. > Chris, thanks for the response. I am currently looking at the XmlCsvReader - it definitely looks like something that I could use. Thanks for your hard work and pointing me in this direction. Can it be modified? (it does appear to be copywritten)... Is there a way to use an .XSD file to describe/interpret the column tags instead of using the first line of the file to define the column names (tags)? This would allow me to fully describe any of the input files, using separate schemas, without needing to modify them before the initial XML conversion. The added bonus would be to use the .XSD files to validate the structure, datatype, cardinality, etc. - could the XmlCsvReader descend from XmlValidatingReader? Also, I came across a really good MSDN link article on customizing the XmlReader: http://msdn.microsoft.com/webservices/building/xmldevelopment/api/default.aspx?pull=/library/en-us/dndotnet/html/custxmlread.asp You are correct about the furture of our B2B. This is just the first step before we "potentially" begin requesting each of our data vendors to supply well-formed XML input streams (preferably via a web service). But, we needed to get this part done reasonably quickly. Thanks again,
The eula is very open. Feel free to use it in your product and make
whatever changes necessary. You could read the XSD and then pass in the column names: XmlCsvReader reader = new XmlCsvReader(uri, nameTable); reader.ColumnNames = new string[] { "foo", "bar", "etc" }; .... The problem is in .NET v1.1 you can't wrap this reader to the XmlValidatingReader. But you could write out the XML, then read it using XmlTextReader, and then the XmlValidatingReader can work over that. [quoted text, click to view] "Glenn Owens" <gowens@nixonpeabody.com> wrote in message news:bc4291aa.0406210923.7e4e2a96@posting.google.com... > "Chris Lovett" <chris@!nospam!.net> wrote in message news:<10d7s2o3f8pvtba@corp.supernews.com>... > > In there user samples on www.gotdotnet.com you will find an XmlCsvReader > > which may be of some help to you for step (1). > > > > Converting the DataSet back to XML is easy, then from XML back to CSV can be > > done with an XSL transform, which I think is included in teh XmlCsvReader > > download. > > > > So yes, this approach makes sense. A lot of folks are standardizing on XML > > for this kind of processing. Hopefully eventually, you'll be able to > > convince your business partner to just send you XML at some point in the > > future, and then you're processing will get a bit more streamlined. > > > > > Chris, thanks for the response. I am currently looking at the > XmlCsvReader - it definitely looks like something that I could use. > Thanks for your hard work and pointing me in this direction. > > Can it be modified? (it does appear to be copywritten)... > > Is there a way to use an .XSD file to describe/interpret the column > tags instead of using the first line of the file to define the column > names (tags)? This would allow me to fully describe any of the input > files, using separate schemas, without needing to modify them before > the initial XML conversion. The added bonus would be to use the .XSD > files to validate the structure, datatype, cardinality, etc. - could > the XmlCsvReader descend from XmlValidatingReader? > > Also, I came across a really good MSDN link article on customizing the > XmlReader: > > http://msdn.microsoft.com/webservices/building/xmldevelopment/api/default.aspx?pull=/library/en-us/dndotnet/html/custxmlread.asp > > You are correct about the furture of our B2B. This is just the first > step before we "potentially" begin requesting each of our data vendors > to supply well-formed XML input streams (preferably via a web > service). But, we needed to get this part done reasonably quickly. > > Thanks again, > Glenn
Don't see what you're looking for? Try a search.
|
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
| home · blog · groups | Questions? Comments? Contact the dn |