Flat File Reader Dropping Rows
I have a CSV file with 2246 rows in it. When I try to read it with a Flat File Source component, it only reads 2239 rows. I've searched the discussion threads and all opinions say the data must be bad, so I set out to validate the data. I wrote a Java program that reads the file byte by byte and I have validated the following: 1) there are 2246 rows 2) each row ends with a <CR><LF> 3) each row has 14 commas in it (separating the 15 fields) 5) other than the <CR>, <LF>, and commas, all other characters are either digits, upper-case letters, lower-case letters, or periods. Here are some other facts: - The SSIS log says "[SSIS.Pipeline] Information: "component "OLE DB Destination" (159)" wrote 2246 rows.", but it truly only wrote 2239. - The package includes a flat file source, derived column transformer, data conversion to convert the character set of two columns, and an Ole DB destinationJames Laskowski
February 28th, 2011 1:20pm

What the data looks like on row 2240 vs 2239? Perhaps one of the fields looks different and you get nulls in the destination (I cannot believe the stat message lies).Arthur My Blog
Free Windows Admin Tool Kit Click here and download it now
February 28th, 2011 1:25pm

Hello, Might the records after line 2239 does not match with records in top rows(number of column, extract columns or sometime text has "," in it and that make it different from other records) and those records will be ignored. Copy those records into new file and try to load and see if there is something strange in data. ThanksAamir
February 28th, 2011 3:42pm

Are the last rows missing or somewhere in the middle? Try to open it in a good/advanced textfile editor (notepad++ or something like that) or in Microsoft Excel to see if there are some strange chars, extra seperators or qualifiers? Please mark the post as answered if it answers your question | My SSIS Blog: http://microsoft-ssis.blogspot.com
Free Windows Admin Tool Kit Click here and download it now
February 28th, 2011 4:24pm

I apologize, gents. This was a programming issue after all. The Data Flow Task was indeed reading and inserting all rows, but there was another component deleting certain rows based on business rules later on in the package.James Laskowski
March 1st, 2011 7:24am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics