Flat files with varying number of columns
Hi all, hoping someone can help me. I have been searching for a solution, but have not been able to make sense of what I have seen. If someone could clearly explain a solution to my issue below, I would be more than grateful. System: Visual Studio 2008 R2 64Bit Situation: Processing data from various customers, each customer flat file has the same core columns that I wish to work with, but each has there own column order and may have additional columns. For example: Customer 1: A,B,C,D 1,2,3,4 Customer 2: A,C,B,D 1,3,2,4 Customer 3: A,B,C,D,E 1,2,3,4,5 My thoughts were to use variables to identify the columns (by column name) using find string (instr) or something similar in VB Script, then using those variables open the data with a flat file connection importing all data to one column. Then use a Derived column block and create the columns I need using substring findstring formula. Although i am having difficulty with the VB script component. This may be completely the wrong way about it? Open to any suggestions.
December 21st, 2011 7:33pm

http://agilebi.com/jwelch/2007/05/08/handling-flat-files-with-varying-numbers-of-columns/ Hope this link helpful! Regards, Eshwar.
Free Windows Admin Tool Kit Click here and download it now
December 21st, 2011 11:29pm

Thanks for comment, unfortunately this manages fine for where data may be missing from a column. But it does not address the two issues I face, being, where columns in different order or additional columns are included in the file. LS
December 21st, 2011 11:33pm

Hi LS, Please refer to the following links talk about the topic: http://social.msdn.microsoft.com/forums/en-US/sqlintegrationservices/thread/53e3f59d-9442-43f9-99c5-37dd964408fc/ http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/3ef5cddb-f4c6-42f6-b301-3b434c3af0d3/ http://www.bidn.com/blogs/DonnyJohns/ssas/1999/flat-file-with-varying-column-lengths-and-number-of-columns Thanks, Eileen
Free Windows Admin Tool Kit Click here and download it now
December 28th, 2011 3:38am

Hi LS, Please refer to the following links talk about the topic: http://social.msdn.microsoft.com/forums/en-US/sqlintegrationservices/thread/53e3f59d-9442-43f9-99c5-37dd964408fc/ http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/3ef5cddb-f4c6-42f6-b301-3b434c3af0d3/ http://www.bidn.com/blogs/DonnyJohns/ssas/1999/flat-file-with-varying-column-lengths-and-number-of-columns Thanks, Eileen
December 28th, 2011 11:24am

Hi Eileen, thanks for the reply, but this does not address the problem. Surely i cannot be the only one who has multiple flat files with varying columns that need to be put through an ETL process.
Free Windows Admin Tool Kit Click here and download it now
February 16th, 2012 4:20am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics