Multiple Flat Files Connection manager - unexpected behavior when using wildcard
Hi
I want to import multiple text files into a destination file. This works fine when I point the connection manager directly to the 3 files that I need to import. However, these 3 files will change and sometimes there may be more or less files. Therefore
I planned to use a wildcard (*). However, when I use a wildcard in the connection manager I
get unexpected results - some of the records get duplicated in the output. I have demonstrated this issue in this
short video clip. Apologies for
the slight buzz / hum on the audio (you can just turn the audio off).
Any help or advice would be appreciated.
Thanks
Steve
November 28th, 2010 3:53am
Hi Steave,
Can you please check the File Name Textbox?
Is should be set as: "C:\myFolder\*.txt"
and not as : "C:\myFolder\File1.txt|C:\myFolder\File2.txt|C:\myFolder\File3.txt"Vishal Gajjar MCITP.DBA \ MCITP.DatabaseDeveloper
Free Windows Admin Tool Kit Click here and download it now
November 28th, 2010 5:13am
Hi Steave,
Can you please check the File Name Textbox?
Is should be set as: "C:\myFolder\*.txt"
and not as : "C:\myFolder\File1.txt|C:\myFolder\File2.txt|C:\myFolder\File3.txt"
from your video it seems that you have set this property as: "C:\myFolder\File1.txt|C:\myFolder\File2.txt|C:\myFolder\*.txt"
this means that:
1. Records from File1 will be loaded, - 3 Records
2. Records from File2 will be loaded, - 3 Records
3. Records from *.txt will be loaded,
3.1.
Records from File1 will be loaded, - 3 Records
3.2.
Records from File2 will be loaded, - 3 Records
3.3.
Records from File3 will be loaded, - 3 Records
that why you are getting 15 records.
Vishal Gajjar MCITP.DBA \ MCITP.DatabaseDeveloper
November 28th, 2010 5:14am
Hi Steave,
Can you please check the File Name Textbox?
Is should be set as: "C:\myFolder\*.txt"
and not as : "C:\myFolder\File1.txt|C:\myFolder\File2.txt|C:\myFolder\File3.txt"
from your video it seems that you have set this property as: "C:\myFolder\File1.txt|C:\myFolder\File2.txt|C:\myFolder\*.txt"
this means that:
1. Records from File1 will be loaded, - 3 Records
2. Records from File2 will be loaded, - 3 Records
3. Records from *.txt will be loaded,
3.1.
Records from File1 will be loaded, - 3 Records
3.2.
Records from File2 will be loaded, - 3 Records
3.3.
Records from File3 will be loaded, - 3 Records
that why you are getting 15 records.
Vishal Gajjar MCITP.DBA \ MCITP.DatabaseDeveloper
Free Windows Admin Tool Kit Click here and download it now
November 28th, 2010 5:14am
Hi Vishal,
That solved it- I had totally missed that there were pipes in the file reference. Thanks for the insight.
Cheers
Steve
November 28th, 2010 1:00pm