dynamically extract and load new columns or calculations into destination
You could use the SSIS API to create packages based on the metadata read from the flat files.
February 18th, 2011 11:50pm

Thanks, Looks like there is a whole realm of programming in SSIS out there which I heard couple of times and did not know much about it. I started reading the MSDN and will see how this goes.DW Guy
Free Windows Admin Tool Kit Click here and download it now
February 18th, 2011 11:57pm

All, I am looking for suggestions on this topic. I am thinking of moving away from SSIS to accomplish this task but want to check with Gurus before I do that. I have a project where I get flat files with some column (Natural Key, Col1, Col2.... column100). Every 2 or 3 months the source system gets upgraded to new version and most of the cases we end up adding some new columns to the source files and some existing calculation involving above pegs get altered slightly and new calcs may get added. Current process(SQL Server + SSIS) is written in such a way all the source column are directly written to one table(RawPeg table) and the all the calcs(Calc Table) involved are written in a different table. So every 2 or 3 months we end up doing a version upgrade project where we add the new column and edit existing calculation or add new calculation. Now the users want to a get away with this version upgrade projects and wants to build more flexibility in the code. He wants to have a config files where if he enter new column into the config files those column need to be automatically loaded into destintion DB tables. Also if he edits the formula or add new formula in the config files those changes have to be reflected automatically, essentially automating the whole version upgrade process. I am aware of Data flow task's inabiliy of dynamicity, so I am planning on using scripting languages and probably bulk data loaders. I want to confirm with some Pundits if they have some other opinions. DW Guy
February 19th, 2011 12:38am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics