Load only updated rows in sql
Hi, Lets say i have 3 spreadsheets loaded to 3 sql tables. These spreadsheets will be loaded monthly in sql tables . now in next run lets say for spreadsheet1 the data is same as previous load and just few rows are updated in it. now how do i load only these updated rows in sql using SSIS? the data in spreadheet2 and 3 is same so i am not calling child packages for 2 and 3. Does anybody happen to know any article on that or any idea? Thanks
September 27th, 2011 4:55am

How many columns are there in each table? And how many rows. If the number of rows/Columns are less you can directly do a merge join on all the columns to get the records that are different(use conditional split to get this).My Blog | Ask Me
Free Windows Admin Tool Kit Click here and download it now
September 27th, 2011 6:37am

If some rows will be added in next run,and not update any row from last run,you can use SSIS to skip the rows from last run. First ,count the row number in the table,and map it to a variable. Second,Transfer the variable's value to system variable,skip row number. Just a thought,you can test it .
September 27th, 2011 6:50am

If some rows will be added in next run,and not update any row from last run,you can use SSIS to skip the rows from last run. Nice idea but how can you be sure that the already existing rows have not been updated?My Blog | Ask Me
Free Windows Admin Tool Kit Click here and download it now
September 27th, 2011 7:07am

I only suppose no update in the already existing rows. If some rows were updated,and there were not any mark on the rows,all the records should be imported and compared.
September 27th, 2011 8:01am

You can use SCD type 1 for this case. The purpose of SCD is for DWH. But, you can use for your need. For instance, Step 1: Upload excel data in to a stage table Step 2: use SCD component connect with Stage table and the Dimension table as your original source table. Step 3: Use SCD type 1. It will update the changes without recording history. or Use SQL statements like MERGEThanks Ayyappan Thangaraj, http://SQLServerRider.wordpress.com
Free Windows Admin Tool Kit Click here and download it now
September 27th, 2011 8:39am

i found this nice article by Jamie Thomson. http://www.sqlis.com/sqlis/post/Get-all-from-Table-A-that-isnt-in-Table-B.aspx Also, if there would be no change in any of the spreadsheet i would have an expression between foreach loop container and an email task for @FileName == "" coz when the file would be loaded first time i would move that file to some other location. So no file no loading. I just want to figure out something else now. When i am getting records into my table i am checking for Validations like if the record length is greater than the one i have then move that record to an error table. but i dont want to load same records again and again. For example if i have got just 2 new records in my spreadsheet i just want to apply validation on those two records and not all the records from spreadsheet as i have already checked those records in my first run. Does anybody have any idea about it? Thanks
September 27th, 2011 4:41pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics