estimate ETL Data Size
Hello
wanted to find out what the best way to calculate or estimate data size of etl proces
Background info
need to determine what spesification of link will be need for an internet based connection (vpn) to a datasource ussing ssis. is there a best way to estimate the needed link or will it just be a resonable guess based on size of current datasource and amount
of data to be extracted
Thank a lot any help or advice will be aprreciated
Regards
Bidev1
June 22nd, 2011 2:00pm
not sure of the use of the term "link" in this instance. "specification of link" ? "An internet based connection (vpn) to a datasource" is nothing more really than using the Internet to form a "virtual private network" by creating a secure and dedicated
connection across the web from two computers or servers that are not phylically connected inside the same domain, building, etc. It makes it seem like it's right next door. "estimate the needed link"?
Help us understand your question better.Todd C - MSCTS SQL Server 2005 - Please mark posts as answered where appropriate.
Free Windows Admin Tool Kit Click here and download it now
June 22nd, 2011 3:03pm
You could probably use the Data Profiling Task (SSIS 2008) to get an understanding of the date being processed in the ETL process. You could then create estimates of data size
http://www.simple-talk.com/sql/ssis/sql-server-2008--ssis-data-profiling-task/
Jeff Wharton
MSysDev (C.Sturt), MDbDsgnMgt (C.Sturt) MCT, MCPD, MCITP, MCDBA
Blog: Mr. Wharty's Ramblings
Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
June 22nd, 2011 3:40pm
not sure of the use of the term "link" in this instance. "specification of link" ? "An internet based connection (vpn) to a datasource" is nothing more really than using the Internet to form a "virtual private network" by creating a secure and dedicated
connection across the web from two computers or servers that are not phylically connected inside the same domain, building, etc. It makes it seem like it's right next door. "estimate the needed link"?
Help us understand your question better.
Todd C - MSCTS SQL Server 2005 - Please mark posts as answered where appropriate.
Basically all am after is best way to determine what the pipe size (speed) between the source system and Dw ,should be :
based on the data measuremnt of extracted data,
a resonable guess based on size of current datasource and amount of data to be extracted,
Free Windows Admin Tool Kit Click here and download it now
June 22nd, 2011 4:21pm
If this is VPN which is a secure communications tunnel over the public internet your effort is redundant because the "pipe" throughput" is not in your control.
Another thing: the package is going to establish the VPN conn, is it? This is strange, when VPN is connecting a user password is used that changes periodically.Arthur My Blog
June 22nd, 2011 5:57pm