CSV file synchronization
hi, My CSV file contains the 23,000 user data.Whether FIM can able to Synchronize all the data in one time.or I need to split the data .If it ,how many thousand records will synchronized ?.
July 28th, 2010 10:37am

FIM can synchronise all records from the CSV directly into the Sync Engine very quickly using the delimited text file MA, although how quickly would depend on your hardware and software configuration. I have always staged CSV data in SQL Server first using SSIS then used the SQL Server MA to transition identity data into the Sync Engine.
Free Windows Admin Tool Kit Click here and download it now
July 28th, 2010 11:41am

Hi There, I've always just used the "Delimited Text File" MA for these types of files. As MMS_Guru mentioned there are some out there who will stage the data into a SQL table however, I generally don't do that. The MA itself will work fine as it is and export the delimited files back in the same format as they came in. Using the database does allow you to query the data more readily if you're looking for certain conditions as well as customize the output by using a different export format than was used for the input format. That said, the database component adds another layer to the synchronization process and if you really wanted to, you can reformat the export of the file by using a standard Extensible Management Agent and writing the code to do the customization. Thanks Blain
July 28th, 2010 3:48pm

There are some interesting advantages when using an SQL MA and SSIS: - you can export your full data collection (ILM/FIMSync natively only exports deltas) - it's easier / faster to support delta imports With a file MA you must be very careful with data input for full import and delta import run profiles. (For example: delta input file to full import might delete a lot of data). Also you need to pay attention to confirming imports when exporting data. This is much more flexible with a SQL table or view... IMHO. But it also depends on the complexity of you data and the amount of updates (per run cycle). These posts handle the same topic: - Synchronizing SQL Server Objects to AD (FYI data origin: csv dump) - Extensible Agent full Export - Export all objects of an MA to file - File MA: export overwrites everything If you need more: searching the ILM forum and FIM forum for SSIS will return interesting results. HTH, Peter Peter Geelen - Sr. Consultant IDA (http://www.fim2010.be) [If a post helps to resolve your issue, please click the "Mark as Answer" of that post or "Helpful" button of that post. By marking a post as Answered or Helpful, you help others find the answer faster.]
Free Windows Admin Tool Kit Click here and download it now
July 28th, 2010 5:57pm

Hi Peter, I think there are a couple things at play here which are pretty important to consider: 1. Is there ever a requirement to do a full data export? 2. Even with the SSIS package, the import of the file has to be handled carefully and set up as to not remove any data that isn't present in the file (in case they do generate a "delta import" file). 3. With a user count of 23,000 users, is the trade off for being able to potentially set up the delta imports (with the change table) a valid concern or can you process just as effectively doing the full import with a delta synchronization? 4. Is the additional layer of technology really something that you want to be supporting for the long term? I admittedly will always go to the lowest common denominator when it comes to a deployment because I'm a true believer in the KISS principle because when I leave, the client has to be able to understand and support it. If the requirements don't drive the additive layer then why would you want to implement it? Thanks Blain
July 29th, 2010 3:48pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics