CSV text qualifier, text qualified, and escape characters
OK. Writing a dataflow to read a CSV file. Connection Manager Editor says format is delimited, text qualifer is <none>. And that's just fine, but when run, a bunch of rows were generating errors. Why? Well, turns out one of the fields contained values sometimes ending with a percent sign, "31%" adjacent to our delimiter - which fwiw is actually a pipe "|" not a comma. Well, it seems like the percent was acting like an escape character. Is that correct? Eliminating the percent fixed the import. Changing to %% fixed the import. And, it seemed that changing the field specification (in the connection manager Advanced tab) to "text qualified = False" also fixed it. But - why did it think that the field was text qualified, when at the connection (file) level, we told it "no text qualifier"? Well, qualifie*r* is not the same as qualifie*d*, right? Ugh. Is this a bug? (2008 R2). Is it a feature??? Should I make it a point in all flat file CSV connections to override all fields to "text qualified = False" - would that have any other side-effects? Thanks. Josh
September 17th, 2010 9:53pm

Was the value surrounded by % like %31%? If yes, then % acted as a text qualifier. This makes the column "text qualified". Does that help?Arthur My Blog
Free Windows Admin Tool Kit Click here and download it now
September 17th, 2010 11:35pm

It was foo|123|something longer than 50 chars and then 31%|bar|987 Look at the "advanced" setting for any of your own flat/csv files and see. Josh
September 18th, 2010 5:45am

Any progress Josh?Best Regards, Uri Dimant SQL Server MVP http://dimantdatabasesolutions.blogspot.com/ http://sqlblog.com/blogs/uri_dimant/
Free Windows Admin Tool Kit Click here and download it now
September 19th, 2010 10:53am

Any progress Josh? Well, I have three workarounds, but it still looks like an SSIS bug to me. Actually a couple of bugs, first the field does not take its default setting from the file, second it does not actually perform text qualification at the field level, and third I'm still not sure when or if the percent sign is an escape character. Josh
September 19th, 2010 5:53pm

could you paste sample rows of csv file here? and I go to check it on my machinehttp://www.rad.pasfu.com
Free Windows Admin Tool Kit Click here and download it now
September 20th, 2010 8:15am

Hi Josh, have a look at http://social.msdn.microsoft.com/Forums/en/sqlintegrationservices/thread/4c90aa26-a897-4c34-850c-279801c54872 M
December 14th, 2010 3:53am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics