Order of precedence in where clause

Im getting an error on this statement:

SELECT *

FROM

[My Table]

WHERE

ISDATE([DATE OF BIRTH]) = 1

AND

CONVERT(DATETIME, [DATE OF BIRTH], 101) < '01/01/1899'

Because its evaluating the second condition first and running into non dates.  Is there a way to change this where clause to guarantee it will exclude the non-dates? Also, because of the way the app was written with dynamic SQL, it has to be done in the where clause.

the error is :

The conversion of a nvarchar data type to a datetime data type resulted in an out-of-range value.

December 19th, 2013 8:15pm

Yes.

Make the ISDATE condition a subquery or CTE.

Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 8:18pm

The optimiser can change the order of execution.

try

where case when isdate([date of birth]) = 1 then CONVERT(DATETIME, [DATE OF BIRTH], 101) else '18990101' end < '18990101'

That's if you can rely on isdate to guarantee a valid date conversion.

December 19th, 2013 8:19pm

The optimiser can change the order of execution.

try

where case when isdate([date of birth]) = 1 then CONVERT(DATETIME, [DATE OF BIRTH], 101) else '18990101' end < '18990101'

That's if you can rely on isdate to guarantee a valid date conversion.

  • Marked as answer by jschroeder Thursday, December 26, 2013 3:04 PM
Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 8:19pm

Thanks
December 19th, 2013 8:23pm

You can't control the order of evaluation.

In SQL 2012, you can use TRY_CONVERT.

http://technet.microsoft.com/en-us/library/hh230993.aspx

Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 8:29pm

I also don't think that subquery will actually help. The optimizer can still re-arrange execution. Only if you subselect into a temp table first.
December 19th, 2013 8:50pm

You may want to read the second article mentioned in this blog post

http://blogs.technet.com/b/wikininjas/archive/2013/12/08/sunday-surprise-let-s-catch-the-errors.aspx

Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 8:54pm

I'm sorry, but a subquery or CTE is not the answer here. The optimizer can, at will, do "predicate pushing" (Bingle it for more info) and push the predicate of the outer query into the inner query and you are back to the exact same execution plan as you began with.

Use Naomi's suggestion to materialize the "safe" rows into a table variable or temp table first and then query that from a second query, or go with Nigel's suggestion (a CASE construct). Your tests will tell you which performs best in your particular situation.

December 19th, 2013 9:40pm

BTW, I tried unmarking Kalman's reply as an answer and it didn't work for me. I have a bug reported in forum's issues because lately I am unable to unpropose or unmark answers. 

Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 9:59pm


Have you considered writing SQL and not dialect? The first thing you missed is that we do not embed blanks in data element names; read ISO-11179 or any book on data modeling. The only date display format allowed in ANSI/ISO Standard SQL is ISO-8601, which is yyyy-mmn-dd; it is one of the most common ISO standards in IT, as it is embedded in so many other standards. 

We do not use the 1970's Sybase string function CONVERT; we have a proper DATE data type now, so dates are not COBOL strings any more! The ANSI/ISO Standard CAST (birth_date AS DATE) will do the conversion, if your data is screwed up. The DATE data type has a range of '0001-01-01' to '9999-12-31', which should be enough for anyone but a museum or an archeologist. Oh, it is also smaller than the old proprietary Sybase DATETIME data type. 

We do not use things like ISDATE() in a correctly build system; we know the data type of the column and have done the data scrubbing with ETL tools or a presentation layer. Do you understand the concept of a tiered architecture? The data in the tables is supposed to be clean. 

What you are getting are kludges and not real help. Your approach is to write 1970's COBOL style code in 1970's Sybase SQL. This will cost you performance, maintainability, portability and the cool kids will laugh at you.  
December 19th, 2013 10:26pm


The only date display format allowed in ANSI/ISO Standard SQL is ISO-8601, which is yyyy-mmn-dd; it is one of the most common ISO standards in IT, as it is embedded in so many other standards. 

'yyyy-mm-dd' is NOT the correct choice for SQL server though

Have a read of this extremely helpful article: http://www.karaszi.com/sqlserver/info_datetime.asp (in particular the Warnings and common misconceptions section)

Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 10:53pm


The only date display format allowed in ANSI/ISO Standard SQL is ISO-8601, which is yyyy-mmn-dd; it is one of the most common ISO standards in IT, as it is embedded in so many other standards. 

'yyyy-mm-dd' is NOT the correct choice for SQL server though

Have a read of this extremely helpful article: http://www.karaszi.com/sqlserver/info_datetime.asp (in particular the Warnings and common misconceptions section)

December 19th, 2013 10:53pm

Exactly. I was meaning to point this out as well.
Free Windows Admin Tool Kit Click here and download it now
December 19th, 2013 10:56pm

Celko,

|dates are not COBOL strings any more

As a matter of fact, yes the date is a COBAL string.  It's being imported from a COBAL system.

This link may help you http://en.wikipedia.org/wiki/Pedant

Also, since I inherited and did not write the app, my options are limited. 

|The first thing you missed is that we do not embed blanks in data element names

The field name is from a COBAL system written decades ago.

| the cool kids will laugh at you. 

I am a cool kid, unlike you.  And I'm the one laughing.

December 20th, 2013 1:21am

I cannot rely on isdate to guarantee a valid date conversion. 
Free Windows Admin Tool Kit Click here and download it now
December 20th, 2013 2:27am

ISDATE will tell you if the value can be converted into a date using one of the conversion styles. If you have mixed styles saved in the database, then you're in trouble.
December 20th, 2013 2:40am

Not using standards is never right. Microsoft is moving to ANSI/ISO Standard temporal data. This means that Nobody should be using the old 1970's Sybase DATETIME data types. The CAST() and future temporal features are all going to be based on the standards, with the old stuff hanging on as a family curse that will have to be kludged constantly. Not portable, not maintainable, not readable to new programmers that did not grow up with 1970's Sybase SQL. Hardly what I would call a best choice. 

The bad news is that we are stuck with DATETIME2 instead of the correct TIMESTAMP. Just like Oracle and its VARCHAR2() from their initial screwup. 

All of these statements return '2003-02-28', as per ANSI/ISO. 

SET LANGUAGE us_english 
SELECT CAST('2003-02-28' AS DATETIME2);

SET LANGUAGE british 
SELECT CAST('2003-02-28' AS DATETIME2);

SET LANGUAGE us_english 
SELECT CAST('02/28/2003' AS DATETIME2);

SET LANGUAGE british 
SELECT CAST('28/02/2003' AS DATETIME2);
Free Windows Admin Tool Kit Click here and download it now
December 20th, 2013 5:41am

Let me state the obvious, that the best thing to do, is to change the data type of "DATE OF BIRTH" to date. When unknown, store NULL.

Then, when filtering on "DATE OF BIRTH", compare against a safe date format, which would be 'YYYYMMDD'.

This and that would lead to:

SELECT my_column
FROM   "My Table"
WHERE  "DATE OF BIRTH" < '18990101'

If you need the query to actually clean up your table, then just take it step by step, and use temporary tables to process the rows and clean the values.

December 20th, 2013 10:37am

Not using standards is never right.

Are you sugesting I rebuild the entire application I've inhereted?  I don't think that's in the company budget for this year. "Have you considered writing" useful replies instead of making normative judgments? Oh, and using this forum as a vehicle for self promotion is "never right" e

Free Windows Admin Tool Kit Click here and download it now
December 20th, 2013 10:47am

Are you suggesting I rebuild the entire application I've inherited? is classic! Lets close our eyes and just kludge it (with help from SQL forums) until it falls apart with deprecation. You will have switched job by then anyway, right?

This sounds like you inherited a classic family curse application written in SQL Server 2000 or earlier and never given adaptive maintenance as releases came out. Say that in your positing! I am suggesting that you begin a policy of SEI style professionalism. Do they still teach software maintenance as part of Software Engineering?

Corrective: maintenance performed to correct faults in hardware or software. Oh, you meant (x >= 0) and and not (x > 0) and so forth. This one explains itself and it is what most people think of when they hear the words software maintenance; it is not the whole picture.

Adaptive: software maintenance performed to make a computer program usable in a changed

environment. Us database guys have it a little easier because in a tiered environment, much of this is done in the presentation layers. Our job is to throw data over the wall in industry standard format, so that any display formatting, special computations, etc can be done by someone else. But if the environment changes, as when European countries switched to the Euro or US retail went to GTIN from UPC, then we have to change our tables.

We also have to re-write code when features are deprecated. Anyone want ot try to use *= in place of LEFT OUTER JOIN?

Perfective: software maintenance performed to improve the performance, maintainability, or other

attributes of a computer program. Functionality is the same. In a procedural language, we might replace a Bubble Sort with a QuickSort.. Better queries and better indexes are part of how we do it in SQL.

Preventative: maintenance performed for the purpose of preventing problems before they occur. This is the red-haired stepchild of software maintenance. Have you worked with people who like disaster? They love to run in and save the day, be a hero. But they could have done something when it was easy and never had the problem.

In SQL, we have constraints that can prevent bad data. How many of your INTEGER columns in your schema have CHECK (x >= 0) or CHECK (x > 0) constraints to help the optimizer? This should cost you a simple ALTER TABLE.

If you know how, you can set up changes with a maintenance program and get the system corrected. 

Also why do you think that professionalism and standards are "self-promotion"?  I promise that I get no kick-back from ISO when you read a standard :) 

December 20th, 2013 1:47pm

Have you considered writing SQL and not dialect?

Have you?


Free Windows Admin Tool Kit Click here and download it now
December 20th, 2013 2:48pm

LOL!  very instructive.  John, I've had this issue multiple time  the solution proposed by Nigel is the one which worked for me all the time.

Statements inside a case are executed sequentially.

"The CASE statement evaluates its conditions sequentially and stops with the first condition whose condition is satisfied."

You must however make sure you do not put potentially erroneous statements in the when:

"In some situations, an expression is evaluated before a CASE statement receives the results of the expression as its input. Errors in evaluating these expressions are possible. Aggregate expressions that appear in WHEN arguments to a CASE statement are evaluated first, then provided to the CASE statement. For example, the following query produces a divide by zero error when producing the value of the MAX aggregate. This occurs prior to evaluating the CASE expression."

Take a look at the remark section of this article:  http://technet.microsoft.com/en-us/library/ms181765.aspx

In other words, if your date manipulations are inside the then statements and you tested the format in the when statements then you should be OK.

  • Edited by Antoine F 12 hours 55 minutes ago
December 20th, 2013 5:32pm

LOL!  very instructive.  John, I've had this issue multiple time  the solution proposed by Nigel is the one which worked for me all the time.

Statements inside a case are executed sequentially.

"The CASE statement evaluates its conditions sequentially and stops with the first condition whose condition is satisfied."

You must however make sure you do not put potentially erroneous statements in the when:

"In some situations, an expression is evaluated before a CASE statement receives the results of the expression as its input. Errors in evaluating these expressions are possible. Aggregate expressions that appear in WHEN arguments to a CASE statement are evaluated first, then provided to the CASE statement. For example, the following query produces a divide by zero error when producing the value of the MAX aggregate. This occurs prior to evaluating the CASE expression."

Take a look at the remark section of this article:  http://technet.microsoft.com/en-us/library/ms181765.aspx

In other words, if your date manipulations are inside the then statements and you tested the format in the when statements then you should be OK.

  • Edited by Antoine F Friday, December 20, 2013 10:30 PM
Free Windows Admin Tool Kit Click here and download it now
December 21st, 2013 1:23am

ANSI/ISO Standard SQL is all I write until I am cornered by a SQL which is not up to speed yet. And even then I put the correct code in a comment, so that the poor bastards who come after me can uncomment and test with the next release. 
December 23rd, 2013 1:29am

>> As a matter of fact, yes the date is a COBAL [sic] string.  It's being imported from a COBAL [sic] system. <<

You misspelled the name of the language :) 

>> Also, since I inherited and did not write the app, my options are limited. <<

The options are much better than they used to be. There are ETL tools, etc. But if all you have is COBOL, it is really good at string manipulations. 

>> { we do not embed blanks in data element names } The field name is from a COBAL [sic] system written decades ago.<<

COBOL uses dashes where SQL uses underscores. That could be a bulk text edit if you need to keep pre-ISO-11179 names.  But there is no excuse for not having a DATE column to which to map the COBOL field. 
Free Windows Admin Tool Kit Click here and download it now
December 23rd, 2013 1:37am

CASE expression, not statement! 

This was trickier than people first think. When we were voting on this, we found that we had to evaluate all of the THEN and ELSE clauses to determine the data type of the expression. But lead to problems with levels of aggregation and forces the THEN and ELSE clauses to be at the same level. ARRGH! 

December 23rd, 2013 1:44am

The only date display format allowed in ANSI/ISO Standard SQL is ISO-8601, which is yyyy-mmn-dd; it is one of the most common ISO standards in IT

SET LANGUAGE british
SELECT CAST('2013-05-21' AS DATETIME)

returns:

Msg 242, Level 16, State 3, Line 2

The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.

Contrary to popular belief, YYYY-MM-DD for DATETIME variables is NOT language-/dateformat-independent in SQL Server! :)

yyyymmdd is the best format  in SQL Server.

Thank you for correcting my spelling. 

Free Windows Admin Tool Kit Click here and download it now
December 24th, 2013 2:03am

The optimiser can change the order of execution.

try

where case when isdate([date of birth]) = 1 then CONVERT(DATETIME, [DATE OF BIRTH], 101) else '18990101' end < '18990101'

That's if you can rely on isdate to guarantee a valid date conversion.


Thanks Nigel.  A variation on this works for me.
December 26th, 2013 6:04pm

'yyyy-mm-dd' is NOT the correct choice for SQL server though

Have a read of this extremely helpful article: http://www.karaszi.com/sqlserver/info_datetime.asp (in particular the Warnings and common misconceptions section)

Instead of referring to an article it is better to state your point.

Why is 'YYYY-MM-DD' ANSI date string not good for SQL Server?

I am looking for a proof not hearsay. After all we are doing Computer Science, aren't we?

Tibor stated the same in another thread (without explanation), but I don't find an explanation in the referenced article.

'YYYYMMDD' & 'YYYY-MM-DD' are both valid ANSI date strings.

Hi Tibor,

>I'm sorry, but a subquery or CTE is not the answer here.

Can you provide a proof?  Thanks.

Free Windows Admin Tool Kit Click here and download it now
December 26th, 2013 6:18pm

Did you check the article I suggested (the article by Saeid Hasani)?

As for Tibor's blog, it provides a sample of yyyy-mm-dd format failing and I just demonstrated it to you in another thread.

December 26th, 2013 8:05pm

Hi Naomi,

Tibor is not Microsoft. The 'YYYY-MM-DD' format is all over BOL and SSMS.

I would not stop using it because it converts differently in some foreign languages when proper conversion style number is not specified.

It converts OK in USA.

Many things are different in date/datetime string representations when dealing with foreign languages, most famous is dmy UK usage as opposed to mdy USA usage.

Free Windows Admin Tool Kit Click here and download it now
December 26th, 2013 8:19pm

The point is that if we know that particular format may not convert correctly when converting to datetime, then why keep suggesting that wrong not 100% safe format when there is one 100% safe?

In my code samples I never use yyyy-mm-dd format answering questions and in our code which I was code reviewing recently I also suggested that safe format to my colleague, so he went ahead and fixed it.

December 26th, 2013 8:35pm

So why isn't BOL using the "safe" format?

BOL: http://technet.microsoft.com/en-us/library/ms186724.aspx

BOL screenshot:

Free Windows Admin Tool Kit Click here and download it now
December 26th, 2013 9:05pm

This is a format used for presentation, not the internal format and not the format that needs to be used for datetime constants that do not have time portion.

Did you find in BOL the implicit constant of yyyy-mm-dd representing datetime value? And even if you do find it, it doesn't mean BOL shows it correctly.

December 26th, 2013 9:32pm

This was a question about order precedence.  Why are you blabbering about about what's taught in software engineering classes?  Frankly, myself and a lot of other people wish you would shut the F up.
Free Windows Admin Tool Kit Click here and download it now
February 12th, 2014 6:50pm

This was a question about order precedence.  Why are you blabbering about about what's taught in software engineering classes?  Frankly, myself and a lot of other people wish you would shut
February 13th, 2014 1:33am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics