CheckDB failed The operating system returned error 1117(The request could not be performed because of an I/O device error.) (Network Steve Forum)

CheckDB failed The operating system returned error 1117(The request could not be performed because of an I/O device error.)

Hello everyone

We had a CheckDB failed error this morning on 3 of our databases.

Of these 3 databases, one is an important database (the DB of our main Sharepoint site collection) with an MDF size of 800Gigs.

This is the message we got : The operating system returned error 1117(The request could not be performed because of an I/O device error.) to SQL Server during a read at offset 0x00000000660000 in file 'filePathHere' ...

Our system admin guys say they dont have any problem with the disks.

Now I am actually backing up this database to have it restored on our test server. I will then run CheckDB against it to verify

For the databases that failed the CheckDB, I noticed they have an extra copie of the DB name NameOfDB_ServerName_FB20150922... if the CheckDB runs again, will it override this DB copy?

Has anyone come across this error before and how have you gone about fixing this issue?

Thank you all in advance

Regards

Edited by fiogeekgurl Friday, September 04, 2015 12:18 AM

September 4th, 2015 12:07am

We would need full result of checkdb/ errorlog to analyse further.

The first and best solution of Fixing corruption issue is restoring with the latest good known backup.

For the databases that failed the CheckDB, I noticed they have an extra copie of the DB name NameOfDB_ServerName_FB20150922... if the CheckDB runs again, will it override this DB copy?

Checkdb always run against a particular DB and how could that override another DB. When you mentioned they have another copy, who have ? and what copy ?

same database with different name inside same instance ?

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 12:59am

See if this matches your case

https://support.microsoft.com/en-us/kb/2519834

September 4th, 2015 1:07am

We would need full result of checkdb/ errorlog to analyse further.

The first and best solution of Fixing corruption issue is restoring with the latest good known backup.

For the databases that failed the CheckDB, I noticed they have an extra copie of the DB name NameOfDB_ServerName_FB20150922... if the CheckDB runs again, will it override this DB copy?

Checkdb always run against a particular DB and how could that override another DB. When you mentioned they have another copy, who have ? and what copy ?

same database with different name inside same instance ?

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 1:12am

See if this matches your case

https://support.microsoft.com/en-us/kb/2519834

September 4th, 2015 1:13am

No, that is not safe, at all. Checkdb didn't finish because it didn't have enough space in tempdb to do its work. Read the beginning and end part carefully. Expand your tempdb and try again.

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 4:44am

I just finished running checkDB directly on the production server and the same databases failed CheckDB:

1. The most important DB - size 800Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_intranet_BK_ServerName..." failed with the following error: "The operating system returned error 1117(The request could not be performed because of an I/O device error.) to SQL Server during a read at offset 0x00000000660000 in file 'G:\\MSSQL10_50.MSSQLSERVER\\MSSQL\\DATA\\FB20150902210608788088\\PRBackup\\DC11-6C8A82FD-C39C-4600-AE6C-B21E54BD43A3\\WSS_Content_Intranet.mdf:MSSQL_DBCC44'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

2. Second datbase, size of 100Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_Conformite_BK_ServerName..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:8158) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:8158) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_Conformite_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_Conformite_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

3. Third database. I restored this database to our test server and ran CheckDB, no errors where returned.

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_ce_BK_ServerName_FB20..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:10560) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:10560) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_ce_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_ce_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Thank you all in advance

Edited by fiogeekgurl 21 hours 37 minutes ago

September 4th, 2015 5:32am

No, that is not safe, at all. Checkdb didn't finish because it didn't have enough space in tempdb to do its work. Read the beginning and end part carefully. Expand your tempdb and

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 5:34am

I just finished running checkDB directly on the production server and the same databases failed CheckDB:

1. The most important DB - size 800Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_intranet_BK_ServerName..." failed with the following error: "The operating system returned error 1117(The request could not be performed because of an I/O device error.) to SQL Server during a read at offset 0x00000000660000 in file 'G:\\MSSQL10_50.MSSQLSERVER\\MSSQL\\DATA\\FB20150902210608788088\\PRBackup\\DC11-6C8A82FD-C39C-4600-AE6C-B21E54BD43A3\\WSS_Content_Intranet.mdf:MSSQL_DBCC44'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

2. Second datbase, size of 100Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_Conformite_BK_ServerName..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:8158) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:8158) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_Conformite_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_Conformite_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

3. Third database. I restored this database to our test server and ran CheckDB, no errors where returned.

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_ce_BK_ServerName_FB20..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:10560) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:10560) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_ce_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_ce_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Thank you all in advance

Your database seems to be corrupt and possible reason would be disk on which data files and log files reside. Can you point this message to storage team and ask them to do another round of checking

You did not posted what minimum level of repair checkdb suggested

September 4th, 2015 6:15am

Check db doesn't create a copy of the database, as you show in the picture above. It creates a database snapshot, but that is an internal thing, and you will not see this (not even in the database snapshot folder). As of SQL 2014, you can see the physical snapshot file(s) in explorer while checkdb is executing - but prior to 2014 you don't even see that (since earlier versions used Alternate Streams in NTFS).

As for the error, the problem is definitely at a level under SQL Server. SQL Server does an I/O request and the operating system returns an error (1117). Hopefully the article posted by Jinu will lead to a solution.

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 6:49am

Hello

Would it be safe to say that my database is not corrupt just by seeing this line :
CHECKDB found 0 allocation errors and 0 consistency errors in database 'WSS_Content_Intranet_20150904'.

This is the complete message after CheckDB was ran.

DBCC results for 'WSS_Content_Intranet_20150904'. Msg 8921, Level 16, State 1, Line 1 Check terminated. A failure was detected while collecting facts. Possibly tempdb out of space or a system table is inconsistent. Check previous errors. DBCC results for 'sys.sysrscols'. There are 2676 rows in 28 pages for object "sys.sysrscols". DBCC results for 'sys.sysrowsets'. There are 321 rows in 6 pages for object "sys.sysrowsets". DBCC results for 'sys.sysallocunits'. There are 402 rows in 11 pages for object "sys.sysallocunits". DBCC results for 'sys.sysfiles1'. There are 2 rows in 1 pages for object "sys.sysfiles1". DBCC results for 'sys.syspriorities'. There are 0 rows in 0 pages for object "sys.syspriorities". DBCC results for 'sys.sysfgfrag'. There are 2 rows in 1 pages for object "sys.sysfgfrag". DBCC results for 'sys.sysphfg'. There are 1 rows in 1 pages for object "sys.sysphfg". DBCC results for 'sys.sysprufiles'. There are 2 rows in 1 pages for object "sys.sysprufiles". DBCC results for 'sys.sysftinds'. There are 0 rows in 0 pages for object "sys.sysftinds". DBCC results for 'sys.sysowners'. There are 20 rows in 1 pages for object "sys.sysowners". DBCC results for 'sys.sysprivs'. There are 139 rows in 1 pages for object "sys.sysprivs". DBCC results for 'sys.sysschobjs'. There are 1798 rows in 33 pages for object "sys.sysschobjs". DBCC results for 'sys.syscolpars'. There are 29821 rows in 595 pages for object "sys.syscolpars". DBCC results for 'sys.sysnsobjs'. There are 1 rows in 1 pages for object "sys.sysnsobjs". DBCC results for 'sys.syscerts'. There are 0 rows in 0 pages for object "sys.syscerts". DBCC results for 'sys.sysxprops'. There are 0 rows in 0 pages for object "sys.sysxprops". DBCC results for 'sys.sysscalartypes'. There are 39 rows in 1 pages for object "sys.sysscalartypes". DBCC results for 'sys.systypedsubobjs'. There are 0 rows in 0 pages for object "sys.systypedsubobjs". DBCC results for 'sys.sysidxstats'. There are 459 rows in 22 pages for object "sys.sysidxstats". DBCC results for 'sys.sysiscols'. There are 1177 rows in 12 pages for object "sys.sysiscols". DBCC results for 'sys.sysbinobjs'. There are 23 rows in 1 pages for object "sys.sysbinobjs". DBCC results for 'sys.sysaudacts'. There are 0 rows in 0 pages for object "sys.sysaudacts". DBCC results for 'sys.sysobjvalues'. There are 2072 rows in 542 pages for object "sys.sysobjvalues". DBCC results for 'sys.sysclsobjs'. There are 19 rows in 1 pages for object "sys.sysclsobjs". DBCC results for 'sys.sysrowsetrefs'. There are 0 rows in 0 pages for object "sys.sysrowsetrefs". DBCC results for 'sys.sysremsvcbinds'. There are 0 rows in 0 pages for object "sys.sysremsvcbinds". DBCC results for 'sys.sysxmitqueue'. There are 0 rows in 0 pages for object "sys.sysxmitqueue". DBCC results for 'sys.sysrts'. There are 1 rows in 1 pages for object "sys.sysrts". DBCC results for 'sys.sysconvgroup'. There are 0 rows in 0 pages for object "sys.sysconvgroup". DBCC results for 'sys.sysdesend'. There are 0 rows in 0 pages for object "sys.sysdesend". DBCC results for 'sys.sysdercv'. There are 0 rows in 0 pages for object "sys.sysdercv". DBCC results for 'sys.syssingleobjrefs'. There are 149 rows in 1 pages for object "sys.syssingleobjrefs". DBCC results for 'sys.sysmultiobjrefs'. There are 31529 rows in 291 pages for object "sys.sysmultiobjrefs". DBCC results for 'sys.sysguidrefs'. There are 0 rows in 0 pages for object "sys.sysguidrefs". DBCC results for 'sys.syscompfragments'. There are 0 rows in 0 pages for object "sys.syscompfragments". DBCC results for 'sys.sysftstops'. There are 0 rows in 0 pages for object "sys.sysftstops". DBCC results for 'sys.sysqnames'. There are 97 rows in 1 pages for object "sys.sysqnames". DBCC results for 'sys.sysxmlcomponent'. There are 99 rows in 1 pages for object "sys.sysxmlcomponent". DBCC results for 'sys.sysxmlfacet'. There are 112 rows in 1 pages for object "sys.sysxmlfacet". DBCC results for 'sys.sysxmlplacement'. There are 18 rows in 1 pages for object "sys.sysxmlplacement". DBCC results for 'sys.sysobjkeycrypts'. There are 0 rows in 0 pages for object "sys.sysobjkeycrypts". DBCC results for 'sys.sysasymkeys'. There are 0 rows in 0 pages for object "sys.sysasymkeys". DBCC results for 'sys.syssqlguides'. There are 0 rows in 0 pages for object "sys.syssqlguides". DBCC results for 'sys.sysbinsubobjs'. There are 3 rows in 1 pages for object "sys.sysbinsubobjs". DBCC results for 'sys.syssoftobjrefs'. There are 5908 rows in 104 pages for object "sys.syssoftobjrefs". DBCC results for 'NameValuePair_Georgian_Modern_Sort_CI_AS'. There are 0 rows in 0 pages for object "NameValuePair_Georgian_Modern_Sort_CI_AS". DBCC results for 'Groups'. There are 574 rows in 15 pages for object "Groups". DBCC results for 'Deps'. There are 702 rows in 28 pages for object "Deps". DBCC results for 'NameValuePair_German_PhoneBook_CI_AS'. There are 0 rows in 0 pages for object "NameValuePair_German_PhoneBook_CI_AS". DBCC results for 'GroupMembership'. There are 9178 rows in 47 pages for object "GroupMembership". DBCC results for 'NameValuePair_Greek_CI_AS'. There are 0 rows in 0 pages for object "NameValuePair_Greek_CI_AS". DBCC results for 'AllDocs'. There are 3159470 rows in 207666 pages for object "AllDocs". CHECKDB found 0 allocation errors and 0 consistency errors in database 'WSS_Content_Intranet_20150904'. Msg 1105, Level 17, State 2, Line 1 Could not allocate space for object 'dbo.SORT temporary run storage: 140737565818880' in database 'tempdb' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup. Msg 9002, Level 17, State 4, Line 1 The transaction log for database 'tempdb' is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases

Edited by fiogeekgurl Friday, September 04, 2015 7:09 AM

September 4th, 2015 7:07am

I just finished running checkDB directly on the production server and the same databases failed CheckDB:

1. The most important DB - size 800Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_intranet_BK_ServerName..." failed with the following error: "The operating system returned error 1117(The request could not be performed because of an I/O device error.) to SQL Server during a read at offset 0x00000000660000 in file 'G:\\MSSQL10_50.MSSQLSERVER\\MSSQL\\DATA\\FB20150902210608788088\\PRBackup\\DC11-6C8A82FD-C39C-4600-AE6C-B21E54BD43A3\\WSS_Content_Intranet.mdf:MSSQL_DBCC44'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

2. Second datbase, size of 100Gig

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_Conformite_BK_ServerName..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:8158) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:8158) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_Conformite_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_Conformite_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

3. Third database. I restored this database to our test server and ran CheckDB, no errors where returned.

Failed:(-1073548784) Executing the query "DBCC CHECKDB(N'WSS_Content_ce_BK_ServerName_FB20..." failed with the following error: "System table pre-checks: Object ID 3. Could not read and latch page (1:10560) with latch type SH. Check statement terminated due to unrepairable error. Unable to read and latch page (1:10560) with latch type SH. 1117(The request could not be performed because of an I/O device error.) failed. CHECKDB found 0 allocation errors and 1 consistency errors not associated with any single object. CHECKDB found 0 allocation errors and 1 consistency errors in database 'WSS_Content_ce_BK_ServerName_FB20150902210608788088'. is the minimum repair level for the errors found by DBCC CHECKDB (WSS_Content_ce_BK_ServerName_FB20150902210608788088).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Thank you all in advance

Edited by fiogeekgurl Friday, September 04, 2015 9:31 AM

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 9:30am

The above post says no corruption. Yes the database 'WSS_Content_Intranet_20150904' is not corrupted

September 4th, 2015 7:22pm

The above post says no corruption. Yes the database 'WSS_Content_Intranet_20150904' is not corrupted

Free Windows Admin Tool Kit Click here and download it now

September 4th, 2015 7:37pm

Hello everyone

Thank you for your replies regarding my problem. After running the CheckDB on a separate test server, I can safely say that the database is not corrupt as the CheckDB results did not find any problems.

The problem is maybe disk-related. I will transfer the link Jinu gave https://support.microsoft.com/en-us/kb/2519834 to our system admin guys to look into.

Regards

Edited by fiogeekgurl 6 hours 34 minutes ago

Marked as answer by fiogeekgurl 6 hours 34 minutes ago

September 4th, 2015 8:35pm

Hello everyone

Thank you for your replies regarding my problem. After running the CheckDB on a separate test server, I can safely say that the database is not corrupt as the CheckDB results did not find any problems.

The problem is maybe disk-related. I will transfer the link Jinu gave https://support.microsoft.com/en-us/kb/2519834 to our system admin guys to look into.

Regards

Edited by fiogeekgurl Saturday, September 05, 2015 12:35 AM

Marked as answer by fiogeekgurl Saturday, September 05, 2015 12:35 AM

Free Windows Admin Tool Kit Click here and download it now

September 5th, 2015 12:34am

This topic is archived. No further replies will be accepted.
Other recent topics