Strange Issue - Windows Server 2008R2
Not sure where to post this, so I am going for the General Forum.
We have a Windows Server 2008 R2 SP1 server. It is our only AD/DNS/DHCP server in the office.
For the last four weeks we have experienced the following issues each Monday morning:
Weekend backups never kick off (Using Windows Backup) on Saturday or Sunday. No errors, they just never start. Friday nights backup starts and finishes normally.
Logging into the actual server (either at the server itself or via Remote Desktop) takes forever (At least 15 minutes).
Once logged it, the server is slow to respond to everything (clicks, program launches, etc).
Print Services start to fail once users start logging in. Print jobs appear to print, but end up in never never land.
Other than the above, user access to the server is fine. File shares work, and everyone is able to access data on the server.
There are no errors in the event log other than Event ID 4005 and Event ID 10154 that start on Monday morning when I try to login to the server.
Server diagnostics on the HP Prolient are coming up fine.
A reboot of the server corrects the issue, and we are fine for the rest of the week.
Any ideas?
Thanks
Ed Fishman
July 11th, 2011 10:48am
1. Weekend backups never kick off (Using Windows Backup) on Saturday
or Sunday. No errors, they just never start. Friday nights backup
starts and finishes normally.
Let's start from this one; download this program to the server (after the reboot so that it will be responsive)
http://technet.microsoft.com/en-us/sysinternals/bb896653
then extract the file(s) to whatever suitable location and leave them there (optionally create a desktop shortcut to ease running the app) at this point, let the server go until the backups won't start failing again, at that point, fire up "process explorer"
the above app and check if there are multiple copies of the "backup" application running at the same time, also check which app is taking up more RAM/CPU
Free Windows Admin Tool Kit Click here and download it now
July 11th, 2011 11:23am
There are no errors in the event log other than Event ID 4005 and
Event ID 10154 that start on Monday morning when I try to login to the
server.
Forgot... could you please copy and paste here a couple of
those events ? See, having the event source and details may
help finding out more infos about them
http://www.eventid.net/display.asp?eventid=4005&source=
http://www.eventid.net/display.asp?eventid=10154&source=
also, and since you're at it; the issue may also be caused by
malware, if that's the case, you may try using this tool
http://connect.microsoft.com/systemsweeper
to scan the system and ensure it's clean from "clandestines"
July 11th, 2011 11:27am
Warning 7/11/2011 10:08:57 AM
Windows Remote Management 10154
None
Error 7/11/2011 8:55:01 AM
Winlogon 4005
None
Note: After the reboot these errors go away and only reappear the following Monday.
Will try and run SystemSweeper tonight. FWIW, I do have a full A/V client (Vipre) on the Server.
Thanks
Ed
Free Windows Admin Tool Kit Click here and download it now
July 11th, 2011 1:15pm
Will do.
Thanks
Ed
July 11th, 2011 1:18pm
Warning 7/11/2011 10:08:57 AMWindows Remote Management 10154None
Error 7/11/2011 8:55:01 AM Winlogon 4005None
Sorry, maybe I was unclear, please, post here the full details related to the events; just to be more clear... open the event viewer, select the relevant event from the list, double click it, click the "copy" button and paste the resulting infos here
As for the error 4005, here you'll find some infos which may help you further digging down the issue and possibly finding its cause and fixing it
http://www.eventid.net/display.asp?eventid=4005&eventno=10171&source=Winlogon&phase=1
http://thebackroomtech.com/2010/08/31/fix-the-windows-logon-process-has-unexpectedly-terminated/
http://technet.microsoft.com/en-us/library/cc734097%28WS.10%29.aspx
notice that the above links refer to "system resources" (amongst other causes) now, here's what I'm suspecting; your scheduled backup starts but doesn't complete (for some reason) and the backup process stays "stuck" in memory; after a while another backup
is started... and over and over, all those processes then eat up system resources and memory causing the system to slow down (due to swapping...) and causing those errors to be logged then, after you reboot the system those "zombie" processes go away and the
system starts back running ok... until all the above doesn't repeat over and over
Notice that I'm suspecting the "backup" since you reported that your backups are failing, but the culprit may also be some other process; again, a look at the box using "process explorer" (see my other message) may help finding what's up
Will try and run SystemSweeper tonight. FWIW, I do have a full A/V
client (Vipre) on the Server.
That's ok, again, the idea is just to ensure that there isn't something which "slipped through" and is causing the issue
Free Windows Admin Tool Kit Click here and download it now
July 12th, 2011 4:22am
Will do.
Thanks
y/w; just a note, ensure to start "procexp" as admin (right click, run as admin) so that you'll be able to see all processes and details
July 12th, 2011 4:27am
As for the error 4005, here you'll find some infos which may help you
further digging down the issue and possibly finding its cause and
fixing it
Forgot (sorry !!) here are some infos about the event 10154
http://technet.microsoft.com/en-us/library/dd348559%28WS.10%29.aspx
Free Windows Admin Tool Kit Click here and download it now
July 12th, 2011 4:33am
I see this in my logs from time to time BUT none of the above causes are true / applicable.
Brute force attacks by bots guessing usernames seems to be the real cause for this as I have picked this up on 3 windows servers (2008 and 2008 R2) which all have adequate resources, uncorrupted registries and nothing wrong as far as services goes.The event
takes place every single time after a bot has been hammering away trying various username and password combinations.
Somehow the bots from time to time throw garbage information at the logon process which causes it to crash and restart.
It is worrying that nobody at Microsoft has picked up this issue and are sticking with the 3 reasons / explanations noted here.
July 19th, 2012 12:56pm