Force reboot of a remote server that has hung shutting down
March 2, 2012 30 Comments
I had an issue last night when a remote server that I was applying windows updates to, hung while it was shutting down. I will still able to ping the server, and access its file shares, but was unable to get RDP access. I was cursing at this point, with the prospect of a long journey the following day to investigate and bring the server back online. I then considered what other steps I could take to try and force the server to reboot remotely. I used the PSTools command psexec to see if I could still get command line access to the remote server, and fortunately I could. Next I tried to force a reboot of the remote server using psexec and the shutdown command as follows, where REMOTE_SERVER_NAME is the name of the remote server that I was trying to reboot:
psexec \\REMOTE_SERVER_NAME shutdown /r /t 01
alternatively you could use:
shutdown /m \\REMOTE_SERVERNAME /r /t 01
This returned the following error:
1115 A system shutdown is in progress
This basically meant that a system shutdown was already in progress, and therefore the command was unable to force a reboot. In the end I used the pskill command to stop the winlogon service on the remote server to try and release whichever process wass causing the server to hang on shutdown. I should stress that this was a last resort, and not something that I would recommend doing unless essential:
pskill \\REMOTE_SERVER_NAME winlogon
Anyway, after another few minutes the remote server did finally restart, although there are a few other things that I should mention that happened in the process. The operating system on this machine was Windows Server 2008 R2. After the server came back up (verified by ping -t REMOTE_SERVER_NAME) I tried to RDP the box again. I was able to enter my credentials and the logon process appeared to start, but after a few seconds the following message appeared on the screen:
Please wait for the Windows Modules Installer
The machine sat like that for quite some time, and then started ‘Configuring Updates’. My RDP session then abruptly ended and the server restarted itself again. Again, when it was back up I tried to RDP the server again and received the ‘Please wait for the windows modules installer message’ for a second time. Thankfully, after a few minutes and another ‘configuring updates’ message, logon continued and ther server was back up and running. On checking the event log and windows update log I was able to verify that all the updates had installed OK, and there were no other errors worthy of note. So in summary, if you want to save yourself a long trip, to most likely press a power or reset switch, you may want to try the above first.
Thank you, you have saved me a trip to site on a weekend
Thank you for the very useful info. By the way, I had to kill the TRUSTEDINSTALLER to get the system to reboot.
same on my side, but i have to do it with the pskill.exe from Sysinternals.
Awesome work, you just saved me from a 2 hour drive in to work!
Excellent suggestion. I ran into exactly this problem, the remote system was partially unresponsive and was clearly not responding to reboot commands. Killing winlogon allowed the reboot to complete/
Thanks
Thanks for this tip! It worked after I did the TRUSTEDINSTALLER kill as well, as mentioned by Geert up above. Phew!
I have tried these options, but I am getting the error “The specified server cannot perform the requested operation.”. But server is reachable thru ping.
Does the account you are logged in with have admin rights on the server you are trying to kill the process on?
I get the following error message when i try to pskill winlogon.
Server ping response is positive
===============================================
Couldn’t access :
The specified network name is no longer available.
Pingback: Force reboot of a remote server | Linewalker Tech Blog
Pingback: Confluence: ITdoc Import
Pingback: Remote shutdown | itdamage
Worked for me – cheers
Worked for me as well. Worth noting for me however is how earlier in my career this was really make or break stuff. Most employers and then later customers didn’t have DRAC-iLo or Managed PDU racks to remote power cycle the gear. Even with VMware and being able to pull up a console it takes so much of the “work” out of fixing this type of situation.
Where I used it THIS time was with a customer who is just now implementing VMware in their environment. In this case it was a physical server, with no DRAC or Managed PDU to bail me out. This tool worked well!
Thanks!
Thanks a lot! It saved me to go to DC, nice one!
Killing the winlogon process worked for me, thanks for the tip. Looks like it turned out to be Windows Updates that was the culprit that caused the botched reboot.
Thanks for the tip, remotely killing the winlogon process using PSkill worked for me and the server rebooted shortly thereafter. I was scratching my head as to why RDP wasn’t working even though telnetting to port 3389 worked. I push installed Dameware Mini Remote to the problematic server and saw on the console “Shutting down Windows…”, this impasse led me to your article.
Thanks for the tip, saved me about 15 minutes of driving I really didn’t want to do. I had to kill the winlogon and trustedinstaller. My OS was 2008 R2 as well.
So awesome! Thanks for this article! Saved me in a very time sensitive server setup scenario.
like everyone else , thanks. You saved me from a drive in to the colo.
Thanks for the tips your advice worked perfectly saved me a 30 min trip to the server on a cold stormy night
this was very helpful! Thank you…. beware of the -s switch on the psshutdown if you only mean to reboot the machine… even if you use the -r the -s will overwrite and shutdown the machine. I made that mistake the first time and then was screwed with the machine inaccessible and shutdown.
Thanks, another tip working very well. How about MS fixing their ‘TrustedInstaller’ blocking reboot after each update…that surely would help most
I had this same issue on Windows server 2008 STD on Hyper-V. The server kept on stuck on “Please wait for the windows modules installer”, after that the server kept on saying corrupted updates and reverting changes, and kept on going into this boot loop.Once the server said Please wait for the windows modules installer, I remotely accessed the Services via the services.msc console and stopped the trustedinstaller service and set it to manual startup.
After that I was able to signon, but only with a local account and not domain account.
After that I to made sure that all updates are ok and no corrupted packages existed, I ran the hotfix “System Update Readiness”
https://support.microsoft.com/en-us/kb/947821
Did a manual reboot and server was stable, and was able to logon with domain account..
Pingback: Getting a hung Windows 2008 R2 server up and running after installing Windows updates. | Bas blogs
Thank chief…you saved my life!
You saved my life… Thank my friend.
It has happened to me, looking at processes, windows updates were installing , I just waited it out. I don’t mind waiting if I know what is going on in the background.
The advice given in the column is very sound.
I suggest the following ONLY as a last resort. If you follow this procedure you run a risk of disk corruption if you happen to execute this command while a disk operation is occurring. Judge the risks and act accordingly.
If killing the winlogon process does not work
=> tried this and it was still showing 1115
used taskkill to kill a csrss process. This forces a blue screen.
=> there were three instances of this process running in sessions other than 0, the system process. No other user processes were running on the machine.
Memo to self: Do not use the shutdown command from the Start Menu. Always issue the shutdown command from a command prompt with the /f switch.
This post is so spot on that it’s making me a little angry. Great advice, has saved me a lot of aggravation.