Force reboot of a remote server that has hung shutting down

I had an issue last night when a remote server that I was applying windows updates to, hung while it was shutting down. I will still able to ping the server, and access its file shares, but was unable to get RDP access. I was cursing at this point, with the prospect of a long journey the following day to investigate and bring the server back online. I then considered what other steps I could take to try and force the server to reboot remotely. I used the PSTools command psexec to see if I could still get command line access to the remote server, and fortunately I could. Next I tried to force a reboot of the remote server using psexec and the shutdown command as follows, where REMOTE_SERVER_NAME is the name of the remote server that I was trying to reboot:

psexec \\REMOTE_SERVER_NAME shutdown /r /t 01

alternatively you could use:

shutdown /m \\REMOTE_SERVERNAME /r /t 01

This returned the following error:

1115 A system shutdown is in progress

This basically meant that a system shutdown was already in progress,  and therefore the command was unable to force a reboot. In the end I used the pskill command to stop the winlogon service on the remote server to try and release whichever process wass causing the server to hang on shutdown. I should stress that this was a last resort, and not something that I would recommend doing unless essential:

pskill \\REMOTE_SERVER_NAME  winlogon

Anyway, after another few minutes the remote server did finally restart, although there are a few other things that I should mention that happened in the process. The operating system on this machine was Windows Server 2008 R2. After the server came back up (verified by ping -t REMOTE_SERVER_NAME) I tried to RDP the box again. I was able to enter my credentials and the logon process appeared to start, but after a few seconds the following message appeared on the screen:

Please wait for the Windows Modules Installer

The machine sat like that for quite some time, and then started ‘Configuring Updates’. My RDP session then abruptly ended and the server restarted itself again. Again, when it was back up I tried to RDP the server again and received the ‘Please wait for the windows modules installer message’ for a second time. Thankfully, after a few minutes and another ‘configuring updates’ message, logon continued and ther server was back up and running. On checking the event log and windows update log I was able to verify that all the updates had installed OK, and there were no other errors worthy of note. So in summary, if you want to save yourself a long trip, to most likely press a power or reset switch, you may want to try the above first.    

 

Advertisements

30 Responses to Force reboot of a remote server that has hung shutting down

  1. Mark says:

    Thank you, you have saved me a trip to site on a weekend

  2. Geert says:

    Thank you for the very useful info. By the way, I had to kill the TRUSTEDINSTALLER to get the system to reboot.

  3. Justin Chambers says:

    Awesome work, you just saved me from a 2 hour drive in to work!

  4. Graham Goddard says:

    Excellent suggestion. I ran into exactly this problem, the remote system was partially unresponsive and was clearly not responding to reboot commands. Killing winlogon allowed the reboot to complete/

    Thanks

  5. Thanks for this tip! It worked after I did the TRUSTEDINSTALLER kill as well, as mentioned by Geert up above. Phew!

  6. Bals says:

    I have tried these options, but I am getting the error “The specified server cannot perform the requested operation.”. But server is reachable thru ping.

  7. Jeff says:

    I get the following error message when i try to pskill winlogon.
    Server ping response is positive
    ===============================================

    Couldn’t access :
    The specified network name is no longer available.

  8. Pingback: Force reboot of a remote server | Linewalker Tech Blog

  9. Pingback: Confluence: ITdoc Import

  10. Pingback: Remote shutdown | itdamage

  11. RichardP says:

    Worked for me – cheers

  12. hughborg707 says:

    Worked for me as well. Worth noting for me however is how earlier in my career this was really make or break stuff. Most employers and then later customers didn’t have DRAC-iLo or Managed PDU racks to remote power cycle the gear. Even with VMware and being able to pull up a console it takes so much of the “work” out of fixing this type of situation.

    Where I used it THIS time was with a customer who is just now implementing VMware in their environment. In this case it was a physical server, with no DRAC or Managed PDU to bail me out. This tool worked well!

    Thanks!

  13. Laz says:

    Thanks a lot! It saved me to go to DC, nice one!

  14. Brenton says:

    Killing the winlogon process worked for me, thanks for the tip. Looks like it turned out to be Windows Updates that was the culprit that caused the botched reboot.

  15. stretchie204 says:

    Thanks for the tip, remotely killing the winlogon process using PSkill worked for me and the server rebooted shortly thereafter. I was scratching my head as to why RDP wasn’t working even though telnetting to port 3389 worked. I push installed Dameware Mini Remote to the problematic server and saw on the console “Shutting down Windows…”, this impasse led me to your article.

  16. wmelton says:

    Thanks for the tip, saved me about 15 minutes of driving I really didn’t want to do. I had to kill the winlogon and trustedinstaller. My OS was 2008 R2 as well.

  17. MJ Hays says:

    So awesome! Thanks for this article! Saved me in a very time sensitive server setup scenario.

  18. gregory15 says:

    like everyone else , thanks. You saved me from a drive in to the colo.

  19. Owen says:

    Thanks for the tips your advice worked perfectly saved me a 30 min trip to the server on a cold stormy night

  20. Anonymous says:

    this was very helpful! Thank you…. beware of the -s switch on the psshutdown if you only mean to reboot the machine… even if you use the -r the -s will overwrite and shutdown the machine. I made that mistake the first time and then was screwed with the machine inaccessible and shutdown.

  21. dublinvince says:

    Thanks, another tip working very well. How about MS fixing their ‘TrustedInstaller’ blocking reboot after each update…that surely would help most

  22. Cornel. says:

    I had this same issue on Windows server 2008 STD on Hyper-V. The server kept on stuck on “Please wait for the windows modules installer”, after that the server kept on saying corrupted updates and reverting changes, and kept on going into this boot loop.Once the server said Please wait for the windows modules installer, I remotely accessed the Services via the services.msc console and stopped the trustedinstaller service and set it to manual startup.
    After that I was able to signon, but only with a local account and not domain account.
    After that I to made sure that all updates are ok and no corrupted packages existed, I ran the hotfix “System Update Readiness”
    https://support.microsoft.com/en-us/kb/947821
    Did a manual reboot and server was stable, and was able to logon with domain account..

  23. Pingback: Getting a hung Windows 2008 R2 server up and running after installing Windows updates. | Bas blogs

  24. Rafael says:

    Thank chief…you saved my life!

  25. Ewerton Rabello says:

    You saved my life… Thank my friend.

  26. Anonymous1 says:

    It has happened to me, looking at processes, windows updates were installing , I just waited it out. I don’t mind waiting if I know what is going on in the background.

  27. Nightkiller says:

    The advice given in the column is very sound.
    I suggest the following ONLY as a last resort. If you follow this procedure you run a risk of disk corruption if you happen to execute this command while a disk operation is occurring. Judge the risks and act accordingly.
    If killing the winlogon process does not work
    => tried this and it was still showing 1115
    used taskkill to kill a csrss process. This forces a blue screen.
    => there were three instances of this process running in sessions other than 0, the system process. No other user processes were running on the machine.
    Memo to self: Do not use the shutdown command from the Start Menu. Always issue the shutdown command from a command prompt with the /f switch.

  28. Lee Sentell says:

    This post is so spot on that it’s making me a little angry. Great advice, has saved me a lot of aggravation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: