By

Bad kernel upgrade on Rackspace Cloud

New – The other day, I woke up to a cloud server incident notification from Rackspace.  One of the cloud server hosts had become unresponsive and needed to be rebooted.  This is nothing unusual.  Our server usually reboots and returns to its normal function. However, I went to check the server and was unable to connect to it via SSH or HTTP.   This was a tad concerning, but nothing to panic about yet.  I checked the MyCloud control panel and saw that the server was shutdown.  A bit odd, but still nothing to worry about.  I rebooted the server and everything appeared fine, but then the server went back to shutdown within a minute.   I was never able to log into the server becuase it never got to the point of starting SSH.  I was able to reboot the server and immediately connect via console on the MyCloud interface.   I was greeted with:

That’s a problem.   To fix the issue, I needed to boot the server into Rescue Mode.  The MyCloud interface provided me with a temporary password to use once it was moved into Rescue Mode.  When logged into the Rescue Mode, I needed to mount the hard disk from the affected server.

This command lists the available disks.   I found the disk that matched the size that I was expecting and mounted it.

I could now edit files on the disk.  Because I knew it was a kernel upgrade issue, I edited /boot/grub/grub.conf to change

to

After saving the file, I used the MyCloud interface to exit rescue mode and rebooted the server.  This change will force Grub to use an different kernel version.  To my relief, the server came back just fine.   I then reinstalled the latest kernel upgrade and rebooted the server to ensure that worked correctly this time.

New Server

Leave a Reply

Your email address will not be published. Required fields are marked *