Please contact your support team if you have a question or need assistance for any Rackspace products, services, or articles.
Earlier this week, we were notified of a potential hypervisor vulnerability (Xen Security Advisory 133: http://xenbits.xen.org/xsa/advisory-133.html and http://venom.crowdstrike.com/) that affects a portion of our First and Next Generation Cloud Servers fleet, as well as Cloud Big Data. Please note that OnMetal Cloud Servers are not affected.
Server Types that ARE Impacted
* FirstGen Cloud Servers running Windows
* NextGen Cloud Servers built from a PVHVM image
Server Types that are NOT Impacted
* FirstGen Cloud Servers running Linux
* NextGen Cloud Servers built from a PV image
We patched the portion of our infrastructure that supports the Cloud Virtual Machine (VM). For the patch to be effective in resolving the vulnerability, the customer VM must be power cycled, either by the customer or by Rackspace. Our preference is that customers do this themselves, and we strongly recommend that customers take this action as quickly as possible.
Given the severity of the vulnerability, customers have less than 24 hours to perform the power cycle themselves. After that window closes, for customers who have not completed this maintenance, Rackspace will force conduct the power cycle. As a number of our customers deploy across multiple regions, regional maintenance events will be staggered so no two regions are affected at the same time. We understand that many customers deploy in a single region. To help customers plan accordingly, a detailed timeline during which the forced Rackspace power cycle will take place for individual VM's will be made available via the First and Next Gen Cloud Servers APIs and Cloud Control Panel on Thursday, 5/14/15.
IMPORTANT NOTE: A SOFT REBOOT IS INSUFFICIENT to make the patch fully effective and resolve this security vulnerability. Details of the recommended power cycle process are outlined at: https://community.rackspace.com/products/f/25/t/5188
We recommend that customers ensure their applications and environments are able to withstand a short interruption in service prior to completing the power cycle of their VM(s). This means that there are no single points of failure in the configuration and that applications are able to gracefully resume service after a server pause process. For a comprehensive description of how customers can prepare for a power cycle, see Community page: https://community.rackspace.com/products/f/25/t/4319.
I received a support ticket listing the effected servers (which is all of them) at 4:34 p.m. Eastern. Between 6 and 7:30 p.m. Eastern, I power cycled all of them (shutdown -h, wait for pings to drop/console connect error, hard reboot via control panel).
I've received a second support ticket at 10:39 p.m. Eastern listing the same servers as being effected. The text of the tickets seems identical except for the blurb changes from "you have 24 hours" to "you have 18 hours". Might this be a duplicate or already queued notification, or should I assume that everything's going to need to reboot again?
I'll answer your question.
Q. I received a support ticket listing the effected servers (which is all of them) at 4:34 p.m. Eastern. Between 6 and 7:30 p.m. Eastern, I power cycled all of them (shutdown -h, wait for pings to drop/console connect error, hard reboot via control panel).
A. If you have already power cycled the servers on your account, then you will not need to do it again. We are communicating through ticket to remind customers of the time remaining on the self-service window. You may receive some additional notifications but you will not need to take further action and we will not power cycle your servers when the maintenance window starts.