Rackspace hosted Exchange suffered a devastating interruption starting December 2, 2022 and is still continuous since 12:37 AM December fourth. Initially described as connectivity and login issues, the assistance was eventually upgraded to announce that they were dealing with a security event.
Rackspace Hosted Exchange Issues
The Rackspace system decreased in the early morning hours of December 2, 2022. At first there was no word from Rackspace about what the problem was, much less an ETA of when it would be solved.
Consumers on Buy Twitter Verification reported that Rackspace was not responding to support e-mails.
This has been rather the day with #Rackspace. Every hosted exchange client has been down for 14 hours or two. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am concerned now that they fell victim to something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace consumer independently messaged me over social media on Friday to relate their experience:
“All hosted Exchange customers down over the past 16 hours.
Not exactly sure the number of companies that is, however it’s substantial.
They’re serving a 554 long delay bounce so individuals emailing in aren’t knowledgeable about the bounce for several hours.”
The main Rackspace status page used a running update of the outage but the preliminary posts had no information aside from there was an outage and it was being investigated.
The first official update was on December second at 2:49 AM:
“We are examining a concern that is impacting our Hosted Exchange environments. More information will be posted as they appear.”
Thirteen minutes later Rackspace started calling it a “connection concern.”
“We are investigating reports of connectivity issues to our Exchange environments.
Users may experience a mistake upon accessing the Outlook Web App (Webmail) and syncing their e-mail client(s).”
By 6:36 AM the Rackspace updates described the continuous problem as “connection and login issues” then later that afternoon at 1:54 PM Rackspace revealed they were still in the “investigation phase” of the interruption, still trying to determine what failed.
And they were still calling it “connectivity and login problems” in their Cloud Office environments at 4:51 PM that afternoon.
Rackspace Recommends Moving to Microsoft 365
Four hours later Rackspace described the circumstance as a “considerable failure”and started using their consumers free Microsoft Exchange Strategy 1 licenses on Microsoft 365 as a workaround up until they understood the issue and could bring the system back online.
The main assistance specified:
“We experienced a significant failure in our Hosted Exchange environment. We proactively shut down the environment to prevent any more problems while we continue work to restore service. As we continue to work through the origin of the issue, we have an alternate service that will re-activate your ability to send and receive emails.
At no cost to you, we will be supplying you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 until additional notice.”
Rackspace Hosted Exchange Security Event
It was not till almost 24 hours later at 1:57 AM on December 3rd that Rackspace formally revealed that their hosted Exchange service was suffering from a security incident.
The announcement even more revealed that the Rackspace professionals had actually powered down and disconnected the Exchange environment.
“After more analysis, we have figured out that this is a security event.
The known effect is separated to a portion of our Hosted Exchange platform. We are taking needed actions to evaluate and secure our environments.”
Twelve hours later on that afternoon they updated the status page with more information that their security team and outside experts were still working on solving the interruption.
Was Rackspace Service Affected by a Vulnerability?
Rackspace has actually not released details of the security event.
A security occasion generally involves a vulnerability and there are two serious vulnerabilities presently in the wile that were patched in November 2022.
These are the 2 most present vulnerabilities:
Microsoft Exchange Server Server-Side Demand Forgery (SSRF) Vulnerability
A Server Side Request Forgery (SSRF) attack permits a hacker to read and change data on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an enemy has the ability to run destructive code on a server.
An advisory published in October 2022 described the effect of the vulnerabilities:
“A confirmed remote aggressor can carry out SSRF attacks to escalate benefits and carry out arbtirary PowerShell code on susceptible Microsoft Exchange servers.
As the attack is targeted against Microsoft Exchange Mailbox server, the enemy can possibly gain access to other resources by means of lateral motion into Exchange and Active Directory site environments.”
The Rackspace failure updates have not shown what the specific issue was, only that it was a security occurrence.
The most current status update as of December 4th stated that the service is still down and customers are motivated to migrate to the Microsoft 365 service.
Rackspace published the following on December 4, 2022 at 12:37 AM:
“We continue to make progress in resolving the incident. The accessibility of your service and security of your data is of high significance.
We have actually dedicated comprehensive internal resources and engaged first-rate external knowledge in our efforts to minimize negative effects to consumers.”
It’s possible that the above noted vulnerabilities are related to the security incident impacting the Rackspace Hosted Exchange service.
There has actually been no announcement of whether customer info has actually been jeopardized. This event is still continuous.
Featured image by Best SMM Panel/Orn Rin