Conclusion of the following
The following measures have been taken since 5 April 23:06 (applicable to Production installations running in the cloud at GX):
- All Production installations have been provided with a patch. This patch closes the vulnerability as it is known so far;
- In addition code has been added to the apache configuration on the frontend server(s). This code blocks requests to "/web/mvc" (the part where the vulnerability resides);
- Finally, the Tomcat web server was updated to the latest version (9.0.62). In this version, a security change has also been implemented in the context of the Spring Framework.
No situations are known in which the vulnerability was exploit.
Software security is a top priority at GX. The recent notification about the vulnerability found in Spring4Shell on Wednesday 30 March therefore reached our product group quickly. From that moment we started checking whether there was any impact for XperienCentral and, if so, how we could provide support as soon as possible.
Updates will follow on this page and questions can be asked through the usual Customer Services channel.
Background
A vulnerability in the "Spring4Shell" has been identified. This functionality allows us to "build a command line (shell) application using the Spring framework". This Spring framework is part of XperienCentral.
We are currently investigating whether this will have an impact on XperienCentral.
The website of the National Cyber Security Centre has prepared a page with in-depth background information: https://www.ncsc.nl/actueel/advisory?id=NCSC-2022-0221.
Update Thursday, March 31 09:15
Start of this article. Via this page we will keep you informed about our research, results, recommendations and actions.
Update 10:04
It seems that the Spring4Shell functionality is only used to a limited extent in the backend (the editorial environment, which is strongly protected from the outside world by default).
This means that so far the front-end (the side that website visitors see) is not affected by this vulnerability.
Update 11:42
In the last hour our product team has been trying to 'exploit' the XC vulnerability themselves. The first reassuring observation is that - at the time of writing - we have not yet succeeded in finding a scenario in which the vulnerability can be easily exploited, despite our extensive knowledge of XperienCentral's security layers.
Update 13:56
In contrast to the message of 10:04 the vulnerability is also present on the frontends (the system that website visitors see).
- If you are using XC R33 (or newer) then there is a lower risk because from that version onwards an extra security filter is included.
- If you are using a release of R32 or older, then the protection of a security filter still applies, albeit an older variant. A malicious party has a small advantage here compared to R33 or higher.
If a malicious party manages to circumvent the above security mechanisms, it seems at the time of writing that the vulnerability is still not exploitable, because in this case the internal operation of XC deviates from the standard.
Update 14:02
Code changes for a patch have been made available internally late this morning.
Update 15:09
So far the Product Group has not been able to exploit the vulnerability on XperienCentral. That is a reassuring sign. However, we are preparing to proactively patch the vulnerability with the following process:
- Future XC releases will of course include a patch for Spring4Shell by default.
- The following applies to XC installations that operate in the cloud at GX:
- By Tuesday 5 April these installations will have been patched. This will be done in the evening hours and will result in only a few minutes of downtime, so that any inconvenience is kept to a minimum.
- If Tuesday 5 April is not convenient for reasons, the next opportunity will be on 19 April (according to maintenance planning).
- If this date is not convenient either, please contact Customer Services.
- Should the risk assessment evolves negatively (following the ongoing investigation and our tests), we will of course speed up these timelines and act accordingly.
- If your XC installation does not operate in the GX cloud, Customer Services will contact you as soon as possible with the advice to install a patch. These are activities that GX can perform when GX has access to the infrastructure. In case GX has no rights (and cannot get rights) to perform this work, then we will deliver the patch, of course together with guidance on how to install it.
Update 15:24
We are checking whether it is possible to set up monitoring in such a way that this functionality contributes to the detection of abuse of this vulnerability.
Update 17:44
Customer Services has started reaching out to all contacts to carry out the tasks associated with the 15:09 update
Update 22:03
Customer Services has contacted a large part of their network with information. If you do not have any additional information yet and you would like to receive it quickly, please contact Customer Services.
Update Friday April 1st 08:00
Our Product team and system administration team continue their search for a way to make use of the found vulnerability on one of our environments. All options and possibilities are tested thoroughly to eventually guarantee that XC (with or without a patch) is not vulnerable.
Update 10:39
Thus far, We have not found a way to abuse the vulnerability Spring4Shell within XC. Together with our product team, our Architects are continuously testing and evaluating the mentioned patch, and implementing improvements if possible.
Update 13:07
Usually, JDK9+ is mentioned as a prerequisite for Spring4Shell vulnerability. This implies that the vulnerability is introduced in Java 9, which classifies Java 8 as invulnerable. However, we found that, for Xperiencentral, it is irrelevant whether the environment runs on Java 8, Java 9 or higher versions.
For security reasons, there will not be any details published in the present article regarding the specific vulnerabilities in the higher Java versions. In conclusion: we also advice a deploy of our patch on versions that run on Java 8.
Update 14:12
The XC environments R23 and older are not eligible for (part of) the mentioned patch due to the absence of a certain security filter in these versions. Due to the fact that these older version are not supported anymore, a different patch needs to be realized by means of Services. In this case, please contact Customer Services for more information.
Update Monday April 4th 09:42
Tomcat (the web server that is often used) has released a new version with a security fix for Spring4Shell. In addition, the release notes show that the current change (also apart from the known vulnerability) is an important one to implement. The advice from Tomcat itself is therefore to upgrade. Today we are testing whether there is an impact for this Tomcat upgrade.
Update 13:36
It seems there is no impact for a Tomcat upgrade. It is now known that version 8.0.13 concerns the 'unsafe' Tomcat. We advise all organisations (who do their own hosting) to upgrade to at least Tomcat 8.5.87.
Update 14:13
Upgrading Tomcat is on our list to be implemented for all installations that operate in the cloud with us (and will be patched by Tuesday April 5th).
Update Tuesday April 5th 08:36
This morning we are checking our patch on internal test environments -for installations running in the GX cloud. Each installation will be tested manually and the results will be logged. We are checking at a high level:
- Is logging into XperienCentral working as intended?
- Is everything started according to the Plugins Panel?
- Does the creation of a page + article under "cut loose" work?
- Can that page be edited and deleted?
- The log files; did everything start correctly and are there no new notable messages?
Update 12:55
Deploys with patches have been tested on internal test environments (in combination with an upgraded Tomcat) and found to be OK. There are no signs of a delay in patching. It is planned that all installations that operate in the GX cloud will be patched by tonight. For installations that run in a private hosting environment Customer Services cooperates to deliver a patched deploy as soon as possible or install it independently (in consultation).
Update 14:34
Customer Services has informed all contacts about the plans for a patch, including text and an explanation about the Spring core vulnerability and how GX acts to limit the impact. Specifically:
On premise
If you manage the XperienCentral hosting yourself (on premise) -or if a third party manages the installation- then:
- Customer Services has already contacted you with an action plan. In the unlikely event that this is not the case, please contact Customer Services directly;
- or a GX person from your scrum team (Technical Consultant, Architect or Business Consultant) supervises this issue from GX.
The speed of this process varies. It can be done within a few hours, but (depending on communication streams, hierarchy or set change windows) it can also take a few days.
XC in the Cloud
If you are using XperienCentral in the GX cloud, you will have a patched installation, with an upgraded Tomcat version, available early tomorrow morning. This is a quick process that requires little interference and coordination because GX handles the patching/deployment, streamlines communication internally, and GX people manage the hosting themselves.
Update 14:53
When all cloud installations have been successfully patched, a report of the issue will take place via Customer Services. The latest status will also be reported on this page.
Update 23:06
All Cloud installations have been patched.
Update April 6th 07:56
Last night's patches have had unanimously positive effects. If you notice anything 'different' during the day, please contact Customer Services immediately; we will check the status promptly.
Update 11:50 - final update
The following is a summary of the mitigating actions taken that have been in place since yesterday evening for Production installations running in the cloud at GX:
- All Production installations have been patched. This patch closes the vulnerability as it is known so far;
- In addition code has been added to the apache configuration on the frontend server(s). This code blocks requests to "/web/mvc" (the part where the vulnerability resides);
- Finally, the Tomcat web server was updated to the latest version (9.0.62). This version also includes a security change related to the Spring Framework.
As reported in previous messages above, these changes have already been tested and implemented on test environments yesterday afternoon.
The above changes are in line with our aim to guarantee the security of the environments as much as possible, even in this specific situation where no-one managed to succeed in exploiting this vulnerability.
This security incident page is hereby closed. If you have any questions, Customer Services (and any GX employee) is of course available.
Wednesday 6 April 13:46
Security incident page closed
Comments
0 comments
Please sign in to leave a comment.