The Java Deserialization Problem

The Java deserialization problem occurs when applications deserialize data from untrusted sources and is one of the most widespread security vulnerabilities to occur over the last couple years.

This article provides a background on the Java deserialization vulnerability and describes the limitations of the existing mitigation techniques. The information presented here is based on the publicly available research of Gabriel Lawrence, Chris Frohoff, Steve Breen, Matthias Kaiser, Alvaro Muñoz and Christian Schneider; the main researchers behind the Java deserialization vulnerabilities.

Toward the end of this article, you will find information that explains why the currently available solutions are not adequate to properly solve the problem in enterprise environments, and how an innovative solution based on runtime virtualization ideally solves this problem.

A brief background

Serialization, or marshalling, is the process of converting a memory object into a stream of bytes in order to store it into the filesystem or transfer it to another remote system. Deserialization, also known as unmarshalling, is the reverse process that converts the serialized stream of bytes back to an object in the memory of the machine. All main programming languages provide facilities to perform native serialization and deserialization and most of them are inherently unsafe.

Recent research by Lawrence, Frohoff, Breen and Kaiser, demonstrated working deserialization attacks on popular Java applications and frameworks that allow Remote Command Execution. To demonstrate their findings they created the ysoserial tool, a proof-of-concept tool for generating payloads that exploit unsafe Java object deserialization. The main driver for their research was the finding of a dangerous class in the Apache Commons Collection library. The name of that class is the InvokerTransformer. This finding gained attention mainly because of the popularity of the Apache Commons Collection (ACC) library. Even CERT (Computer Emergency Response Teams) issued a Vulnerability Note for the vulnerability in the ACC library. Any application, server or framework that depended on the Apache Commons Collection was potentially vulnerable. JBoss, WebLogic, WebSphere and Jenkins were only just a few of the affected systems.

Explaining all the internal details of the deserialization exploits goes beyond the scope of this article and this has already been done with great success. However, a few important concepts need to be explained that will allow us to identify the gaps in the existing mitigations and highlight the necessity for a new solution to the problem.

How does the InvokerTransformer allow an attacker to exploit the system?

The InvokerTransformer’s goal is to transform objects in a collection by invoking a method using reflection. Attackers abuse this functionality and manage to invoke any method they want. The deserialization proof of concept exploit tool ysoserial abuses the InvokerTransformer and instead of transforming a collection object, it invokes the Runtime.exec() that executes arbitrary commands on the target system.

In order to abuse the InvokerTransformer and make it invoke arbitrary dangerous methods, such as the Runtime.exec(), a specially crafted method sequence needs to be created by the attacker. Each method in the sequence is called a “gadget” and the malicious sequence of method calls is known as a “gadget chain”. In the case of the Apache Commons Collections, the InvokerTransformer is a gadget in the malicious gadget chain. Note that every time a new gadget chain is identified, the ysoserial tool is updated to showcase how these gadget chains work.

One important thing to note here is that most gadget chains utilise gadgets from third-party components, such as the InvokerTransformer from the Apache Commons Collection. However, there are some gadget chains that contain no third-party gadgets. These gadget chains contain only JRE gadgets. Nothing more than just a vulnerable version of the JVM is required for such chains to exploit the system. Such gadget chains are often called golden gadget chains as they can easily bypass existing mitigation solutions.

The attack mechanism can be summarized in the following steps:

• A vulnerable application accepts user-supplied serialized objects.
• An attacker creates a malicious gadget chain, serializes it into a stream of bytes and sends it to the application.
• The vulnerable application reads the received stream of bytes and tries to construct the object. This operation is called “deserialization”.
• During deserialization, the gadget chain is executed, resulting in a compromised system.

What is the impact of such a system compromise?

Depending on the payload, a gadget chain can perform Remote Code Injection, Remote Command Execution, Denial of Service, etc. In most cases, the exploit is possible without any authentication. Finally, note that an attack on a server like WebLogic could impact all its running web applications. For these reasons, Java deserialization vulnerabilities are considered to be critical vulnerabilities with a CVSS score from 7.5 up to 10, depending the environment.

What is it that makes the above scenario vulnerable to attacks?

Only two criteria are required for a Java deserialization vulnerability to be introduced:

• The software must accept and unmarshal serialized data from a location where an attacker has access to.
• “Unsafe” classes (gadgets) must exist in the classpath of the application.

This means that it is not enough for an application to just use a “vulnerable” version of the Apache Commons Collection in order to be vulnerable. It must also deserialize data from unsafe locations.

This is exactly why the Apache Foundation claims that the InvokerTransformer and other such classes that implement a certain functionality cannot be blamed for this vulnerability. It is the combination of both these criteria that introduces the vulnerability. The InvokerTransformer by itself is not vulnerable.

How did Apache react to the finding of this vulnerability?

Even though Apache stated that the InvokerTransformer cannot be blamed for this vulnerability, they hardened the InvokerTransformer by removing its ability to be serialized. However, this means that such a change breaks backwards compatibility and any application that was depended on serializing the InvokerTransformer would break. To overcome this limitation the Apache community decided to introduce a system property that will restore the previous, potentially unsafe behavior of the InvokerTransformer. Therefore, with Apache’s fix a system cannot both use the InvokerTransformer for serialization and be protected at the same time. It must be either one or the other, based on a system property.

What needs to be understood here is that by not allowing the InvokerTransformer to be serialized, attackers will not be able to use the InvokerTransformer anymore to craft malicious gadget chains.

Does this solve the problem at its core? No. It merely patches the problem temporarily.

There is a Greek expression that says that if you have a headache, cutting off your head will not solve your problem. This is exactly what happened here with the InvokerTransformer. Disabling its serializability is not the proper way to solve the problem, it might break your application and does not automatically make your system safe. Also note that the InvokerTransformer is not the only known gadget. Several other have been identified and many more will be found in the future. Disabling a class that could be used as a gadget will only create a never-ending Whack-a-Mole game.

The ysoserial exploit kit is a good example that demonstrates this never-ending game. Currently it contains 27 gadget chains that utilize several distinct gadgets. Disabling the InvokerTransformer does not solve your problem since there are more than 21 other gadget chains that do not use the InvokerTransformer and could potentially compromise your system. To make things even worse, golden chains, that contain only JRE gadgets, cannot be blindly disabled or removed as this will most likely break the application because of the missing required functionality.

Additionally, the transitive dependencies of third-party components create a library sprawl which makes the problem of identifying and disabling “dangerous” classes even more complicated.

Variations of Java Deserialization attacks

At this point it is important to introduce three variations of the Java deserialization attacks in order to better understand the impact of these attacks.

• Blind deserialization attacks that aim to extract data from the target system in environments where the system is behind a network firewall that blocks outgoing connections or when strict Security Manager policies are in place.
• Asynchronous (or stored) deserialization attacks that store the gadget chains in a database or a message queue. The gadget chains will be executed when the target system reads data from the database or the message queue and deserializes them.
• Deferred-execution deserialization attacks that do not execute the gadget chains during deserialization, but rather after deserialization has completed. This is usually achieved via the finalize() method during garbage-collection.

The situation gets even worse because most of the known DoS (Denial of Service) deserialization attacks were classified as “won’t fix” by Oracle and a few other vendors such as Red Hat. Having a production infrastructure with vulnerable software whose vendors refuse to provide a fix is the worst situation for any enterprise! What makes this even worse is that most of these DoS deserialization attacks cannot be mitigated using simple techniques such as whitelisting. A good example of this attack is the SerialDOS exploit created by Wouter Coekaerts.

What is the proper fix?

Is there a solution that solvesthe problem and stops all of the various types of deserialization attacks? According to CERT “Developers need to re-architect their applications.” Obviously, such a fix requires significant code changes, time, effort and money to achieve this. If changing the source code and the architecture of the application is an option then this is the preferred approach. However, bear in mind that even if an application does not perform any deserialization in its own components, most servers, frameworks and third-party components do. So, it is extremely difficult to be 100% certain that the whole stack does not and will never perform deserialization without breaking existing required functionality.

Especially for enterprise production environments with hundreds of deployed instances, making any source code changes is often not feasible to implement. Typically, for enterprise production environments, any security solution that requires code changes and more than a few minutes of deployment time is not acceptable, especially for critical vulnerabilities such as the deserialization vulnerability. Enterprise solutions need accurate protection, fast and without requiring source code changes.

CERT alternatively suggests that blocking the network port using a firewall might solve the problem in some cases. However, in most cases this is not applicable. For example, the deserialization exploits in JBoss, WebLogic, WebSphere, etc run on the HTTP port of the web server. Which means that blocking that port will render the server useless. Also, such solution cannot protect against blind deserialization attacks. Therefore, blocking the network port is not a viable option.

How did the vendors of the affected systems solve the issue?

Without going into much detail of every affected software, the following list shows how some vendors handled the issue:

Also note that there were even cases where the vendors refused to create a fix for the issue either because they do not acknowledge the problem as their own problem or the affected system is an old version that is no longer supported.

How can customers with old or legacy versions of affected systems be protected against the Java deserialization attacks?

If the vendors cannot provide patches and the customers cannot make any source code changes, then how can such production systems be protected? The following are the currently available options.

• Web Application Firewalls – WAFs are not helpful here because they have no application context since they can only examine the input and the output of the application. Applying heuristics on the incoming requests is guaranteed to produce false positives and false negatives. Any security solution that has no application context and operates outside of the application cannot adequately mitigate deserialization attacks
• RASP vendors and Java agents that either disable deserialization completely or apply blacklisting / whitelisting on the classes that are getting deserialized.

Some of these agent solutions break the Oracle Binary Code License Agreement and, therefore, are improperly deployed in production. Even if that was proper, by completely disabling native deserialization system-wide, this will likely break all except the most trivial Java applications.

Why blacklisting and whitelisting are bad solutions to the problem

Any security solution that depends on blacklisting of dangerous classes requires profiling of the application in order to verify that these classes are not utilized by the application. Without first profiling the application, it is not possible to blacklist a class because the risk of breaking the functionality of the application is significant. Additionally, adopting a negative security model means that you will never be sure that you have blacklisted everything.

The list of blocked signatures has to be maintained constantly and frequently and by definition it does not protect any unpublished, zero-day exploits. Any security solution that promotes a blacklisting strategy as a solution to deserialization attacks is doomed to fail since it plays the Whack-a-Mole game. Blacklisting is a poor strategy regardless if it occurs at the application layer, the JVM layer or the network layer.

Whitelisting is a better approach than blacklisting. However, to apply whitelisting, profiling of the application is again required. In this case, the white list will be a really big list of classes. Such big lists are difficult to manage, especially for enterprise environments. In addition, every time the application needs to upgrade to a newer release, the profiling needs to be performed again and a new white list needs to be created. This considerably complicates the deployment of new releases in production. This usually leads to white lists that are not updated and, in turn, produces false positives. Finally, even if an enterprise decides to accept the effort to constantly profile their infrastructure and maintain whitelists, they are still vulnerable to golden gadget chains and to Denial of Service deserialization attacks.

Another suggested mitigation is to blindly block (or whitelist) process forking and file/network IO. Even though this approach will reduce the impact of a deserialization attack, it does not protect against blind attacks for data exfiltration nor Denial of Service deserialization attacks.

Finally, some researchers suggest that using an ad-hoc Security Manager can help mitigate these attacks. However, the truth is that even though it is a good first mitigation step, it is insufficient because of its many limitations.

• Security Managers are known to be easily bypassed.
• It does not protect deferred attacks where the execution of the payload is executed after deserialization, for example via the finalize() method.
• Most DoS deserialization attacks cannot be mitigated by the Security Manager.
• To effectively utilise the Security Manager, another type of white list needs to be created and maintained; thus this approach suffers from the same limitations of the whitelisting.

A new solution

Clearly there is a need for a better security solution to address this critical vulnerability. The following is a list of requirements for what could be described as the ideal security solution for Java deserialization attacks:

• Must work with zero source code changes
• Must work with no application profiling
• Must work with no configuration or tuning (no blacklists or whitelists)
• Must allow applications to be updated / upgraded without redeployment effort
• Must work with existing hardware and application stack
• Must allow applications to use the “dangerous” classes / gadgets, as long as they are used for legitimate functionality
• Must not break existing application functionality or binary compatibility
• Must not produce any false positives or false negatives
• Must protect against all known gadget chains (all ysoserial payloads)
• Must protect even against unpublished, zero-day gadget chains with no configuration
• Must protect against golden gadget chains
• Must protect against gadget chains that have been classified as “won’t fix” by the vendors
• Must protect against blind deserialization attacks
• Must protect against Denial of Service deserialization attacks
• Must protect both the Serializable and the Externalizable interfaces
• Must protect attacks via any end-point (such as HTTP, RMI, JMS, JNDI, etc)
• Must protect the full application stack (the JRE, the server, the framework, the application and all the dependent libraries)
• Must protect against deferred deserialization attacks (such as via the finalize())
• Must protect against lateral / stored deserialization attacks (such as via databases)
• Must not depend exclusively on the Security Manager
• Must support all versions and releases of Java

  

Lastly, the solution must be easy to use and to deploy as well as to achieve all the above without incurring any noticeable performance overhead. In other words, the solution must be production-ready.

The above list of requirements can be very helpful to anyone who might want to evaluate the effectiveness and the usability of a deserialization mitigation solution.

Compiler based RASP is a new application security approach that remediates Java object deserialization attacks and fulfills all the above requirements.

Using a Waratek and by turning on a the Deserial rule, the full application stack is automatically protected against Java deserialization attacks, both known or unknown (zero-day).

This is achieved by enclosing the serialized data within a dynamic, smart compartment inside Waratek that provides full runtime visibility. This smart compartment is active for the duration of each deserialization operations as well as after the deserialization has completed, on specific events such as during garbage collection. The compartment enforces isolation and contextual access control that allows any legitimate functionality to run normally but prohibits any gadget chain to abuse and compromise the system. Because of this fine-grained compartmentalization, unsafe classes, such as the InvokerTransformer, are allowed to be used normally by systems that depend on this functionality, without risking the system to be compromised by any malicious gadget chains.

All the above is achieved without having to make any application source code changes, any configuration, any profiling, any black or white listing with no false positives or negatives and without breaking existing functionality.

The feature also remediates golden gadget chains (JRE-only gadgets), blind attacks, Denial of Service, asynchronous / lateral attacks, as well as deferred-execution attacks.

The protection is achieved with minimum performance overhead and can be deployed on any Java release.