Java serialization filters need an allowlist first-run process

Java deserialization remains dangerous because the input controls object construction and can reach gadget chains in libraries the application did not intend to expose. The platform's serialization filter mechanism provides a useful constraint. It is not a switch that makes every ObjectInputStream safe, and a copied reject list can create more trouble than it prevents.

What the JVM can filter

The Java serialization filter mechanism screens incoming streams of serialized objects. A process-wide filter can be configured with the jdk.serialFilter system property or the corresponding security property. The pattern language can accept or reject class names, packages, and modules, while also limiting array size, graph depth, and references.

The filter is applied when the JVM's serialization machinery processes an object stream. Individual applications can also set an ObjectInputFilter on a stream or use a filter factory. The ObjectInputFilter API is the place to enforce application-specific context when a global pattern is too coarse.

Serialization filtering is not the same as safe data exchange. A filtered stream can still contain attacker-controlled values that reach application logic. Parsing JSON, XML, or a bounded binary format with explicit schema validation is usually a better design for new protocols.

Start with an inventory, not a known-bad list

A deny list is attractive because it preserves existing behavior for most classes. It is also brittle. A new library version can introduce another gadget class, and an attacker can choose from classes the maintainers did not enumerate. An allowlist is harder to build, but it defines the classes the application actually expects.

Instrument a staging system with representative traffic. Record the class names and resource limits used by the application's own serialization paths. Separate legitimate serialization from Java RMI, distributed cache formats, session replication, and framework internals, because those paths may need different filters.

The JVM-wide property is useful as a broad baseline. It can cap graph depth, reference count, and array size even when the class list remains incomplete. Apply these limits first, then narrow the class list in an application-specific filter. Test the limits with the largest legitimate object graph rather than an average request.

Roll out with observable failures

A safe first configuration might reject unexpected classes while allowing the application's model package and JDK primitives. Do not guess the package tree. Generate it from staging observations and code review. A filter like this is illustrative, not a production template:

java -Djdk.serialFilter="maxdepth=20;maxrefs=10000;maxarray=100000;com.example.dto.*;java.lang.*;!*" \ -jar app.jar

The final pattern order matters. Test the exact property in the same JVM and application classloader used by production. A class that appears allowed in a unit test may come from a different module or a dynamically loaded plug-in.

Log filter rejections with the stream context and the application request identifier, but do not log the serialized bytes. A burst of rejected classes during rollout can identify a missing legitimate class or an unexpected client. Treat a missing class as a design review, not an automatic reason to add a broad package wildcard.

Keep a separate compatibility test for session replication and cache services. Those components can fall back or fail in ways that are not visible in the main application logs. If a cache or session format cannot be filtered safely, isolate it in a process with the narrowest possible privileges and avoid exposing its port to untrusted networks.

Make filtering part of a larger change

Review every ObjectInputStream and every library that accepts serialized objects. Remove serialization from new API boundaries where an explicit format is possible. Set network access and process privileges so that a successful gadget chain cannot reach every secret or host. Keep the filter configuration under version control and test it after dependency upgrades.

The serialization filter is a strong safety net when the application has a known object model and passes bounded input. It is a poor substitute for design changes, and it is not a reason to accept serialized input from an untrusted source. The useful outcome is a smaller attack surface with failures that the team can see and explain.