“ClassNotFoundException: sun.security.provider.Sun

Question

A DoFn in our Dataflow pipeline contains a type with a Random field pointing to a SecureRandom instance, and that field fails to deserialize when running in the Dataflow service using DataflowPipelineRunner. (stack trace below)

We create the SecureRandom using its default ctor, which happens to hand back an instance that uses sun.security.provider.Sun as its java.security.Provider (see SecureRandom#getProvider). SecureRandom extends Random, which is serializable.

The Dataflow service chokes when trying to deserialize this class because it can't create sun.security.provider.Sun.

Looking closer at the stack trace, I see that deserialization happens through com.google.apphosting.runtime.security.UserClassLoader, and now my theory is that this classloader doesn't allow loading of sun.* classes, or at least this particular sun.* class.

java.lang.IllegalArgumentException: unable to deserialize com.example.Example@13e88d
    at com.google.cloud.dataflow.sdk.util.SerializableUtils.deserializeFromByteArray(SerializableUtils.java:73)
    at com.google.cloud.dataflow.sdk.util.SerializableUtils.clone(SerializableUtils.java:88)
    at com.google.cloud.dataflow.sdk.transforms.ParDo$Bound.<init>(ParDo.java:683)
    [...]
    Caused by: java.lang.ClassNotFoundException: sun.security.provider.Sun
    at com.google.apphosting.runtime.security.UserClassLoader.loadClass(UserClassLoader.java:442)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:375)
    at java.lang.Class.forName0(Native Method)
    [...]

Answer 1

The problem is that sun.security.provider.Sun doesn't appear on the App Engine JRE whitelist, so the classloader can't instantiate instances of it:

https://cloud.google.com/appengine/docs/java/jrewhitelist

But luckily you can still say new SecureRandom() in the same environment.

To work around the problem, we added a custom de/serialization hook to the class with the Random field. Simplified example:

class Example implements Serializable {

  // See comments on {@link #writeObject} for why this is transient.
  // Should be treated as final, but can't be declared as such.
  private transient Random random;

  //
  // [Guts of the class go here...]
  //

  /**
   * Serialization hook to handle the transient Random field.
   */
  private void writeObject(ObjectOutputStream out) throws IOException {
    out.defaultWriteObject();
    if (random instanceof SecureRandom) {
      // Write a null to tell readObject() to create a new
      // SecureRandom during deserialization; null is safe to use
      // as a placeholder because the constructor disallows null
      // Randoms.
      //
      // The dataflow cloud environment won't deserialize
      // SecureRandom instances that use sun.security.provider.Sun
      // as their Provider, because it's a system
      // class that's not on the App Engine whitelist:
      // https://cloud.google.com/appengine/docs/java/jrewhitelist
      out.writeObject(null);
    } else {
      out.writeObject(random);
    }
  }

  /**
   * Deserialization hook to initialize the transient Random field.
   */
  private void readObject(ObjectInputStream in)
      throws IOException, ClassNotFoundException {
    in.defaultReadObject();
    Object newRandom = in.readObject();
    if (newRandom == null) {
      // writeObject() will write a null if the original field was
      // SecureRandom; create a new instance to replace it. See
      // comments in writeObject() for background.
      random = new SecureRandom();
      random.nextDouble(); // force seeding
    } else {
      random = (Random) newRandom;
    }
  }
}