异常场景

错误堆栈如下：

Caused by: java.io.InvalidClassException: org.apache.iceberg.BaseFile; local class incompatible: stream classdesc serialVersionUID = -1234567891234562001, local class serialVersionUID = -2345678901234560014
        at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:560)
        at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2020)
        at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1870)
        at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2020)
        at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1870)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2201)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
        at java.base/java.io.ObjectInputStream.readArray(ObjectInputStream.java:2134)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1675)
        at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2496)
        at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2390)
        at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2228)
        at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1687)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:489)
        at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
        ... 4 more

异常分析

这种错误常见于序列化和反序列化的类的serialVersionUID不一致的情况。

可通过serialver工具查看某个类的serialVersionUID，命令行如下

serialver -classpath "{classpath}" {类全限定名}

# 例如
# serialver -classpath "/data/spark/lib1/*:/data/spark/lib2/*"   org.apache.iceberg.BaseFile

最常见的是，写使用一个版本的jar包，而读使用另一个版本的jar包，这两个版本的类不一致，则serialVersionUID不一致，读的时候就会报上述错误。