在Python3中反序列化Python2的datetime

Question

我选择使用pickle（加上base64和TCP套接字）来在我的python3代码和旧版python2代码之间传输数据，但我在处理datetime对象时遇到了麻烦：

在PY3中生成的对象在PY2中可以正常解码，但反过来却在调用datetime构造函数时引发了TypeError错误，接着在load_reduce函数中又出现了UnicodeEncodeError错误。

一个简短的测试程序和日志，包括PY2和PY3的pickle输出，可以在这个gist中找到。

我在PY2中使用pickle.dumps(reply, protocol=2)
然后在PY3中使用pickle._loads(pickled, fix_imports=True, encoding='latin1')
（尝试过None和utf-8都没有成功）
原生的cPickle的loads解码也失败了，我只是为了调试使用了纯python的_loads。

这是一个datetime的bug吗？也许datetime.__getstate__/__setstate__的实现不兼容？

对代码的任何评论都欢迎...

补充

PY-3.4.0的pickle：

 0: \x80 PROTO      2
 2: c    GLOBAL     'datetime datetime'
21: q    BINPUT     0
23: c    GLOBAL     '_codecs encode'
39: q    BINPUT     1
41: X    BINUNICODE u'\x07\xde\x07\x11\x0f\x06\x11\x05\n\x90'
58: q    BINPUT     2
60: X    BINUNICODE u'latin1'
71: q    BINPUT     3
73: \x86 TUPLE2
74: q    BINPUT     4
76: R    REDUCE
77: q    BINPUT     5
79: \x85 TUPLE1
80: q    BINPUT     6
82: R    REDUCE
83: q    BINPUT     7
85: .    STOP

PY-2.7.6的pickle：

 0: \\x80 PROTO      2
 2: c    GLOBAL     'datetime datetime'
21: q    BINPUT     0
23: U    SHORT_BINSTRING '\\x07\xc3\x9e\\x07\\x11\\x0f\\x06\\x11\\x05\\n\\x90'
35: q    BINPUT     1
37: \\x85 TUPLE1
38: q    BINPUT     2
40: R    REDUCE
41: q    BINPUT     3
43: ]    EMPTY_LIST
44: q    BINPUT     4
46: N    NONE
47: \\x87 TUPLE3
48: q    BINPUT     5
50: .    STOP

PY-3.4.0的pickle.load_reduce：

def load_reduce(self):
    stack = self.stack
    args = stack.pop()
    func = stack[-1]
    try:
        value = func(*args)
    except:
        print(sys.exc_info())
        print(func, args)
        raise
    stack[-1] = value
dispatch[REDUCE[0]] = load_reduce

PY-3.4.0的datetime pickle支持：

# Pickle support.

def _getstate(self):
    yhi, ylo = divmod(self._year, 256)
    us2, us3 = divmod(self._microsecond, 256)
    us1, us2 = divmod(us2, 256)
    basestate = bytes([yhi, ylo, self._month, self._day,
                       self._hour, self._minute, self._second,
                       us1, us2, us3])
    if self._tzinfo is None:
        return (basestate,)
    else:
        return (basestate, self._tzinfo)

def __setstate(self, string, tzinfo):
    (yhi, ylo, self._month, self._day, self._hour,
     self._minute, self._second, us1, us2, us3) = string
    self._year = yhi * 256 + ylo
    self._microsecond = (((us1 << 8) | us2) << 8) | us3
    if tzinfo is None or isinstance(tzinfo, _tzinfo_class):
        self._tzinfo = tzinfo
    else:
        raise TypeError("bad tzinfo state arg %r" % tzinfo)

def __reduce__(self):
    return (self.__class__, self._getstate())

datetime serialization deserialization pickle tcp sockets python3 cPickle Python2

在Python3中反序列化Python2的datetime

补充

1 个回答

撰写回答