回溯序列化库。
tblib的Python项目详细描述
回溯序列化库。
- 免费软件:BSD许可证
它允许您:
- Pickle跟踪并引发异常 在不同的过程中有腌制的回溯。这样可以在运行时更好地处理错误 多进程上的代码(想象一下多进程、台球、期货、芹菜等)。
- 从字符串创建回溯对象(方法from_string。不使用酸洗。
- 序列化到普通dict(方法from_dict和to_dict)的回溯。不使用酸洗。
- 引发从上述源创建的回溯。
请再次注意,使用pickle支持是完全可选的。你要全权负责 如果您决定使用pickle支持,则会出现安全问题。
安装
pip install tblib
文档
Pickling tracebacks
note:输出的回溯对象被剥夺了一些属性(如变量)。但你可以提出例外 那些回溯或打印它们-应该覆盖99%的用例。
>>> from tblib import pickling_support >>> pickling_support.install() >>> import pickle, sys >>> def inner_0(): ... raise Exception('fail') ... >>> def inner_1(): ... inner_0() ... >>> def inner_2(): ... inner_1() ... >>> try: ... inner_2() ... except: ... s1 = pickle.dumps(sys.exc_info()) ... >>> len(s1) > 1 True >>> try: ... inner_2() ... except: ... s2 = pickle.dumps(sys.exc_info(), protocol=pickle.HIGHEST_PROTOCOL) ... >>> len(s2) > 1 True >>> try: ... import cPickle ... except ImportError: ... import pickle as cPickle >>> try: ... inner_2() ... except: ... s3 = cPickle.dumps(sys.exc_info(), protocol=pickle.HIGHEST_PROTOCOL) ... >>> len(s3) > 1 True
Unpickling
>>> pickle.loads(s1) (<...Exception'>, Exception('fail'...), <traceback object at ...>) >>> pickle.loads(s2) (<...Exception'>, Exception('fail'...), <traceback object at ...>) >>> pickle.loads(s3) (<...Exception'>, Exception('fail'...), <traceback object at ...>)
Raising
>>> from six import reraise >>> reraise(*pickle.loads(s1)) Traceback (most recent call last): ... File "<doctest README.rst[14]>", line 1, in <module> reraise(*pickle.loads(s2)) File "<doctest README.rst[8]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail >>> reraise(*pickle.loads(s2)) Traceback (most recent call last): ... File "<doctest README.rst[14]>", line 1, in <module> reraise(*pickle.loads(s2)) File "<doctest README.rst[8]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail >>> reraise(*pickle.loads(s3)) Traceback (most recent call last): ... File "<doctest README.rst[14]>", line 1, in <module> reraise(*pickle.loads(s2)) File "<doctest README.rst[8]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail
What if we have a local stack, does it show correctly ?
是的,有:
>>> exc_info = pickle.loads(s3) >>> def local_0(): ... reraise(*exc_info) ... >>> def local_1(): ... local_0() ... >>> def local_2(): ... local_1() ... >>> local_2() Traceback (most recent call last): File "...doctest.py", line ..., in __run compileflags, 1) in test.globs File "<doctest README.rst[24]>", line 1, in <module> local_2() File "<doctest README.rst[23]>", line 2, in local_2 local_1() File "<doctest README.rst[22]>", line 2, in local_1 local_0() File "<doctest README.rst[21]>", line 2, in local_0 reraise(*exc_info) File "<doctest README.rst[11]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail
It also supports more contrived scenarios
类似于带有语法错误的回溯:
>>> from tblib import Traceback >>> from examples import bad_syntax >>> try: ... bad_syntax() ... except: ... et, ev, tb = sys.exc_info() ... tb = Traceback(tb) ... >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[58]>", line 1, in <module> reraise(et, ev, tb.as_traceback()) File "<doctest README.rst[57]>", line 2, in <module> bad_syntax() File "...tests...examples.py", line 18, in bad_syntax import badsyntax File "...tests...badsyntax.py", line 5 is very bad ^ SyntaxError: invalid syntax
或其他导入失败:
>>> from examples import bad_module >>> try: ... bad_module() ... except: ... et, ev, tb = sys.exc_info() ... tb = Traceback(tb) ... >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[61]>", line 1, in <module> reraise(et, ev, tb.as_traceback()) File "<doctest README.rst[60]>", line 2, in <module> bad_module() File "...tests...examples.py", line 23, in bad_module import badmodule File "...tests...badmodule.py", line 3, in <module> raise Exception("boom!") Exception: boom!
或者由于超过递归限制而导致的回溯(这里是 强制类型和值跨平台保持一致性:
>>> def f(): f() >>> try: ... f() ... except RuntimeError: ... et, ev, tb = sys.exc_info() ... tb = Traceback(tb) ... >>> reraise(RuntimeError, RuntimeError("maximum recursion depth exceeded"), tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[32]>", line 1, in f def f(): f() File "<doctest README.rst[32]>", line 1, in f def f(): f() File "<doctest README.rst[32]>", line 1, in f def f(): f() ... RuntimeError: maximum recursion depth exceeded
Reference
tblib.Traceback
它由pickling_support使用。如果你想要更多的灵活性,你也可以使用它:
>>> from tblib import Traceback >>> try: ... inner_2() ... except: ... et, ev, tb = sys.exc_info() ... tb = Traceback(tb) ... >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[21]>", line 6, in <module> reraise(et, ev, tb.as_traceback()) File "<doctest README.rst[21]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail
tblib.Traceback.to_dict
您可以使用to_dict方法和from_dict类方法来 将回溯转换为可由stdlib序列化的词典 json.jsondecoder:
>>> import json >>> from pprint import pprint >>> try: ... inner_2() ... except: ... et, ev, tb = sys.exc_info() ... tb = Traceback(tb) ... tb_dict = tb.to_dict() ... pprint(tb_dict) {'tb_frame': {'f_code': {'co_filename': '<doctest README.rst[...]>', 'co_name': '<module>'}, 'f_globals': {'__name__': '__main__'}}, 'tb_lineno': 2, 'tb_next': {'tb_frame': {'f_code': {'co_filename': ... 'co_name': 'inner_2'}, 'f_globals': {'__name__': '__main__'}}, 'tb_lineno': 2, 'tb_next': {'tb_frame': {'f_code': {'co_filename': ... 'co_name': 'inner_1'}, 'f_globals': {'__name__': '__main__'}}, 'tb_lineno': 2, 'tb_next': {'tb_frame': {'f_code': {'co_filename': ... 'co_name': 'inner_0'}, 'f_globals': {'__name__': '__main__'}}, 'tb_lineno': 2, 'tb_next': None}}}}
tblib.Traceback.from_dict
基于前面的示例:
>>> tb_json = json.dumps(tb_dict) >>> tb = Traceback.from_dict(json.loads(tb_json)) >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[21]>", line 6, in <module> reraise(et, ev, tb.as_traceback()) File "<doctest README.rst[21]>", line 2, in <module> inner_2() File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail
tblib.Traceback.from_string
>>> tb = Traceback.from_string(""" ... File "skipped.py", line 123, in func_123 ... Traceback (most recent call last): ... File "tests/examples.py", line 2, in func_a ... func_b() ... File "tests/examples.py", line 6, in func_b ... func_c() ... File "tests/examples.py", line 10, in func_c ... func_d() ... File "tests/examples.py", line 14, in func_d ... Doesn't: matter ... """) >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[42]>", line 6, in <module> reraise(et, ev, tb.as_traceback()) File "...examples.py", line 2, in func_a func_b() File "...examples.py", line 6, in func_b func_c() File "...examples.py", line 10, in func_c func_d() File "...examples.py", line 14, in func_d raise Exception("Guessing time !") Exception: fail
如果您使用strict=False选项,那么解析就有点松懈:
>>> tb = Traceback.from_string(""" ... File "bogus.py", line 123, in bogus ... Traceback (most recent call last): ... File "tests/examples.py", line 2, in func_a ... func_b() ... File "tests/examples.py", line 6, in func_b ... func_c() ... File "tests/examples.py", line 10, in func_c ... func_d() ... File "tests/examples.py", line 14, in func_d ... Doesn't: matter ... """, strict=False) >>> reraise(et, ev, tb.as_traceback()) Traceback (most recent call last): ... File "<doctest README.rst[42]>", line 6, in <module> reraise(et, ev, tb.as_traceback()) File "bogus.py", line 123, in bogus File "...examples.py", line 2, in func_a func_b() File "...examples.py", line 6, in func_b func_c() File "...examples.py", line 10, in func_c func_d() File "...examples.py", line 14, in func_d raise Exception("Guessing time !") Exception: fail
tblib.decorators.return_error
>>> from tblib.decorators import return_error >>> inner_2r = return_error(inner_2) >>> e = inner_2r() >>> e <tblib.decorators.Error object at ...> >>> e.reraise() Traceback (most recent call last): ... File "<doctest README.rst[26]>", line 1, in <module> e.reraise() File "...tblib...decorators.py", line 19, in reraise reraise(self.exc_type, self.exc_value, self.traceback) File "...tblib...decorators.py", line 25, in return_exceptions_wrapper return func(*args, **kwargs) File "<doctest README.rst[5]>", line 2, in inner_2 inner_1() File "<doctest README.rst[4]>", line 2, in inner_1 inner_0() File "<doctest README.rst[3]>", line 2, in inner_0 raise Exception('fail') Exception: fail
这有什么用?假设您正在使用这样的多处理:
# Note that Python 3.4 and later will show the remote traceback (but as a string sadly) so we skip testing this. >>> import traceback >>> from multiprocessing import Pool >>> from examples import func_a >>> pool = Pool() # doctest: +SKIP >>> try: # doctest: +SKIP ... for i in pool.map(func_a, range(5)): ... print(i) ... except: ... print(traceback.format_exc()) ... Traceback (most recent call last): File "<doctest README.rst[...]>", line 2, in <module> for i in pool.map(func_a, range(5)): File "...multiprocessing...pool.py", line ..., in map ... File "...multiprocessing...pool.py", line ..., in get ... Exception: Guessing time ! <BLANKLINE> >>> pool.terminate() # doctest: +SKIP
不是很有用吧?我们来解决这个问题:
>>> from tblib.decorators import apply_with_return_error, Error >>> from itertools import repeat >>> pool = Pool() >>> try: ... for i in pool.map(apply_with_return_error, zip(repeat(func_a), range(5))): ... if isinstance(i, Error): ... i.reraise() ... else: ... print(i) ... except: ... print(traceback.format_exc()) ... Traceback (most recent call last): File "<doctest README.rst[...]>", line 4, in <module> i.reraise() File "...tblib...decorators.py", line ..., in reraise reraise(self.exc_type, self.exc_value, self.traceback) File "...tblib...decorators.py", line ..., in return_exceptions_wrapper return func(*args, **kwargs) File "...tblib...decorators.py", line ..., in apply_with_return_error return args[0](*args[1:]) File "...examples.py", line 2, in func_a func_b() File "...examples.py", line 6, in func_b func_c() File "...examples.py", line 10, in func_c func_d() File "...examples.py", line 14, in func_d raise Exception("Guessing time !") Exception: Guessing time ! <BLANKLINE> >>> pool.terminate()
好多了!
What if we have a local call stack ?
>>> def local_0(): ... pool = Pool() ... for i in pool.map(apply_with_return_error, zip(repeat(func_a), range(5))): ... if isinstance(i, Error): ... i.reraise() ... else: ... print(i) ... >>> def local_1(): ... local_0() ... >>> def local_2(): ... local_1() ... >>> try: ... local_2() ... except: ... print(traceback.format_exc()) Traceback (most recent call last): File "<doctest README.rst[...]>", line 2, in <module> local_2() File "<doctest README.rst[...]>", line 2, in local_2 local_1() File "<doctest README.rst[...]>", line 2, in local_1 local_0() File "<doctest README.rst[...]>", line 5, in local_0 i.reraise() File "...tblib...decorators.py", line 20, in reraise reraise(self.exc_type, self.exc_value, self.traceback) File "...tblib...decorators.py", line 27, in return_exceptions_wrapper return func(*args, **kwargs) File "...tblib...decorators.py", line 47, in apply_with_return_error return args[0](*args[1:]) File "...tests...examples.py", line 2, in func_a func_b() File "...tests...examples.py", line 6, in func_b func_c() File "...tests...examples.py", line 10, in func_c func_d() File "...tests...examples.py", line 14, in func_d raise Exception("Guessing time !") Exception: Guessing time ! <BLANKLINE>
Other weird stuff
清除回溯工作(Python3.4及更高版本):
>>> tb = Traceback.from_string(""" ... File "skipped.py", line 123, in func_123 ... Traceback (most recent call last): ... File "tests/examples.py", line 2, in func_a ... func_b() ... File "tests/examples.py", line 6, in func_b ... func_c() ... File "tests/examples.py", line 10, in func_c ... func_d() ... File "tests/examples.py", line 14, in func_d ... Doesn't: matter ... """) >>> import traceback, sys >>> if sys.version_info > (3, 4): ... traceback.clear_frames(tb)
学分
- mitsuhiko/jinja2用于找到创建回溯对象的方法。
更改日志
1.4.0(2019-05-02)
- 删除对生命周期结束的Python3.3的支持。
- 修复了Python3.7的测试。由Elliott Sales de Andrade in提供 #36。
- 修复了twised的兼容性问题(twisted.python.failure.Failure需要一个co_code属性)。
1.3.2(2017-04-09)
- 添加对pypy3.5-5.7.1-beta的支持。以前的AttributeError: 'Frame' object has no attribute 'clear'可以筹集。见Pypy 发布#2532。
1.3.1(2017-03-27)
- 修复了由于超过递归限制而对回溯的处理。 修复#15。
1.3.0(2016-03-08)
- 添加了Traceback.from_string。
1.1.0(2015-07-27)
1.0.0(2015-03-30)
- 在回溯中添加了to_dict方法和from_dict类方法。 Beckjake在#5中的贡献。