高QPS场景下Python importlib包的性能问题

2024-09-29 22:00:58 发布

您现在位置:Python中文网/ 问答频道 /正文

我在程序中使用importlib包动态执行用户函数,如下所示

modules = defaultdict()
if module_path not in modules:
    modules[module_path] = importlib.import_module(module_path)

当我对我的程序进行压力测试时,我发现import_模块包在高QPS场景下存在性能问题。下面是我使用line_profiler工具进行的比较,底部是高QPS场景(QPS=10)

我现在很困惑,想知道这种现象的原因吗?我该怎么解决呢

# low QPS
Timer unit: 1e-06 s

Total time: 0.842017 s
File: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py
Function: import_module at line 109

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   109                                           def import_module(name, package=None):
   110                                               """Import a module.
   111
   112                                               The 'package' argument is required when performing a relative import. It
   113                                               specifies the package to use as the anchor point from which to resolve the
   114                                               relative import to an absolute import.
   115
   116                                               """
   117         1          1.0      1.0      0.0      level = 0
   118         1          2.0      2.0      0.0      if name.startswith('.'):
   119                                                   if not package:
   120                                                       msg = ("the 'package' argument is required to perform a relative "
   121                                                              "import for {!r}")
   122                                                       raise TypeError(msg.format(name))
   123                                                   for character in name:
   124                                                       if character != '.':
   125                                                           break
   126                                                       level += 1
   127         1     842014.0 842014.0    100.0      return _bootstrap._gcd_import(name[level:], package, level)

# high QPS (10)
Timer unit: 1e-06 s

Total time: 6.16965 s
File: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py
Function: import_module at line 109

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   109                                           def import_module(name, package=None):
   110                                               """Import a module.
   111
   112                                               The 'package' argument is required when performing a relative import. It
   113                                               specifies the package to use as the anchor point from which to resolve the
   114                                               relative import to an absolute import.
   115
   116                                               """
   117         1          2.0      2.0      0.0      level = 0
   118         1          2.0      2.0      0.0      if name.startswith('.'):
   119                                                   if not package:
   120                                                       msg = ("the 'package' argument is required to perform a relative "
   121                                                              "import for {!r}")
   122                                                       raise TypeError(msg.format(name))
   123                                                   for character in name:
   124                                                       if character != '.':
   125                                                           break
   126                                                       level += 1
   127         1    6169641.0 6169641.0    100.0      return _bootstrap._gcd_import(name[level:], package, level)

希望能得到你的答案,谢谢


Tags: thetonameimportpackageiftimeis

热门问题