<p>两年前,我对这个问题发表评论很久之后,这个问题就一直困扰着我!我自己最近也遇到了同样的问题,我发现文档非常稀少,我想你们中的大多数人一定都经历过。所以我试着研究一下<a href="https://github.com/pypa/setuptools/tree/master/setuptools" rel="nofollow noreferrer">setuptools</a>和{a2}的源代码,看看能不能找到一种或多或少标准化的方法来回答你提出的这两个问题。在</p>
<hr/>
<p>你问的第一个问题</p>
<blockquote>
<p><em>Question #1</em>: how to run a terminal command (i.e., <code>make</code> in my case) during the build process of the package, using setuptools/distutils?</p>
</blockquote>
<p>有很多方法,它们都涉及在调用<code>setup</code>时设置<code>cmdclass</code>。<code>setup</code>的参数<code>cmdclass</code>必须是将根据发行版的构建或安装需要执行的命令名与继承自<a href="https://docs.python.org/3/distutils/apiref.html#module-distutils.cmd" rel="nofollow noreferrer">^{<cd5>}</a>基类的类之间的映射(作为补充说明,<code>setuptools.command.Command</code>类是从<code>distutils</code>'<code>Command</code>类派生的,因此可以直接从<code>setuptools</code>实现派生)。在</p>
<p><code>cmdclass</code>允许您定义任何命令名,如<a href="https://stackoverflow.com/a/43013234/7253166">ayoon</a>所做的,然后在从命令行调用<code>python setup.py install-option="customcommand"</code>时具体执行它。问题在于,当试图通过<code>pip</code>或调用<code>python setup.py install</code>安装包时,它不是标准命令。实现这一点的标准方法是检查<code>setup</code>将尝试在正常安装中执行哪些命令,然后重载特定的<code>cmdclass</code>。在</p>
<p>从<a href="https://github.com/pypa/setuptools/blob/bb71fd1bed9f5e5e239ef99be82ed57e9f9b1dda/setuptools/__init__.py#L126" rel="nofollow noreferrer">^{<cd16>}</a>和<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/core.py" rel="nofollow noreferrer">^{<cd17>}</a>开始,<code>setup</code>将运行命令it <a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/core.py#L145:L148" rel="nofollow noreferrer">found in the command line</a>,这让我们假设只是一个普通的<code>install</code>。在<code>setuptools.setup</code>的情况下,这将触发一系列测试,这些测试将查看是否要求助于对<code>distutils.install</code>命令类的简单调用,如果没有发生这种情况,它将尝试运行<a href="https://github.com/pypa/setuptools/blob/master/setuptools/command/install.py#L109" rel="nofollow noreferrer">^{<cd22>}</a>。反过来,这个命令可以做很多事情,但关键是决定是否调用<code>build_clib</code>、<code>build_py</code>和/或<code>build_ext</code>命令。如果需要,<code>distutils.install</code>只运行<code>build</code>,它还运行<a href="https://github.com/pypa/setuptools/blob/master/setuptools/command/build_clib.py" rel="nofollow noreferrer">^{<cd23>}</a>、<a href="https://github.com/pypa/setuptools/blob/master/setuptools/command/build_py.py" rel="nofollow noreferrer">^{<cd24>}</a>和/或{a11}。这意味着无论您使用<code>setuptools</code>还是<code>distutils</code>,如果需要从源代码构建,命令<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/command/build_clib.py" rel="nofollow noreferrer">^{<cd23>}</a>、<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/command/build_py.py" rel="nofollow noreferrer">^{<cd24>}</a>和/或<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/command/build_ext.py" rel="nofollow noreferrer">^{<cd25>}</a>将被运行,因此我们希望用<code>setup</code>的{<cd1>}重载这些命令,问题就变成这三个命令中的哪一个。在</p>
<ul>
<li><code>build_py</code>用于“构建”纯python包,因此我们可以安全地忽略它。在</li>
<li><code>build_ext</code>用于生成声明的扩展模块,这些模块通过对<code>setup</code>函数的调用的<code>ext_modules</code>参数传递。如果我们希望重载这个类,构建每个扩展的主方法是<a href="https://github.com/pypa/setuptools/blob/master/setuptools/command/build_ext.py#L190" rel="nofollow noreferrer">^{<cd42>}</a>(或者对于distutils,<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/command/build_ext.py#L485" rel="nofollow noreferrer">here</a>)</li>
<li><code>build_clib</code>用于生成声明的库,这些库通过调用<code>libraries</code>参数传递给<code>setup</code>函数。在本例中,我们应该用派生类重载的主要方法是<a href="https://github.com/pypa/setuptools/blob/master/setuptools/command/build_clib.py#L23" rel="nofollow noreferrer">^{<cd46>}</a>方法(<a href="https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/distutils/command/build_clib.py#L181" rel="nofollow noreferrer">here</a>表示<code>distutils</code>)。在</li>
</ul>
<p>我将分享一个示例包,它通过Makefile使用<code>setuptools</code><code>build_ext</code>命令,通过Makefile构建一个toyc静态库。这种方法可以适应使用<code>build_clib</code>命令,但是您必须签出<code>build_clib.build_libraries</code>的源代码。在</p>
<p><strong>设置.py</strong></p>
<pre><code>import os, subprocess
import setuptools
from setuptools.command.build_ext import build_ext
from distutils.errors import DistutilsSetupError
from distutils import log as distutils_logger
extension1 = setuptools.extension.Extension('test_pack_opt.test_ext',
sources = ['test_pack_opt/src/test.c'],
libraries = [':libtestlib.a'],
library_dirs = ['test_pack_opt/lib/'],
)
class specialized_build_ext(build_ext, object):
"""
Specialized builder for testlib library
"""
special_extension = extension1.name
def build_extension(self, ext):
if ext.name!=self.special_extension:
# Handle unspecial extensions with the parent class' method
super(specialized_build_ext, self).build_extension(ext)
else:
# Handle special extension
sources = ext.sources
if sources is None or not isinstance(sources, (list, tuple)):
raise DistutilsSetupError(
"in 'ext_modules' option (extension '%s'), "
"'sources' must be present and must be "
"a list of source filenames" % ext.name)
sources = list(sources)
if len(sources)>1:
sources_path = os.path.commonpath(sources)
else:
sources_path = os.path.dirname(sources[0])
sources_path = os.path.realpath(sources_path)
if not sources_path.endswith(os.path.sep):
sources_path+= os.path.sep
if not os.path.exists(sources_path) or not os.path.isdir(sources_path):
raise DistutilsSetupError(
"in 'extensions' option (extension '%s'), "
"the supplied 'sources' base dir "
"must exist" % ext.name)
output_dir = os.path.realpath(os.path.join(sources_path,'..','lib'))
if not os.path.exists(output_dir):
os.makedirs(output_dir)
output_lib = 'libtestlib.a'
distutils_logger.info('Will execute the following command in with subprocess.Popen: \n{0}'.format(
'make static && mv {0} {1}'.format(output_lib, os.path.join(output_dir, output_lib))))
make_process = subprocess.Popen('make static && mv {0} {1}'.format(output_lib, os.path.join(output_dir, output_lib)),
cwd=sources_path,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=True)
stdout, stderr = make_process.communicate()
distutils_logger.debug(stdout)
if stderr:
raise DistutilsSetupError('An ERROR occured while running the '
'Makefile for the {0} library. '
'Error status: {1}'.format(output_lib, stderr))
# After making the library build the c library's python interface with the parent build_extension method
super(specialized_build_ext, self).build_extension(ext)
setuptools.setup(name = 'tester',
version = '1.0',
ext_modules = [extension1],
packages = ['test_pack', 'test_pack_opt'],
cmdclass = {'build_ext': specialized_build_ext},
)
</code></pre>
<p><strong>测试包</p>
^{pr2}$
<p><strong>测试软件包</strong></p>
<pre><code>from __future__ import absolute_import, print_function
import test_pack_opt.test_ext
</code></pre>
<p><strong>测试软件包opt/src/Makefile</strong></p>
<pre><code>LIBS = testlib.so testlib.a
SRCS = testlib.c
OBJS = testlib.o
CFLAGS = -O3 -fPIC
CC = gcc
LD = gcc
LDFLAGS =
all: shared static
shared: libtestlib.so
static: libtestlib.a
libtestlib.so: $(OBJS)
$(LD) -pthread -shared $(OBJS) $(LDFLAGS) -o $@
libtestlib.a: $(OBJS)
ar crs $@ $(OBJS) $(LDFLAGS)
clean: cleantemp
rm -f $(LIBS)
cleantemp:
rm -f $(OBJS) *.mod
.SUFFIXES: $(SUFFIXES) .c
%.o:%.c
$(CC) $(CFLAGS) -c $<
</code></pre>
<p><strong>test_pack_opt/src/test.c</strong></p>
<pre><code>#include <Python.h>
#include "testlib.h"
static PyObject*
test_ext_mod_test_fun(PyObject* self, PyObject* args, PyObject* keywds){
testlib_fun();
return Py_None;
}
static PyMethodDef TestExtMethods[] = {
{"test_fun", (PyCFunction) test_ext_mod_test_fun, METH_VARARGS | METH_KEYWORDS, "Calls function in shared library"},
{NULL, NULL, 0, NULL}
};
#if PY_VERSION_HEX >= 0x03000000
static struct PyModuleDef moduledef = {
PyModuleDef_HEAD_INIT,
"test_ext",
NULL,
-1,
TestExtMethods,
NULL,
NULL,
NULL,
NULL
};
PyMODINIT_FUNC
PyInit_test_ext(void)
{
PyObject *m = PyModule_Create(&moduledef);
if (!m) {
return NULL;
}
return m;
}
#else
PyMODINIT_FUNC
inittest_ext(void)
{
PyObject *m = Py_InitModule("test_ext", TestExtMethods);
if (m == NULL)
{
return;
}
}
#endif
</code></pre>
<p><strong>test_pack_opt/src/testlib.c</strong></p>
<pre><code>#include "testlib.h"
void testlib_fun(void){
printf("Hello from testlib_fun!\n");
}
</code></pre>
<p><strong>test_pack_opt/src/testlib.h</strong></p>
<pre><code>#ifndef TESTLIB_H
#define TESTLIB_H
#include <stdio.h>
void testlib_fun(void);
#endif
</code></pre>
<p>在本例中,我想使用定制Makefile构建的c库只有一个函数,它将<code>"Hello from testlib_fun!\n"</code>打印到stdout。<code>test.c</code>脚本是python和这个库的单个函数之间的一个简单接口。我的想法是告诉<code>setup</code>我想构建一个名为<code>test_pack_opt.test_ext</code>的c扩展,它只有一个源文件:<code>test.c</code>接口脚本,我还告诉扩展必须链接到静态库<code>libtestlib.a</code>。最重要的是我结束了使用<code>specialized_build_ext(build_ext, object)</code>加载<code>build_ext</code>cmdclass。只有当您希望能够调用<code>super</code>来分派给父类方法时,<code>object</code>的继承才是必需的。<code>build_extension</code>方法以一个<code>Extension</code>实例作为第二个参数,为了更好地处理其他需要<code>build_extension</code>行为的<code>Extension</code>实例,我检查这个扩展是否有特殊的名称,如果它没有,我调用<code>super</code>的{<cd42>}方法。在</p>
<p>对于特殊库,我只使用<code>subprocess.Popen('make static ...')</code>调用Makefile。传递给shell的命令的其余部分只是将静态库移动到某个默认位置,在该位置库应该能够将其链接到已编译扩展的其余部分(该扩展也是使用<code>super</code>的<code>build_extension</code>方法编译的)。在</p>
<p>正如您可以想象的那样,您可以用很多不同的方式来组织这些代码,将它们全部列出是没有意义的。我希望这个例子能够说明如何调用Makefile,以及在标准安装中应该重载哪个<code>cmdclass</code>和{<cd8>}派生类来调用<code>make</code>。在</p>
<hr/>
<p>现在,进入问题2。在</p>
<blockquote>
<p><em>Question #2</em>: how to ensure, that such terminal command is executed only if the corresponding extra1 is specified during the installation process?</p>
</blockquote>
<p>这可能是因为不推荐使用的<code>features</code>参数<code>setuptools.setup</code>。标准的方法是根据满足的要求尝试安装包。^{<cd76>列出强制要求,<code>extras_requires</code>列出可选要求。例如从<a href="http://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies" rel="nofollow noreferrer">^{<cd9>} documentation</a></p>
<pre><code>setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
</code></pre>
<p>您可以通过调用<code>pip install Project-A[PDF]</code>来强制安装可选的必需包,但是如果由于某种原因,名为extra的<code>'PDF'</code>的要求事先得到满足,<code>pip install Project-A</code>将以相同的<code>"Project-A"</code>功能结束。这意味着“Project-A”的安装方式并不是针对命令行中指定的每一个额外的命令而定制的,“Project-A”将始终尝试以相同的方式安装,并且可能由于不可用的可选要求而导致功能减少。在</p>
<p>据我所知,这意味着,为了只在指定了[extra1]的情况下编译和安装模块X,您应该将模块X作为一个单独的包发送,并通过<code>extras_require</code>依赖它。假设模块X将在<code>my_package_opt</code>中交付,您的<code>my_package</code>的设置应该如下所示</p>
<pre><code>setup(
name="my_package",
...
extras_require={
'extra1': ["my_package_opt"],
}
)
</code></pre>
<hr/>
<p>很抱歉,我的回答太长了,但我希望能有所帮助。请不要犹豫指出任何概念性错误或命名错误,因为我主要是从<code>setuptools</code>源代码中推断出来的。在</p>