为什么使用ThreadPoolExecutor的Dynesty多处理不使用所有内核？

Question

我尝试运行一个简单的示例，使用嵌套采样和dynesty。你知道吗

我从github安装了dynesty：https://github.com/joshspeagle/dynesty

---

计算机设置

操作系统：Mac OSX El Capitan（10.11.6）
CPU:8核
内存：16.0 GB

gcc:4.8.5通过conda install gcc

---

问题设置

我运行了下面的代码（模拟数据、设置优先级/可能性、提交到dynesty）。你知道吗

为了建立多处理，我使用了multiprocessing.Pool、concurrent.futures.ProcessPoolExecutor和concurrent.futures.ThreadPoolExecutor。你知道吗

我尝试了Jupyter Lab、ipython和（脚本）python run_dynesty_test.py中的代码。你知道吗

问题：整个脚本运行良好；但是dynesty/python开始使用所有核心；然后它慢慢开始使用越来越少的核心。最后，大约5分钟后，dynesty/python几乎正好使用1个内核。你知道吗

证据：htop开始读取780%，然后是550%，然后是350%，然后是100%的CPU—在其余的操作中它保持在100%的CPU；除了每隔一分钟读取一次外，htop将读取约250-300%的CPU。你知道吗

---

问题

为什么dynesty/ThreadPoolExecutor/python/etc没有一直使用我的所有内核？你知道吗

---

涉及动态和多处理的代码段

with ThreadPoolExecutor(max_workers=cpu_count()-1) as executor:

    sampler = dynesty.DynamicNestedSampler(
                            loglike,
                            prior,
                            ndim=ndims,
                            nparam=ndims,
                            bound='multi',
                            sample='unif',
                            pool=executor,
                            queue_size=cpu_count())

    sampler.run_nested(nlive_init=100,nlive_batch=100)

    res = sampler.results

---

设置测试的完整脚本

from __future__ import absolute_import,\
              unicode_literals, print_function

from multiprocessing import set_start_method
set_start_method('forkserver')

import dynesty
import math
import os
import threading, subprocess

from sys import platform
from numpy import pi, sin, cos, linspace
from pylab import *#;ion()

from multiprocessing import Pool, cpu_count

if not os.path.exists("chains"): os.mkdir("chains")

# plotting
import matplotlib
from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

def gaussian1Dp(cube):
    center = cube[0]
    width  = cube[1]
    height = cube[2]
    return lambda y: height*np.exp(-0.5*(( (center - y) / width)**2))

np.random.seed(42)

param0a= -0.5
param0b= 0.5
param1a= 0.1
param1b= 0.1
param2a= 0.8
param2b= 0.8

yunc  = 0.1
nPts  = int(100)
nThPts= int(1e3)

xmin  = -1
xmax  =  1
dx    = 0.1*(xmax - xmin)

yuncs = np.random.normal(yunc, 1e-2 * yunc, nPts)
thdata= np.linspace(xmin-dx, xmax+dx, nThPts)

xdata = np.linspace(xmin,xmax,nPts)

ydata = model([param0a,param1a,param2a])(xdata) \
        + model([param0b,param1b,param2b])(xdata)

yerr  = np.random.normal(0, yuncs, nPts)
zdata = ydata + yerr

figure(figsize=(10,10))
plot(thdata, model([param0a,param1a,param2a])(thdata) \
        + model([param0b,param1b,param2b])(thdata))
errorbar(xdata, zdata, yunc*ones(zdata.size), fmt='o')
show()

def prior(cube):
    cube[0] = cube[0]*2 - 1
    cube[1] = cube[1]*2
    cube[2] = cube[2]*2

    return cube

def loglike(cube):
    modelNow = model(cube)(xdata)
    return -0.5*((modelNow - ydata)**2./yuncs**2.).sum()

from concurrent.futures import ThreadPoolExecutor,\
                               ProcessPoolExecutor

if __name__ == '__main__':
    if not os.path.exists("chains"): os.mkdir("chains")
    n_params = len(parameters)
    ndims = 3

    with ThreadPoolExecutor(max_workers=cpu_count()-1) as executor:

        sampler = dynesty.DynamicNestedSampler(
                                loglike,
                                prior,
                                ndim=ndims,
                                nparam=ndims,
                                bound='multi',
                                sample='unif',
                                pool=executor,
                                queue_size=cpu_count())

        sampler.run_nested(nlive_init=100, nlive_batch=100)

        res = sampler.results

from dynesty import plotting as dyplot

# evidence check
fig, axes = dyplot.runplot(res, color='red',
                lnz_truth=lnz_truth,
                truth_color='black',
                logplot=True)

fig.tight_layout()

joblib.dump(res,'dynesty_double_gaussian_test_results.joblib.save')