使用线程在c中同时调用多个Pythonextensions

Question

我有一个c程序，它使用两个线程。现在我想在两个线程中调用一个Python函数。当我在一个线程中只调用一个python函数时，它确实可以工作，但是当同时调用这两个函数时，它会出现以下错误：
致命的Python错误：已跟踪GC对象

环顾四周，我发现这是由python函数引起的，而垃圾收集器并没有区分两者。现在我似乎找不到任何关于这个话题的例子。我确实找到了一些可以帮助我的线索，但同样没有关于如何实施它们的例子。我想我需要让GC知道两者之间的区别，方法类似于malloc（）

我想我可以使用以下函数PyObject_GC_New（），但我不知道如何实现它

在我的代码下面，当您注释掉speakerPlay（）中的audioSend（）并将其替换为例如print语句时，它就会起作用

#include <stdio.h>
#include <Python.h>
#include <numpy/ndarrayobject.h>
#include <pthread.h>

typedef struct{
    int length;
    int duration;
}args;

void* micRecord(void* val);
void* calc(void* val);
void* speakerPlay(void* val);

npy_float** audioReceive(int length);
int audioSend(int length);

int main(){
  // init python imports
  Py_Initialize();
  import_array();

  int length = 2;
  int duration = 10;

  args* tArgs = &(args){.length = length, .duration = duration};

  pthread_t recordHandle;
  if( pthread_create(&recordHandle , NULL, micRecord, tArgs))exit( EXIT_FAILURE );

  pthread_t playHandle;
  if( pthread_create(&playHandle , NULL, speakerPlay, tArgs))exit( EXIT_FAILURE );

  void* result;
  if(pthread_join(recordHandle, &result)==-1)
  {
      exit(EXIT_FAILURE);
      printf("could not join\n");
  }

  if(pthread_join(playHandle, &result)==-1)
  {
      exit(EXIT_FAILURE);
      printf("could not join\n");
  }
  printf("Threads joined\n");

  Py_Finalize();
  return 0;
}

void* micRecord(void* rArgs){
  printf("in micRecord\n\n");

  args tArgs = *(args*)rArgs;
  int length = tArgs.length;
  int duration = tArgs.duration;

  npy_float** tempArray;

  for(int i = 0; i<duration; i++){
    tempArray = audioReceive(length);
  }

  printf("micRecord thread done\n");
  pthread_exit(NULL);
}

void* speakerPlay(void* pArgs){
  printf("in speakerPlay\n\n");

  args tArgs = *(args*)pArgs;
  int length = tArgs.length;
  int duration = tArgs.duration;

  for(int i = 0; i < duration; i++){
    audioSend(length);
    //printf("sending\n");
  }

  printf("Speaker thread done\n");
  pthread_exit(NULL);
}

npy_float** audioReceive(int length){
  //variables init
  PyObject *pName, *pModule, *pFunc, *pArgs;
  PyObject* pValue;
  npy_float** array;

  // import the .py file in which the to-call python function is located
  pName = PyUnicode_FromString("receiveSound");
  pModule = PyImport_Import(pName);
  Py_DECREF(pName);
  if(pModule == NULL) printf("its null\n");

  if (pModule != NULL) {
    // Get the reference to the to-call python function and checks if its callable
    pFunc = PyObject_GetAttrString(pModule, "recordSound");
    if (pFunc && PyCallable_Check(pFunc)) {
      // set arguments you want to pass along to the python function
      pArgs = PyTuple_New(1);
      PyTuple_SetItem(pArgs, 0, PyLong_FromLong(length));

      // call the python function and return the numpy array in pValue
      pValue = PyObject_CallObject(pFunc, pArgs);
      if (pValue == NULL) {
        Py_DECREF(pFunc);
        Py_DECREF(pModule);
        PyErr_Print();
        fprintf(stderr,"Call failed\n");
        return NULL;
      }

      // get the type description from the array content
      PyArray_Descr *descr;
      descr = PyArray_DescrFromType(PyArray_TYPE(pValue));

      // convert the numpy array to, a by c-compiler useable format, npy_float array
      if (PyArray_AsCArray(&pValue, (void*)&array, PyArray_DIMS(pValue), PyArray_NDIM(pValue), descr) < 0)  {
        PyErr_SetString(PyExc_TypeError, "error converting to c array");
        return NULL;
      }
      //printf("input\n%"NPY_FLOAT_FMT"\n%"NPY_FLOAT_FMT"\n%"NPY_FLOAT_FMT"\n%"NPY_FLOAT_FMT"\n", array[0], array[5], array[10], array[12]);
      Py_DECREF(pValue);
    }
    // if there was no such function in the .py file
    else {
        if (PyErr_Occurred())
            PyErr_Print();
        fprintf(stderr, "Cannot find function \n");
    }
    Py_XDECREF(pFunc);
    Py_DECREF(pModule);
  }

  // if there was no such .py file
  else {
    PyErr_Print();
    fprintf(stderr, "Failed to load \\n");
    return NULL;
  }

  return array;
}

int audioSend(int length){
  // init variables
  PyObject *pName, *pModule, *pFunc;
  PyObject *pValue;

  // import the .py file in which the to-call python function is located
  pName = PyUnicode_FromString("ss");
  pModule = PyImport_Import(pName);
  Py_DECREF(pName);
  if(pModule == NULL) printf("its null\n");

  if (pModule != NULL) {
      // Get the reference to the to-call python function and checks if its callable
      pFunc = PyObject_GetAttrString(pModule, "playSound");
      if (pFunc && PyCallable_Check(pFunc)) {

        // call the python function to play the sound by reading the array
        pValue = PyObject_CallObject(pFunc, NULL);

        // check if array is played
        if (pValue != NULL) {
            Py_DECREF(pValue);
        }

        // if the array was not correctly read
        else {
            Py_DECREF(pFunc);
            Py_DECREF(pModule);
            PyErr_Print();
            fprintf(stderr,"Call failed\n");
            return 1;
        }
      }

      // if no such function exists in the .py file
      else {
          if (PyErr_Occurred())
              PyErr_Print();
          fprintf(stderr, "Cannot find function \n");
      }
      Py_XDECREF(pFunc);
      Py_DECREF(pModule);
    }
    // if no such .py file exists
    else {
        PyErr_Print();
        fprintf(stderr, "Failed to load \\n");
        return 1;
    }
  return 0;
}

下面是两个python代码文件，它们非常基本。它们包含在我的设备上的receiveSound.py和ss.py文件中

receiveSound.py：

import numpy as np
from array import array

def recordSound(length=50):

    print ("-----------------RECORDING SOUND------------------")

    # Make the array `rec_array`
    recArray = np.array([[0,1,2,3,4], [5,6,7,8,9]], dtype=np.float)

    return recArray

党卫军

def playSound():

    print ("----------------SOUND IS PLAYED------------------")#recArray[49]
    return 1

这是我的makefile，basicANC.c是我的c代码

basicANCmake: basicANC.c
    gcc -o basicANC -I/usr/include/python3.6m basicANC.c -lpython3.6m -lpthread
run: basicANCmake
    PYTHONPATH=. ./basicANC

Answer 1

Python通过一个全局锁（恰当地称为“全局解释器锁”）保护其对象空间，防止并发访问。为了从宿主C程序的多个线程中使用相同的嵌入式Python解释器，您需要尊重并适应这一点。该文档包含a section describing these requirements。有很多方法可以对所有这些进行细粒度控制，但是，根据文档

[t]he typical idiom for calling into Python from a C thread is:

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

/* Perform Python actions here. */
result = CallSomeFunction();
/* evaluate result or handle exception */

/* Release the thread. No Python API allowed beyond this point. */
PyGILState_Release(gstate);

我建议通过更新micRecord()和speakerPlay()函数，将对audioReceive()和audioSend()的每个调用包装在PyGILState_Ensure()和PyGILState_Release()之间，从而将该模式合并到程序中

更新：

另外,

• 根据您使用的Python版本的不同，您可能需要在使用任何线程和GIL操作函数之前调用PyEval_InitThreads()。您应该在同一线程中调用Py_Initialize()后执行此操作。即使在实际上不需要它的Python版本上，这样做也是安全的

• 在其他线程获取GIL之前，您可能需要主线程来释放GIL。在主线程中调用PyGILState_Check()函数将确认（或反驳）这一点。有几种方法可以释放GIL，但最简单的可能是将main()的部分括起来，该部分创建并连接Py_BEGIN_ALLOW_THREADS和Py_END_ALLOW_THREADS之间的子线程。请阅读这些宏的文档，了解有关这些宏的详细信息以及如何使用它们