如何在仍显示摄像机记录的情况下，对照训练过的模型检查图像？

Question

我有一个python程序，它在帮助下OpenCV (cv2)打开一个摄像头会话。我设置了一个感兴趣的区域，在该区域中，从摄像机记录中提取图像，并以经过训练的keras模型作为参数进行传递，以获得预测。我的问题是，我怎样才能每10秒检查一次模型？我尝试使用time.sleep(10)将整个窗口冻结（因为它发生在while循环中）10秒钟。这意味着整个录制过程每10秒停止一次，而我希望能够持续录制，并仅每10秒检查一次模型。 这是我目前的代码：

import cv2
import numpy as np
from tensorflow import keras
import time

#import playsound

# wait for the sound to finish playing?
blocking = True

model = keras.models.load_model("model2.h5")
cam = cv2.VideoCapture(0)

####region of interest dimensions
startX =  800
startY = 0
finishX = 1200
finishY = 400
while(1):
    ret, frame = cam.read()
    if ret:
        ### displays video recording and region of interest
        frame = cv2.flip(frame,1)
        display = cv2.rectangle(frame.copy(),(startX,startY),(finishX,finishY),(0,0,255),2) 
        cv2.imshow('Total Input',display)
        ROI = frame[startY:finishY, startX:finishX].copy()
        cv2.imshow('Region of Interest', ROI)
         
        #pauses for 10 seconds
        time.sleep(10)

        img = cv2.resize(display, (128, 128)) #R
        img = img.reshape(1, 128, 128, 3)
        predictions = model.predict(img) # Make predictions towards the test set
        predicted_label = np.argmax(predictions) # Get index of the predicted label from prediction
        print(predicted_label)
        if cv2.waitKey(10) & 0xFF == ord('q'):
          break

cam.release()
cv2.destroyAllWindows()

我在考虑是否应该使用线程，但我对python中的线程并不十分熟悉。有人知道如何做到这一点吗

Answer 1

您需要有一个变量来指示您上次进行预测的时间。然后，在每次迭代中，将时间与该变量进行比较，以确定是否是进行另一次预测的正确时间

使用time.time（）：

我使用了time.time()方法来跟踪时间，该方法返回的时间是秒：

last_prediction_time = time.time()

以下是我在while循环中所做的操作：

# Make predictions every 10 seconds:

if last_prediction_time - time.time() >= 10:
    img = cv2.resize(display, (128, 128))  # R
    img = img.reshape(1, 128, 128, 3)
    predictions = model.predict(img)  # Make predictions towards the test set
    predicted_label = np.argmax(predictions)  # Get index of the predicted label from prediction
    print(predicted_label)

    # update the last prediction time
    last_prediction_time = time.time()

一切都在一起：

import cv2
import numpy as np
from tensorflow import keras
import time

# import playsound

# wait for the sound to finish playing?
blocking = True

model = keras.models.load_model("model2.h5")
cam = cv2.VideoCapture(0)

####region of interest dimensions
startX = 800
startY = 0
finishX = 1200
finishY = 400

last_prediction_time = time.time()

while (1):
    ret, frame = cam.read()
    if ret:
        ### displays video recording and region of interest
        frame = cv2.flip(frame, 1)
        display = cv2.rectangle(frame.copy(), (startX, startY), (finishX, finishY), (0, 0, 255), 2)
        cv2.imshow('Total Input', display)
        ROI = frame[startY:finishY, startX:finishX].copy()
        cv2.imshow('Region of Interest', ROI)

        # Make predictions every 10 seconds:

        if last_prediction_time - time.time() >= 10:
            img = cv2.resize(display, (128, 128))  # R
            img = img.reshape(1, 128, 128, 3)
            predictions = model.predict(img)  # Make predictions towards the test set
            predicted_label = np.argmax(predictions)  # Get index of the predicted label from prediction
            print(predicted_label)

            # update the last prediction time
            last_prediction_time = time.time()

        if cv2.waitKey(10) & 0xFF == ord('q'):
            break

cam.release()
cv2.destroyAllWindows()

Answer 2

您可以在while循环中运行一个检查，查看是否已经过了10秒，然后运行函数并重置计时器。大概是这样的：

from datetime import datetime

last_time = datetime.now()

while(1):
    # Other stuff

    if (datetime.now() - last_time).total_seconds() > 10:
        last_time = datetime.now()
        img = cv2.resize(display, (128, 128)) #R
        img = img.reshape(1, 128, 128, 3)
        predictions = model.predict(img) # Make predictions towards the test set
        predicted_label = np.argmax(predictions) # Get index of the predicted label from prediction
        print(predicted_label)
    if cv2.waitKey(10) & 0xFF == ord('q'):
      break

Answer 3

我通常使用帧ID进行此操作。基本上，您的模型只会在n帧之后进行预测。下面是如何使用它的代码。您可以编辑要跳过的帧数：

frame_id =0    
while(1):
  frame_id +=1
  ret, frame = cam.read()
  if ret:
    ### displays video recording and region of interest
    frame = cv2.flip(frame,1)
    display = cv2.rectangle(frame.copy(),(startX,startY),(finishX,finishY),(0,0,255),2) 
    cv2.imshow('Total Input',display)
    ROI = frame[startY:finishY, startX:finishX].copy()
    cv2.imshow('Region of Interest', ROI)
     
    #pauses for 10 seconds
    time.sleep(10)

    img = cv2.resize(display, (128, 128)) #R
    img = img.reshape(1, 128, 128, 3)
    if fram_id % 10 == 0:
        predictions = model.predict(img) # Make predictions towards the test set
        predicted_label = np.argmax(predictions) # Get index of the predicted label from prediction
        print(predicted_label)
    if cv2.waitKey(10) & 0xFF == ord('q'):
      break