无法在pyspark中加载模型并使用grp提供服务

Question

我正在尝试使用grpc提供一个spark葡萄酒分类模型。在

我已经训练过模特了。 我已经用flask编写了在http服务器上服务的代码。在

现在，我试着用grpc来服务它，试图了解性能的提高。在

但我得到了以下错误-

服务器端-

ERROR:root:Exception calling application: classify() missing 1 required positional argument: 'context'
Traceback (most recent call last):
  File "/media/dosi/Data/mlframework/venv/grpc/lib/python3.5/site-packages/grpc/_server.py", line 375, in _call_behavior
    return behavior(argument, context), True
TypeError: classify() missing 1 required positional argument: 'context'

客户端-

^{pr2}$

我有以下疑问，但我认为这不应该是错误的原因-

• grpc安装在python虚拟环境中。也许，在执行spark提交时，我没有使用虚拟env。我不认为这是问题所在，因为在这种情况下，我不应该导入grpc，更不用说启动服务器和运行客户端了。在

服务器代码-

import time, sys,  os
from pyspark import SparkContext, SparkConf
from pyspark.sql import SparkSession, SQLContext
import config
import grpc
import wine_pb2
import wine_pb2_grpc
from concurrent import futures

_ONE_DAY_IN_SECONDS = 60 * 60 * 24

class WineClassifier(wine_pb2_grpc.WineClassifierServicer):

  def classify(self, request, context):
    return wine_pb2.WineClass(request.mg)

def init_spark_context():
    # load spark context
    conf = SparkConf().setAppName("wine-app-server")
    # IMPORTANT: pass aditional Python modules to each worker
    sc = SparkContext(conf=conf)
    spark = SparkSession \
    .builder \
    .appName("Python Spark SQL basic example") \
    .config("spark.some.config.option", "some-value") \
    .getOrCreate()

    return sc, spark


def run_server():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier, server)
    server.add_insecure_port('[::]:50051')
    server.start()
    try:
        while True:
            time.sleep(_ONE_DAY_IN_SECONDS)
    except keyboardInterrupt:
        server.stop(0)

if __name__ == "__main__":
    # Init spark context and load libraries
    sc, spark = init_spark_context()

    # start web server
    run_server()

原型文件-

syntax="proto3";

service WineClassifier {
  rpc classify (WineAttributes) returns (WineClass) {}
}

message WineAttributes {
    double alcohol = 1;
    double malic_acid = 2;
    double ash = 3;
    double alcalinity = 4;
    int32 mg = 5;
    double phenols = 6;
    double flavanoids = 7;
    double flavaniods = 8;
    double nf_phenols = 9;
    double proanthocyanins = 10;
    double color_intensity = 11;
    double hue = 12;
    double diluted = 13;
    double proline = 14;
}

message WineClass {
  int32 class = 1;
}

你知道这里出什么问题了吗？在

Answer 1

我认为问题是在wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier, server)中传递的是WineClassifier类对象，而不是{}类的实例。尝试将该行改为wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier(), server)。在