权值量化后的张量流模型误差

Question

在量化模型的权重，然后将其保存为TF服务之后，我能够得到预测，尽管比以前慢了一些。你知道吗

然而，在Quantizing权重和模型，然后保存它以供服务和使用之后，我从客户机得到以下错误 在V100上运行tf服务docker容器（tensorflow/服务：最新gpu)-客户和模型

_Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNIMPLEMENTED details = "Broadcast between [1,9,4] and [221,1,4] is not supported yet. [[{{node anchors_3/add_2/eightbit}}]] [[{{node filtered_detections/map/while/non_max_suppression_47/NonMaxSuppressionV3}}]]" debug_error_string = "{"created":"@1552349235.607723578","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"Broadcast between [1,9,4] and [221,1,4] is not supported yet.\n\t [[{{node anchors_3/add_2/eightbit}}]]\n\t [[{{node filtered_detections/map/while/non_max_suppression_47/NonMaxSuppressionV3}}]]","grpc_status":12}"

在TF服务端，我得到以下错误

2019-03-12 00:07:13.149087: I external/org_tensorflow/tensorflow/core/kernels/quantized_add_op.cc:546] ndims=3 2019-03-12 00:07:13.149233: I external/org_tensorflow/tensorflow/core/kernels/quantized_add_op.cc:547] bcast.x_reshape()=[1,9,4] 2019-03-12 00:07:13.149309: I external/org_tensorflow/tensorflow/core/kernels/quantized_add_op.cc:549] bcast.y_reshape()=[221,1,4] 2019-03-12 00:07:13.149348: I external/org_tensorflow/tensorflow/core/kernels/quantized_add_op.cc:551] bcast.x_bcast()=[221,1,1] 2019-03-12 00:07:13.149385: I external/org_tensorflow/tensorflow/core/kernels/quantized_add_op.cc:553] bcast.y_bcast()=[1,9,1]

模型是一种图像检测模型

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input_image'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, -1, -1, 3)
        name: input_1_2:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['filtered_detections/map/TensorArrayStack/TensorArrayGatherV3:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 300, 4)
        name: filtered_detections/map/TensorArrayStack/TensorArrayGatherV3:0
    outputs['filtered_detections/map/TensorArrayStack_1/TensorArrayGatherV3:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 300)
        name: filtered_detections/map/TensorArrayStack_1/TensorArrayGatherV3:0
    outputs['filtered_detections/map/TensorArrayStack_2/TensorArrayGatherV3:0'] tensor_info:
        dtype: DT_INT32
        shape: (-1, 300)
        name: filtered_detections/map/TensorArrayStack_2/TensorArrayGatherV3:0
  Method name is: tensorflow/serving/predict