由于格式错误，Flume HTTPSource拒绝JSON正文

Question

我有一个Flume代理配置，其中我使用HTTPSource从特定服务接收事件数据。出于测试目的，我在Python中创建了一个名为data的字符串对象的静态JSON结构（请参阅下面的代码片段1），并将该对象发送到带有适当头的flume，但是flume总是返回400个错误的请求错误（见下面的代码片段2）。下面的代码段3中提供了关联的flume执行异常消息。在

问题：谁能告诉我静态json请求有什么问题导致flume HTTPSource拒绝它？我是否还遗漏了一些与json数据无关的问题？谢谢。在

SNIPPET 1（Python脚本生成包含静态JSON数据的虚拟HTTP请求）

 import urllib2, json

 serviceName = "serviceA"
 timestamp = datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d %H:%M:%S')

 body = "{ \"service\":\"" + serviceName + "\" }"
 print("BODY: " +  body)

 //My JSON data
 data = "[{ \"headers\" : { \"timestamp\" : \"" + timestamp + "\" }, \"body\" : " + body + " }]"
 print("DATA: "  + data)

 req = urllib2.Request("http://10.1.0.100:5140")
 req.add_header('Content-Type', 'application/json')
 response = urllib2.urlopen(req, data)  

SNIPPET 2-python脚本的执行输出

^{pr2}$

SNIPPET 3-Flume执行输出中的异常消息

  2016-01-13 12:26:48,653 (34313572@qtp-604003190-13)
  [WARN -     org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:      242)] Received bad request from client. 
  org.apache.flume.source.http.HTTPBadRequestException: Request has invalid JSON Syntax.
  at org.apache.flume.source.http.JSONHandler.getEvents(JSONHandler.java:119)
  at  org.apache.flume.source.http.HTTPSource$FlumeHTTPServlet.doPost(HTTPSource.java:240)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:814)
  at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
  at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  at org.mortbay.jetty.Server.handle(Server.java:326)
  at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
  at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
  at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
  at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
  at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
  at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  Caused by: com.google.gson.JsonSyntaxException:  java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line  1 column 67
  at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
  at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
  at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:81)
  at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:60)
  at com.google.gson.Gson.fromJson(Gson.java:795)
  at com.google.gson.Gson.fromJson(Gson.java:761)
  at org.apache.flume.source.http.JSONHandler.getEvents(JSONHandler.java:117)
... 17 more
  Caused by: java.lang.IllegalStateException: Expected a string but was  BEGIN_OBJECT at line 1 column 67
  at com.google.gson.stream.JsonReader.nextString(JsonReader.java:464)
  at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:349)
  at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:337)
  at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
  at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)

Answer 1

正如@martynw在他的评论中指出的，JSONHandler期望并验证JSON具有某种结构。在

这在JavaDocs of JSONHandler中描述：

JSONHandler for HTTPSource that accepts an array of events. This handler throws exception if the deserialization fails because of bad format or any other reason. Each event must be encoded as a map with two key-value pairs.

1. headers - the key for this key-value pair is "headers". The value for this key is another map, which represent the event headers. These headers are inserted into the Flume event as is.

2. body - The body is a string which represents the body of the event. The key for this key-value pair is "body". All key-value pairs are considered to be headers. An example:

[{"headers" : {"a":"b", "c":"d"},"body": "random_body"}, {"headers" : {"e": "f"},"body": "random_body2"}]