java应用程序引擎映射减少控制器回调上的异常
在我们的AppEngineJava实例上运行MapReduce作业时，它会在一段时间后暂停，我得到一个异常，请求太大。从例外情况来看，它不清楚如何解决这个问题，因为问题发生在洗牌阶段，而洗牌阶段应该是无缝的
我了解实体大小、超时等方面的限制，但我不确定在哪里向API指出这一点
我使用的代码与代码here非常相似，它只计算不同的属性属性，让servlet几乎保持不变
这就是我得到的：
2012-12-02 13:18:53.632 /mapreduce/controllerCallback 500 535ms 0kb AppEngine-Google; (+http://code.google.com/appengine)
0.1.0.2 - - [02/Dec/2012:03:18:53 -0800] "POST /mapreduce/controllerCallback HTTP/1.1" 500 0 "http://xxx/mapreduce/controllerCallback" "AppEngine-Google; (+http://code.google.com/appengine)" "xxx.com" ms=535 cpu_ms=288 queue_name=default task_name=16764257756372630651 instance=00c61b117cc2653091fefc2f9b795b790f1a
I 2012-12-02 13:18:53.203
com.google.appengine.tools.mapreduce.impl.shardedjob.ShardedJobRunner pollTaskStates: Polling task states for job 002b0ae1-643b-420d-b3cd-20f03cc054ff-reduce, sequence number 166
W 2012-12-02 13:18:53.630
/mapreduce/controllerCallback
com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:494)
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:392)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:781)
at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:461)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:703)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:338)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:330)
at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:458)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:823)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.success(RpcStub.java:808)
at com.google.net.rpc3.impl.client.RpcClientInternalContext.runCallbacks(RpcClientInternalContext.java:898)
at com.google.net.rpc3.impl.client.RpcClientInternalContext.finishRpcAndNotifyApp(RpcClientInternalContext.java:803)
at com.google.net.rpc3.impl.client.RpcNetChannel.afterFinishingActiveRpc(RpcNetChannel.java:1063)
at com.google.net.rpc3.impl.client.RpcNetChannel.finishRpc(RpcNetChannel.java:911)
at com.google.net.rpc3.impl.client.RpcNetChannel.handleResponse(RpcNetChannel.java:2267)
at com.google.net.rpc3.impl.client.RpcNetChannel.messageReceived(RpcNetChannel.java:2068)
at com.google.net.rpc3.impl.client.RpcNetChannel.access$2000(RpcNetChannel.java:143)
at com.google.net.rpc3.impl.client.RpcNetChannel$TransportCallback.receivedMessage(RpcNetChannel.java:3129)
at com.google.net.rpc3.impl.client.RpcChannelTransportData$TransportCallback.receivedMessage(RpcChannelTransportData.java:599)
at com.google.net.rpc3.impl.wire.RpcBaseTransport.receivedMessage(RpcBaseTransport.java:417)
at com.google.apphosting.runtime.udrpc.UdrpcTransport$ClientAdapter.receivedMessage(UdrpcTransport.java:424)
at com.google.apphosting.runtime.udrpc.UdrpcTransport.dispatchPacket(UdrpcTransport.java:265)
at com.google.apphosting.runtime.udrpc.UdrpcTransport.readPackets(UdrpcTransport.java:217)
at com.google.apphosting.runtime.udrpc.UdrpcTransport$1.run(UdrpcTransport.java:81)
at com.google.net.eventmanager.AbstractFutureTask$Sync.innerRun(AbstractFutureTask.java:260)
at com.google.net.eventmanager.AbstractFutureTask.run(AbstractFutureTask.java:121)
at com.google.net.eventmanager.EventManagerImpl.runTask(EventManagerImpl.java:578)
at com.google.net.eventmanager.EventManagerImpl.internalRunWorkerLoop(EventManagerImpl.java:1002)
at com.google.net.eventmanager.EventManagerImpl.runWorkerLoop(EventManagerImpl.java:884)
at com.google.net.eventmanager.WorkerThreadInfo.runWorkerLoop(WorkerThreadInfo.java:136)
at com.google.net.eventmanager.EventManagerImpl$WorkerThread.run(EventManagerImpl.java:1855)
C 2012-12-02 13:18:53.631
Uncaught exception from servlet
com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:494)
at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:392)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:781)
at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:461)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:703)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:338)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:330)
at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:458)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:823)
at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.success(RpcStub.java:808)
at com.google.net.rpc3.impl.client.RpcClientInternalContext.runCallbacks(RpcClientInternalContext.java:898)
at com.google.net.rpc3.impl.client.RpcClientInternalContext.finishRpcAndNotifyApp(RpcClientInternalContext.java:803)
at com.google.net.rpc3.impl.client.RpcNetChannel.afterFinishingActiveRpc(RpcNetChannel.java:1063)
at com.google.net.rpc3.impl.client.RpcNetChannel.finishRpc(RpcNetChannel.java:911)
at com.google.net.rpc3.impl.client.RpcNetChannel.handleResponse(RpcNetChannel.java:2267)
at com.google.net.rpc3.impl.client.RpcNetChannel.messageReceived(RpcNetChannel.java:2068)
at com.google.net.rpc3.impl.client.RpcNetChannel.access$2000(RpcNetChannel.java:143)
at com.google.net.rpc3.impl.client.RpcNetChannel$TransportCallback.receivedMessage(RpcNetChannel.java:3129)
at com.google.net.rpc3.impl.client.RpcChannelTransportData$TransportCallback.receivedMessage(RpcChannelTransportData.java:599)
at com.google.net.rpc3.impl.wire.RpcBaseTransport.receivedMessage(RpcBaseTransport.java:417)
at com.google.apphosting.runtime.udrpc.UdrpcTransport$ClientAdapter.receivedMessage(UdrpcTransport.java:424)
at com.google.apphosting.runtime.udrpc.UdrpcTransport.dispatchPacket(UdrpcTransport.java:265)
at com.google.apphosting.runtime.udrpc.UdrpcTransport.readPackets(UdrpcTransport.java:217)
at com.google.apphosting.runtime.udrpc.UdrpcTransport$1.run(UdrpcTransport.java:81)
at com.google.net.eventmanager.AbstractFutureTask$Sync.innerRun(AbstractFutureTask.java:260)
at com.google.net.eventmanager.AbstractFutureTask.run(AbstractFutureTask.java:121)
at com.google.net.eventmanager.EventManagerImpl.runTask(EventManagerImpl.java:578)
at com.google.net.eventmanager.EventManagerImpl.internalRunWorkerLoop(EventManagerImpl.java:1002)
at com.google.net.eventmanager.EventManagerImpl.runWorkerLoop(EventManagerImpl.java:884)
at com.google.net.eventmanager.WorkerThreadInfo.runWorkerLoop(WorkerThreadInfo.java:136)
at com.google.net.eventmanager.EventManagerImpl$WorkerThread.run(EventManagerImpl.java:1855)
我找到了this，但我没有使用任何实体，我猜创建实体是为了存储数据，但因为我不确定在哪里定义应该写入多少数据
# 1 楼答案
我在运行Map Reduce时也会遇到同样的异常。 问题在于Map Reduce包含大量元数据，这些元数据是在作业期间生成的，之后数据将保存为数据存储中的单个实体。 当实体变得大于允许值时，将抛出异常，并停止Map Reduce
我的问题是我写了很多日志，比如
在地图的主体中减少。 解决方案是减少日志大小和数量
# 2 楼答案
我最近也遇到了同样的问题，并将其追溯到我的Mapper类。该类的成员数据在片之间被序列化/形成数据存储（我想），我有大量的数据，超过1MB。它与我的类正在进行的对数据存储的任何显式调用无关
# 3 楼答案
在一次调用中可以写入的最大实体数为500个，目前，实体的大小最多为1MB。这里描述了这一点：
Quotas_and_Limits
因此，您试图在实体中存储太多数据，因此会引发此异常。 以下是一些您可以尝试的解决方法：