启动映射作业时遇到错误

Question

开始地图作业时，我遇到了这个错误。

ERROR    2015-05-11 06:03:45,719 webapp2.py:1528] __init__() got an unexpected keyword argument '_user_agent'
Traceback (most recent call last):
  File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1076, in __call__
    handler = self.handler(request, response)
  File "/home/rshah/work/python/web/fourtop/mapreduce/base_handler.py", line 85, in __init__
    _user_agent=self._DEFAULT_USER_AGENT))
  File "/home/rshah/google_appengine/google/appengine/datastore/datastore_rpc.py", line 105, in positional_wrapper
    return wrapped(*args, **kwds)
TypeError: __init__() got an unexpected keyword argument '_user_agent'

下面是如何启动map函数。

control.start_map(name='Export Device Health Logs',
      handler_spec='data_process.process_health_logs',
      reader_spec='mapreduce.input_readers.DatastoreInputReader',
      mapper_parameters={"input_reader":{'entity_kind': 'models.DeviceHealth','email':user.email,'to_date':to_date,'from_date':from_date},'output_writer': {'bucket_name': bucket_name,
                'content_type': 'text/plain'}},
      shard_count=1,
      output_writer_spec='mapreduce.output_writers.GoogleCloudStorageConsistentOutputWriter',
      mapreduce_parameters={'done_callback':'/tasks/mapreduce/done/export_health_data',
                              'done_callback_queue':'mapreduce-done'})

*处理程序代码：- 此代码读取设备健康模型的每一行，并为其准备一个CSV文件。 在完成对数据模型行的迭代之后，我们准备CSV文件并通过电子邮件发送给用户

def process_health_logs(health_logs):
  ctx = context.get()
  params = ctx.mapreduce_spec.mapper.params
  to_date = params.get('to_date')
  from_date = params.get('from_date')

  to_date = datetime.datetime.strptime(str(to_date),'%Y-%m-%d').date()
  from_date = datetime.datetime.strptime(str(from_date),'%Y-%m-%d').date()
  error_date = health_logs.error_date if health_logs.error_date else health_logs.added_date
  date  = error_date.date()

  if date >= from_date and date <= to_date:
    added_date = time_util.utc_to_local(error_date, health_logs.place.timezone)
    device_id = health_logs.device_id
    user = health_logs.user.email
    place = health_logs.place.name
    unique_device_id = health_logs.unique_device_id 
    error_type = health_logs.error_type
    error_level = health_logs.error_level
    error_description = health_logs.error_description

    row_data=[
              added_date,
              device_id, 
              user,
              place, 
              unique_device_id, 
              error_type,
              error_level,
              error_description
             ]
    row_data = [gtools.str_or_empty(x).encode('utf-8').replace(',', ';') for x in row_data]
    yield ','.join(row_data) + '\n'

Answer 1

很可能您使用的是旧的cloudstorage模块，与mapreduce模块不兼容。你知道吗