用python将xls文件发送到S3

Question

我在python程序中创建了一个.xls文件，我试图用boto将其推送到S3

我的代码：

just_models = [{"key": "1"}, {"key2": 2}]

df_just_models = pd.DataFrame(just_models)
# Notice desired file extension.
just_models = df_just_models.to_excel('../file.xls')

def save_just_models():
    # Save to S3
    date_file = datetime.now().strftime("%Y-%m-%d")
    s3 = boto3.resource('s3')
    obj = s3.Object('directory', 'indice_na/just_models' + date_file + '_indice_na.xls')
    obj.put(Body=just_models)
save_just_models()

我的错误：

    Traceback (most recent call last):
  File "indicena.py", line 11985, in <module>
    save_just_models()
  File "indicena.py", line 11984, in save_just_models
    obj.put(Body=just_models)
  File "/home/bolgi/.local/lib/python3.8/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/home/bolgi/.local/lib/python3.8/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/home/bolgi/.local/lib/python3.8/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/bolgi/.local/lib/python3.8/site-packages/botocore/client.py", line 598, in _make_api_call
    request_dict = self._convert_to_request_dict(
  File "/home/bolgi/.local/lib/python3.8/site-packages/botocore/client.py", line 646, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
  File "/home/bolgi/.local/lib/python3.8/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Body, value: None, type: <class 'NoneType'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object

错误来自obj.put()，但我不知道如何确切地解决它

Answer 1

是的，您可以从内存直接写入S3。无需将xls文件保存在本地硬盘上。这是可能的，因为您可以写入BytesIO而不是文件，然后将BytesIO发送到S3，如下所示。默认的大熊猫XLSX编写器被弃用，所以你可以考虑使用新的一个，比如^ {CD3>}：

import io

import pandas as pd

just_models = [{"key": "1"}, {"key2": 2}]

df_just_models = pd.DataFrame(just_models)

mem_file = io.BytesIO();

df_just_models.to_excel(mem_file, engine='xlsxwriter')

def save_just_models():
    # Save to S3
    date_file = datetime.now().strftime("%Y-%m-%d")
    s3 = boto3.resource('s3')
    obj = s3.Object('directory', 'indice_na/just_models' + date_file + '_indice_na.xls')
    obj.put(Body=mem_file.getvalue())
    
save_just_models()