擅长:python、mysql、java
<p>为什么不直接复制(S3->;S3)并节省内存和I/O呢?在</p>
<pre><code>import awswrangler as wr
SOURCE_PATH = "s3://..."
TARGET_PATH = "s3://..."
objs = wr.s3.list_objects(path=SOURCE_PATH)
wr.s3.copy_listed_objects(
objects_paths=objs,
source_path=SOURCE_PATH,
target_path=TARGET_PATH,
mode="append", # Could be append, overwrite or overwrite_partitions
)
</code></pre>
<p><a href="https://github.com/awslabs/aws-data-wrangler" rel="nofollow noreferrer">Reference</a></p>