没有项目描述
s3bro的Python项目详细描述
#S3兄弟！一个方便的s3 cli工具
概述
是你的第三个朋友（兄弟）。您经常会发现，为了对s3执行任务，您必须运行复杂的cli/aws命令。假设您需要从s3 glacier存储类恢复所有密钥，如果您尝试过此任务，您可能已经意识到，aws cli不允许以“批处理”的简单方式执行此任务。
随后，您将需要使用aws cli命令的组合，并将其管道化到其他命令，以便获得所需的结果。除了这种复杂性，它将通过密钥恢复密钥，非常缓慢。
然而，s3bro为您提供了一个解决上述问题和其他常见情况的方法，在这些情况下，您必须通过使用多处理/线程来运行复杂的s3命令。这意味着您可以比使用常规方法更快地加速任务。哦，我有没有提到…以一种更优雅的方式。这是一个python cli，它将为您抽象大部分工作。
为什么要运行以下两个命令来擦除存储桶：
>>>
aws s3api list-object-versions --bucket rsavordelli --output json --query
'Versions\[\].\[Key, VersionId\]' | jq -r '.\[\] | "--key '\\''" + .\[0\] + "'\\''
--version-id " + .\[1\]' | xargs -L1 aws s3api delete-object --bucket rsavordelli
aws s3api list-object-versions --bucket rsavordelli --output json --query
'DeleteMarkers\[\].\[Key, VersionId\]' | jq -r '.\[\] | "--key '\\''" + .\[0\] + "'\\''
--version-id " + .\[1\]' | xargs -L1 aws s3api delete-object --bucket rsavordelli
如果你能运行这个？
>>> s3bro purge --bucket rsavordelli
如果你需要从冰川存储类中恢复所有密钥呢？或者只是其中的一部分？
>>>
➜ ~ s3bro restore --bucket yourBucket --prefix glacier --days 10 --type Expedited --exclude .html --workers 10
Initiating Expedited restore for yourBucket/glacier...
Restoring keys for 10 days
Versions: False
===========================
Restoration completed: glacier/River Flows In You - A Love Note (CM Remix).mp3 until "Sat, 03 Mar 2018 00:00:00 GMT"
Submitting restoration request: glacier/asd.js
Restoration completed: glacier/Yiruma - River Flows In You (English Version).mp3 until "Sat, 03 Mar 2018 00:00:00 GMT"
Restoration completed: glacier/River Flows In You- Lindsey Stirling.mp3 until "Sat, 03 Mar 2018 00:00:00 GMT"
Restoration completed: glacier/River Flows In You - A Love Note (Ryan Wong Remix).mp3 until "Sat, 03 Mar 2018 00:00:00 GMT"
Restoration completed: glacier/ until "Sat, 03 Mar 2018 00:00:00 GMT"
Restoration completed: glacier/Endless Love {Piano Version} | Beautiful Piano.mp3 until "Sat, 03 Mar 2018 00:00:00 GMT"
Total keys proccessed: 7 in 5.44s
安装
^{tt1}$
来源
^{tt2}$
^{tt3}$
注意：对于python3，由于在python3中更改了unicode处理，您可能会遇到一些问题。根据计算机运行的区域设置，这些问题的解决方案是不同的。 通常“export lang=en_us.utf-8”会解决（放置您的lang）。更多信息：http://click.pocoo.org/5/python3/
示例
# s3bro restore --help # s3bro restore --bucket bucketName --prefix myglacierPrefix --days 20 --type Bulk # s3bro restore --bucket bucketName --prefix myglacierPrefix --days 20 --type Standard --include .css --versions # s3bro restore --bucket bucketName --prefix myglacierPrefix --days 20 --type Expedited --permanent-restore --storage-class ONEZONE_IA # s3bro restore -b bucketName -p 123 --days 2 --type Expedited --permanent-restore --restore-to-bucket DestbucketName --storage-class ONEZONE_IA # s3bro purge --bucket bucketName # s3bro scan-objects --bucket bucketName # s3bro scan-objects-v2 --bucket bucketName --make-private # s3bro scan-bucket --all # s3bro tail --bucket bucketName --timeout 1
命令
恢复
Restore S3 keys in Glacier Storage class
还原选项
>>>
Usage: s3bro restore [OPTIONS] [RESTORE]...
restore S3 objects from Glacier Storage Class
Options:
-b, --bucket TEXT bucket name [required]
-p, --prefix TEXT prefix
-d, --days INTEGER Days to keep the restore [required]
-t, --type [Standard|Expedited|Bulk]
restore type (Tier) [required]
-v, --versions / --no-versions [--no-versions is DEFAULT] - this option
will make the restore to include all
versions excluding delete markers
-pr, --permanent-restore Move keys ALREADY restored from Glacier back
to a storage class of your choice
-rtb, --restore-to-bucket TEXT Copy keys ALREADY restored to a different
bucket. It can only be used in combination
with --permanent-restore
--storage-class [STANDARD|STANDARD_IA|ONEZONE_IA]
The StorageClass type to use with
--permanent-restore [default is STANDARD]
-urd, --update-restore-date / --do-not-update-restore-date
If passed, it will change the restore date
for already restored key
-in, --include TEXT Only restore keys that matches with a given
string, you can add multiples times by
passing --include multiple times
-ex, --exclude TEXT Do not restore if key name matches with a
given pattern,you can add multiple patterns
by inputting
--workers INTEGER How many helpers to include in task, default
is 10
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
还原详细信息
选项–日志级别可用于调试错误/行为。
>>> DEBUG - similar to boto3 debug level with additional information WARNING - will print some threading information and Keys excluded during the iteration (exclude, include, storage-class, delete-marker, etc)
- 选项–workers允许您指定将使用列表的工人数量。计算每个核心最多5个工人
- 选项–update restore date可用于“扩展”已还原的密钥。它将向对象发送一个新的“到期”日期
- 选项–永久还原将数据从Glacier复制回您选择的存储类（与–存储类结合使用）
净化
Delete all keys in the bucket - as simple as that. It will delete versions, delete markers. Everything
清除选项
>>>
Usage: s3bro purge [OPTIONS] [PURGE]...
delete all the bucket content
Options:
-b, --bucket TEXT Bucket name [required]
-p, --prefix TEXT prefix name - optional
--yes first confirmation
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
清除详细信息
- 剧本有两个确认。第一个可以通过-是的。第二个请求您确认bucket名称。
- 如果在bucket中使用key:s3bro_delete和value:yes创建标记，则可以避免第二次确认。这将通过bucket名称确认。
扫描桶
scan bucket ACLs
扫描存储桶选项
>>>
Usage: s3bro scan-bucket [OPTIONS] [SCAN_BUCKET]...
scan bucket ACLs
Options:
-b, --bucket TEXT Bucket name
-A, --all Scan permissions for all your buckets (don't
combine -b with -A)
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
扫描对象
scan object ACLs
扫描对象选项
>>>
Usage: s3bro scan-objects [OPTIONS] [SCAN_OBJECTS]...
scan object ACLs
Options:
-b, --bucket TEXT Bucket name [required]
-p, --prefix TEXT prefix name - optional
--workers INTEGER How many helpers to include in task, default
is 10
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
扫描对象详细信息
- 扫描对象仅扫描对象的当前版本
扫描对象详细信息
- 扫描对象仅扫描对象的当前版本
scan-objects-v2
scan-objects-v2 is a simplified version of scan-objects and introduce new features like –make-private (make public keys, private). It’s focused on looking only for Public Keys (Everyone’s access), it will not print permission to another aws accounts.
扫描对象-v2选项
>>>
Usage: s3bro scan-objects-v2 [OPTIONS] [SCAN_OBJECTS_V2]...
scan object ACLs (V2) - The V2 only look for Everyone permissios, while
the scan-objects will look for all ACLs - The V2 is capable to reset ACLs
back to private (Everyone)
Options:
-b, --bucket TEXT Bucket name [required]
-p, --prefix TEXT prefix name - optional
-mp, --make-private Make all keys with public ACL private
-v, --versions / --no-versions [--no-versions is DEFAULT] - this option
will make the restore to include all
versions excluding delete markers
--workers INTEGER How many helpers to include in task, default
is 10
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
扫描对象-v2详细信息
- 扫描对象支持版本
- –设为private在对象中放置一个私有acl
尾部
s3 logs in “real-time” through S3 Events (for puts and deletes only)
选项
>>>
Usage: s3bro tail [OPTIONS] [TAIL]...
tail is an S3 real-time logging tool. It makes use of S3 events (for puts and deletes only)
Options:
-b, --bucket TEXT Bucket name [required]
-t, --timeout INTEGER How much time (in minutes) to run, it will destroy
the resources created after this time [required]
--help Show this message and exit.
细节
基本上它的作用是：
- 创建SQS
- 创建S3事件通知
- 连接到队列并继续检索消息，直到达到超时时间为止。
- 删除创建的资源
>>> --timeout is in minutes >>> it only works for PUTs and Deletes (s3 events does not support GET requests)
查找未加密的
find unencrypted keys in a bucket
查找未加密选项
>>>
Usage: s3bro find-unencrypted [OPTIONS] [FIND_UNENCRYPTED]...
find unencrypted keys in a bucket (ServerSideEncryption)
Options:
-b, --bucket TEXT Bucket name [required]
-p, --prefix TEXT prefix name - optional
-v, --versions / --no-versions [--no-versions is DEFAULT] - this option
will make the restore to include all
versions excluding delete markers
--workers INTEGER How many helpers to include in task, default
is 10
--log-level [INFO|ERROR|DEBUG|WARNING]
logging type
--help Show this message and exit.
查找未加密的详细信息
- 它只打印未加密密钥的详细信息。如果要检查所有密钥加密状态，请运行–日志级别警告
查找未加密的详细信息
- 它只打印未加密密钥的详细信息。如果要检查所有密钥加密状态，请运行–日志级别警告
欢迎加入QQ群-->： 979659372
