boto_rsync 0.8.1

概要

boto rsync是对boto的s3put脚本的粗略改编 重新设计以更接近rsync。它的目标是提供 类似rsync的boto的s3和google存储接口包装器。

默认情况下，脚本递归工作，文件之间的差异是 通过比较文件大小（例如rsync的-递归和-仅大小）进行检查 选项）。如果文件存在于目的地，但其大小不同 然后将覆盖源（除非使用-w选项）。

安装

要安装，只需：

pip install boto_rsync

• 您需要安装Python2.5+和pip。
• 您可能必须是pip的根（或使用sudo）才能将脚本安装到 $path中的全局可执行目录。
• pip应该会自动为您安装boto，但是高级用户可以找到 在这里：http://github.com/boto/boto/

用法

boto-rsync [OPTIONS] SOURCE DESTINATION

源和目标可以是：

• 指向目录或特定文件的本地路径
• 指向目录或特定键的自定义s3或gs url，格式为 s3://bucketname/path/or/key
• 使用两个s3url从s3传输到s3
• 使用两个GS URL进行GS到GS的传输

示例

boto-rsync [OPTIONS] /local/path/ s3://bucketname/remote/path/

或：

boto-rsync [OPTIONS] gs://bucketname/remote/path/or/key /local/path/

或：

boto-rsync [OPTIONS] s3://bucketname/ s3://another_bucket/

选项

-a KEY, --access-key KEY
                      Your Access Key ID. If not supplied, boto will look
                      for an environment variable or a credentials file.
-s SECRET, --secret-key SECRET
                      Your Secret Key. If not supplied, boto will look for
                      an environment variable or a credentials file.
--anon                Connect without credentials (S3 only). Useful if
                      working with others' buckets that have a global
                      read/write ACL.
--endpoint HOST       Specify a specific S3 endpoint to connect to via
                      boto's "host" connection argument (S3 only).
-g GRANT, --grant GRANT
                      A canned ACL policy that will be granted on each file
                      transferred to S3/GS. The value provided must be one
                      of the "canned" ACL policies supported by S3/GS:
                      private, public-read, public-read-write (S3 only), or
                      authenticated-read
-m METADATA [METADATA ...], --metadata METADATA [METADATA ...]
                      One or more "Name: value" pairs specifying what
                      metadata to set on each file transferred to S3/GS.
                      Note: Be sure to end your args with "--" if this is
                      the last argument specified so that SOURCE and
                      DESTINATION can be read properly. e.g. boto-rsync -m
                      "Content-Type: audio/mpeg" "Content-Disposition:
                      attachment" -- ./path/ s3://bucket/
-r, --reduced         Enable reduced redundancy on files copied to S3.
-e, --encrypt-keys    Enable server-side encryption on files copied to S3
                      (only applies when S3 is the destination).
-p, --preserve-acl    Copy the ACL from the source key to the destination
                      key (only applies in S3/S3 and GS/GS transfer modes).
-w, --no-overwrite    No files will be overwritten, if the file/key exists
                      on the destination it will be kept. Note that this is
                      not a sync--even if the file has been updated on the
                      source it will not be updated on the destination.
--glob                Interpret the tail end of SOURCE as a filename pattern
                      and filter transfers accordingly. Note: If globbing a
                      local path, make sure that your CLI's automatic
                      filename expansion is disabled (typically accomplished
                      by enclosing SOURCE in quotes, e.g. "/path/*.zip").
--no-recurse          Do not recurse into directories.
--skip-dirkeys        When syncing to S3 or GS, skip the creation of keys
                      which represent "directories" (an empty key ending in
                      "/" for S3 or "_$folder$" for GS).
--ignore-empty        Ignore empty (0-byte) keys/files/directories. This
                      will skip the transferring of empty directories and
                      keys/files whose size is 0. Warning: S3/GS often uses
                      empty keys with special trailing characters to specify
                      directories.
--delete              Delete extraneous files from destination dirs after
                      the transfer has finished (e.g. rsync's --delete-
                      after).
-n, --dry-run         No files will be transferred, but informational
                      messages will be printed about what would have
                      happened.
-v, --verbose         Print additional informational messages.
-d LEVEL, --debug LEVEL
                      Level 0 means no debug output (default), 1 means
                      normal debug output from boto, and 2 means boto debug
                      output plus request/response output from httplib.
-h, --help            show the help message and exit

已知问题和限制

• 通过检查大小，假设键/文件之间的差异only。
• 由于目录在s3/gs中工作的性质，一些非标准的文件夹 结构可能无法正确传输。空目录也可以是 在某些情况下被忽视。如果有疑问，请先使用“-n”。
• 支持简单的“globbing”（例如/path/*.zip），但可能行为异常 在某些系统上。有关详细信息，请参见“–glob”选项的帮助文本。
• 此时，脚本没有利用boto的“multipart” 转移方法。（欢迎拉取请求！）

免责声明和警告

这是alpha软件-请记住首先使用“-n”选项！

本软件按“原样”提供，无任何形式的保证，明示或 默示的，包括但不限于适销性、适用性的保证 为了特殊目的和不侵权。在任何情况下，作者不得 对任何索赔、损害赔偿或其他责任负责，无论是在 合同，侵权行为或其他，由 软件或软件中的使用或其他交易。

最新更改