谷歌云存储：向GCS上传字符串时CRC32C和MD5不匹配

Question

在尝试上载JSON字符串并覆盖GCS bucket中的现有对象时，出现以下错误

google.api_core.exceptions.BadRequest: 400 POST https://storage.googleapis.com/upload/storage/v1/b/cc-freshdesk/o?uploadType=multipart: {
  "error": {
    "code": 400,
    "message": "Provided CRC32C \"i8Z/Pw==\" doesn't match calculated CRC32C \"mVn0oQ==\".",
    "errors": [
      {
        "message": "Provided CRC32C \"i8Z/Pw==\" doesn't match calculated CRC32C \"mVn0oQ==\".",
        "domain": "global",
        "reason": "invalid"
      },
      {
        "message": "Provided MD5 hash \"6NMASNWhbd4WlIj/tWK4Sw==\" doesn't match calculated MD5 hash \"9H5THzsUBARmhzw5NjjgNw==\".",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}
: ('Request failed with status code', 400, 'Expected one of', <HTTPStatus.OK: 200>)

请在下面查找代码段：

storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
config_blob = bucket.blob(destination_blob_name)
config_blob.upload_from_string(json.dumps(config_data,indent=4), content_type='text/plain')

有人能帮我理解为什么会发生这个问题吗

Answer 1

以防9个月后有人需要。 您通常不希望使用两个不同的blob。很多时候，你必须在两个方向上进行多次读写。因此，我坚决反对使用这种方法。 您只需通过调用“reload（）”显式刷新CRC32校验和即可：

storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
config_blob = bucket.get_blob(destination_blob_name)
config_blob.reload()
config_blob.upload_from_string(json.dumps(config_data,indent=4), 
content_type='text/plain')

Answer 2

要复制您遇到的错误，请执行以下操作：

import json
from google.cloud import storage

client = storage.Client()
bucket = client.get_bucket('some-bucket')

# blob1 object
blob1 = bucket.get_blob('file.json')

# downloads content
blob1_string = blob1.download_as_string()

# converts to dict and update content
blob1_obj = json.loads(blob1_string)
blob1_obj['some-key'] = 'some value'

# upload using same blob instance
blob1.upload_from_string(json.dumps(blob1_obj))

# throws error like this `Provided MD5 hash "Ax9olGoqOSb7Nay2LNkCSQ==\" #doesn't match calculated MD5 hash \"XCMPR0o7NdgmI5zN1fMm6Q==\".",

您可能正在使用相同的blob来下载和上载内容。要防止此错误，您需要创建两个blob实例：

import json
from google.cloud import storage

client = storage.Client()
bucket = client.get_bucket("some-bucket")

# blob1 object   for downloading contents
blob1 = bucket.get_blob('file.json')

blob1_string = blob1.download_as_string()
# Convert to dictionary
blob1_obj = json.loads(blob1_string)
# Add stuff
blob1_obj['some-key'] = 'some value'

# blob2 object   for uploading contents
blob2 = bucket.get_blob('file.json')

blob2.upload_from_string(json.dumps(blob1_obj))

# no error