java板条箱数据库表在从板条箱4.2.7升级到板条箱4.3.4时出错

Question

在恢复快照或将cratedb从4.2.7升级到4.3.4的更高版本时遇到以下错误

SnapshotRestoreException[[repo_name:snapshot_name/******] cannot restore index [schema_name.table_name] because it cannot be upgraded]

org.elasticsearch.snapshots.SnapshotRestoreException: [repo_name:snapshot_name/******] cannot restore index [schema_name.table_name] because it cannot be upgraded
    at org.elasticsearch.snapshots.RestoreService$1.execute(RestoreService.java:273)
    at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
    at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:681)
    at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:307)
    at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:207)
    at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:139)
    at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:153)
    at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:191)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:258)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:221)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.IllegalStateException: unable to upgrade the mappings for the index [[repo_name:snapshot_name/******]]
    at org.elasticsearch.cluster.metadata.MetadataIndexUpgradeService.checkMappingsCompatibility(MetadataIndexUpgradeService.java:175)
    at org.elasticsearch.cluster.metadata.MetadataIndexUpgradeService.upgradeIndexMetadata(MetadataIndexUpgradeService.java:98)
    at org.elasticsearch.snapshots.RestoreService$1.execute(RestoreService.java:270)
    ... 12 more
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [default]: Mapping definition for [dynamicFieldName3] has unsupported parameters:  [fields : {keyword={type=keyword, ignore_above=256}}]
    at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:285)
    at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:272)
    at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:251)
    at org.elasticsearch.cluster.metadata.MetadataIndexUpgradeService.checkMappingsCompatibility(MetadataIndexUpgradeService.java:171)
    ... 14 more
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Mapping definition for [dynamicFieldName3] has unsupported parameters:  [fields : {keyword={type=keyword, ignore_above=256}}]
    at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:153)
    at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:147)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:215)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:163)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parse(ObjectMapper.java:134)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:207)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:163)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parse(ObjectMapper.java:134)
    at org.elasticsearch.index.mapper.ArrayTypeParser.parse(ArrayTypeParser.java:51)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:207)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:163)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parse(ObjectMapper.java:134)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:207)
    at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:163)
    at org.elasticsearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:103)
    at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:101)
    at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:89)
    at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:283)

下面是示例表创建模式

CREATE TABLE IF NOT EXISTS TABLE_NAME (
  "id" STRING,
  "field1" STRING NOT NULL,
  "field2" STRING NOT NULL,
  "field3" OBJECT NOT NULL,
  "createdAt" TIMESTAMP NOT NULL,
  "updatedAt" TIMESTAMP NOT NULL,
  PRIMARY KEY ("field1", "field2")
)

由于field3是动态对象，因此允许向其中添加更多属性

因此，现在将数据插入表后，当前模式如下

CREATE TABLE IF NOT EXISTS TABLE_NAME (
  "id" STRING,
  "field1" STRING NOT NULL,
  "field2" STRING NOT NULL,
  "field3" OBJECT(DYNAMIC) AS (
      "dynamicField1" BOOLEAN,
      "dynamicField2" TEXT,
      "dynamicField3" TEXT INDEX USING FULLTEXT,
      "dynamicField4" TEXT
   )  NOT NULL,
  "createdAt" TIMESTAMP NOT NULL,
  "updatedAt" TIMESTAMP NOT NULL,
  PRIMARY KEY ("field1", "field2")
)
...
...
...

我已经拍了一张快照，现在我正试图恢复/升级到一个更高的版本，它面临上述错误

更新1：

SELECT DISTINCT version['created'], version['upgraded'] FROM "information_schema"."tables" WHERE table_schema not IN ('sys','information_schema','pg_catalog') limit 100;

Result from query:
version['created']  version['upgraded']
4.0.12              4.2.7 
4.1.5               4.2.7 
4.2.7               NULL 

SELECT distinct min_lucene_version FROM sys.shards limit 100;

Result from query:

min_lucene_version
8.0.0 
8.5.1 
8.4.0

Answer 1

上述错误

[fields : {keyword={type=keyword, ignore_above=256}}]

提示，您可能希望升级/恢复使用CrateDB版本创建的表/快照<；3.0到4。x

4。x仅支持使用3创建的表。x及以上 第三条。x仅支持使用2创建的表。x及以上

快照基本上只是Lucene索引/段的副本

如注释中所述，您应该运行以下查询来检查创建表的版本：

SELECT DISTINCT version['created'], version['upgraded'] FROM "information_schema"."tables" WHERE table_schema not IN ('sys','information_schema','pg_catalog')

SELECT distinct min_lucene_version FROM sys.shards

---

如果只有较旧的表受到影响，我建议使用4.2.7重新索引这些表，然后运行升级

SHOW CREATE table old;
INSERT INTO new SELECT * FROM old;
 or
COPY old TO;
COPY new FROM;