java在Google BigQuery中查找非查询作业的数据行计数
我用这个code将CSV数据从云存储加载到一个新的BigQuery表中。加载操作已成功完成,但我需要有关数据计数的度量
import com.google.cloud.bigquery.*;
import com.google.cloud.bigquery.JobInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LoadCsvFromGcs {
private static final Logger LOGGER = LoggerFactory.getLogger(LoadCsvFromGcs.class);
private static long csvRowCount;
public static void loadCsvFromGcs(
String datasetName, String tableName, String sourceUri, Schema schema) {
try {
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
CsvOptions csvOptions = CsvOptions.newBuilder().setSkipLeadingRows(1).build();
TableId tableId = TableId.of(datasetName, tableName);
LoadJobConfiguration loadConfig =
LoadJobConfiguration.newBuilder(tableId, sourceUri, csvOptions).setSchema(schema)
.setMaxBadRecords(999999999).build();
Job job = bigquery.create(JobInfo.of(loadConfig));
job = job.waitFor();
csvRowCount = job.getQueryResults().getTotalRows();
LOGGER.info(Long.toString(csvRowCount));
if (job.isDone()) {
LOGGER.info(sourceUri + " CSV from GCS successfully added during load append job");
} else {
LOGGER.error("BigQuery was unable to load into the table due to an error:"
+ job.getStatus().getError() + " , " + sourceUri);
System.exit(-1);
}
} catch (BigQueryException | InterruptedException e) {
LOGGER.error(sourceUri + " Column not added during load append \n" + e);
System.exit(-1);
}
}
}
我查看了GoogleBigQuery文档,发现job类的度量可以通过get functions检索。 然而,当我尝试时,我得到以下错误
Exception in thread "main" java.lang.UnsupportedOperationException: Getting query results is supported only for QUERY jobs
如何找出非查询作业加载了多少行数据之类的指标
共 (0) 个答案