有 Java 编程相关的问题?

你可以在下面搜索框中键入要查询的问题!

java在Google BigQuery中查找非查询作业的数据行计数

我用这个code将CSV数据从云存储加载到一个新的BigQuery表中。加载操作已成功完成,但我需要有关数据计数的度量

import com.google.cloud.bigquery.*;
import com.google.cloud.bigquery.JobInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LoadCsvFromGcs {

    private static final Logger LOGGER = LoggerFactory.getLogger(LoadCsvFromGcs.class);
    private static long csvRowCount;

    public static void loadCsvFromGcs(
            String datasetName, String tableName, String sourceUri, Schema schema) {
        try {
            BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
            CsvOptions csvOptions = CsvOptions.newBuilder().setSkipLeadingRows(1).build();

            TableId tableId = TableId.of(datasetName, tableName);
            LoadJobConfiguration loadConfig =
                    LoadJobConfiguration.newBuilder(tableId, sourceUri, csvOptions).setSchema(schema)
                            .setMaxBadRecords(999999999).build();
            Job job = bigquery.create(JobInfo.of(loadConfig));
            job = job.waitFor();

            csvRowCount = job.getQueryResults().getTotalRows();
            LOGGER.info(Long.toString(csvRowCount));

            if (job.isDone()) {
                LOGGER.info(sourceUri + " CSV from GCS successfully added during load append job");
            } else {
                LOGGER.error("BigQuery was unable to load into the table due to an error:"
                        + job.getStatus().getError() + " , " + sourceUri);
                System.exit(-1);
            }
        } catch (BigQueryException | InterruptedException e) {
            LOGGER.error(sourceUri + " Column not added during load append \n" + e);
            System.exit(-1);
        }
    }
}

我查看了GoogleBigQuery文档,发现job类的度量可以通过get functions检索。 然而,当我尝试时,我得到以下错误

Exception in thread "main" java.lang.UnsupportedOperationException: Getting query results is supported only for QUERY jobs

如何找出非查询作业加载了多少行数据之类的指标


共 (0) 个答案