Hadoop失败错误：java。lang.ArrayIndexOutofBounds异常：5

Question

我使用Hadoop MapReduce来计算每年的最小值和最大值，但是当我运行程序时，我得到了错误：FAILED Error: java.lang.ArrayIndexOutOfBoundsException: 5

我认为这是因为我的数据中有空值，因为当没有空值时程序运行良好

因此，在我的map函数中，我编写if语句来检查是否有头以及是否有空值：

 public static class ExposureMapper
        extends Mapper<Object, Text, Text, MinMaxExposure> {

    private Text year = new Text();
    private double minexposure;
    private Double maxexposure;

    private MinMaxExposure outPut = new MinMaxExposure();

    public void map(Object key, Text value, Context context
    ) throws IOException, InterruptedException {
        try {
            //Some condition satisfying it is header
            if (value.toString().contains("Product")) {
                return;
            } else if(value.toString()==null) {
               return;
            }
            else{
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        String[] solarFields = value.toString().split(",");

        year.set(solarFields[2]);
        minexposure = Double.parseDouble(solarFields[5]);
        maxexposure = Double.parseDouble(solarFields[5]);

        try {
            outPut.setMinExposure(minexposure);
            outPut.setMaxExposure(maxexposure);
            context.write(year, outPut);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

但同样的错误也会发生。。。 是因为value.toString()==null不是检查空值的正确方法吗

编辑：

19/06/07 00:14:30 INFO mapreduce.Job: Task Id : attempt_1527224104960_0812_m_000000_1, Status : FAILED
Error: java.lang.ArrayIndexOutOfBoundsException: 5
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:50)
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:23)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)

Answer 1

如果value.toString().split(",");的元素少于六个，那么solarFields[5]将不是元素，因此您将看到一个ArrayIndexOutOfBoundsException

创建solarFields后，应立即检查其长度：

if (solarFields == null || solarFields.length < 6) {
    return;
}

您还希望确保Double.parseDouble(solarFields[5]);不会抛出NumberFormatException：

Double exposure;
try {
    exposure = Double.parseDouble(solarFields[5]);
} catch (NumberFormatException e) {
    return;
}