用于使用Java查找字符串的windows MapReduce
我试图从一个文本文件中搜索一个特定的字符串，以及该字符串的出现，但在运行此代码后，我在io之间得到classCastException。可写的
Error: java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at searchaString.SearchDriver$searchMap.map(SearchDriver.java:1)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
16/04/30 02:48:17信息地图还原。作业：映射0%减少0% 16/04/30 02:48:23信息地图还原。作业：任务Id:尝试\u 1461630807194\u 0021\u m\u000000\u 2，状态：失败 错误：java。lang.ClassCastException:org。阿帕奇。hadoop。木卫一。无法将LongWritable强制转换为组织。阿帕奇。hadoop。木卫一。正文
package samples.wordcount;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
//import org.apache.hadoop.util.GenericOptionsParser;
//import org.apache.hadoop.mapred.lib.NLineInputFormat;
import java.io.IOException;
import java.util.Iterator;
public class WordCount {
public static void main(String[] args) throws Exception {
@SuppressWarnings("unused")
JobClient jobC =new JobClient();
Configuration conf = new Configuration();
//String args[] = parser.getRemainingArgs();
Job job = Job.getInstance(conf);
job.setJobName("WordCount");
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setReducerClass(IntSumReducer.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);
//job.setInputFormatClass(TextInputFormat.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
/*String MyWord = args[2];
TokenizerMapper.find = MyWord;*/
System.exit(job.waitForCompletion(true) ? 0:1);
}
public static class TokenizerMapper extends Mapper<Text, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
// private Text word = new Text();
static String find="txt was not created";
public int i;
public void map(Text key, Text value,OutputCollector<Text, IntWritable> output,Reporter reporter) throws IOException, InterruptedException
{
String cleanLine = value.toString();
String[] cleanL =cleanLine.split("home");
output.collect(new Text(cleanL[1]), one);
}
}
public static class IntSumReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output,Reporter reporter)
throws IOException, InterruptedException {
int sum = 0;
String wordText="txt was not created";
while(values.hasNext()) {
Boolean check = values.toString().contains("txt was not created");
if(check)
{
String[] cleanL =values.toString().split("\\.");
for(String w : cleanL)
{
if(w.length()>=wordText.length())
{
String wrd = w.substring(0,wordText.length());
if(wrd.equals(wordText))
{
IntWritable value=values.next();
sum += value.get();
}
}
}
}
}
output.collect(key,new IntWritable(sum));
}
}
}
我是这个MapReduce的新手，不知道怎么做
这也是我的文本文件的外观：
tab/hdhdhd/hip/home。slkj。斯基德。dgsyququ/djkdjjd****未创建文本** 我必须搜索特定的文本
请回复
如果您分享一些解决方案，请简要说明我应该在代码中更改什么
谢谢
# 1 楼答案
新映射器：公共类TokenizerMapper扩展映射器
你的写作方法是 续写（新文本（cleanL[1]），一份
“一”不是一回事。要么改变你的签名如下 公共类TokenizerMapper扩展了映射器并编写如下
续写（新文本（cleanL[1]），新文本（“一”）
或
公共类TokenizerMapper扩展了映射器并写入为
cont.write（新文本（cleanL[1]），新IntWritable（1）
# 2 楼答案
您已给出Mapper类的签名，如下所示
公共静态类TokenizerMapper扩展映射器
map方法采用的输入键是行的字节偏移量。例如，如果以下是ur文件的内容
你好，世界
地图功能将第一行的字节偏移量（十六进制）作为键和“Hello World！”作为价值。字节偏移量是一个长值的kinf
将输入键更改为LongWritable