在Windows上集成ApacheNutch和MySQL的java
我正在尝试将ApacheNutch2.1与Windows8平台上的Mysql服务器集成。我正在学习教程http://nlp.solutions.asia/?p=180。我对apache-nutch-2.1做了以下更改
- 我下载了apache-nutch-2.1-src。拉链和提取李>
常春藤/常春藤中未注释的以下内容。xml
<dependency org="mysql" name="mysql-connector-java" rev="5.1.18" conf="*->default"/>
注释了mysql conf/gora的sql属性,并添加了mysql conf/gora的gora属性。财产
gora.sqlstore.jdbc.driver=com.mysql.jdbc.Driver gora.sqlstore.jdbc.url=jdbc:mysql://localhost:3306/nutch? createDatabaseIfNotExist=true gora.sqlstore.jdbc.user=root gora.sqlstore.jdbc.password=root
- 向conf/nutch站点添加属性。xml
- 从命令提示符执行ant运行时命令。它创建了/runtime目录李>
- 添加种子。包含www.apache的/runtime/local/url目录中的txt文件。纳奇。组织价值李>
- 添加了+^http://([a-z0-9]*)*纳奇。组织/到两个域URL筛选器。txt和正则表达式urlfilter。/runtime/local/conf目录中的txt文件李>
当我运行命令开始通过cygwin终端爬行时。。发生以下异常:
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Abhijeet\mapred\staging\Abhijeet530509219\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:50)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:219)
at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:68)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:136)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:250)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawler.main(Crawler.java:257)
我在互联网上搜索到Hadoop不适用于Windows,这没关系,因为我没有使用Hadoop存储数据。我正在使用Mysql
有人能告诉我我做错了什么吗
# 1 楼答案
我已经在windows和Linux上使用了Nutch2。要在Windows上运行它,您需要安装此Haddop 1.0.3修补程序:https://github.com/congainc/patch-hadoop_7682-1.0.x-win