首页 新闻 会员 周边

如何在eclipse设置本地文件系统文件路径名和要在HDFS中写入的SequenceFile文件路径名

0
悬赏园豆:80 [已关闭问题] 关闭于 2013-03-27 21:41

实战Hadoop第十二章的实例:基于Hadoop的MapReduce在blast算法基因序列比对的应用,

源码: http://www.chinacloud.cn/show.aspx?id=7173&cid=50 第12章  综合实战2——生物信息学应用

请问我要把本地文件系统中FASTA格式的文件转换成SequenceFile文件写入到HDFS路径下,这样设置路径正确吗?
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
String file= args[0]="file:///home/lin/hadoop/input/lin"; //这一句就是输入路径的,本地文件系统下的lin文件
File fastaFile = new File(file);System.out.println("reading fasta data from file: "+file);
Path path = new Path(args[1]);
System.out.println("writing SequenceFile format NDA sequences to file: "+path.toString());
System.out.println("Please wait...");
Fasta2SequenceFileConverter converter = new Fasta2SequenceFileConverter(fastaFile, fs, conf, path);

报错如下:reading fasta data from file: file:///home/lin/hadoop/input/lin
writing SequenceFile format NDA sequences to file: hdfs://localhost:9000/user/lin/fastaInFile/out
Please wait...
java.io.FileNotFoundException: file:/home/lin/hadoop/input/lin (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at java.io.FileReader.<init>(FileReader.java:55)
at Fasta2SequenceFileConverter.<init>(Fasta2SequenceFileConverter.java:77)
at Fasta2SequenceFileConverter.main(Fasta2SequenceFileConverter.java:450)
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:9000/user/lin/fastaInFile, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:301)
at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:470)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:365)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:484)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
at Fasta2SequenceFileConverter.<init>(Fasta2SequenceFileConverter.java:82)
at Fasta2SequenceFileConverter.main(Fasta2SequenceFileConverter.java:450)

 

求大虾指点,跪谢了!

< >
分享
所有回答(1)
0

OK了,输入路径是本地文件系统下FASTA文件的存放路径,输出路径仍然是本地文件系统下的文件路径,但是要加入输出文件名

/home/××/hadoop/input/文件名 /home/××/output/out

sixthsister | 园豆:204 (菜鸟二级) | 2013-03-27 21:40
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册