How to run Nutch in Hadoop installed in pseudo-distributed mode

Question

I have Nutch 1.13 installed on my Ubuntu. I can run a crawl in standalone mode. It successfully runs and produces the desired results but I have no idea how to run it in hadoop now? I have Hadoop installed in pseudo distributed mode and I want to run a Nutch crawl with Hadoop and monitor it. How can I do it? There are a lot of tutorials for running it in standalone mode but I couldn't find any clear instructions on how Can I run it in Hadoop except that I have to use "Nutch Job" after I build it with ant.

Answer 1

Make sure you have built Nutch from source i.e. don't use the binary release which works only in local mode. Once you've compile with

ant clean runtime

go to runtime/deploy/bin and run the scripts as usual.

NB you need to modify the conf files prior to recompiling.