I have to ingest in hadoop cluster large number of files for testing what is the best way to do it

0 votes
Jul 8, 2020 in Big Data Hadoop by sonali
 • 120 points • 1,213 views

1 answer to this question.

0 votes

Hi@sonali,

It depends on what kind of testing you want to do. Say if you are trying to perform some SQL type command then go for HiveQL, Scala, etc. Or if you want to use that file in another platform like Spark. Then simply upload your data in HDFS Cluster and use it.

answered Jul 8, 2020 by MD
 • 95,460 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is the best way to integrate SAS with Hadoop without losing the parallel processing capacity of Hadoop

One of the major pushes at SAS ...READ MORE

answered Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,239 views
0 votes
1 answer

What is the default location of Hadoop log files & how can I change that location?

The default directory of Hadoop log file ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
 • 13,490 points • 21,277 views
0 votes
1 answer

What is the command to check the number of cores in Spark?

Go to your Spark Web UI & ...READ MORE

answered May 17, 2018 in Big Data Hadoop by Shubham
 • 13,490 points • 7,024 views
0 votes
1 answer

What is the best functional language to do Hadoop Map-Reduce?

down voteacceptedBoth Clojure and Haskell are definitely ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 907 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,780 views
0 votes
1 answer

What is the standard way to create files in your hdfs file-system?

Well, it's so easy. Just enter the below ...READ MORE

answered Sep 23, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,599 views
0 votes
1 answer

What is the command to count number of lines in a file in hdfs?

hadoop fs -cat /example2/doc1 | wc -l READ MORE

answered Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,220 points • 4,028 views
+1 vote
2 answers

Sir if i have no knowledge of any programming language then it is good or bad for me but i have learned Big Data and Hadoop so i can get a job or not

Hey Rahul,  You have mentioned that you have no ...READ MORE

answered May 6, 2019 in Big Data Hadoop by poonam
 • 1,479 views
0 votes
1 answer

I was installing Hadoop for windows everything went smoothly but when I tried to format NameNode it displayed an error of failing to start NameNode. What should I do?

Hi@alex, You may set the wrong IP in ...READ MORE

answered Oct 12, 2020 in Big Data Hadoop by MD
 • 95,460 points • 917 views
0 votes
1 answer

How to find the number of blocks for a file in Hadoop?

Hi@akhtar, You can use Hadoop file system command to ...READ MORE

answered Oct 13, 2020 in Big Data Hadoop by MD
 • 95,460 points • 2,271 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.