Question

Why we use --split-by command in Sqoop?

Answer 1

The command --split-by is used to specify the column of the table used to generate splits for imports. This means that it specifies which column will be used to create the split while importing the data into the cluster.

Basically it is used to improve the import performance to achieve faster parallelism.

Answer 2

In simple explanation,

When specify SPLIT_BY only for sqoop'ing the whole table [which would be select * from table]. As this not condition based we need a logic to divide the data and process them in more than one node.

So we select a column based on which data can be divided base on a range [for integers] & characters [alphabets], to avoid huge amount of the data being concentrated on one mapper/node.

Bigdata is specifically meant to achieve parallelism.