I have large no of files. Data is NASDAQ Stock data from 1999 till date. I have data in folders. Ex: 1999 Folders will have months and month folder will have files.
So I will have 25 Years folders, and in each year folder I have 12 months folders and in each month I will have files.
How do I load these files into Hadoop in a single shot and also, is there a way to load all the files into one folder in Hadoop/Spark.