How to read HDFS Files from python code? I use the below method to read from local file system but it fails to read HDFS files given HDFS file path.
localFileSystem = ‘/home/classic/data/’ ## For reading the file from local file system
with open("".join([input, ‘mappingFile.pkl’]), mode=‘rb’) as fp:
pd_mappingFile = cpick.load(fp)
hdfsFileSystem = ‘/user/classic/data/’ ## For reading the file from HDFS file system but the above open method is unable to read
How do I read HDFS file from python code?
Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs
- Click here for access to state of the art 13 node Hadoop and Spark Cluster