I am trying to read text file from remote hdfs in labs.itversity.com from my local environment. I could get it successfully run when reading from local filesystem.
Can you please let me know how I can authenticate when connecting to labs hdfs? Here is my code that reads the file:
textFile = sc.textFile(“hdfs://188.8.131.52:50070/user/simhalagna25/demo/data/nyse/NYSE_1997.txt”)
py4j.protocol.Py4JJavaError: An error occurred while calling o20.partitions.
: java.net.ConnectException: Call From apples-MacBook-Pro.local/192.168.3.107 to gw01.itversity.com:50070 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)