Uploading my files to lab

Uploading my files to lab
2.0 2


Hi Team,

How can i import or download my test files to lab, to practce or let me know where i can find sample data(different formats JSON, XML , CSV etc) to practice.

I mean from my local system to my folder on lab



@gvenkatanarayana To manage your files use winscp from windows. Or you can use Ambari File view as well.
Host Name: gw01.itversity.com
Port : 22
User name and password from here


Similar issue , I would say for me. I am trying to execute below command from my local to move data to my lab terminal, but I get an error

scp -r lakshmit@gw01.itversity.com:/data/retail_db /Users/itversity/Research

it says /Users/itversity/Research folder not found and it does not let me create these folders also. Let me know what to to.

I am going thru below video in itversity

Big Data Workshop - 08 - Spark - Writing Spark applications using Scala - Compute Daily Revenue


And I just try to copy retail_db data to my /home/lakshmit , I get an error again

scp -r lakshmit@gw01.itversity.com:/data/retail_db /home/lakshmit

it asked for password credentials and I copy paste password from Big data labs , I see this error

/home/lakshmit: Operation not supported


@Lakshmi use command like bellow
scp file/to/copy/location username@gw01.itversity.com:~
Then give the password.
In Your home directory you can find the files.Then you can copy into hdfs by using command
hadoop fs -put /home/username/file/location/ /user/username/destination/location/


That worked thanks ,and I moved the files to HDFS also. Then , when am in my local mac , in sbt console,
here is what I tried , to create orders RDD

and I get a exception , when I tried the take action on the RDD . Please look into , my hdfs fsck command on the orders file shows me location . The hdfs url , I have in below sc.textFile is correct ??

scala> val orders = sc.textFile(“hdfs://wn04.itversity.com:50010/user/lakshmit/research/data/retail_db/orders”)

scala> orders.take(5).foreach(println)

java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: “Yuvarajs-MBP.attlocal.net/”; destination host is: “wn04.itversity.com”:50010;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
at org.apache.hadoop.ipc.Client.call(Client.java:1351)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy20.getFileInfo(Unknown Source)



i am getting below iisue


@gvenkatanarayana Which client application causing the above issue for you. When you are getting the above problem.



while connecting winscp


@gvenkatanarayana Please try filezilla instead of winscp. Winscp is not supporting for some users.



No use with filezilla

can you try from ur side for my login


@gvenkatanarayana I am able to connect winscp as well filezilla. Please check your network firewall from your side and provider side.If you are trying to connect in corporate network it might be blocking.