Listing hdfs database from within pyspark

hi is there a way to run hdfs dfs -ls /data/… like commands from within pyspark session:

would >>> hdfs dfs -ls /data/… work?

Hi @xyz111,

Please go through the following video.

And also please watch all the videos in the following playlist to understand the labs.

Hi the video takes care of using hive warehouse to run sparkDF and sparkSQL queries…i am after if you did not have hive warehouse then how do you go about it…