Spark session is not working

I have created spark session as below. though it is giving some warning errors still created.

spark = SparkSession.
… builder.
… appName(‘CSV Example’).
… master(‘local’).
… getOrCreate()
20/08/24 02:38:44 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to “WARN”

Now when reading json data or text, it is throwing error path doest not exist.

spark.read.json(’/user/pradeepkumar/samples/orders.json’)
Traceback (most recent call last):
File “”, line 1, in
File “/home/pradeepkumar/.local/lib/python3.6/site-packages/pyspark/sql/readwriter.py”, line 300, in json
return self._df(self._jreader.json(self._spark._sc._jvm.PythonUtils.toSeq(path)))
File “/home/pradeepkumar/.local/lib/python3.6/site-packages/py4j/java_gateway.py”, line 1305, in call
answer, self.gateway_client, self.target_id, self.name)
File “/home/pradeepkumar/.local/lib/python3.6/site-packages/pyspark/sql/utils.py”, line 137, in deco
raise_from(converted)
File “”, line 3, in raise_from
pyspark.sql.utils.AnalysisException: Path does not exist: file:/user/pradeepkumar/samples/orders.json;

but these location really exist.

[pradeepkumar@gw03 ~]$ hadoop fs -ls /user/pradeepkumar/samples
Found 13 items
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-12 02:29 /user/pradeepkumar/samples/daily_revenue_avro_python
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-18 03:11 /user/pradeepkumar/samples/daily_revenue_save
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-12 02:54 /user/pradeepkumar/samples/daily_revenue_txt_python
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-18 03:12 /user/pradeepkumar/samples/daily_revenue_write
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-23 13:12 /user/pradeepkumar/samples/order_items_jdbc
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-23 05:37 /user/pradeepkumar/samples/orders.csv
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-23 06:23 /user/pradeepkumar/samples/orders.json
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-09 14:32 /user/pradeepkumar/samples/revenue_per_order_compressed
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-09 14:30 /user/pradeepkumar/samples/revenue_per_order_id
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-09 15:04 /user/pradeepkumar/samples/revenue_per_order_json
-rw-r–r-- 2 pradeepkumar hdfs 98 2020-07-19 06:22 /user/pradeepkumar/samples/sample
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-22 09:57 /user/pradeepkumar/samples/wordcount
drwxr-xr-x - pradeepkumar hdfs 0 2020-08-22 10:06 /user/pradeepkumar/samples/wordcount_csv

Kindly suggest how to resolve this issue?

Hi @Pradeepkumar_C_G,

where you are trying to run these lines of code-
i tried it on jupyter notebook and it works for me without giving any warning.

and while naming to a directory please do not use (.) it might be consider as file extension.