Spark is not executing basic commands

i have just purchased bigdata labs am executing basic commands like collect() , take whatever it is it throwing errors as immutable please rectify it

my lab id sagar4477

@Sagar_Bunny - Please paste the full code to recreate the issue.

[sagar4477@gw01 ~]$ spark-shell
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set
Spark1 will be picked by default
Java HotSpot™ 64-Bit Server VM warning: INFO: os::commit_memory(0x0000779688c00000, 716177408, 0) failed; error=‘Cannot allocate memory’ (errno=12)

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (mmap) failed to map 716177408 bytes for committing reserved memory.

An error report file with more information is saved as:

/home/sagar4477/hs_err_pid23560.log

[sagar4477@gw01 ~]$ spark-shell spark.uo.port = 56984
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set
Spark1 will be picked by default
Java HotSpot™ 64-Bit Server VM warning: INFO: os::commit_memory(0x000072406d000000, 716177408, 0) failed; error=‘Cannot allocate memory’ (errno=12)

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (mmap) failed to map 716177408 bytes for committing reserved memory.

An error report file with more information is saved as:

/home/sagar4477/hs_err_pid24832.log

[sagar4477@gw01 ~]$

Hi,

I am unable to launch spark shell due to memory issues. Please find the error below.

[debjanis@gw01 ~]$ spark-shell --master yarn --packages com.databricks:spark-avro_2.10:2.0.1 --conf spark.ui.port=22322
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set
Spark1 will be picked by default
Java HotSpot™ 64-Bit Server VM warning: INFO: os::commit_memory(0x00006f3289b00000, 175112192, 0) failed; error=‘Cannot allocate memory’ (errno=12)
[debjanis@gw01 ~]$

scala> val s = List(1 to 25)
s: List[scala.collection.immutable.Range.Inclusive] = List(Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25))
scala> s.collect()
:28: error: not enough arguments for method collect: (pf: PartialFunction[scala.collection.immutable.Range.Inclusive,B])(implicit bf: scala.collection.generic.CanBuildFrom[Lis
t[scala.collection.immutable.Range.Inclusive],B,That])That.
Unspecified value parameter pf.
s.collect()
^
scala> val s = List(1 to 25)
s: List[scala.collection.immutable.Range.Inclusive] = List(Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25))
scala> val b = s.filter(x => x % 2 == 0)
:27: error: value % is not a member of scala.collection.immutable.Range.Inclusive
val b = s.filter(x => x % 2 == 0)
^

what is the command to copy files from local in it versity
lab val s = sc.textFile(“file:///user/sagar4477/departments”)

its showing file doesn’t exist

[sagar4477@gw01 ~]$ ls -ltr
total 404
-rw-r–r-- 1 sagar4477 students 14124 Jul 17 17:18 categories.java
-rw-r–r-- 1 sagar4477 students 27626 Jul 17 17:19 customers.java
-rw-r–r-- 1 sagar4477 students 11573 Jul 17 17:19 departments.java
-rw-r–r-- 1 sagar4477 students 22555 Jul 17 17:19 order_items.java
-rw-r–r-- 1 sagar4477 students 16344 Jul 17 17:20 orders.java
-rw-r–r-- 1 sagar4477 students 20949 Jul 17 17:20 products.java
-rw-r–r-- 1 sagar4477 students 118 Jul 17 19:03 wordcount.txt
drwxr-xr-x 2 sagar4477 students 4096 Jul 17 23:39 practice
-rw-r–r-- 1 sagar4477 students 22126 Jul 18 12:36 hs_err_pid23560.log
-rw-r–r-- 1 sagar4477 students 22099 Jul 18 12:37 hs_err_pid24832.log
-rw-r–r-- 1 sagar4477 students 25100 Jul 18 12:38 hs_err_pid25550.log
-rw-r–r-- 1 sagar4477 students 25052 Jul 18 12:38 hs_err_pid25608.log
-rw-r–r-- 1 sagar4477 students 22128 Jul 18 12:39 hs_err_pid26359.log
-rw-r–r-- 1 sagar4477 students 21652 Jul 18 12:41 hs_err_pid27349.log
-rw-r–r-- 1 sagar4477 students 22077 Jul 18 12:43 hs_err_pid28848.log
-rw-r–r-- 1 sagar4477 students 22072 Jul 18 12:44 hs_err_pid30012.log
-rw-r–r-- 1 sagar4477 students 22092 Jul 18 12:45 hs_err_pid30456.log
-rw-r–r-- 1 sagar4477 students 22091 Jul 18 12:45 hs_err_pid30541.log
-rw-r–r-- 1 sagar4477 students 22141 Jul 18 12:45 hs_err_pid30690.log
-rw-r–r-- 1 sagar4477 students 725 Jul 18 12:53 derby.log
[sagar4477@gw01 ~]$