Launching spark cluster on ec2

apache-spark

#1

Hi all. Am getting this error when I try launching spark cluster on ec2 (new aws account. No problem with other account):ERROR:boto:400 Bad Request
ERROR:boto:b’<?xml version="1.0" encoding="UTF-8"?>\nInvalidVpcID.NotFoundThe vpc ID ‘vpc-3ee01b46’ does not ex ista80c9aa6-21ae-4469-9a86-f6d1dbf5c963</R equestID>’
Setting up security groups…
Creating security group spark-cluster-play-master
Traceback (most recent call last):
File “./spark_ec2.py”, line 1532, in
main()
File “./spark_ec2.py”, line 1524, in main
real_main()
File “./spark_ec2.py”, line 1353, in real_main
(master_nodes, slave_nodes) = launch_cluster(conn, opts, cluster_name)
File “./spark_ec2.py”, line 502, in launch_cluster
master_group = get_or_make_group(conn, cluster_name + “-master”, opts.vpc_id )
File “./spark_ec2.py”, line 369, in get_or_make_group
return conn.create_security_group(name, “Spark EC2 group”, vpc_id)
File “C:\Users\name\Desktop\Shell\spark-1.6.3-bin-hadoop2.6\ec2\lib\boto-2. 34.0\boto\ec2\connection.py”, line 3004, in create_security_group
SecurityGroup, verb=‘POST’)
File “C:\Users\name\Desktop\Shell\spark-1.6.3-bin-hadoop2.6\ec2\lib\boto-2. 34.0\boto\connection.py”, line 1204, in get_object
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

<?xml version="1.0" encoding="UTF-8"?>

InvalidVpcID.NotFoundThe vpc ID ’ vpc-3ee01b46’ does not exista80c9aa6-21ae- 4469-9a86-f6d1dbf5c963


Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster