BYR Achieve · 镜像论坛

spark standalone模式，一台master，一台worker； sparkcontext是这样的： val conf = new SparkConf().setAppName("PageRank").set("spark.driver.host", "10.209.73.235").setMaster("spark://10.77.109.181:7077").set("spark.executor.memory", "200m").setSparkHome("/data1/spark-1.3.1-bin-hadoop2.3").setJars(List("out\\artifacts\\PageRank_jar\\PageRank.jar")) 在intellij上显示： 15/06/16 14:12:56 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 1521.0 B, free 133.6 MB) 15/06/16 14:12:56 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.56.1:7979 (size: 1521.0 B, free: 133.6 MB) 15/06/16 14:12:56 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0 15/06/16 14:12:56 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:839 15/06/16 14:12:56 INFO DAGScheduler: Submitting 2 missing tasks from Stage 1 (MapPartitionsRDD[1] at map at PageRank.scala:22) 15/06/16 14:12:56 INFO TaskSchedulerImpl: Adding task set 1.0 with 2 tasks 15/06/16 14:12:56 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, 10.77.109.182, PROCESS_LOCAL, 1453 bytes) 15/06/16 14:12:56 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, 10.77.109.182, PROCESS_LOCAL, 1453 bytes) 15/06/16 14:12:56 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, 10.77.109.182, PROCESS_LOCAL, 1461 bytes) 15/06/16 14:12:56 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 3, 10.77.109.182, PROCESS_LOCAL, 1465 bytes) 15/06/16 14:13:57 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, 10.77.109.182): java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) woker log如下： 15/06/16 14:10:34 INFO Executor: Fetching http://192.168.56.1:7959/jars/PageRank.jar with timestamp 1434435169701 15/06/16 14:11:34 INFO Executor: Fetching http://192.168.56.1:7959/jars/PageRank.jar with timestamp 1434435169701 15/06/16 14:11:34 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0) java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) at sun.net.www.http.HttpClient.New(HttpClient.java:308) at sun.net.www.http.HttpClient.New(HttpClient.java:326) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1167) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1103) log中有个192.168.56.1的地址，但是我的ip地址都是10.77.109开头的，不知道什么原因；还有不太明白sparkconf里面的setJars的含义，在intellij上面提交会driver会把application jar包分发到每一台worker node上吗？

[问题]intellij上提交spark应用的问题