13518219792
问题：
网站建设哪家好，找成都创新互联公司！专注于网页设计、网站建设、微信开发、成都微信小程序、集团企业网站建设等服务项目。为回馈新老客户创新互联还提供了色尼免费建站欢迎大家使用！
用spark-submit以yarn-client方式提交任务，在集群的某些节点上的任务出现连接超时的错误，排查过各种情况后，确定在防火墙配置上出现问题。
原因：
我猜测是python程序启动后，作为Server，hadoop中资源调度是以java程序作为Client端访问，
Python启动的Server端需要接受localhost的client访问。
当你从一台linux主机向自身发送数据包时,实际上的数据包是通过虚拟的lo接口来发送接受的,而不会通过你的物理网卡 eth0/eth2....，此时防火墙就要允许来自本地lo接口的数据包，需要加入以下配置允许Python Server接受来自本地lo接口的数据包，然后解决该问题。
iptables -A INPUT -i lo -j ACCEPT 添加iptables规则,允许来自于lo接口的数据包
任务的部分报错日志
16/07/25 13:56:44 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev d62701d4d05dfa6115bbaf8d9dff002df142e62d] 16/07/25 13:56:44 INFO Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 16/07/25 13:56:44 INFO Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 16/07/25 13:56:44 INFO Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 16/07/25 13:56:44 INFO Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 16/07/25 13:56:44 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 16/07/25 13:57:47 WARN python.PythonWorkerFactory: Failed to open socket to Python daemon: java.net.ConnectException: 连接超时 at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.(Socket.java:425) at java.net.Socket. (Socket.java:241) at org.apache.spark.api.python.PythonWorkerFactory.createSocket$1(PythonWorkerFactory.scala:75) at org.apache.spark.api.python.PythonWorkerFactory.liftedTree1$1(PythonWorkerFactory.scala:90) at org.apache.spark.api.python.PythonWorkerFactory.createThroughDaemon(PythonWorkerFactory.scala:89) at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:62) at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:135) at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:101) at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:70) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:297) at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) at org.apache.spark.api.python.PairwiseRDD.compute(PythonRDD.scala:342) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:297) at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:88) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 16/07/25 13:57:47 WARN python.PythonWorkerFactory: Assuming that daemon unexpectedly quit, attempting to restart 16/07/25 13:58:51 ERROR executor.Executor: Exception in task 0.0 in stage 0.0 (TID 0)
参考地址：
http://stackoverflow.com/questions/15659132/connection-refused-between-a-python-server-and-a-java-client
http://stackoverflow.com/questions/26297551/connecting-python-and-java-via-sockets/38605208#38605208
http://www.zybang.com/question/9ab66451988eb2768194817f25a0b7a9.html
