spark-submit yarn-client 运行失败

发布于 2022-01-05 19:55:03 字数 12967 浏览 776 评论 1

2015-08-26 16:48:55,941 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - Changing view acls to: root

2015-08-26 16:48:55,946 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - Changing modify acls to: root

2015-08-26 16:48:55,947 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)

2015-08-26 16:48:56,403 INFO [sparkDriver-akka.actor.default-dispatcher-4] slf4j.Slf4jLogger (Slf4jLogger.scala:applyOrElse(80)) - Slf4jLogger started

2015-08-26 16:48:56,456 INFO [sparkDriver-akka.actor.default-dispatcher-4] Remoting (Slf4jLogger.scala:apply$mcV$sp(74)) - Starting remoting

2015-08-26 16:48:56,816 INFO [sparkDriver-akka.actor.default-dispatcher-5] Remoting (Slf4jLogger.scala:apply$mcV$sp(74)) - Remoting started; listening on addresses :[akka.tcp://sparkDriver@hadoop05:54033]

2015-08-26 16:48:56,820 INFO [sparkDriver-akka.actor.default-dispatcher-5] Remoting (Slf4jLogger.scala:apply$mcV$sp(74)) - Remoting now listens on addresses: [akka.tcp://sparkDriver@hadoop05:54033]

2015-08-26 16:48:56,832 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'sparkDriver' on port 54033.

2015-08-26 16:48:56,857 INFO [main] spark.SparkEnv (Logging.scala:logInfo(59)) - Registering MapOutputTracker

2015-08-26 16:48:56,870 INFO [main] spark.SparkEnv (Logging.scala:logInfo(59)) - Registering BlockManagerMaster

2015-08-26 16:48:56,889 INFO [main] storage.DiskBlockManager (Logging.scala:logInfo(59)) - Created local directory at /tmp/spark-local-20150826164856-d539

2015-08-26 16:48:56,893 INFO [main] storage.MemoryStore (Logging.scala:logInfo(59)) - MemoryStore started with capacity 530.3 MB

2015-08-26 16:48:57,347 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

2015-08-26 16:48:57,538 INFO [main] spark.HttpFileServer (Logging.scala:logInfo(59)) - HTTP File server directory is /tmp/spark-8ea80eb5-5ae9-4f11-8c5f-4616d14fc3d3

2015-08-26 16:48:57,543 INFO [main] spark.HttpServer (Logging.scala:logInfo(59)) - Starting HTTP Server

2015-08-26 16:48:57,603 INFO [main] server.Server (Server.java:doStart(272)) - jetty-8.y.z-SNAPSHOT

2015-08-26 16:48:57,624 INFO [main] server.AbstractConnector (AbstractConnector.java:doStart(338)) - Started SocketConnector@0.0.0.0:50271

2015-08-26 16:48:57,624 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'HTTP file server' on port 50271.

2015-08-26 16:48:57,739 INFO [main] server.Server (Server.java:doStart(272)) - jetty-8.y.z-SNAPSHOT

2015-08-26 16:48:57,752 INFO [main] server.AbstractConnector (AbstractConnector.java:doStart(338)) - Started SelectChannelConnector@0.0.0.0:4040

2015-08-26 16:48:57,753 INFO [main] util.Utils (Logging.scala:logInfo(59)) - Successfully started service 'SparkUI' on port 4040.

2015-08-26 16:48:57,756 INFO [main] ui.SparkUI (Logging.scala:logInfo(59)) - Started SparkUI at http://hadoop05:4040

2015-08-26 16:48:57,885 INFO [main] spark.SparkContext (Logging.scala:logInfo(59)) - Added JAR /home/hbase/cdrp-0.0.2-SNAPSHOT.jar at http://10.2.98.144:50271/jars/cdrp-0.0.2-SNAPSHOT.jar with timestamp 1440578937884

2015-08-26 16:48:57,886 INFO [main] spark.SparkContext (Logging.scala:logInfo(59)) - Added JAR /home/hbase/oozie-client-4.0.0-cdh5.2.0.jar at http://10.2.98.144:50271/jars/oozie-client-4.0.0-cdh5.2.0.jar with timestamp 1440578937886

2015-08-26 16:48:58,052 INFO [main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at /0.0.0.0:8032

2015-08-26 16:48:58,283 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Requesting a new application from cluster with 7 NodeManagers

2015-08-26 16:48:58,298 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Verifying our application has not requested more than the maximum memory capability of the cluster (9990 MB per container)

2015-08-26 16:48:58,299 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Will allocate AM container, with 1408 MB memory including 384 MB overhead

2015-08-26 16:48:58,300 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Setting up container launch context for our AM

2015-08-26 16:48:58,305 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Preparing resources for our AM container

2015-08-26 16:48:58,442 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Source and destination file systems are the same. Not copying file:/opt/cloudera/parcels/CDH-5.3.5-1.cdh5.3.5.p0.4/jars/spark-assembly-1.2.0-cdh5.3.5-hadoop2.5.0-cdh5.3.5.jar

2015-08-26 16:48:58,526 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Setting up the launch environment for our AM container

2015-08-26 16:48:58,731 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - Changing view acls to: root

2015-08-26 16:48:58,732 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - Changing modify acls to: root

2015-08-26 16:48:58,732 INFO [main] spark.SecurityManager (Logging.scala:logInfo(59)) - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)

2015-08-26 16:48:58,735 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Submitting application 42 to ResourceManager

2015-08-26 16:48:58,776 INFO [main] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(236)) - Submitted application application_1440484199657_0042

2015-08-26 16:48:59,782 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:48:59,790 INFO [main] yarn.Client (Logging.scala:logInfo(59)) -

client token: N/A

diagnostics: N/A

ApplicationMaster host: N/A

ApplicationMaster RPC port: -1

queue: root.root

start time: 1440578938748

final status: UNDEFINED

tracking URL: http://hadoop05:8088/proxy/application_1440484199657_0042/

user: root

2015-08-26 16:49:00,794 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:01,798 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:02,803 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:03,807 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:04,812 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:05,818 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:06,823 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:07,828 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:08,832 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:09,837 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:10,844 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:11,849 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: ACCEPTED)

2015-08-26 16:49:12,853 INFO [main] yarn.Client (Logging.scala:logInfo(59)) - Application report for application_1440484199657_0042 (state: FAILED)

2015-08-26 16:49:12,855 INFO [main] yarn.Client (Logging.scala:logInfo(59)) -

client token: N/A

diagnostics: Application application_1440484199657_0042 failed 2 times due to AM Container for appattempt_1440484199657_0042_000002 exited with exitCode: 1 due to: Exception from container-launch.

Container id: container_1440484199657_0042_02_000001

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)

at org.apache.hadoop.util.Shell.run(Shell.java:455)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

.Failing this attempt.. Failing the application.

ApplicationMaster host: N/A

ApplicationMaster RPC port: -1

queue: root.root

start time: 1440578938748

final status: FAILED

tracking URL: http://hadoop05:8088/cluster/app/application_1440484199657_0042

user: root

Exception in thread "main" org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.

at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:102)

at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:58)

at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:140)

at org.apache.spark.SparkContext.<init>(SparkContext.scala:335)

at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)

at com.goodwill.cdr.hadoop.spark.PatientCountJHMK.execute(PatientCountJHMK.java:56)

at com.goodwill.cdr.hadoop.spark.PatientCountJHMK.main(PatientCountJHMK.java:40)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)

at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)

at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)