15/09/19 10:28:44 WARN cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
This message in log means that Hadoop cluster does not have resources which are asked by Job.
I submitted Spark job on my test local cluster , and got this message.
Additional logs also show following ,
15/09/19 10:23:20 INFO yarn.YarnRMClient: Registering the ApplicationMaster 15/09/19 10:23:20 INFO yarn.YarnAllocator: Will request 2 executor containers, each with 1 cores and 1408 MB memory including 384 MB overhead 15/09/19 10:23:20 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:1408, vCores:1>) 15/09/19 10:23:20 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:1408, vCores:1>) 15/09/19 10:23:20 INFO yarn.ApplicationMaster: Started progress reporter thread with (heartbeat : 5000, initial allocation : 200) intervals
My cluster had only 3 GB RAM so Yarn cannot allocated what Spark was asking
No comments:
Post a Comment
Please share your views and comments below.
Thank You.