Spark - 名称选项尚不清楚
我不明白这个选项。似乎是执行者的最大人数。
如果群集中的节点上没有足够的内存，则该数字无需做任何事情，而执行者却少于我要求的。
有人可以澄清一下吗？
I do not understand this option. It seems like its the maximum number of executors.
If there is not enough memory on the nodes in the cluster this number is doing nothing and there are fewer executors than I'm asking for.
Can someone clarify this please?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
选项
- 在计算执行者数量的基础架构支持从工作节点上的可用内存中支持的执行者数量后，使用了NUM-EXECTOR。计算可以按照说明进行
The option
--num-executorsis used after we calculate the number of executors our infrastructure supports from the available memory on the worker nodes.The calculation can be performed as stated here. This wuill let you know the number of executors supported by your hadoop infrastructure or your the queue that has been assigned to your team.