spark-submit master为yarn。命令如下:
spark-submit --master yarn --deploy-mode cluster --class com.fooww.research.houserank.recommend.data.RentData --jars /root/fangwu/mysql-connector-java-8.0.16.jar,jedis-2.7.0.jar,spark-redis-0.3.2.jar --conf spark.default.parallelism=10 --conf spa
rk.cores.max=10 --conf spark.executor.cores=1 --conf spark_executor_instances=5 --conf num-executor=5 fangwu-spark-1.0.0.jar
为什么执行之后,还是只分配了两个executor?
--conf num-executor=5
改成 --num-executor 5