Skip to content

实现一个onYarn的程序 : 有时候我们的standalone的可能会考虑在Yarn上部署,利用内部Yarn支持的DockerContainer的和标签的特性,实现特定机器快速部署和动态的增加节点

Notifications You must be signed in to change notification settings

shangwen/StandaloneOnYarn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Standalone on Yarn项目

编译执行执行脚本./bin/build.sh即可,就可以在目录build看到压缩包

提交时需要配置

yarn.standalone.master.address=spark://master_host:7077 #master的地址
yarn.standalone.hdfs.path=hdfs://ns1/tmp/yarnStandalone/ #压缩包的目录
yarn.standalone.appmaster.tar=standalone-yarn.zip #am的jar包名字
yarn.standalone.tar=standalone.tar #worker的jar包的名字

提交

standalone-yarn.sh startAmServer

增加worker

sh standalone-yarn.sh reserveWorkers -num 15

About

实现一个onYarn的程序 : 有时候我们的standalone的可能会考虑在Yarn上部署,利用内部Yarn支持的DockerContainer的和标签的特性,实现特定机器快速部署和动态的增加节点

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published