请教使用supervisor启动pyspider运行中phantomjs进程数量过多问题

Question

下面是supervisor的配置

[program:pyspider]
command=/usr/local/bin/pyspider -c /root/config.json
autostart=true
autorestart=true
user=root
directory=/root

不使用supervisor来启动pyspider,phantomjs进程为2个.
使用supervisor后,phantomjs进程超过6个以上,并且在代码界面进行测试也会卡死,只能kill掉pyspider和phantomjs才行.
环境是Ubuntu14.04X64
python2.7.6
phantomjs1.9.0
vps内存768M

Answer 1

[group:pyspider]
program=pyspider-webui,pyspider-scheduler,pyspider-processor,pyspider-result_worker,pyspider-fetcher,pyspider-phantomjs
priority=999

[program:pyspider-webui]
command=/usr/local/bin/pyspider/run.py -c /root/config.json webui
directory=/root
autostart=true
autorestart=true
priority=905
user=root

[program:pyspider-scheduler]
command=/usr/local/bin/pyspider/run.py -c /root/config.json scheduler
directory=/root
autostart=true
autorestart=true
priority=900
user=root

[program:pyspider-processor]
command=/usr/local/bin/pyspider/run.py -c /root/config.json processor
directory=/root
autostart=true
autorestart=true
priority=903
user=root

[program:pyspider-result_worker]
command=/usr/local/bin/pyspider/run.py -c /root/config.json result_worker
directory=/root
autostart=true
autorestart=true
priority=904
user=root

[program:pyspider-fetcher]
command=/usr/local/bin/pyspider/run.py -c /root/config.json --phantomjs-proxy="localhost:25555" fetcher
directory=/root
autostart=true
autorestart=true
priority=902
user=root

[program:pyspider-phantomjs]
command=/usr/local/bin/pyspider/run.py -c /root/config.json phantomjs
directory=/root
autostart=true
autorestart=true
priority=901
user=root

Answer 2

尝试轻量级的，系统自带，不用配置文件，直接命令

nohup /usr/local/bin/pyspider -c /root/config.json &

输出调试信息都在nohup.out中查看