重新配置scalr
[root@localhost bin]# /opt/scalr-server/bin/scalr-server-ctl reconfigure
Recipe: scalr-server::group_mysql_enabled_post
* supervisor_service[mysql] action enable
* template[/opt/scalr-server/etc/supervisor/conf.d/mysql.conf] action create (up to date)
- Enabling supervisor_service[mysql]
* supervisor_service[mysql] action start
================================================================================
Error executing action `start` on resource 'supervisor_service[mysql]'
================================================================================
RuntimeError
------------
service mysql not in state RUNNING after 20 tries
Cookbook Trace:
---------------
/opt/scalr-server/embedded/cookbooks/supervisor/providers/service.rb:164:in `wait_til_state'
/opt/scalr-server/embedded/cookbooks/supervisor/providers/service.rb:45:in `block in class_from_file'
Resource Declaration:
---------------------
# In /opt/scalr-server/embedded/cookbooks/scalr-server/recipes/group_mysql_enabled_post.rb
3: supervisor_service 'mysql' do
4: command "#{node[:scalr_server][:install_root]}/embedded/bin/pidproxy" \
5: " #{run_dir_for node, 'mysql'}/mysql.pid" \
6: " #{node[:scalr_server][:install_root]}/embedded/bin/mysqld_safe" \
7: " --defaults-file=#{etc_dir_for node, 'mysql'}/my.cnf" \
8: " --basedir=#{node[:scalr_server][:install_root]}/embedded"
9: stdout_logfile "#{log_dir_for node, 'supervisor'}/mysql.log"
10: stderr_logfile "#{log_dir_for node, 'supervisor'}/mysql.err"
11: action [:enable, :start]
12: autostart true
13: startsecs 10
14: subscribes :restart, 'user[mysql_user]' if service_is_up?(node, 'mysql')
15: end
16:
Compiled Resource:
------------------
# Declared in /opt/scalr-server/embedded/cookbooks/scalr-server/recipes/group_mysql_enabled_post.rb:3:in `from_file'
supervisor_service("mysql") do
action [:enable, :start]
updated true
retries 0
retry_delay 2
default_guard_interpreter :default
declared_type :supervisor_service
cookbook_name :"scalr-server"
recipe_name "group_mysql_enabled_post"
command "/opt/scalr-server/embedded/bin/pidproxy /opt/scalr-server/var/run/mysql/mysql.pid /opt/scalr-server/embedded/bin/mysqld_safe --defaults-file=/opt/scalr-server/etc/mysql/my.cnf --basedir=/opt/scalr-server/embedded"
stdout_logfile "/opt/scalr-server/var/log/supervisor/mysql.log"
stderr_logfile "/opt/scalr-server/var/log/supervisor/mysql.err"
autostart true
startsecs 10
service_name "mysql"
process_name "%(program_name)s"
numprocs 1
numprocs_start 0
priority 999
autorestart :unexpected
startretries 3
exitcodes [0, 2]
stopsignal :TERM
stopwaitsecs 10
stdout_logfile_maxbytes "50MB"
stdout_logfile_backups 10
stdout_capture_maxbytes "0"
stderr_logfile_maxbytes "50MB"
stderr_logfile_backups 10
stderr_capture_maxbytes "0"
serverurl "AUTO"
end
Running handlers:
[2016-08-09T16:53:29+08:00] ERROR: Running exception handlers
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] Unable to record event with remote Sentry server (Faraday::ConnectionFailed - execution expired)
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:923:in `block in connect'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/timeout.rb:101:in `call'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/timeout.rb:101:in `timeout'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:923:in `connect'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:852:in `start'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/2.1.0/net/http.rb:1375:in `request'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:82:in `perform_request'
[2016-08-09T16:53:31+08:00] ERROR: ** [Raven] /opt/scalr-server/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:40:in `block in call'
- Raven::Chef::SentryHandler
Running handlers complete
[2016-08-09T16:53:31+08:00] ERROR: Exception handlers complete
[2016-08-09T16:53:31+08:00] FATAL: Stacktrace dumped to /opt/scalr-server/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 29 resources updated in 91.728195093 seconds
[2016-08-09T16:53:32+08:00] ERROR: supervisor_service[mysql] (scalr-server::group_mysql_enabled_post line 3) had an error: RuntimeError: service mysql not in state RUNNING after 20 tries
[2016-08-09T16:53:32+08:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
杀掉和mysql相关进程
[root@localhost bin]# ps -ef | grep mysql
root 5865 1 0 16:31 ? 00:00:00 /opt/scalr-server/embedded/bin/python /opt/scalr-server/embedded/bin/pidproxy /opt/scalr-server/var/run/mysql/mysql.pid /opt/scalr-server/embedded/bin/mysqld_safe --defaults-file=/opt/scalr-server/etc/mysql/my.cnf --basedir=/opt/scalr-server/embedded
root 5896 5865 0 16:31 ? 00:00:00 /bin/sh /opt/scalr-server/embedded/bin/mysqld_safe --defaults-file=/opt/scalr-server/etc/mysql/my.cnf --basedir=/opt/scalr-server/embedded
scalr-m+ 6339 5896 0 16:31 ? 00:00:09 /opt/scalr-server/embedded/bin/mysqld --defaults-file=/opt/scalr-server/etc/mysql/my.cnf --basedir=/opt/scalr-server/embedded --datadir=/opt/scalr-server/var/lib/mysql --plugin-dir=/opt/scalr-server/embedded/lib/plugin --user=scalr-mysql --log-error=/opt/scalr-server/var/log/mysql/error.log --pid-file=/opt/scalr-server/var/run/mysql/mysql.pid --socket=/opt/scalr-server/var/run/mysql/mysql.sock --port=6280
root 18338 4993 0 16:53 pts/0 00:00:00 grep --color=auto mysql
[root@localhost bin]# kill -9 5865
[root@localhost bin]# kill -9 5896
[root@localhost bin]# kill -9 6339
[root@localhost bin]# ps -ef | grep mysql
root 18877 4993 0 16:54 pts/0 00:00:00 grep --color=auto mysql
杀掉所有scalr相关进程
[root@localhost bin]# ps -ef | grep scalr | grep -v grep | awk '{print $2}' | xargs kill -9
reference:
使用xargs与awk联合使用批量杀进程,很方便
检查是否已经杀光
[root@localhost bin]# ps -ef | grep scalr
root 19659 4993 0 16:55 pts/0 00:00:00 grep --color=auto scalr
再次重新配置
[root@localhost bin]# /opt/scalr-server/bin/scalr-server-ctl reconfigure
检查80端口
[root@localhost bin]# netstat -antup | grep :80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 20079/httpd
tcp 0 0 10.200.44.123:57654 184.173.242.46:80 TIME_WAIT -
tcp 0 1 10.200.44.123:57665 184.173.242.46:80 FIN_WAIT1 -
tcp 0 0 10.200.44.123:50577 151.101.88.249:80 TIME_WAIT -
tcp 0 0 10.200.44.123:43737 202.114.18.160:80 ESTABLISHED 21494/python
tcp 0 0 10.200.44.123:57675 184.173.242.46:80 TIME_WAIT -
tcp 0 0 10.200.44.123:57678 184.173.242.46:80 ESTABLISHED 20092/python
tcp 0 0 10.200.44.123:43681 202.114.18.160:80 TIME_WAIT -
udp 0 0 0.0.0.0:805 0.0.0.0:* 1085/rpcbind
udp6 0 0 :::805 :::* 1085/rpcbind
看到pid是20079,然后查看20079进程的依赖关系
[root@localhost bin]# ps -ef | grep 20079
root 20079 20075 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
scalr-a+ 20140 20079 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
scalr-a+ 20141 20079 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
scalr-a+ 20142 20079 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
scalr-a+ 20143 20079 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
scalr-a+ 20144 20079 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND
root 22265 4993 0 16:58 pts/0 00:00:00 grep --color=auto 20079
[root@localhost bin]# pstree | grep httpd -A 10
| |-httpd---5*[httpd]
| |-memcached---5*[{memcached}]
| |-php---2*[{php}]
| |-pidproxy---mysqld_safe---mysqld---38*[{mysqld}]
| |-python---dbqueue_event---{dbqueue_event}
| |-python---msg_sender---msg_sender---{msg_sender}
| |-python---analytics_polle---{analytics_polle}
| |-python---szr_upd_service---{szr_upd_service}
| |-python---load_statistics
| |-python---load_statistics---plotter---103*[{plotter}]
| |-python---analytics_proce---{analytics_proce}
[root@localhost bin]# pstree | grep httpd -C 5
|-rsyslogd---2*[{rsyslogd}]
|-rtkit-daemon---2*[{rtkit-daemon}]
|-smartd
|-sshd
|-supervisord-+-crond
| |-httpd---5*[httpd]
| |-memcached---5*[{memcached}]
| |-php---2*[{php}]
| |-pidproxy---mysqld_safe---mysqld---38*[{mysqld}]
| |-python---dbqueue_event---{dbqueue_event}
| |-python---msg_sender---msg_sender---{msg_sender}
[root@localhost bin]# ps -ef | grep superv
root 20075 1 0 16:56 ? 00:00:00 /opt/scalr-server/embedded/bin/python /opt/scalr-server/embedded/bin/supervisord -c /opt/scalr-server/etc/supervisor/supervisord.conf
root 22472 4993 0 16:59 pts/0 00:00:00 grep --color=auto superv
下次再遇到可以尝试
/opt/scalr-server/embedded/bin/httpd -f /opt/scalr-server/etc/httpd/httpd.conf -DFOREGROUND