I’m modernising a 12+ year old PHP code base, and to improve the application performance, I decided to run some tasks asynchronously. It’s notoriously difficult to write asynchronous PHP code, so I took an alternative approach using Beanstalk queues, where a task execution is deferred to a later time. This approach has its pros and cons, but for now it’s good enough for this system.
A worker watches the queue for new jobs, and when those arrive, it processes them. The worker - a simple PHP script - must be run all the time, which can be done in many different ways. The easiest solution would be setting up a cronjob, but it has a few drawbacks (e.g. properly monitoring the tasks, handling script crashes, etc). I thought about using monit or Upstart/Systemd, but it was definitely overkill. Then I got to supervisord: it’s almost as simple as cron, but with many more features.
Installing supervisord
To install supervisord, you must have Python installed. You can install it either via pip or through a package manager, although these packages tend to be outdated.
pip install supervisor
If you install it using pip, you will also need to setup a init script. The official repository contains the scripts for the most popular Linux distros.
For Ubuntu, you need to copy the script to /etc/init.d/supervisord, and then update its permissions:
sudo chmod +x /etc/init.d/supervisord
Next, you need to enable the service:
sudo update-rc.d supervisord defaults
sudo service supervisord start
Configuration
To configure supervisord, you need to create a new configuration file. supervisord comes with the echo_supervisord_conf command, which bootstraps this process:
sudo echo_supervisord_conf > /etc/supervisord.conf
After that, append the following to /etc/supervisord.conf. This setting will inform supervisord to look for our monitors in the directory /etc/supervisord.d/.
[include]
files=/etc/supervisord.d/*.conf
Then, create the monitors directory:
mkdir -p /etc/supervisord.d/
Finally, create a directory to store the monitors’ logs:
mkdir -p /var/log/supervisord
Process control
Creating a new monitor is trivial - you only need to create a new configuration file and reload supervisord.
Below is a monitor with very basic settings:
[program:my-queue]
command=php /home/mordor/www/script.php
numprocs=1
numprocs_start=1
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/queue.err.log
stdout_logfile=/var/log/supervisor/queue.out.log
- [program:my-queue]: the program name (it will be used as the display name in
supervisord). - command: the command
supervisordwill run. - numprocs: the number of instances of the program that we want to run.
- numprocs_start: the number of instances of the program that we want to run at the start of the program.
- autostart: specifies if the process should be automatically started when
supervisordstarts. - autorestart: specifies if the process should be automatically restarted if it exits with an exit code different than 0 or 2.
- stderr_logfile: the path of the stderr log.
- stdout_logfile: the path of the stdout log.
After creating your monitor, you must reload supervisord by running supervisorctl update.
Managing processes
supervisord also provides the supervisorctl command line utility. All the available commands can be found in the documentation.
Start/stop/restart all workers
supervisorctl [start|stop|restart] all
Start/stop/restart a single worker
supervisorctl [start|stop|restart] <worker>
Reread configs and restart supervisord
supervisorctl update
Monitor status
supervisorctl status