I’m modernising a 12+ year old PHP code base, and to improve the application performance, I decided to run some tasks asynchronously. It’s notoriously difficult to write asynchronous PHP code, so I took an alternative approach using Beanstalk queues, where a task execution is deferred to a later time. This approach has its pros and cons, but for now it’s good enough for this system.
A worker watches the queue for new jobs, and when those arrive, it processes them. The worker - a simple PHP script - must be run all the time, which can be done in many different ways. The easiest solution would be setting up a cronjob, but it has a few drawbacks (e.g. properly monitoring the tasks, handling script crashes, etc). I thought about using monit or Upstart/Systemd, but it was definitely overkill. Then I got to supervisord: it’s almost as simple as cron, but with many more features.
To install supervisord, you must have Python installed. You can install it either via
pip or through a package manager, although these packages tend to be outdated.
pip install supervisor
If you install it using
pip, you will also need to setup a init script. The official repository contains the scripts for the most popular Linux distros.
For Ubuntu, you need to copy the script to
/etc/init.d/supervisord, and then update its permissions:
sudo chmod +x /etc/init.d/supervisord
Next, you need to enable the service:
sudo update-rc.d supervisord defaults sudo service supervisord start
supervisord, you need to create a new configuration file.
supervisord comes with the
echo_supervisord_conf command, which bootstraps this process:
sudo echo_supervisord_conf > /etc/supervisord.conf
After that, append the following to /etc/supervisord.conf. This setting will inform
supervisord to look for our monitors in the directory /etc/supervisord.d/.
Then, create the monitors directory:
mkdir -p /etc/supervisord.d/
Finally, create a directory to store the monitors’ logs:
mkdir -p /var/log/supervisord
Creating a new monitor is trivial - you only need to create a new configuration file and reload
Below is a monitor with very basic settings:
[program:my-queue] command=php /home/mordor/www/script.php numprocs=1 numprocs_start=1 autostart=true autorestart=true stderr_logfile=/var/log/supervisor/queue.err.log stdout_logfile=/var/log/supervisor/queue.out.log
- [program:my-queue]: the program name (it will be used as the display name in
- command: the command
- numprocs: the number of instances of the program that we want to run.
- numprocs_start: the number of instances of the program that we want to run at the start of the program.
- autostart: specifies if the process should be automatically started when
- autorestart: specifies if the process should be automatically restarted if it exits with an exit code different than 0 or 2.
- stderr_logfile: the path of the stderr log.
- stdout_logfile: the path of the stdout log.
After creating your monitor, you must reload
supervisord by running
supervisord also provides the
supervisorctl command line utility. All the available commands can be found in the documentation.
Start/stop/restart all workers
supervisorctl [start|stop|restart] all
Start/stop/restart a single worker
supervisorctl [start|stop|restart] <worker>
Reread configs and restart