The Relay API gateway optionally requires a queuing system, which means tasks get queued in a central data store and worked on after a request has finished. The tasks can be processes using one or more workers on multiple machines in parallel.
This requires two extra deployment related tasks:
1) One or more worker tasks have to be run in the background and automatically restarted if they stop. 2) On deployment the worker processes have to be restarted to use the new code.
In the bundle configuration set the
queue_dsn key to a DSN supported by the
Symfony messenger component
At the moment we only support the redis and doctrine transports:
This transport requires the Redis PHP extension (>=4.3) and a running Redis server (^5.0).
# config/packages/dbp_relay_core.yaml dbp_relay_core: # redis[s]://[pass@][ip|host|socket[:port]] queue_dsn: 'redis://localhost:6379'
This creates a redis stream automatically when active.
In case of doctrine you have to install
composer require symfony/doctrine-messenger
then create a doctrine connection and point the
queue_dsn to that connection:
# config/packages/doctrine.yaml doctrine: dbal: connections: worker_queue_connection_name: url: 'mysql://db:secret@mariadb:3306/db'
# config/packages/dbp_relay_core.yaml dbp_relay_core: queue_dsn: 'doctrine://worker_queue_connection_name'
It will automatically create a new database table when active.
Keep your secrets in the .env.local file
To provide easier maintainability for your project it is recommended to keep all secrets in the .env.local file. In this case, you could add a variable QUEUE_URL='mysql://db:secret@mariadb:3306/db' to your .env.local file, and reference it using url: '%env(QUEUE_URL)%'
Run the workers
Start a worker using
./bin/console dbp:relay:core:queue:work my-worker-01
It will automatically exit after a specific amount 0f time or after a specific number of processed tasks.
- You need to take care of restarting it automatically.
- Each active worker needs to have a unique name passed as the first argument which should stay the same across restarts.
Restart the workers
After deployment run
This will signal the workers to exit after the current task, which means they will be restarted by supervisor and will run the newly deployed code.
Manage Workers with Supervisor
Symfony recommends to use Supervisor to handle worker jobs and restart them.
sudo apt-get install supervisor
;/etc/supervisor/conf.d/queue-worker.conf [program:queue-work] command=php /path/to/your/app/bin/console dbp:relay:core:queue:work "%(program_name)s_%(process_num)02d" user=user numprocs=2 startsecs=0 autostart=true autorestart=true process_name=%(program_name)s_%(process_num)02d
user to the Unix user on your server.
Testing your Setup
After everything is set up you can create a few dummy tasks and see if they get handled by the workers:
./bin/console dbp:relay:core:queue:test --count 10 --delay 3