Contents
Introduction
In this section, we'll provide a real life example of configuration setup for development purposes.
Let's say we're confronted to a problem setting up a development platform for the dev team. Every developer needs its own instance to debug tasks and workflows. One of the possible solutions would be to install standalone evQueue for each developer (on VM for example), but the maintenance wouldn't be easy.
Instead, we'll use a clustered environment on one machine using unix sockets and configuration file variables.
Environment description
For this setup, all developers share a common server.
Every developer has its own tasks directory /data/dev-login/evqueue-tasks in which we put tasks executed by evQueue.
We install only one evQueue instance that is system wide. This engine will be launched several times with different configurations to provide a dedicated instance to all developers.
Configuration
We will now use a special evQueue feature: it's possible to set environment variables that will be used to replace parts of the configuration file.
The configuration file can contain variables between braces. If this variable is found in environment variables, it will be replaced by the environment variable value.
We will use 2 variables that are computed by a bash script:
- {login}
Is the developer login - {port}
The port the websocket server will listen on - {cluster}
Is the cluster description, it contains the connection descriptor of all nodes
Now we will describe the modifications made to the standard configuration file to setup our development cluster.
First, for everything to work properly, we need a separate IPC queue for each instance:
The IPC queue ID will be computed dynamically from the directory inode. The directory MUST exist.
Next, we want each developer to have its own task directory. Relative paths in tasks will be relative to this direcvtory.
Each node must have a unique socket for cluster to work. This will also be used to access API.
Each developper must have its own port to access websockets interface :
Avoid pid file from being overridden:
Last thing is the cluster configuration:
Launching the cluster
Now we have everything ready. We need a bash script to set variables and launch the cluster instances.
# Discover users
USERS=`ls /data/*`
# Build cluster description
CLUSTER=""
for i in $USERS
do
if [ "$CLUSTER"!= "" ]
then
CLUSTER="$CLUSTER,"
fi
CLUSTER="${CLUSTER}unix:///data/$i/evqueue.socket"
done
# Start engine instances
port=5100
for i in $USERS
do
login=$i cluster=$CLUSTER port={port} /usr/bin/evqueue --config /etc/evqueue.multiuser.conf --daemon
port=$((port+1))
done
As you can see, a new evQueue instance will be launched for each directory found under /data so every developer will have its own instance.
This bash script sets the {login}, {port} and {cluster} variables that will be used for replacement in the configuration file.
How it works
After you've launched the cluster, every developer will have its own evQueue node. The configuration of tasks and workflows will be shared by all instances so everyone can see every task and workflow.
The frontend configuration should be configured with all the cluster nodes (see clustering) but you only need to install it once. It will be shared by all the developers. In this sample, websockets ports start with 5100 and increase.
A developer who wants to launch a new workflow instance must contact its own node with the following connection string: unix:///data/dev-login/evqueue.socket. Of course dev-login must be replaced by the real login.
Tasks must be created with a relative path, as processmanager.tasks.directory will be prepended to this path. This way every developer will execute tasks located in its personal directory. Even when executing a workflow created by someone else, the tasks will be launched from the developer personal directory.
For code sharing (tasks) we use a git repository shared amongst all developers. That way, once a workflow is ready, tasks are pushed to the repository and every developer can use it.