Activity Runner

Introduction

The SheetKraft Activity Runner component does all the actual work of executing activities run by the user from the SheetKraft portal. The broad flow looks like this:

  1. User opens an activity sheet, provides inputs, and starts the activity

  2. Portal captures the inputs in the SheetKraft database and queues the activity in the database

  3. Activity Runner picks up the activity from the queue and runs it. Any outputs produced are stored in the database.

  4. Portal displays the outputs to the user

The activity runner is a Windows Service. This means that it keeps running even when there is no work for it to do. It has a configured number of threads. Each thread can run one activity at a time. Thus an activity runner with 4 threads can run 4 activities in parallel. When there are no running activities, the threads are in an inactive state and consume very few resources.

Apart from the threads dedicated to running activities, the activity runner has additional threads. For example, if email triggers are configured, additional threads are employed to listen for incoming emails on the inboxes being monitored. When an email is received that matches specific conditions, these threads queue the activity in the database.

There is also a coordinating thread that gets new work and assigns it to one of the threads dedicated for running activities. Whenever a new activity is queued into the database, either by the portal or by other threads in the activity runner itself, the coordinating thread is woken up from its inactive state. It checks how many threads are currently available for running activities and attempts to pick up that many activities from the queue. The activities are then assigned to the free threads and the coordinating thread goes back to its inactive state. Whenever any of the threads finishes running an activity, it wakes up the coordinating thread so that it can be assigned more work.

It is possible to have more than one instance of the activity runner, either on different machines or even on the same machine. An activity runner can be configured to run activities from specific modules only or to ignore activities from specific modules. By creating different activity runners for different modules, it is possible to ensure that too many activities getting queued from one module do not prevent activities from other modules from running. By adding activity runners on multiple machines, the overall capacity of the system to run activities in parallel increases. However, it should be noted that all activity runners work on the same database, and if multiple activities attempt to write to the same data at the same time, they will not be able to run in parallel or be able to run successfully despite having multiple runners.

Details of the activity queue

An activity runner is configured with a GUID (Globally Unique Identifier) and a url on which it listens for requests from the portal. When the activity runner starts up, it registers its unique id and its url in the database. Whenever the portal queues an activity, it sends a wakeup request to all registered urls.

The activity queue is just a database table with information about the activity to be run. Each entry in the queue is assigned a run id and begins its life in the queued status. When the coordinating thread of an activity runner wakes up and requests activities from the queue, entries are allocated to the runner. This means that the entries are associated with the unique id of the runner and the status changes to running. Once the activity runner finishes running the activities that were allocated to it, the status of the activities changes to succeeded or failed.

If the activity runner is stopped cleanly, it stops picking up any new activities, and then waits until all the activities that it was already running complete, and then it shuts down. While one activity runner is stopped, other activity runners can still pick up activities on the queue. Even if there are no other activity runners, the queued activities will be picked up the next time the activity runner starts again.

However if the activity runner is terminated, any activities it was running at the time get terminated too. There is no opportunity to update their status in the database. The next time the activity runner starts up, it sees the activities that are associated with its unique id but are neither succeeded nor failed. These activities are then marked as abandoned.