Troubleshooting Activity Runner

Activities remain in queued state

Normally,  when a user runs an activity from the website, it goes into a queued state. The website then sends a wakeup request to all ActivityRunners that it knows about and one of the runners picks up the activity. If all runners are busy, this can take some time, but if a runner is available, picking up should be almost instantaneous.

If sending the wakeup request fails for some reason, the activity runner will remain in a queued state and will not get picked up automatically unless the runner is restarted. This is because the runner checks for activities in the queue at startup, after finishing the running of any activity or when it gets a wakeup request. If it is not running any activity and it does not get a wakeup request, it will never pick up an activity on its own.

When a runner gets a wakeup request, this is logged in SheetKraft.log with a message of 'Received wakup'. If such a log entry is not found, there may be some network issue like a firewall that is blocking the request. This happens most commonly when an activity runner is installed on a different machine from the primary application server. One way to test this is to run the command Invoke-WebRequest -Uri http://ip:port/wakeup -Method post -TimeoutSec 10 on PowerShell. ip and port need to be replaced with appropriate values targeting the runner that is not receiving the wakeup request. This command is expected to return a successful but empty response (HTTP status code 204). If it times out or fails in any other way, there is a network problem that can be shown to the IT. The command can be run both from the primary application server and from the machine with the additional activity runner. If the command works from the activity runner machine but fails from the primary application server, this is a clear indication of some network problem like a misconfigured firewall.