Skip to content

Increasing Management Console worker threads

Management Console is a resource intensive application serving hundreds of Agents and tens of Jobs. For the purposes of optimization and stability, the Management Console process is split into several subprocesses: a main thread and a number of workers. Workers perform communication between the Agents and Management Console: send out jobs' configuration, collect statistics and events, update database entries, etc., while the main thread accumulates data from the workers.

By default, the Management Console runs one main thread and two workers. Usually this is sufficient for most use cases; however, in some cases it's advisable to increase the number of workers.

Determine the number of workers needed

These are general recommendations on the number of workers in relation to the number of connected Agents:

Agents (range)         Workers (total) Main Worker RAM Regular Worker RAM (each) Estimated Total RAM (MC)
500 - 1 000 2 6 GB 1 GB 8 GB
1 000 - 2 000 3 6 GB 1 GB 9 GB
2 000 - 4 000 4 6 GB 1 GB 10 GB
4 000 - 5 000 5 8-10 GB 1-1.5 GB 15.25 GB
5 000 - 7 500 6 12-14 GB ~1.5 GB 22 GB
7 500 - 11 500 7 18-20 GB 1.5-2 GB 31.25 GB
11 500 - 16 000 8 24 GB ~2 GB 40 GB
16 000 - 27 000 9 28 GB 2-2.5 GB 48.25 GB
27 000 - 40 000 10 32 GB 2-3 GB 57 GB

Info

The table above is a general recommendation and applies to Resilio deployments running 3 to 7 Jobs. The more Jobs you have, the more workers you may need. If you have more than 7 Jobs, contact support for recommendations.

Tip

One of the symptoms that you may need more workers is general slowness of the Management Console UI.

Determine the current number of workers

To determine the number of workers currently running, check the processes on the host running the Management Console:

  • Windows: Open Task Manager and look for Node.js JavaScript Runtime processes.

    Node Js

  • Linux:

    Run the command:

    ps aux | grep node

    vboxuser@primary-storage:~$ ps aux | grep node
root        2314  0.0  0.0   2936  2316 ?        Ss   10:21   0:00 fusermount3 -o  w,nosuid,nodev,fsname=portal,auto_unmount,subtype=portal -- /run/user/1000/doc
root        3224  0.7  4.8 1374128 264356 ?      Ssl  10:22   2:14 /opt/resilio-connect-server/nodejs/bin/node --max_old_space_size=6144 --no-deprecation /opt/resilio-connect-server/server/server.js run --appdata /opt/resilio-connect-server/var
root        3236  0.0  3.0 1333156 166024 ?      Sl   10:22   0:08 /opt/resilio-connect-server/nodejs/bin/node --max_old_space_size=6144 --no-deprecation /opt/resilio-connect-server/server/server.js run --appdata /opt/resilio-connect-server/var --worker_id=0-stats --worker_role=REPORTING_DATA_STORAGE
root        3243  0.0  2.8 1136428 157296 ?      Sl   10:22   0:04 /opt/resilio-connect-server/nodejs/bin/node --max_old_space_size=2048 --no-deprecation /opt/resilio-connect-server/server/server.js run --appdata /opt/resilio-connect-server/var --worker_id=0-report_building --worker_role=REPORT_BUILDING
root        3250  0.3  3.1 1335112 172972 ?      Sl   10:22   0:55 /opt/resilio-connect-server/nodejs/bin/node --max_old_space_size=1024 --no-deprecation /opt/resilio-connect-server/server/server.js run --appdata /opt/resilio-connect-server/var --worker_id=0 --worker_role=AGENT
root        3251  0.0  3.1 1141068 169920 ?      Sl   10:22   0:07 /opt/resilio-connect-server/nodejs/bin/node --max_old_space_size=1024 --no-deprecation /opt/resilio-connect-server/server/server.js run --appdata /opt/resilio-connect-server/var --worker_id=0 --worker_role=PUBLIC_API_GATEWAY
vboxuser    4486 25.0  0.0  18148  2584 pts/0    S+   15:05   0:00 grep --color=auto node

Increase the number of workers

Warning: Proceed with caution

Misconfiguration of workers may negatively impact your deployment. Contact support for assistance.

Impact of increasing the number of workers

Increasing the number of workers will increase CPU and system memory usage. Management Console normally utilizes two CPU cores (one for the main thread and one for worker processes). Management Console will utilize one additional core per extra worker.

By default the Management Console requires ~3 GB of system memory for the main thread and ~1 GB per worker thread. It is recommended to increase the system memory allocation for the main process by ~1 GB for each extra worker.

To increase the number of workers:

  1. Stop the Management Console process. Ensure the process stops cleanly; do not force kill it.

  2. Locate and open the Management Console's configuration file:

    • Windows: C:\ProgramData\Resilio\ConnectServer\resilio-connect-server.conf
    • Linux: ${resilio_connect_server_dir}/var/resilio-connect-server.conf

  3. Add the following lines at the top level (or edit them if they already exist). Make sure to preserve JSON formatting.

    "agentWorkers":  
{
  "number": 1,  
  "maxMemory": 1024  
}

    where:

    • number is the total number of workers to spawn
    • maxMemory is the amount of RAM (in MB) allowed for each worker thread to use.

  4. Start the Management Console and verify the number of workers.

Increase Management Console's main process memory allocation

Windows

To increase the maximum system memory allocated to the main process on Windows:

Note

Applicable to Resilio Active Everywhere 3.8.2 and newer. For information on adjusting the main process memory allocation on older versions, see the note at the end of this section.

  1. Stop the Management Console service.

  2. Browse your file system and open the srvctrl.exein a text editor.

    Note

    By default, the srvctrl.exe file is located in the C:\Program Files\Resilio Connect Server\ folder.

  3. Search for the line containing max_old_space_size parameter, replace it with CUSTOM_MEMORY_LIMIT and adjust its value (megabytes).

    @echo off
set NODE_ENV=production

node --CUSTOM_MEMORY_LIMIT=6144 --no-deprecation server\server.js %*

  4. Save the file, then start the Management Console service.

Adjusting Management Console's main process memory allocation on Resilio Active Everywhere older than 3.8.2

  1. Stop the Management Console service.
  2. Open regedit and navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\resilio-connect-server.

  3. Edit the ImagePath key and adjust the value of the --max_old_space_size parameter (megabytes), for example:

    "C:\Program Files\Resilio Connect Server\node.exe" "--max_old_space_size=3072" "--no-deprecation" "C:\Program Files\Resilio Connect Server\server\server.js" "run"

  4. Close regedit, then start the Management Console service.

Linux

To increase the maximum system memory allocated to the main process on Linux:

  1. Stop the Management Console service.

    • Management Console installed from a DEB or RPM package:

      sudo systemctl stop resilio-connect-server

    • Management Console installed from a tarball archive:

      sudo ./srvctrl stop

  2. Edit the srvctrl shell script, and adjust the value of the CUSTOM_MEMORY_LIMIT parameter.

    #!/bin/sh

CUSTOM_NODE_ARGS=
CUSTOM_MEMORY_LIMIT=12288

# Do not edit below this line
MEMORY_LIMIT=6144
if [ -n "$CUSTOM_MEMORY_LIMIT" ] && [ "$MEMORY_LIMIT" -lt "$CUSTOM_MEMORY_LIMIT" ]; then
    MEMORY_LIMIT=$CUSTOM_MEMORY_LIMIT
fi

DIST_PATH=$(dirname $0)
SERVER_DIST_PATH=$DIST_PATH/server
export PATH=$DIST_PATH/nodejs/bin:$PATH
export NODE_ENV=production
node --max_old_space_size=$MEMORY_LIMIT --no-deprecation $CUSTOM_NODE_ARGS "$SERVER_DIST_PATH/server.js" $*

    Resilio Active Everywhere older than 3.8.2

    For Resilio Active Everywhere older than 3.8.2, set max_old_space_size parameter value directly in the start script.

    \#!/bin/sh

DIST_PATH=$(dirname $0)
SERVER_DIST_PATH=$DIST_PATH/server
export PATH=$DIST_PATH/nodejs/bin:$PATH
export NODE_ENV=production
node --max_old_space_size=3072 "$SERVER_DIST_PATH/server.js" $*

  3. Save the file, then start the Management Console service:

    • Management Console installed from a DEB or RPM package:

      sudo systemctl start resilio-connect-server

    • Management Console installed from a tarball archive:

      sudo ./srvctrl start