Skip to content

Configuring Linux OS cache servers

Linux cache server utilizes FUSE based solution to provide access to files to third-party applications that are not physically present on cache server. Together with Ganesha NFS server it allows to access this virtual file system over NFS v3 or v4. Linux cache server is used in a File caching or Hybrid work Jobs.

Feature availability

Available in: Resilio Active Everywhere 4.0.0
Supported for: Linux x64. Linux arm64 is supported starting with v4.2.0.
Not supported: docker installations for Gateway.

Resilio Agent installation

Root is mandatory

Run the Resilio Agent as root. When installing via packages - make sure to update your .service files to run agent as root.

Install Resilio Agent via package or tarball unpacking. For details, see Install Agent.

Third party packages installation

Resilio Agent allows the exposure of gateway shares using either NFS (via Ganesha NFS server) or Samba (via Samba server). The specific requirements for additional libraries depend on the chosen configuration:

  • NFS and Samba setup: Certain libraries must be installed regardless of whether NFS or Samba is used.
  • NFS-Specific setup: Additional dependencies may be required for NFS functionality. Ensure that all necessary libraries are installed based on your environment to guarantee proper operation of the gateway shares.

Ubuntu Rocky LinuxOracle Linux 9Oracle Linux 8CentosAmazon Linux 2023

Required: fuse3 ``libfuse2 (for pre-4.1 Agent version) lsof
Additionally for NFS-Specific Setup: nfs-ganesha-5 nfs-ganesha- vfs``software-properties-common

Follow these commands to connect necessary repositories deploy all of the above. All commands must be started as root. If any of packages informs it's installed already - ignore the message.

apt install -y software-properties-common
apt update && sudo apt-get upgrade -y
apt install -y fuse3 libfuse2 nfs-ganesha nfs-ganesha-vfs

ensure /usr/bin/lsof is available either as binary or as a symbolic link

Required: fuse3``libfuse2 (for pre-4.1 Agent version) lsof
Additionally for NFS-Specific Setup: :libnfsidmap libntirpc libprometheus-cpp libtirpc libwbclient, nfs-ganesha nfs-ganesha-vfs nfs-utils rpcbind

Follow these commands to connect necessary repositories deploy all of the above. All commands must be started as root. If any of packages informs it's installed already - ignore the message.

setenforce 0
dnf install centos-release-nfs-ganesha5  
dnf install fuse3  
dnf install dbus-tools
dnf download libnfsidmap.x86_64 libntirpc.x86_64 libprometheus-cpp.x86_64 libtirpc.x86_64 libwbclient.x86_64 nfs-ganesha.x86_64 nfs-ganesha-vfs.x86_64 nfs-utils.x86_64 rpcbind.x86_64
rpm -i --nodeps libnfsidmap.x86_64
rpm -i --nodeps libntirpc.x86_64
rpm -i --nodeps libprometheus-cpp.x86_64
rpm -i --nodeps libtirpc.x86_64
rpm -i --nodeps libwbclient.x86_64
rpm -i --nodeps nfs-ganesha.x86_64
rpm -i --nodeps nfs-ganesha-vfs.x86_64
rpm -i --nodeps nfs-utils.x86_64
rpm -i --nodeps rpcbind.x86_64

NFS-Specific Setup:libnfsidmap libntirpc libprometheus-cpp libtirpc libwbclient nfs-ganesha nfs-ganesha-vfs nfs-utils rpcbind

Follow these commands to connect necessary repositories deploy all of the above. All commands must be started as root. If any of packages informs it's installed already - ignore the message.

setenforce 0
wget https://internal.resilio.com/install/centos-release-nfs-ganesha5.rpm
rpm -i --nodeps centos-release-nfs-ganesha5.rpm
dnf download libnfsidmap.x86_64 libntirpc.x86_64 libprometheus-cpp.x86_64 libtirpc.x86_64 libwbclient.x86_64 nfs-ganesha.x86_64 nfs-ganesha-vfs.x86_64 nfs-utils.x86_64 rpcbind.x86_64
rpm -i --nodeps libnfsidmap.x86_64
rpm -i --nodeps libntirpc.x86_64
rpm -i --nodeps libprometheus-cpp.x86_64
rpm -i --nodeps libtirpc.x86_64
rpm -i --nodeps libwbclient.x86_64
rpm -i --nodeps nfs-ganesha.x86_64
rpm -i --nodeps nfs-ganesha-vfs.x86_64
rpm -i --nodeps nfs-utils.x86_64
rpm -i --nodeps rpcbind.x86_64

Old Ganesha NFS

Oracle Linux 8 only allows installation of Ganesha NFS v3.5. For a newer version of Ganesha NFS ensure you are running Oracle Linux 9+

Required:fuse lsof
Additionally NFS-Specific Setup: glusterfs-server oracle-gluster-release- el8 nfs-ganesha-vfs

Follow these commands to connect necessary repositories deploy all of the above. All commands must be started as root:

setenforce 0        
dnf install -y lsof fuse3
dnf install -y oracle-gluster-release-el8
dnf config-manager --enable ol8_addons ol8_UEKR6 ol8_appstream
dnf install -y glusterfs-server nfs-ganesha-vfs
setsebool -P ganesha_use_fusefs 1

Required:fuse lsof
Additionally NFS-Specific Setup: glusterfs-server oracle-gluster-release- el8 nfs-ganesha-vfs

Follow these commands to connect necessary repositories deploy all of the above. All commands must be started as root:

setenforce 0
dnf install -y centos-release-nfs-ganesha5
dnf install -y libnfsidmap libntirpc libprometheus-cpp libtirpc libwbclient nfs-ganesha nfs-ganesha-vfs nfs-utils rpcbind

Required: fuse fuse-libs
Additionally NFS-Specific Setup: rpcbind nfs-utils libtirpc libwbclient libnfsidmap nfs-ganesha nfs-ganesha-vfs libntirpc libprometheus-cpp

Follow these commands to connect necessary repositiries deploy all of the above. All commands must be started as root:

dnf install rpcbind nfs-utils libtirpc libwbclient libnfsidmap fuse3 fuse-libs
wget https://internal.resilio.com/install/centos-release-nfs-ganesha5.rpm
rpm -i --nodeps centos-release-nfs-ganesha5.rpm
mkdir packages
cd packages
dnf download nfs-ganesha nfs-ganesha-vfs libntirpc libprometheus-cpp
rpm -i --nodeps -v *.rpm

No standard NFS daemon

Exposing Resilio virtual TSS folder is not supported with standard NFS daemon. Use Ganesha NFS v5 or newer instead.

Configure Ganesha NFS server

Make export directory writable for all

chmod 0777 /mnt/ganesha-test

Ensure nfs-ganesha server is stopped, then edit the file /etc/ganesha/ganesha.conf. Use example config file for Ganesha as a template

NFSv4 {  
  Only_Numeric_Owners = true;  
      }  
EXPORT
  {
  Export_Id = 12345;
  Path = /mnt/ganesha-test;
  Pseudo = /mnt/ganesha-test;
  Protocols = 3,4;
  Access_Type = RW;
  Squash = none;
  FSAL {
         Name = VFS;
       }
  }

Configuration file supports several EXPORTs. For each new unique job location a new EXPORT must be added to the config. Export_Id, Path and Pseudo must be different for every export.

The folder that represents Resilio job does not have to be a root of exported folder, it may be a part of subtree.

Start nfs-ganesha to apply new configuration
systemctl restart nfs-ganesha

Ubuntu 25.04

Additional configuration is required for Ubuntu 25 installations. Ensure that job mounted folder is added to Apparmor profile:

  1. Update the Apparmor /etc/apparmor.d/fusermount3 profile config file. Include the job folder, or ensure that it's included by default rules already. Restart the Apparmor service with systemctl restart apparmor.
    Note, next Apparmor package update will rewrite it.
  2. Put fusermount3 profile in complain mode.
  3. Disable fusermount3 profile. For more information on Apparmour Profile, see Ubuntu documentation - AppArmor.

NFS server delay

By default both NFS server and NFS clients cache FS objects metadata, so NFS users will not see new files until those caches are up-to-date. Having shorter TTL for cache entries can help to see new files sooner, but it affects overall performance. Default Ganesha NFS rescan interval is 60s, it can be configured in /etc/ganesha/ganesha.conf, for example 5s :

EXPORT
  {
  ...
  Attr_Expiration_Time = 5;
  ...
  }

NFS client cache entry TTL can be set in mount options, for example 5s: actimeo=5

Configure NFS clients accessing Ganesha NFS exports

Mount exported NFS folder from another Linux host

mount -v -o async,nconnect=16,noatime <server>:/mnt/ganesha-test /mnt/ganesha-test

Ensure read and write operations work normally and cause files to hydrate in Resilio virtual folder (number of local files in Management Console will grow every time you open a new file). Mount exported NFS folder on several Linux hosts and make sure Agent does not stuck under load.

Configure Samba server

The following guide is verified for Ubuntu and Debian distros. Install Samba server

apt install -y samba

Configure Samba. Add the following to /etc/samba/smb.conf, where path /mnt/storage/share1 is the Resilio job folder

[share1]
comment = any comment  
read only = no  
path = /mnt/storage/share1  
guest ok = no

To ensure the config is valid, run as root:

testparm

Add samba users. This step can be skipped if you already have a configured Samba server with users' access to it. Note, the users must exist in /etc/passwd. Run the following command as root, it will prompt for a password for each user:

smbpasswd -a user1

Make sure shared folder is mounted:

# mount | grep resiliofs  
resiliofs on /mnt/storage/share1 type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

Restart samba server:

systemctl restart smbd

Configure Samba clients accessing the samba shares

Mount configured folder from a Linux host:

USER=user1 PASSWD=mysecurepassword123 mount //my-samba-server/share1 /mnt/smb

Ensure read and write operations work normally and cause files to hydrate in Resilio virtual folder as number of local files in Management Console will grow every time you open a new file.

Configuring File caching/Hybrid work job

Linux cache server can be added to the job only after the Agent and 3rd party packages were installed and properly configured. Two locations must be selected for Caching Gateway agent.

Warning

The selected paths cannot be changed later after the job is saved. For more information, see Applying any job changes or restarting the Gateway Agent.

Linux path: it is the "Path" matching Ganesha or Samba config file above, or a directory inside it. Is the file access location which is made available over NFS/Samba remotely. Files are virtually represented by this location for further access. This path must exist and be empty.
Linux cache: the actual file download location, not available for file access.

Syncing home folders

For the synchronization of Linux users’ home folders it is recommended to add the following lines to File and folder filter in Job profile.

.Xauthority
.Xauthority*

Applying any job changes or restarting the Gateway Agent

Should you need to apply any changes in the job or restart Resilio Agent or its host, please follow the pattern below:

  • Stop any application / daemon using the NFS share exported by Ganesha
  • (optional, just speeds things up) Unmount exported shares
  • Stop nfs-ganesha daemon
  • Stop Resilio Agent / restart host

Start all components in the opposite order:

  • Start Resilio Agent first
  • Start nfs-ganesha daemon
  • Ensure exported servers are mounted on Linux machines that will consume the data
  • Start your applications that will work with the data on Ganesha exports

Note

When changing the mount point, while cache folder stays the same, the new mount point will be remounted and initial indexing will be triggered, the file counters will start from zero. All files in the cache folder will be listed and reused as the pre-seeded scenario.