Synchronize local files with Cryptomator Vault

Among thousands of files on my computer, I have some with special meanings for me. Photos, videos, papers, and spreadsheets – they occupy dedicated places within my filesystem. That’s the problem – they exist only on my hard drive. However, I sometimes need a quick access to these files from my smartphone or other computers. The solution seems to be simple: migrating to the cloud. But what if I definitely don’t want to keep these files outside?

Personally, I don’t tap into the cloud storage too much. That’s my private choice that may change in the future. But cloud storage is so powerful, cheap and widely available. It would be nice to use available space on free storage providers, especially in a case when I need some data outside the home.

This is the time when encryption comes in.

Meet Cryptomator

Last time I found a nice project called Cryptomator. It’s a client-side encryption software which supports transparent encryption. That means you have easy access to your encrypted files as if they are decrypted. But it doesn’t mean that they are insecure – they are decrypted on-the-fly on demand. Moreover, Cryptomator stores each file separately. That’s cool, cloud-friendly solution!

I started playing with this and I decided to make a small experiment. I’ve tried to write a simple mirroring solution that once a day synchronizes a bunch of my local files to the encrypted Cryptomator’s vault.

Cryptomator nad macOS’ Automator

I use macOS, so I started by running Automator, Apple’s solution to easily create workflows. Automator is the great tool to create a custom task which interacts with the macOS.

Because Cryptomator is a GUI application, I tried to set up a semi-automatic workflow, similar to following

Run Cryptomator
Wait for X seconds and let user manually unlock vault
Copy files to vault
Close Cryptomator

Single blocks within the Automator allow me to create a workflow like above. Unfortunately, it has several problems.

It was not fully automated
Files always be copied regardless of changes
Workflow didn’t recognize the WebDAV destination after the re-run

These problems enforce me to find other solution. Ideally when Cryptomator would offer a CLI access to manipulate the vault directly by e.g. shell script. After a small research, I found the project cryptomator/cli on GitHub. That was exactly what I was looking for.

Synchronize local files with encrypted vault

CLI for Cryptomator allows us to easily start the WebDAV server as an access layer to the encrypted vault. This server could be running in the background. To get a quick access to the files you need to connect to it. I decided to mount it as an external drive. Therefore, I can interact with this virtual space like with a normal directory.

Platform and Software requirements

Java – Cryptomator is written in Java languages
JCE unlimited strength policy files
Cryptomator CLI JAR file

Install JCE unlimited strength policy files

If you use homebrew you can install the necessary policy files by running:

brew cask install jce-unlimited-strength-policy

Unfortunately, it didn’t work for me so I had to go a manually way. According to Readme file, I extracted files directly to the /lib/security. The `` directory depends on your installation but in most cases, it should be in /Library/Java/JavaVirtualMachines/< jdk_version_of_your_pc >/Contents/Home/jre, according to this StackOverflow answer.

Flow of mirroring files to the encrypted vault

The flow is quite simple:

Run cryptomator-cli and start WebDAV server
Create a dedicated directory as mount point within the filesystem
Attach WebDAV server to new directory
Synchronize files from specific directory using rsync
Unmount the WebDAV server
Remove unnecessary directory
Exit Cryptomator

As you see there is no a big deal and basic shell script can handle it. And that was the goal.

Prototype the mirror script

Configuration

First, I set up some configuration variables. They are widely used in the whole script.

CRYPTOMATOR_PATH="/usr/local/bin/cryptomator-cli.jar"
DIR_TO_SYNC_PATH="$HOME/Private"
MOUNT_PATH="$HOME/VAULT_$RANDOM"
VAULT_PATH="$HOME/PATH/TO/ENCRYPTED/VAULT"
VAULT_PASSWORD="`security find-generic-password -a KEYCHAIN_ENTRY_VAULT_PASSPHRASE -w`"
VAULT_NAME="vault"
BIND_HOST="localhost"
BIND_PORT="8198"

I think they are self-described and they don’t need more comments.

Note about variables
Some of above variables are used as constants. In the shell there is no difference between them. As you can see, names of variables describe the value rather than containing the value. Because of that, they are change-resistant as long as the context is matched. For instance, instead of DOCUMENTS_DIR I use DIR_TO_SYNC_PATH, because I could change the directory and the previus name would not match. You can read more about it in my post: Do your constants make sense?.

The one variable is interesting – VAULT_PASSWORD. Instead of hard-coding plaintext password within the script, I use security tool. This is a macOS tool to retrieve data and password from your keychain. In this case, the script is looking for entry KEYCHAIN_ENTRY_VAULT_PASSPHRASE (you can also adjust this name).

Keychain is very restrictive. You cannot retrieve anything from it, even using the security tool. Before the first use, macOS will ask for permissions to keychain access.

Run Cryptomator CLI in the background

Using above configuration you can run WebDAV server which serves the content of your vault:

/usr/bin/java -jar $PATH_TO_CRYPTOMATOR \
--vault $VAULT_NAME=$VAULT_PATH  \
--password $VAULT_NAME=$VAULT_PASSWORD \
--bind $BIND_HOST \
--port $BIND_PORT &

PID=($!)

The PID will be needed in the very last step to finally go server down.

Mount WebDAV server to the userspace

To ensure easy access to the vault you can mount it like a normal storage. The mount point may be anywhere, but I recommend keeping it in your home directory. You will not a root access to mount it. Moreover, the vault contains your private data, right?

mkdir -p $MOUNT_PATH 
/sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH

Tip:
Instead of using mount_webdav you can also use a mount -t davfscommand.

Synchronization private data with the vault

The easy approach is to copy all data from a private dir to the mounted vault. Although it looks fine, it enforces full cloud synchronization, because all files are modifying. Instead of copy, you can use the rsync tool. Below command uses recursive synchronization and preserves modification times, so it’s cloud-friendly. Additionally, you can exclude some files, e.g. temporary files.

/usr/bin/rsync -rvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/

Cleanup

The very last point is to unmount WebDAV server, remove the mounting point and eventually kill the Cryptomator. Let’s do this!

/sbin/umount $MOUNT_PATH
rm -rf $MOUNT_PATH
kill $PID

Put these parts together

Final script has some extra echo statements for debugging purposes. It has also a couple of timeouts and checks to ensure that every step has proceeded properly.

I noticed that sometimes WebDAV is not mounted correctly to the file system. I don’t know the reason, but as a workaround, I created a loop that tries mount the vault after the short pause.

n=0
until [ $n -ge 5 ]
do
    echo "Try $n: Mount http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ to $MOUNT_PATH"
    /sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH && break
    n=$[$n+1]
    sleep 1
done

If the vault is mounted then we’ll exit the loop, thanks to the logical ANDoperator and break statement. As you see, the maximum count of trials is 5, so in the worst case, the script will continue the execution without a mounted vault. To avoid synchronize files with non-mounted directory I created an extra check.

if /sbin/mount | grep $MOUNT_PATH > /dev/null; then
    /usr/bin/rsync -hrvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/
    /sbin/umount $MOUNT_PATH
else
    echo "WebDAV not mounted correctly..."
fi

Eventually, the whole script looks like below.

#!/bin/sh

CRYPTOMATOR_PATH="/usr/local/bin/cryptomator-cli.jar"
DIR_TO_SYNC_PATH="$HOME/Private"
MOUNT_PATH="$HOME/VAULT_$RANDOM"
VAULT_PATH="$HOME/PATH/TO/ENCRYPTED/VAULT"
VAULT_PASSWORD="`security find-generic-password -a KEYCHAIN_ENTRY_VAULT_PASSPHRASE -w`"
VAULT_NAME="vault"
BIND_HOST="localhost"
BIND_PORT="8198"

/usr/bin/java -jar $PATH_TO_CRYPTOMATOR \
--vault $VAULT_NAME=$VAULT_PATH  \
--password $VAULT_NAME=$VAULT_PASSWORD \
--bind $BIND_HOST \
--port $BIND_PORT &

PID=($!)

echo "Cryptomator started under PID: $PID"
sleep 5
echo "Create new mounting point: $MOUNT_PATH"
mkdir -p $MOUNT_PATH
echo "Attach WebDAV interface to mount point"

n=0
until [ $n -ge 5 ]
do
    echo "Try $n: Mount http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ to $MOUNT_PATH"
    /sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH && break
    n=$[$n+1]
    sleep 1
done

if /sbin/mount | grep $MOUNT_PATH > /dev/null; then
    /usr/bin/rsync -hrvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/
    /sbin/umount $MOUNT_PATH
else
    echo "WebDAV not mounted correctly..."
fi

echo "Removing mounting point"
rm -rf $MOUNT_PATH
echo "Close cryptomator"
kill $PID

Link the script to macOS

Store vault passphrase in the keychain

You should store a passphrase to your vault in the keychain. I use a Systemkeychain. After running above script for the first time you’ll be asked for the permission to access this specific entry. To avoid prompting every time when a script is executing, consider adding security to the allowed application list.

Schedule execution

I’m very used to cron, but I read that this is not a recommended solution in macOS. To define scheduled jobs you should use launchd daemon.

Start by creating the plist job file. In my case this is pl.skrajewski.backup.plist.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>pl.skrajewski.backup</string>

<key>Program</key>
<string>/usr/local/bin/custom/backup.sh</string>

<key>StartCalendarInterval</key>
<dict>
    <key>Hour</key>
    <integer>22</integer>
    <key>Minute</key>
    <integer>45</integer>
</dict>

<key>StandardErrorPath</key>
<string>/var/log/vault-backup.error.log</string>

<key>StandardOutPath</key>
<string>/var/log/vault-backup.log</string>
</dict>
</plist>

This file seems to be more complicated than simple crontab entry but is more powerful. In truth, this is a basic XML file. If you want to know more about the possible configuration options, please take a look on this launchd site.

I briefly describe above file:

Label is a mandatory key and should be unique within the launchdinstance.
Program contains complete path to the executable.
StartCalendarInterval allows you to define the specific time when the job run.
StandardErrorPath is a path to file for stderr.
StandardOutPath is a path to file dedicated for stdout.

Load the job into the launchd daemon

Load the plist definition into the launchd is very simple:

sudo launchctl load pl.skrajewski.backup.plist

And that’s it. The job is currently managed by launchd. If you have an error Path had bad ownership/permissions don’t worry! Change the owner of this file to root:wheel by running the below command

sudo chown root:wheel pl.skrajewski.backup.plist

and try to load the file again. If you want to change the definition of your job, remember about the unload it first using launchctl unload.

Summary

You can find both of this files on GitHub Gists.

Cryptomator is a good piece of software. Moreover, it’s open source. I like the idea of a transparent encryption because it significantly facilitates access to encrypted files. The client-side encryption also put a limit on ways how your data is used by cloud storage providers. This is a double-edged weapon – your data is secure, but nobody helps you if you lost the master key.

When it comes to the proposed mirroring solution – this is only my experiment. I treat this more like curio rather than a way to synchronize files. That’s why I mirror my files instead of storing them directly in cryptomator’s vault. When I wrote this article, Cryptomator CLI was in 0.3.1-Pre-release version. It’s not stable yet but I continue my tests.

I can’t wait for the JavaScript library for the Cryptomator. I’d like to see web-based clients and possible integration with popular cloud storage providers. Client-side encryption is good for folks, not companies that live from their users’ data. Maybe this is the reason why we don’t have this kind of encryption yet?