Among thousands of files on my computer, I have some with special meanings for me. Photos, videos, papers, and spreadsheets – they occupy dedicated places within my filesystem. That’s the problem – they exist only on my hard drive. However, I sometimes need a quick access to these files from my smartphone or other computers. The solution seems to be simple: migrating to the cloud. But what if I definitely don’t want to keep these files outside?
Personally, I don’t tap into the cloud storage too much. That’s my private choice that may change in the future. But cloud storage is so powerful, cheap and widely available. It would be nice to use available space on free storage providers, especially in a case when I need some data outside the home.
This is the time when encryption comes in.
Last time I found a nice project called Cryptomator. It’s a client-side encryption software which supports transparent encryption. That means you have easy access to your encrypted files as if they are decrypted. But it doesn’t mean that they are insecure – they are decrypted on-the-fly on demand. Moreover, Cryptomator stores each file separately. That’s cool, cloud-friendly solution!
I started playing with this and I decided to make a small experiment. I’ve tried to write a simple mirroring solution that once a day synchronizes a bunch of my local files to the encrypted Cryptomator’s vault.
If you don’t know what Cryptomator exactly is and how it works, take a look on its website.
Cryptomator nad macOS’ Automator
I use macOS, so I started by running Automator, Apple’s solution to easily create workflows. Automator is the great tool to create a custom task which interacts with the macOS.
Because Cryptomator is a GUI application, I tried to set up a semi-automatic workflow, similar to following
- Run Cryptomator
- Wait for X seconds and let user manually unlock vault
- Copy files to vault
- Close Cryptomator
Single blocks within the Automator allow me to create a workflow like above. Unfortunately, it has several problems.
- It was not fully automated
- Files always be copied regardless of changes
- Workflow didn’t recognize the WebDAV destination after the re-run
These problems enforce me to find other solution. Ideally when Cryptomator would offer a CLI access to manipulate the vault directly by e.g. shell script. After a small research, I found the project cryptomator/cli on GitHub. That was exactly what I was looking for.
Synchronize local files with encrypted vault
CLI for Cryptomator allows us to easily start the WebDAV server as an access layer to the encrypted vault. This server could be running in the background. To get a quick access to the files you need to connect to it. I decided to mount it as an external drive. Therefore, I can interact with this virtual space like with a normal directory.
Platform and Software requirements
- Java – Cryptomator is written in Java languages
- JCE unlimited strength policy files
- Cryptomator CLI JAR file
Install JCE unlimited strength policy files
If you use homebrew you can install the necessary policy files by running:
brew cask install jce-unlimited-strength-policy
Unfortunately, it didn’t work for me so I had to go a manually way. According to Readme file, I extracted files directly to the
/lib/security. The `` directory depends on your installation but in most cases, it should be in
/Library/Java/JavaVirtualMachines/< jdk_version_of_your_pc >/Contents/Home/jre, according to this StackOverflow answer.
Flow of mirroring files to the encrypted vault
The flow is quite simple:
- Run cryptomator-cli and start WebDAV server
- Create a dedicated directory as mount point within the filesystem
- Attach WebDAV server to new directory
- Synchronize files from specific directory using
- Unmount the WebDAV server
- Remove unnecessary directory
- Exit Cryptomator
As you see there is no a big deal and basic shell script can handle it. And that was the goal.
Prototype the mirror script
First, I set up some configuration variables. They are widely used in the whole script.
CRYPTOMATOR_PATH="/usr/local/bin/cryptomator-cli.jar" DIR_TO_SYNC_PATH="$HOME/Private" MOUNT_PATH="$HOME/VAULT_$RANDOM" VAULT_PATH="$HOME/PATH/TO/ENCRYPTED/VAULT" VAULT_PASSWORD="`security find-generic-password -a KEYCHAIN_ENTRY_VAULT_PASSPHRASE -w`" VAULT_NAME="vault" BIND_HOST="localhost" BIND_PORT="8198"
I think they are self-described and they don’t need more comments.
Some of above variables are used as constants. In the shell there is no difference between them. As you can see, names of variables describe the value rather than containing the value. Because of that, they are change-resistant as long as the context is matched. For instance, instead of
DIR_TO_SYNC_PATH, because I could change the directory and the previus name would not match. You can read more about it in my post: Do your constants make sense?.
The one variable is interesting –
VAULT_PASSWORD. Instead of hard-coding plaintext password within the script, I use security tool. This is a macOS tool to retrieve data and password from your keychain. In this case, the script is looking for entry
KEYCHAIN_ENTRY_VAULT_PASSPHRASE (you can also adjust this name).
Keychain is very restrictive. You cannot retrieve anything from it, even using the
security tool. Before the first use, macOS will ask for permissions to keychain access.
Run Cryptomator CLI in the background
Using above configuration you can run WebDAV server which serves the content of your vault:
/usr/bin/java -jar $PATH_TO_CRYPTOMATOR \ --vault $VAULT_NAME=$VAULT_PATH \ --password $VAULT_NAME=$VAULT_PASSWORD \ --bind $BIND_HOST \ --port $BIND_PORT & PID=($!)
PID will be needed in the very last step to finally go server down.
Mount WebDAV server to the userspace
To ensure easy access to the vault you can mount it like a normal storage. The mount point may be anywhere, but I recommend keeping it in your home directory. You will not a root access to mount it. Moreover, the vault contains your private data, right?
mkdir -p $MOUNT_PATH /sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH
Instead of using
mount_webdavyou can also use a
mount -t davfscommand.
Synchronization private data with the vault
The easy approach is to copy all data from a private dir to the mounted vault. Although it looks fine, it enforces full cloud synchronization, because all files are modifying. Instead of copy, you can use the
rsync tool. Below command uses recursive synchronization and preserves modification times, so it’s cloud-friendly. Additionally, you can exclude some files, e.g. temporary files.
/usr/bin/rsync -rvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/
The very last point is to unmount WebDAV server, remove the mounting point and eventually kill the Cryptomator. Let’s do this!
/sbin/umount $MOUNT_PATH rm -rf $MOUNT_PATH kill $PID
Put these parts together
Final script has some extra
echo statements for debugging purposes. It has also a couple of timeouts and checks to ensure that every step has proceeded properly.
I noticed that sometimes WebDAV is not mounted correctly to the file system. I don’t know the reason, but as a workaround, I created a loop that tries mount the vault after the short pause.
n=0 until [ $n -ge 5 ] do echo "Try $n: Mount http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ to $MOUNT_PATH" /sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH && break n=$[$n+1] sleep 1 done
If the vault is mounted then we’ll exit the loop, thanks to the logical
break statement. As you see, the maximum count of trials is 5, so in the worst case, the script will continue the execution without a mounted vault. To avoid synchronize files with non-mounted directory I created an extra check.
if /sbin/mount | grep $MOUNT_PATH > /dev/null; then /usr/bin/rsync -hrvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/ /sbin/umount $MOUNT_PATH else echo "WebDAV not mounted correctly..." fi
Eventually, the whole script looks like below.
#!/bin/sh CRYPTOMATOR_PATH="/usr/local/bin/cryptomator-cli.jar" DIR_TO_SYNC_PATH="$HOME/Private" MOUNT_PATH="$HOME/VAULT_$RANDOM" VAULT_PATH="$HOME/PATH/TO/ENCRYPTED/VAULT" VAULT_PASSWORD="`security find-generic-password -a KEYCHAIN_ENTRY_VAULT_PASSPHRASE -w`" VAULT_NAME="vault" BIND_HOST="localhost" BIND_PORT="8198" /usr/bin/java -jar $PATH_TO_CRYPTOMATOR \ --vault $VAULT_NAME=$VAULT_PATH \ --password $VAULT_NAME=$VAULT_PASSWORD \ --bind $BIND_HOST \ --port $BIND_PORT & PID=($!) echo "Cryptomator started under PID: $PID" sleep 5 echo "Create new mounting point: $MOUNT_PATH" mkdir -p $MOUNT_PATH echo "Attach WebDAV interface to mount point" n=0 until [ $n -ge 5 ] do echo "Try $n: Mount http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ to $MOUNT_PATH" /sbin/mount_webdav http://$BIND_HOST:$BIND_PORT/$VAULT_NAME/ $MOUNT_PATH && break n=$[$n+1] sleep 1 done if /sbin/mount | grep $MOUNT_PATH > /dev/null; then /usr/bin/rsync -hrvtP --update --exclude "~*" $DIR_TO_SYNC_PATH/ $MOUNT_PATH/ /sbin/umount $MOUNT_PATH else echo "WebDAV not mounted correctly..." fi echo "Removing mounting point" rm -rf $MOUNT_PATH echo "Close cryptomator" kill $PID
Link the script to macOS
Store vault passphrase in the keychain
You should store a passphrase to your vault in the keychain. I use a Systemkeychain. After running above script for the first time you’ll be asked for the permission to access this specific entry. To avoid prompting every time when a script is executing, consider adding
security to the allowed application list.
I’m very used to cron, but I read that this is not a recommended solution in macOS. To define scheduled jobs you should use
Start by creating the plist job file. In my case this is
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>pl.skrajewski.backup</string> <key>Program</key> <string>/usr/local/bin/custom/backup.sh</string> <key>StartCalendarInterval</key> <dict> <key>Hour</key> <integer>22</integer> <key>Minute</key> <integer>45</integer> </dict> <key>StandardErrorPath</key> <string>/var/log/vault-backup.error.log</string> <key>StandardOutPath</key> <string>/var/log/vault-backup.log</string> </dict> </plist>
This file seems to be more complicated than simple crontab entry but is more powerful. In truth, this is a basic XML file. If you want to know more about the possible configuration options, please take a look on this launchd site.
I briefly describe above file:
Labelis a mandatory key and should be unique within the launchdinstance.
Programcontains complete path to the executable.
StartCalendarIntervalallows you to define the specific time when the job run.
StandardErrorPathis a path to file for stderr.
StandardOutPathis a path to file dedicated for stdout.
Load the job into the launchd daemon
Load the plist definition into the launchd is very simple:
sudo launchctl load pl.skrajewski.backup.plist
And that’s it. The job is currently managed by launchd. If you have an error
Path had bad ownership/permissions don’t worry! Change the owner of this file to
root:wheel by running the below command
sudo chown root:wheel pl.skrajewski.backup.plist
and try to load the file again. If you want to change the definition of your job, remember about the
unload it first using
You can find both of this files on GitHub Gists.
Cryptomator is a good piece of software. Moreover, it’s open source. I like the idea of a transparent encryption because it significantly facilitates access to encrypted files. The client-side encryption also put a limit on ways how your data is used by cloud storage providers. This is a double-edged weapon – your data is secure, but nobody helps you if you lost the master key.
When it comes to the proposed mirroring solution – this is only my experiment. I treat this more like curio rather than a way to synchronize files. That’s why I mirror my files instead of storing them directly in cryptomator’s vault. When I wrote this article, Cryptomator CLI was in 0.3.1-Pre-release version. It’s not stable yet but I continue my tests.