s3fs on Linux
Mount a bucket as a folder on Linux.
s3fs (s3fs-fuse) mounts a Zata bucket as a folder on Linux. You can then use it with normal commands such as ls, cp and mkdir. The examples use Indore (https://idr01.zata.ai); for a bucket in Mumbai, use https://bom01.zata.ai.
/etc/fstab).Before you start
- A bucket — see Create a bucket.
- An access key from the same region as the bucket — see Access keys.
- A Linux machine where you can use
sudo.
Connection settings
| Setting | Value |
|---|---|
Endpoint (url=) | https://idr01.zata.ai for Indore, https://bom01.zata.ai for Mumbai |
| Path-style | use_path_request_style |
| Keys | in the file /etc/passwd-s3fs, as YOUR_ACCESS_KEY:YOUR_SECRET_KEY — the key must be from the same region as the bucket |
Set up s3fs
- Ubuntu / Debian
- RHEL / CentOS
sudo apt update
sudo apt install -y s3fs
s3fs --version
sudo yum install -y s3fs-fuse
s3fs --version
Open the keys file:
sudo nano /etc/passwd-s3fs
Add your keys on one line in this format, then save with Ctrl+O, Enter, Ctrl+X:
YOUR_ACCESS_KEY:YOUR_SECRET_KEY
Make the file readable only by root. s3fs refuses a keys file that others can read.
sudo chmod 600 /etc/passwd-s3fs
Replace my-bucket with your bucket name:
sudo mkdir -p /mnt/zata
sudo s3fs my-bucket /mnt/zata -o passwd_file=/etc/passwd-s3fs -o url=https://idr01.zata.ai -o use_path_request_style
To let normal (non-root) users open the files, add -o allow_other -o uid=$(id -u),gid=$(id -g) to the command.
Test it
mountpoint /mnt/zata
echo "Hello from s3fs and Zata" | sudo tee /mnt/zata/test.txt
ls -l /mnt/zata/
mountpoint prints /mnt/zata is a mountpoint. The file test.txt also appears in your bucket in the Zata console under Buckets.
Mount at boot
Add this line to /etc/fstab (sudo nano /etc/fstab), with your bucket name and endpoint:
my-bucket /mnt/zata fuse.s3fs _netdev,passwd_file=/etc/passwd-s3fs,url=https://idr01.zata.ai,use_path_request_style 0 0
Test the line without rebooting. Unmount first if the bucket is still mounted:
sudo umount /mnt/zata
sudo mount /mnt/zata
mountpoint /mnt/zata
Unmount
sudo umount /mnt/zata
If it says target is busy, close any terminal or app that is using the folder, then try again.
Limits
s3fs makes a bucket look like a folder, but it is not a full disk:
- Editing a file in place re-uploads the whole file, so changes to large files are slow.
- Renaming or moving a large folder is slow: each file is copied and then deleted.
- With a sub-user key, mount the bucket by its name as above. The sub-user needs a permission that can list the bucket — Upload Only cannot mount it, because s3fs lists the bucket when it starts.
For many files or large transfers, use rclone. For fast reads and writes of whole files, see goofys.
Troubleshooting
| Problem | Fix |
|---|---|
403 Access Denied | Check the keys in /etc/passwd-s3fs and that the key can use this bucket. The key must be from the bucket's region. |
| The mount fails with no clear error | Run it in the foreground to see why: add -f -o dbglevel=info to the mount command. |
Permission denied when a normal user opens files | Add -o allow_other and -o uid=$(id -u),gid=$(id -g) to the mount command. |
RequestTimeTooSkewed or signature errors | Your computer's clock is more than about 15 minutes off. Turn on time sync. See Clock and credentials. |
For other errors, see Access denied and Endpoint and region.