Migration Tool

migrate-to-photovault

A one-shot CLI written in Go that migrates photos and videos from your existing bucket into the PhotoVault layout, generating 512px JPEG thumbnails on the fly.

Interactive setup wizard Resumable ETag dedup ffmpeg for video
Install

Run this one-liner in your terminal — it downloads the right binary for your OS and architecture and launches the setup wizard immediately:

Bash 
curl -fsSL https://photo-vault-release.s3.cubbit.eu/migrator/install.sh | bash

No root required. Credentials are never written to disk.

▸ Prefer to inspect the script before running it?

Download, read, then execute manually:

Bash 
curl -fsSL https://photo-vault-release.s3.cubbit.eu/migrator/install.sh -o install.sh
cat install.sh          # inspect
bash install.sh         # run when satisfied

install.sh content:

Shell 
#!/bin/sh
set -e

OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
case "$ARCH" in
  x86_64)          ARCH=amd64 ;;
  arm64|aarch64)   ARCH=arm64 ;;
esac

URL="https://photo-vault-release.s3.cubbit.eu/migrator/migrate-to-photovault_${OS}_${ARCH}"
DEST="./migrate-to-photovault"

echo "Downloading migrate-to-photovault for ${OS}/${ARCH}…"
curl -fsSL "$URL" -o "$DEST"
chmod +x "$DEST"
echo "Done — run ./migrate-to-photovault --help to get started"
How It Works
1

Scan destination

Before any upload starts, the tool lists all existing keys under {dst-prefix}media/ and loads their ETags. This lets it decide in a single pass whether a file is new, already uploaded, or a genuine collision.

2

List source objects

All objects in src-bucket under the optional src-prefix are listed. Keys already in the resume log are skipped instantly — no download.

3

Process each file (up to --concurrency in parallel)

  • Download to a temporary file
  • Extract DateTimeOriginal from EXIF (falls back to LastModified)
  • Allocate destination key {prefix}media/YYYY/MM/DD/HH-mm-ss-mmm[_N].ext using ETag comparison:
ETag match → already uploaded, skip (logged for future runs)
ETag mismatch → different file, use _2, _3…
→ Free key → reserve and proceed
  • Generate 512px JPEG thumbnail for all photo formats (including HEIC/RAW) (imaging for photos, ffmpeg for videos)
  • Upload thumbnail to {prefix}thumbnails/…
  • Server-side CopyObject if same endpoint, otherwise re-upload
  • Optionally delete source with --delete-source
4

Session & resume

The session config is saved at ~/.photovault-migrator/config.json (no credentials — structural params only). On the next run you are prompted:

Found a previous migration session:

├─ Source:      my-bucket (prefix: "")

├─ Destination: photovault-prod (prefix: "photo-vault/")

├─ Started:     2024-03-15 14:30

├─ Last run:    2024-03-15 15:12

└─ Processed:   500 files

Resume? [Y/n]

Press Enter to resume. Press n to start fresh (clears the resume log). Pass --no-resume to skip the prompt entirely.

Destination Layout

After migration your bucket will look like this (with the default photo-vault/ prefix):

Text 
photo-vault/
├── media/
│   └── 2024/
│       └── 03/
│           └── 15/
│               ├── 14-30-45-123.jpg        ← original photo
│               ├── 14-30-45-123.mp4        ← original video
│               └── 14-30-45-123_2.jpg      ← same-millisecond collision
└── thumbnails/
    └── 2024/
        └── 03/
            └── 15/
                ├── 14-30-45-123.jpg        ← photo thumbnail (512px JPEG)
                ├── 14-30-45-123_2.jpg      ← photo thumbnail same-millisecond collision (512px JPEG)
                └── 14-30-45-123_VIDEO.jpg  ← video thumbnail
All Flags
Flag Default Description
--src-bucket* — Source bucket name
--src-access-key* env SRC_ACCESS_KEY Source access key
--src-secret-key* env SRC_SECRET_KEY Source secret key
--dst-bucket* — Destination bucket name
--src-endpoint https://s3.cubbit.eu Source endpoint URL
--src-prefix "" Migrate only keys under this path
--dst-endpoint same as src Destination endpoint
--dst-prefix photo-vault/ PhotoVault prefix in destination
--dst-access-key same as src Destination access key
--dst-secret-key same as src Destination secret key
--region eu-west-1 Region
--concurrency 10 Parallel workers
--dry-run false Print planned actions without writing anything
--no-resume false Ignore any saved session and start fresh
--delete-source false Delete source object after successful migration
* required flag
Notes

All photo formats supported

HEIC, RAW, and all other photo formats are fully supported. Thumbnails are always generated as 512px JPEG. The original file is moved as-is without any conversion.

Video thumbnails require ffmpeg

ffmpeg must be in your PATH. If it is not found, the video is still migrated but without a thumbnail.

Credentials are never persisted

The session config saved at ~/.photovault-migrator/ contains only structural parameters (buckets, prefix, region). Your access keys are never written to disk.

⌃C

Safe interruption

Pressing Ctrl+C lets the current batch of in-flight files finish cleanly. The resume log is flushed before exit, so no work is lost.

Verbose logs

During the live UI, per-file details are written to migration.log in the current directory. Tail it in a second terminal for real-time output.

Ready to start?

Install the tool, run it once, and open the web gallery to browse your entire library.

Open the Web App Read the FAQ