Backup and Recovery in YugabyteDB Using Distributed Snapshots: A Step-by-Step Guide
===================================================================================
A distributed snapshot is a consistent cut of the data taken across every node
in a YugabyteDB cluster at the same logical point in time. It is the fastest
backup method the database offers, and it behaves very differently from an
Oracle RMAN backup or a PostgreSQL pg_dump.
The reason it is fast: creating a snapshot does not copy any data. YugabyteDB
creates hard links to the existing SST files on the same storage volumes where
the data already lives. Both backup and restore are close to instantaneous
regardless of database size.
The trade-off follows directly from that. Because the snapshot lives on the
same disks as the data, it protects you from logical errors - a bad DELETE, a
wrong UPDATE, a failed deployment - but not from file system corruption or
hardware failure. For that you must move the snapshot off the cluster, which is
covered in section D.
Environment used in this guide: the three-node cluster built in the previous
post - ybnode1 (10.0.0.11), ybnode2 (10.0.0.12), ybnode3 (10.0.0.13), all
running YugabyteDB with masters on port 7100.
SCOPE
-----
- Creating and listing in-cluster snapshots
- Restoring a database from an in-cluster snapshot
- Understanding what a snapshot restore does NOT recover
- Moving a snapshot to external storage and restoring it on another cluster
- Point-in-time recovery using snapshot schedules
- YCQL equivalents and common errors
BACKUP METHOD COMPARISON
------------------------
Method Speed Protects against Off-cluster
--------------------- ---------- ------------------- -----------
In-cluster snapshot Instant Logical errors No
Snapshot + export Fast Logical + hardware Yes
PITR schedule Instant Logical errors, No
to any point in
the retention window
ysql_dump Slow Everything Yes
In practice you use more than one. A PITR schedule covers the "someone ran
DELETE without a WHERE clause ten minutes ago" case. An exported snapshot on
object storage covers the "we lost the data centre" case.
A. THE SNAPSHOT LIFECYCLE
create_database_snapshot -> snapshot in COMPLETE state
| |
| +--> restore_snapshot (in-place)
| |
| +--> export_snapshot ---+
| |
+--> delete_snapshot v
external storage
(S3 / NFS / tape)
|
v
import_snapshot on new cluster
|
v
restore_snapshot
B. CREATE AN IN-CLUSTER SNAPSHOT
--------------------------------
Step 1 - Create some test data
Connect with ysqlsh and build a database we can safely damage:
./bin/ysqlsh -h 10.0.0.11 -p 5433 -U yugabyte
yugabyte=# CREATE DATABASE snaptest;
yugabyte=# \c snaptest
snaptest=# CREATE TABLE employees (
emp_id int PRIMARY KEY,
emp_name text,
dept text);
snaptest=# INSERT INTO employees VALUES
(1,'Ashok','DBA'),
(2,'Ravi','Apps'),
(3,'Meena','Cloud');
snaptest=# SELECT count(*) FROM employees;
Step 2 - Create the snapshot
Snapshots for YSQL are taken at database level. Backing up an individual table
is not supported on the YSQL side - note the ysql. prefix on the database name,
which is what tells yb-admin this is a YSQL database and not a YCQL keyspace.
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
create_database_snapshot ysql.snaptest
Output:
Started snapshot creation: 0d4b4935-2c95-4523-95ab-9ead1e95e794
Record that UUID. It is how you check, restore, export or delete the snapshot.
Step 3 - Confirm the snapshot completed
The create command returns immediately, but the snapshot itself completes
asynchronously. Never treat the returned UUID as proof of a usable backup -
always verify the state:
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
list_snapshots
Output:
Snapshot UUID State Creation Time
0d4b4935-2c95-4523-95ab-9ead1e95e794 COMPLETE 2026-08-16 09:20:38.214201
Only a snapshot in COMPLETE state is restorable. If it is still CREATING, wait
and run list_snapshots again. This command also shows any restore operations
and their states, which is how you monitor a restore in progress.
C. RESTORE FROM AN IN-CLUSTER SNAPSHOT
--------------------------------------
Step 1 - Simulate the failure
snaptest=# DELETE FROM employees;
DELETE 3
snaptest=# SELECT count(*) FROM employees;
count
-------
0
Step 2 - Restore
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
restore_snapshot 0d4b4935-2c95-4523-95ab-9ead1e95e794
Output:
Started restoring snapshot: 0d4b4935-2c95-4523-95ab-9ead1e95e794
Restoration id: 5a9bc559-2155-4c38-ac8b-b6d0f7aa1af6
The restore is in-place. It rolls the existing database in the same cluster
back to its state at snapshot time. There is no separate target - anything
written after the snapshot is discarded.
Step 3 - Confirm the restore finished
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
list_snapshots
The restoration appears with its own state. Wait for RESTORED before you let
applications reconnect.
Step 4 - Verify the data
snaptest=# SELECT * FROM employees;
emp_id | emp_name | dept
--------+----------+-------
1 | Ashok | DBA
2 | Ravi | Apps
3 | Meena | Cloud
Step 5 - Delete the snapshot when no longer needed
Snapshots never expire. They are retained for the life of the cluster and each
one holds disk space that would otherwise be reclaimed by compaction, so an
unmanaged pile of snapshots slowly inflates your storage bill.
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
delete_snapshot 0d4b4935-2c95-4523-95ab-9ead1e95e794
IMPORTANT LIMITATION - SCHEMA CHANGES ARE NOT RESTORED
------------------------------------------------------
The in-cluster restore reverts data changes, not schema changes. If you take a
snapshot, DROP a table, and then restore the snapshot, the table does not come
back.
This surprises DBAs coming from RMAN, where a restore returns the whole
database to its earlier state. Plan around it in one of two ways:
- Export the snapshot to external storage (section D), which carries the schema
in a separate dump file, or
- Use point-in-time recovery (section E), which does handle schema rollback.
Either way: take a snapshot immediately after every schema change, and keep an
independent schema dump. A snapshot whose schema you cannot reproduce is only
half a backup.
D. MOVE A SNAPSHOT TO EXTERNAL STORAGE
--------------------------------------
This is the procedure that turns a snapshot into a real backup - one that
survives losing the cluster, and can be restored onto a different cluster.
Step 1 - Record the catalog version
snaptest=# SELECT yb_catalog_version();
yb_catalog_version
--------------------
13
Note this number. You will compare against it in step 4.
Step 2 - Create the in-cluster snapshot
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
create_database_snapshot ysql.snaptest
Wait for COMPLETE via list_snapshots.
Step 3 - Dump the YSQL schema
./postgres/bin/ysql_dump \
-h 10.0.0.11 \
--include-yb-metadata \
--serializable-deferrable \
--create \
--schema-only \
--dbname snaptest \
--file snaptest_schema.sql
Flag by flag:
--include-yb-metadata
Emits YugabyteDB-specific attributes such as tablet split points and
colocation settings. Without it the restored schema loses its distribution
properties and you get a functionally correct but differently sharded
database.
--serializable-deferrable
Takes the dump at a consistent snapshot without blocking writers.
--create
Includes the CREATE DATABASE statement, so the target cluster does not need
the database pre-created.
--schema-only
Structure only. The data comes from the snapshot files, not from this dump.
Step 4 - Re-check the catalog version
snaptest=# SELECT yb_catalog_version();
If this does not match what you recorded in step 1, a DDL statement ran during
the backup and the snapshot is not guaranteed to be consistently restorable.
Start the whole procedure again. Do not skip this check - it is the only thing
standing between you and a backup that fails on restore day.
Step 5 - Export the snapshot metadata
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
export_snapshot 0d4b4935-2c95-4523-95ab-9ead1e95e794 snaptest.snapshot
This writes a metadata file describing which tables and tablets belong to the
snapshot. It does not contain the data.
Step 6 - Copy the tablet snapshot data off the nodes
This is the manual part. The snapshot files live under each tablet server's
data directory, in this structure:
<fs_data_dir>/yb-data/tserver/data/rocksdb/
table-<table_id>/
tablet-<tablet_id>.snapshots/
<snapshot_id>/
Example:
cp -r /yugabyte01/YUGABYTE/data1/yb-data/tserver/data/rocksdb/ \
table-00004000000030008000000000004003/ \
tablet-b0de9bc6a4cb46d4aaacf4a03bcaf6be.snapshots/ \
0d4b4935-2c95-4523-95ab-9ead1e95e794/ \
/backup/snaptest/
Two things that save time here:
- You only need the leader tablet on each node. Every replica holds identical
data, so copying all three replicas triples your backup size for nothing.
- Get the table_id values from the master admin UI at
http://10.0.0.11:7000/tables - they are UUIDs, not table names.
If your cluster has several data directories in --fs_data_dirs, repeat for each
one.
Step 7 - Copy the metadata files
Move both snaptest_schema.sql and snaptest.snapshot to the same external
location as the tablet data. All three parts are required for a restore.
At this point you can safely delete the in-cluster snapshot to reclaim space.
E. RESTORE FROM EXTERNAL STORAGE ONTO ANOTHER CLUSTER
-----------------------------------------------------
Step 1 - Make sure the target database does not exist
yugabyte=# DROP DATABASE IF EXISTS snaptest;
Step 2 - Recreate the schema
./bin/ysqlsh -h 10.0.0.11 --echo-all --file=snaptest_schema.sql
This is the step that recovers schema changes - the piece the in-cluster
restore cannot do.
Step 3 - Import the snapshot metadata
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
import_snapshot snaptest.snapshot snaptest
Output includes the ID mapping, which you need for the next step:
Successfully applied snapshot.
Object Old ID New ID
Keyspace 000040000000300080000000000000 000040000000300080000000000000
Table 000040000000300080000000004003 000040000000300080000000004001
Tablet 0 b0de9bc6a4cb46d4aaacf4a03bcaf6be 50046f422aa6450ca82538e919581048
Snapshot 0d4b4935-2c95-4523-95ab-... 6beb9c0e-52ea-4f61-89bd-...
The new cluster assigned fresh table, tablet and snapshot IDs. Keep this
mapping in front of you.
Step 4 - Copy the tablet data into the new locations
Using the mapping above, place each old tablet's snapshot contents into the
directory named for the corresponding new tablet ID:
scp -r /backup/snaptest/table-00004000000030008000000000004003/ \
tablet-b0de9bc6a4cb46d4aaacf4a03bcaf6be.snapshots/ \
0d4b4935-2c95-4523-95ab-9ead1e95e794/* \
10.0.0.11:/yugabyte01/YUGABYTE/data1/yb-data/tserver/data/rocksdb/ \
table-00004000000030008000000000004001/ \
tablet-50046f422aa6450ca82538e919581048.snapshots/ \
6beb9c0e-52ea-4f61-89bd-c160ec02c729/
Copy the contents of the snapshot folder, not the folder itself. Repeat for
every tablet peer, and for any read replica cluster.
Step 5 - Restore
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
restore_snapshot 6beb9c0e-52ea-4f61-89bd-c160ec02c729
Use the NEW snapshot ID from the import output, not the original one.
F. POINT-IN-TIME RECOVERY WITH SNAPSHOT SCHEDULES
-------------------------------------------------
Manual snapshots only let you go back to the moments you happened to take one.
A snapshot schedule takes them automatically and lets you restore to any point
inside the retention window - including schema state.
Step 1 - Create a schedule
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
create_snapshot_schedule 60 1440 ysql.snaptest
The two numbers are minutes: take a snapshot every 60 minutes, retain for 1440
minutes (24 hours). Tighter intervals give a finer recovery granularity at the
cost of more retained snapshots and more disk.
The command returns a schedule ID - store it with your runbooks.
Step 2 - List schedules
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
list_snapshot_schedules
Step 3 - Restore to a point in time
Relative time - "put it back to five minutes ago":
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
restore_snapshot_schedule <schedule_id> minus 5m
Absolute time, using Unix microseconds, when you know exactly when the bad
statement ran.
Before restoring, confirm no other restore is already running against the same
database - concurrent restores on one keyspace produce unpredictable results.
Step 4 - Remove a schedule
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
delete_snapshot_schedule <schedule_id>
Choose your restore target as close to the incident as you can. Everything
written between your target time and now is lost - that gap is your real RPO,
not the schedule interval.
G. YCQL EQUIVALENTS
-------------------
The YCQL side supports table-level granularity, which YSQL does not.
Whole keyspace:
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
create_keyspace_snapshot my_keyspace
Single table with its indexes:
./bin/yb-admin \
--master_addresses 10.0.0.11:7100,10.0.0.12:7100,10.0.0.13:7100 \
create_snapshot my_keyspace my_table
list_snapshots, restore_snapshot, delete_snapshot, export_snapshot and
import_snapshot behave identically for both APIs.
TROUBLESHOOTING QUICK REFERENCE
-------------------------------
Symptom Most likely cause
-------------------------------- ---------------------------------------
Snapshot stuck in CREATING A tablet server is down or unreachable;
check list_all_tablet_servers
Restore completes, table missing Schema change after the snapshot -
in-cluster restore cannot recover DDL
Catalog version changed DDL ran during the backup; restart the
export procedure
import_snapshot fails on target Schema not applied first, or applied
without --include-yb-metadata
Restore appears to do nothing Used the old snapshot ID instead of the
new one from import_snapshot output
Disk usage keeps climbing Old snapshots never deleted; they pin
SST files against compaction
PRACTICAL RECOMMENDATIONS
-------------------------
- Run a PITR schedule on every production database. It is the cheapest
insurance against human error.
- Export a snapshot to external storage on a fixed cadence. In-cluster
snapshots share the fate of the disks they sit on.
- Take a manual snapshot immediately before and after any schema change or
application release.
- Keep an independent ysql_dump schema backup alongside every exported
snapshot.
- Script the delete_snapshot cleanup. Nothing expires on its own.
- Rehearse the external restore on a test cluster before you need it. The ID
mapping step in section E is where an untested procedure fails.
