Single-node RF=1 yugabyted or pinned three-node Docker Compose cluster for repeatable YugabyteDB development?
0 reputation · 09 May 2020, 16:09 UTC
0 reputation · 09 May 2020, 16:09 UTC
We are standardizing a local YugabyteDB development environment that every engineer can tear down and recreate identically. The goal is a scripted setup where schema migrations and seed data apply cleanly on every fresh start, and where the database version is pinned so upgrades never surprise us mid-sprint.
Two documented options are on the table. The first is a single-node yugabyted instance started with a dedicated --base_dir and replication factor 1, which boots fast and keeps state isolated in one directory we can wipe. The second is a three-node RF=3 cluster via the Docker Compose setup, with the release image tag pinned and per-node volumes either mounted or recreated per run. Our application connects over YSQL on port 5433 using a standard PostgreSQL driver.
The tension is parity versus cost: RF=1 never exercises replication, tablet placement, or node failover, so it may hide behavior our production RF=3 cluster exhibits, while RF=3 is slower to provision and heavier on laptops. We also need to decide whether mounted volumes help or hurt repeatability, since stale cluster metadata could survive between runs.
Which topology do teams typically pick for development when production runs RF=3, and what breaks in practice when testing only against RF=1? Should repeatability favor wiping data directories on every start, or preserving volumes? Does pinning the image tag fully protect against on-disk format changes across upgrades?
Use single-node yugabyted with RF=1 and a fresh temporary data directory for every developer machine. Reserve the three-node Docker Compose RF=3 cluster for CI pipelines and pre-merge integration tests. This split gives you sub-second startup for the inner loop while still exercising distributed behavior where it matters—before code merges.
Testing exclusively against RF=1 leaves these production code paths unexercised:
yb_read_from_followers=on or AS OF FOLLOWER clauses silently read from the leader.yb_servers() returns one row; any tooling that assumes multiple zones breaks.If your application uses any of the above, you must run the three-node cluster in CI.
Mounted volumes hurt repeatability. They preserve stale cluster UUIDs, master leadership leases, and tablet metadata that cause Cluster already initialized errors or schema-version mismatches after an image upgrade. The cleanest pattern:
# Developer machine (Makefile target or shell alias)
yugabyted destroy --base_dir=/tmp/yb_dev 2>/dev/null || true
yugabyted start --base_dir=$(mktemp -d) --replication_factor=1
# then run idempotent migration/seed scripts against localhost:5433
In CI, use the official three-node compose file with no persistent volumes:
docker compose -f yugabyte-db-cluster.yaml up --force-recreate --remove-orphans
# wait for health checks, then run the SAME migration/seed scripts
Add docker compose down -v in the teardown step to guarantee a clean slate.
Pinning yugabytedb/yugabyte:2024.1.0-b123 (or better, by digest sha256:…) fixes the binary, but YugabyteDB may change RocksDB SST layout, catalog schema, or tablet metadata between minor versions. A clean wipe is still required when the pinned tag/digest changes. Check the release notes for the specific version at docs.yugabyte.com to see if an on-disk format change is listed.
yugabyted status shows Replication Factor: 1 and Tablet Servers: 1.docker compose exec yb-master-1 yb-admin --master_addresses yb-master-1:7100,yb-master-2:7100,yb-master-3:7100 list_tablet_servers returns three entries.SELECT * FROM yb_servers(); on port 5433 — RF=1 returns one row, RF=3 returns three with cloud,region,zone populated.docker compose kill yb-tserver-2) and verify the app reconnects without error.Cluster already initialized errors appear.Does your application enable yb_read_from_followers or use AS OF FOLLOWER reads in production? If yes, you cannot rely on RF=1 even for daily development—run the three-node cluster locally (or accept that those code paths are CI-only).
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.