Concurrent Writes

When multiple writers commit to the same table, iceberg-go uses optimistic concurrency control: every commit is validated against the table state it was built on. If another writer committed first, the commit is rejected rather than silently clobbering their snapshot.

Conflict detection

Each producer (Append, Overwrite, Delete, RowDelta, RewriteDataFiles) runs a set of validators before the commit lands. They check that the files the operation assumed (added, deleted, or filtered) still match the current table state. A failed validation surfaces as one of:

• table.ErrCommitFailed — the base snapshot moved on; the commit can be retried after refreshing (see below).
• table.ErrCommitDiverged — terminal. The base snapshot is no longer on the branch at all, so a retry cannot reconcile the change.
• table.ErrTransactionUnusable — this transaction can't be committed again. If the error also matches ErrCommitFailed, retry in a new transaction from a reloaded table.

import (
    "context"
    "errors"

    "github.com/apache/iceberg-go/table"
)

_, err := txn.Commit(context.Background())
switch {
case errors.Is(err, table.ErrTransactionUnusable) && errors.Is(err, table.ErrCommitFailed):
    // a concurrent change: retry in a new transaction from a reloaded table
case errors.Is(err, table.ErrCommitDiverged), errors.Is(err, table.ErrTransactionUnusable):
    // unrecoverable: rebuild the operation from the latest table
case errors.Is(err, table.ErrCommitFailed):
    // retriable: refresh and try again (the retry loop does this for you)
case errors.Is(err, table.ErrCommitRejected):
    // the catalog wrote nothing: fix the cause, then call Commit again
}

A commit can also fail with table.ErrCommitRejected, which means the catalog rejected it before writing anything, for example an update the catalog can't apply or a REST 400, 401, 403 or 404. The commit is not retried, and the transaction can still be committed.

Isolation levels

Delete and update operations are validated under an isolation level, set per operation through table properties:

serializable rejects the commit if any concurrent snapshot added data matching the operation's filter. snapshot is more permissive — it only rejects when concurrent deletes touch the same files.

Automatic retry

iceberg-go can refresh the table and replay the operation against the latest snapshot between attempts. On each retry the table metadata is reloaded, the producer's validators re-run against the fresh snapshot, and the commit is re-submitted only if it is still valid. RollbackToSnapshot, ExpireSnapshots and AssertRefSnapshotID pin the refs they guard, so a retry does not rebase those assertions. If a guarded ref has moved, the commit fails with ErrCommitFailed instead of retrying. An ExpireSnapshots that removes nothing adds no guards.

Retry is off by default (commit.retry.num-retries is 0). Opt in by setting the retry properties — see Manifest and commit in the configuration reference for the full list:

// Opt in to retries (e.g. 4 attempts after the first) for a contended table.
_, err := cat.CreateTable(ctx, ident, schema,
    catalog.WithProperties(iceberg.Properties{
        "commit.retry.num-retries": "4",
    }),
)

Catalog support: the REST, SQL, Glue, Hive, and Hadoop catalogs report commit conflicts, including requirements that fail because of a concurrent change, as ErrCommitFailed, so the retry loop engages on all of them. A commit on another branch that takes the staged snapshot's sequence number or first-row-id is a conflict too, and the retry renumbers the snapshot.

Only the implicit assertion on the commit branch's head is rebased after a refresh. Other requirements, including the schema, spec, sort order, and pinned refs, must still hold. Otherwise, the commit fails without another attempt and the transaction is unusable.

After a conflict, calling Commit again on the same transaction refreshes the table and replays the staged changes like the next retry attempt, so it can rebase onto a peer's commit even with retries disabled. Each call gets the configured retry budget. A commit carrying delete-file removals that cannot be replayed resubmits only if its branch has not moved. The rejections after a refresh that make the transaction unusable are listed on ErrTransactionUnusable.

A failed commit keeps the files it staged until a later commit lands. A transaction you abandon leaves them for orphan file removal, as one with retries disabled always has. An abandoned manifest rewrite also leaves the manifests that later attempts merged. TableCommit refuses a transaction whose commit failed, since the multi-table path does not refresh. Call Commit to resume it, or build a new transaction.

Because an ExpireSnapshots that removes something pins the commit branch, a transaction that also appends fails instead of rebasing onto a peer's append. This is deliberate, since the expiry was computed from the old head.