Types & Handles

TypeDescription
KVStoreOpaque handle to an open database.
KVColumnFamilyOpaque handle to a column family.
KVIteratorOpaque handle to a key-value iterator.
KVStoreStatsStatistics counters (see Diagnostics).
KVStoreConfigConfiguration struct for kvstore_open_v2.

Error Codes

CodeValueDescription
KVSTORE_OK0Operation succeeded.
KVSTORE_ERROR1Generic error.
KVSTORE_BUSY5Database locked by another connection.
KVSTORE_LOCKED6Database locked within the same connection.
KVSTORE_NOMEM7Memory allocation failed.
KVSTORE_READONLY8Database is read-only.
KVSTORE_CORRUPT11Database file is corrupted.
KVSTORE_NOTFOUND12Key or column family not found.
KVSTORE_PROTOCOL15Database lock protocol error.

All codes are aliases for the corresponding SQLITE_* values.

Constants

Journal Modes

ConstantValueDescription
KVSTORE_JOURNAL_DELETE0Rollback journal mode.
KVSTORE_JOURNAL_WAL1Write-Ahead Logging — recommended.

Sync Levels

ConstantValueDescription
KVSTORE_SYNC_OFF0No fsync — fastest, not crash-safe.
KVSTORE_SYNC_NORMAL1WAL-safe — survives process crash (default).
KVSTORE_SYNC_FULL2Power-safe — fsync on every commit.

Checkpoint Modes

ConstantValueDescription
KVSTORE_CHECKPOINT_PASSIVE0Non-blocking copy; may not flush all frames.
KVSTORE_CHECKPOINT_FULL1Wait for writers, then copy all frames.
KVSTORE_CHECKPOINT_RESTART2Like FULL, then reset WAL write position.
KVSTORE_CHECKPOINT_TRUNCATE3Like RESTART, then truncate WAL file to zero.

Other

ConstantValueDescription
KVSTORE_MAX_COLUMN_FAMILIES64Maximum column families per database.
KVSTORE_NO_TTL-1Sentinel: key exists but has no expiry.

Configuration — KVStoreConfig

typedef struct KVStoreConfig KVStoreConfig;
struct KVStoreConfig {
  int journalMode;   /* KVSTORE_JOURNAL_WAL (default) or KVSTORE_JOURNAL_DELETE */
  int syncLevel;     /* KVSTORE_SYNC_NORMAL (default), _OFF, or _FULL */
  int cacheSize;     /* page cache in pages (0 = default = 2000 pages ≈ 8 MB) */
  int pageSize;      /* page size in bytes (0 = 4096; new databases only) */
  int readOnly;      /* 1 = open read-only; default 0 */
  int busyTimeout;   /* ms to retry on SQLITE_BUSY (0 = fail immediately) */
  int walSizeLimit;  /* auto-checkpoint every N commits in WAL mode (0 = off) */
};
FieldDefaultNotes
journalModeKVSTORE_JOURNAL_WALWAL strongly recommended.
syncLevelKVSTORE_SYNC_NORMALSafe against process crash.
cacheSize2000 (~8 MB)Larger cache improves read-heavy loads.
pageSize4096 bytesIgnored for existing databases.
readOnly0
busyTimeout0 (fail immediately)Set > 0 for multi-process workloads.
walSizeLimit0 (disabled)Auto-checkpoint every N committed writes.

Database Lifecycle

kvstore_open_v2

int kvstore_open_v2( const char *zFilename, /* path to file; NULL = in-memory */ KVStore **ppKV, /* out: handle */ const KVStoreConfig *pConfig /* NULL = all defaults */ );

Open or create a database with full configuration control. Pass NULL for pConfig to use all defaults (WAL mode, NORMAL sync, 8 MB cache).

/* default open */
KVStore *kv;
kvstore_open_v2("mydb.db", &kv, NULL);

/* fully configured */
KVStoreConfig cfg = {0};
cfg.journalMode = KVSTORE_JOURNAL_WAL;
cfg.syncLevel   = KVSTORE_SYNC_FULL;
cfg.cacheSize   = 4000;
cfg.busyTimeout = 5000;
kvstore_open_v2("mydb.db", &kv, &cfg);

/* read-only */
KVStoreConfig ro = {0};
ro.readOnly = 1;
kvstore_open_v2("mydb.db", &kv, &ro);
Returns

KVSTORE_OK on success, error code otherwise.

kvstore_open

int kvstore_open( const char *zFilename, KVStore **ppKV, int journalMode /* KVSTORE_JOURNAL_WAL or _DELETE */ );

Simplified open. Equivalent to kvstore_open_v2 with only journalMode set; all other fields use defaults.

KVStore *kv = NULL;
kvstore_open("mydata.db", &kv, KVSTORE_JOURNAL_WAL);

kvstore_close

int kvstore_close(KVStore *pKV);

Close the database and free all resources. Any uncommitted transaction is rolled back. Runs a WAL checkpoint before closing.

Key-Value Operations — Default Column Family

kvstore_put

int kvstore_put( KVStore *pKV, const void *pKey, int nKey, const void *pValue, int nValue );

Insert or update a key-value pair. If the key already exists, its value is replaced. Keys and values are binary-safe. Without an explicit transaction, each call auto-commits.

kvstore_get

int kvstore_get( KVStore *pKV, const void *pKey, int nKey, void **ppValue, int *pnValue );

Retrieve a value by key. The returned buffer is heap-allocated by SNKV. Caller must free with sqliteFree() / snkv_free().

void *value = NULL; int vlen = 0;
int rc = kvstore_get(kv, "user:1", 6, &value, &vlen);
if (rc == KVSTORE_OK) {
    printf("%.*s\n", vlen, (char*)value);
    sqliteFree(value);
}
Returns

KVSTORE_OK if found · KVSTORE_NOTFOUND if key does not exist.

kvstore_delete

int kvstore_delete(KVStore *pKV, const void *pKey, int nKey);

Delete a key. Returns KVSTORE_NOTFOUND if the key did not exist.

kvstore_exists

int kvstore_exists(KVStore *pKV, const void *pKey, int nKey, int *pExists);

Check if a key exists without reading its value. More efficient than kvstore_get for existence checks. Sets *pExists to 1 if found, 0 otherwise.

Column Family Management

Column families provide multiple logical key-value namespaces within a single file. Each is stored in its own B-tree. Maximum KVSTORE_MAX_COLUMN_FAMILIES (64) per database.

kvstore_cf_create

int kvstore_cf_create(KVStore *pKV, const char *zName, KVColumnFamily **ppCF);

Create a new column family. zName max 255 characters, must be unique. *ppCF receives the handle.

kvstore_cf_open

int kvstore_cf_open(KVStore *pKV, const char *zName, KVColumnFamily **ppCF);

Open an existing column family. Returns KVSTORE_NOTFOUND if it does not exist.

kvstore_cf_get_default

int kvstore_cf_get_default(KVStore *pKV, KVColumnFamily **ppCF);

Get a handle to the default column family. Always exists; created automatically on open.

kvstore_cf_drop

int kvstore_cf_drop(KVStore *pKV, const char *zName);

Delete a column family and all its data permanently. The default column family cannot be dropped. Also removes any hidden TTL index CFs.

kvstore_cf_list

int kvstore_cf_list(KVStore *pKV, char ***pazNames, int *pnCount);

List all column families. Caller must free each name and the array with sqliteFree().

char **names = NULL; int count = 0;
kvstore_cf_list(kv, &names, &count);
for (int i = 0; i < count; i++) {
    printf("%s\n", names[i]);
    sqliteFree(names[i]);
}
sqliteFree(names);

kvstore_cf_close

void kvstore_cf_close(KVColumnFamily *pCF);

Release the column family handle. Does not delete data.

CF Key-Value Operations

Identical to the default-CF variants but take a KVColumnFamily* handle.

int kvstore_cf_put(KVColumnFamily *pCF, const void *pKey, int nKey, const void *pValue, int nValue); int kvstore_cf_get(KVColumnFamily *pCF, const void *pKey, int nKey, void **ppValue, int *pnValue); int kvstore_cf_delete(KVColumnFamily *pCF, const void *pKey, int nKey); int kvstore_cf_exists(KVColumnFamily *pCF, const void *pKey, int nKey, int *pExists);
Note: Caller must free the value from kvstore_cf_get with sqliteFree().

Iterators

Iterate all key-value pairs in ascending lexicographic key order.

kvstore_iterator_create / kvstore_cf_iterator_create

int kvstore_iterator_create(KVStore *pKV, KVIterator **ppIter); int kvstore_cf_iterator_create(KVColumnFamily *pCF, KVIterator **ppIter);

Create a forward iterator for the default or a specific column family.

kvstore_iterator_first / next / eof

int kvstore_iterator_first(KVIterator *pIter); int kvstore_iterator_next(KVIterator *pIter); int kvstore_iterator_eof(KVIterator *pIter); /* returns 1 at end */

Position at first entry, advance to next, or test for end-of-sequence.

kvstore_iterator_key / value

int kvstore_iterator_key(KVIterator *pIter, void **ppKey, int *pnKey); int kvstore_iterator_value(KVIterator *pIter, void **ppValue, int *pnValue);

Read the current key or value. Pointers are owned by the iterator — do not free them. Valid until the next next() or close() call.

KVIterator *it = NULL;
kvstore_iterator_create(kv, &it);
kvstore_iterator_first(it);
while (!kvstore_iterator_eof(it)) {
    void *key, *val; int klen, vlen;
    kvstore_iterator_key(it, &key, &klen);
    kvstore_iterator_value(it, &val, &vlen);
    printf("%.*s\n", klen, (char*)key);
    kvstore_iterator_next(it);
}
kvstore_iterator_close(it);

kvstore_iterator_close

void kvstore_iterator_close(KVIterator *pIter);

Release the iterator and its cursor. Always call this when done.

Prefix Iterators

Pre-positioned at the first key matching the prefix. Stop automatically when keys no longer match. Do not call kvstore_iterator_first() — read directly, then call next().

int kvstore_prefix_iterator_create( KVStore *pKV, const void *pPrefix, int nPrefix, KVIterator **ppIter ); int kvstore_cf_prefix_iterator_create( KVColumnFamily *pCF, const void *pPrefix, int nPrefix, KVIterator **ppIter );
KVIterator *it = NULL;
kvstore_prefix_iterator_create(kv, "user:", 5, &it);
/* already at first match — do NOT call first() */
while (!kvstore_iterator_eof(it)) {
    void *key; int klen;
    kvstore_iterator_key(it, &key, &klen);
    kvstore_iterator_next(it);
}
kvstore_iterator_close(it);

Reverse Iterators

Traverse keys in descending order. Use kvstore_iterator_last() and kvstore_iterator_prev() instead of first() and next(). All key/value/eof/close accessors work identically.

int kvstore_reverse_iterator_create(KVStore *pKV, KVIterator **ppIter); int kvstore_cf_reverse_iterator_create(KVColumnFamily *pCF, KVIterator **ppIter); /* Reverse prefix — pre-positioned at last matching key */ int kvstore_reverse_prefix_iterator_create( KVStore *pKV, const void *pPrefix, int nPrefix, KVIterator **ppIter ); int kvstore_cf_reverse_prefix_iterator_create( KVColumnFamily *pCF, const void *pPrefix, int nPrefix, KVIterator **ppIter ); int kvstore_iterator_last(KVIterator *pIter); /* seek to last key */ int kvstore_iterator_prev(KVIterator *pIter); /* move to previous key */
/* Full reverse scan */
KVIterator *it = NULL;
kvstore_reverse_iterator_create(kv, &it);
kvstore_iterator_last(it);
while (!kvstore_iterator_eof(it)) {
    void *key; int klen;
    kvstore_iterator_key(it, &key, &klen);
    kvstore_iterator_prev(it);
}
kvstore_iterator_close(it);

/* Reverse prefix — do NOT call last(), already positioned */
kvstore_reverse_prefix_iterator_create(kv, "user:", 5, &it);
while (!kvstore_iterator_eof(it)) {
    kvstore_iterator_prev(it);
}
kvstore_iterator_close(it);

Iterator Seek

Jump an open iterator to an arbitrary key in O(log N) time. Works on forward and reverse iterators, including prefix iterators.

kvstore_iterator_seek

int kvstore_iterator_seek(KVIterator *pIter, const void *pKey, int nKey);

Reposition the iterator without closing and reopening it.

  • Forward iterator — seek to the first key >= (pKey, nKey).
  • Reverse iterator — seek to the last key <= (pKey, nKey).

If the seek lands outside a prefix iterator’s prefix, the iterator immediately enters the eof state. Call kvstore_iterator_key / kvstore_iterator_next as usual after a successful seek.

ReturnMeaning
KVSTORE_OKSeek succeeded (check eof to know if a key was found).
KVSTORE_ERRORInvalid arguments.
/* forward: scan from "m" onwards */
KVIterator *it = NULL;
kvstore_iterator_create(kv, &it);
kvstore_iterator_seek(it, "m", 1);
while (!kvstore_iterator_eof(it)) {
    void *key; int klen;
    kvstore_iterator_key(it, &key, &klen);
    kvstore_iterator_next(it);
}
kvstore_iterator_close(it);

/* reverse: scan from "m" backwards */
kvstore_reverse_iterator_create(kv, &it);
kvstore_iterator_seek(it, "m", 1);
while (!kvstore_iterator_eof(it)) {
    kvstore_iterator_prev(it);
}
kvstore_iterator_close(it);

Transactions

int kvstore_begin(KVStore *pKV, int wrflag); /* 1=write, 0=read */ int kvstore_commit(KVStore *pKV); int kvstore_rollback(KVStore *pKV);

Explicit ACID transactions for batching operations atomically. Without an explicit transaction, each put/delete auto-commits. Wrapping bulk writes in a single transaction is significantly faster.

kvstore_begin(kv, 1);
kvstore_put(kv, "key1", 4, "val1", 4);
kvstore_put(kv, "key2", 4, "val2", 4);
kvstore_commit(kv);   /* both writes are atomic */
KVSTORE_BUSY: Set cfg.busyTimeout > 0 to auto-retry on lock contention, or handle the return code and retry manually.

TTL / Key Expiry

Per-key expiry with zero overhead for stores that never use TTL. Hidden index CFs are created lazily on first use and are invisible in kvstore_cf_list.

kvstore_put_ttl / kvstore_get_ttl

int64_t kvstore_now_ms(void); /* current time in ms since Unix epoch */ int kvstore_put_ttl( KVStore *pKV, const void *pKey, int nKey, const void *pValue, int nValue, int64_t expire_ms /* absolute ms since epoch; 0 = no TTL */ ); int kvstore_get_ttl( KVStore *pKV, const void *pKey, int nKey, void **ppValue, int *pnValue, int64_t *pnRemaining /* may be NULL */ );
/* write key that expires in 30 minutes */
int64_t exp = kvstore_now_ms() + 30 * 60 * 1000;
kvstore_put_ttl(kv, "sess:abc", 8, token, tlen, exp);

/* read with lazy expiry */
void *val = NULL; int nVal = 0; int64_t rem = 0;
int rc = kvstore_get_ttl(kv, "sess:abc", 8, &val, &nVal, &rem);
if (rc == KVSTORE_OK) { /* rem = ms remaining */ sqliteFree(val); }

kvstore_ttl_remaining

int kvstore_ttl_remaining(KVStore *pKV, const void *pKey, int nKey, int64_t *pnRemaining);
Return*pnRemainingMeaning
KVSTORE_OKKVSTORE_NO_TTL (-1)Key exists, no expiry.
KVSTORE_OK0Key just expired (lazy delete performed).
KVSTORE_OKN > 0N ms remain.
KVSTORE_NOTFOUNDKey does not exist.

kvstore_purge_expired

int kvstore_purge_expired(KVStore *pKV, int *pnDeleted); /* pnDeleted may be NULL */

Scan the expiry index and delete all expired keys in one write transaction. O(expired keys) — uses sorted expire-time prefix to stop at first non-expired entry.

CF-level TTL: All four functions have CF equivalents operating on a KVColumnFamily* handle. Each user CF maintains independent TTL indexes; purging CF A never touches CF B.

kvstore_cf_put_ttl / kvstore_cf_get_ttl

int kvstore_cf_put_ttl( KVColumnFamily *pCF, const void *pKey, int nKey, const void *pValue, int nValue, int64_t expire_ms /* absolute ms since epoch; 0 = no TTL */ ); int kvstore_cf_get_ttl( KVColumnFamily *pCF, const void *pKey, int nKey, void **ppValue, int *pnValue, int64_t *pnRemaining /* may be NULL */ );

CF-level equivalents of kvstore_put_ttl / kvstore_get_ttl. Semantics are identical; the TTL indexes are created lazily per CF and are fully independent.

kvstore_cf_ttl_remaining

int kvstore_cf_ttl_remaining( KVColumnFamily *pCF, const void *pKey, int nKey, int64_t *pnRemaining /* must not be NULL */ );

Return remaining TTL for a key in this column family without fetching the value. Return codes and *pnRemaining semantics are identical to kvstore_ttl_remaining.

kvstore_cf_purge_expired

int kvstore_cf_purge_expired(KVColumnFamily *pCF, int *pnDeleted); /* pnDeleted may be NULL */

Scan and delete all expired keys in this column family only, in a single write transaction. O(expired keys). Purging CF A never affects other column families.

Conditional Insert (Put-If-Absent)

Atomically insert a key-value pair only when the key does not already exist. Expired keys (lazy TTL) are treated as absent. Supports an optional TTL on the new entry.

kvstore_put_if_absent / kvstore_cf_put_if_absent

int kvstore_put_if_absent( KVStore *pKV, const void *pKey, int nKey, const void *pValue, int nValue, int64_t expire_ms, /* 0 = no TTL */ int *pInserted /* 1=inserted, 0=already existed; may be NULL */ ); int kvstore_cf_put_if_absent(KVColumnFamily *pCF, /* ... same params ... */);
expire_msBehaviour
0No TTL — new entry is permanent.
> 0Absolute expiry in ms since Unix epoch (kvstore_now_ms() + delta_ms).

The existence check and the write are performed inside a single write transaction, so no other writer can insert the same key concurrently.

int inserted = 0;
kvstore_put_if_absent(kv, "lock", 4, "owner", 5, 0, &inserted);
if (inserted) {
    puts("acquired lock");
} else {
    puts("lock already held");
}

/* with TTL: session token that expires in 1 hour */
int64_t exp = kvstore_now_ms() + 3600 * 1000;
kvstore_put_if_absent(kv, session_id, sid_len, token, tok_len, exp, NULL);

Bulk Clear

Remove every key-value pair from a column family in a single atomic write transaction. TTL index entries are also cleared. Much faster than iterating and deleting individually.

kvstore_clear / kvstore_cf_clear

int kvstore_clear(KVStore *pKV); int kvstore_cf_clear(KVColumnFamily *pCF);

Truncates the column family’s B-tree in O(pages) time using SQLite’s BtreeClearTable — the B-tree structure (root page) is preserved so future inserts work immediately without re-opening.

Precondition: No open iterators pointing at the column family may exist when clear is called. BtreeClearTable invalidates all cursors on the cleared table; accessing a stale cursor is undefined behaviour.
/* flush a rate-limit namespace at midnight */
kvstore_cf_clear(rate_limit_cf);

/* wipe everything in the default CF */
kvstore_clear(kv);

Key Count

Count keys in a column family without iterating over them. Uses SQLite’s sqlite3BtreeCount which reads only page headers — O(pages) ≈ O(N/100).

kvstore_count / kvstore_cf_count

int kvstore_count(KVStore *pKV, int64_t *pnCount); int kvstore_cf_count(KVColumnFamily *pCF, int64_t *pnCount);

Counts only the main data B-tree of the column family. TTL index CFs are separate B-trees and are not included. Expired-but-not-yet-purged keys are included in the count — call kvstore_purge_expired first for an exact live count.

Uncommitted writes: If a write transaction is active, the count reflects committed data only. Commit first for an accurate count that includes pending writes.
int64_t n;
kvstore_count(kv, &n);
printf("default CF has %lld entries\n", (long long)n);

kvstore_purge_expired(kv, NULL);
kvstore_count(kv, &n);
printf("live entries after purge: %lld\n", (long long)n);

Diagnostics & Maintenance

kvstore_errmsg

const char *kvstore_errmsg(KVStore *pKV);

Return the last error message string. Do not free. Valid until the next operation on the same handle.

kvstore_stats / kvstore_stats_reset

int kvstore_stats(KVStore *pKV, KVStoreStats *pStats); int kvstore_stats_reset(KVStore *pKV); /* zero all cumulative counters */

kvstore_stats fills the KVStoreStats struct. All counters are cumulative from open (or last reset). kvstore_stats_reset zeros them; nDbPages is a live gauge and is never zeroed.

FieldTypeDescription
nPutsuint64_tTotal put operations (includes put_ttl, put_if_absent).
nGetsuint64_tTotal get operations (includes get_ttl).
nDeletesuint64_tTotal delete operations.
nIterationsuint64_tTotal iterators created.
nErrorsuint64_tTotal errors encountered.
nBytesReaduint64_tTotal value bytes returned by get operations.
nBytesWrittenuint64_tTotal (key + value) bytes written by put operations.
nWalCommitsuint64_tWrite transactions successfully committed.
nCheckpointsuint64_tWAL checkpoints performed (auto + explicit).
nTtlExpireduint64_tKeys lazily expired on get / exists calls.
nTtlPurgeduint64_tKeys removed by kvstore_purge_expired.
nDbPagesuint64_tCurrent total pages in the database file (gauge, not zeroed by reset).

kvstore_integrity_check

int kvstore_integrity_check(KVStore *pKV, char **pzErrMsg);

Walk the entire B-tree and verify structural integrity. Returns KVSTORE_OK or KVSTORE_CORRUPT with details in *pzErrMsg. Caller must free *pzErrMsg with sqliteFree().

kvstore_sync

int kvstore_sync(KVStore *pKV);

Force all pending changes to disk. If a write transaction is active, performs a commit-and-reopen cycle to flush the WAL.

kvstore_incremental_vacuum

int kvstore_incremental_vacuum(KVStore *pKV, int nPage);

Reclaim unused pages and shrink the database file. Pass 0 to free all unused pages, or a positive nPage to limit work per call (good for latency-sensitive apps).

kvstore_incremental_vacuum(kv, 0);    /* free all */
kvstore_incremental_vacuum(kv, 50);   /* incremental: 50 pages */

kvstore_checkpoint

int kvstore_checkpoint(KVStore *pKV, int mode, int *pnLog, int *pnCkpt);

Copy WAL frames back into the main database file. pnLog and pnCkpt may be NULL. Requires no active write transaction. Non-WAL databases: no-op returning KVSTORE_OK.

int nLog = 0, nCkpt = 0;
kvstore_checkpoint(kv, KVSTORE_CHECKPOINT_PASSIVE, &nLog, &nCkpt);
printf("%d frames total, %d checkpointed\n", nLog, nCkpt);

/* reclaim WAL disk space */
kvstore_checkpoint(kv, KVSTORE_CHECKPOINT_TRUNCATE, NULL, NULL);

Memory Management

MacroDescription
sqliteMalloc(n)Allocate n zero-initialised bytes.
sqliteFree(p)Free memory allocated by SNKV.
sqliteRealloc(p, n)Resize allocation.
sqliteStrDup(s)Duplicate a C string (equivalent to sqlite3_mprintf("%s", s)).
snkv_malloc(n)Alias for sqliteMalloc (single-header build).
snkv_free(p)Alias for sqliteFree (single-header build).
Rule: Any pointer returned by kvstore_get, kvstore_cf_get, kvstore_cf_list, or kvstore_integrity_check must be freed with sqliteFree(). Iterator key/value pointers are iterator-owned — do NOT free them.

Thread Safety

  • All kvstore_* calls are mutex-protected — a single KVStore handle is safe across threads.
  • In WAL mode, multiple readers run concurrently with a single writer.
  • Only one write transaction can be active at a time; concurrent writes return KVSTORE_BUSY.
  • For maximum throughput, each thread can open its own KVStore to the same file.
  • KVIterator handles are not thread-safe — do not share across threads.

Installation

# Install from PyPI
pip install snkv

# Or build from source
cd python
python3 setup.py build_ext --inplace

Quick Start

from snkv import KVStore

with KVStore("mydb.db") as db:
    db["hello"] = "world"
    print(db["hello"].decode())      # world
    print(db.get("missing"))         # None

    for key, value in db:
        print(key, value)              # b'hello' b'world'

Constants

Journal Mode

ConstantDescription
JOURNAL_WALWrite-Ahead Logging — concurrent readers (default).
JOURNAL_DELETERollback journal mode.

Sync Level

ConstantDescription
SYNC_OFFNo fsync — fastest, not crash-safe.
SYNC_NORMALSurvives process crash (default).
SYNC_FULLfsync on every commit — strongest durability.

Checkpoint Mode

ConstantDescription
CHECKPOINT_PASSIVENon-blocking copy (default).
CHECKPOINT_FULLWait for all readers, then copy all frames.
CHECKPOINT_RESTARTLike FULL, then reset WAL write position.
CHECKPOINT_TRUNCATELike RESTART, then truncate WAL to zero bytes.

TTL

ConstantValueDescription
NO_TTL-1Returned by ttl() when key has no expiry.

Exceptions

snkv.Error           ← base class for all SNKV errors
├── snkv.NotFoundError  (also subclass of KeyError)
├── snkv.BusyError
├── snkv.LockedError
├── snkv.ReadOnlyError
└── snkv.CorruptError
ExceptionRaised when
ErrorGeneric SNKV error (base class).
NotFoundErrorKey does not exist; also a KeyError.
BusyErrorDatabase locked and busy_timeout expired.
LockedErrorWrite lock conflict with a concurrent transaction.
ReadOnlyErrorWrite attempted on a read-only store.
CorruptErrorIntegrity check detected corruption.

KVStore

The main entry point. Opens or creates a key-value store at a given path. Always use as a context manager to ensure proper cleanup.

Opening a Store — KVStore(path, *, **config)

KVStore( path = None, # str path or None for in-memory journal_mode = JOURNAL_WAL, sync_level = SYNC_NORMAL, cache_size = 2000, # pages (~8 MB default) page_size = 4096, # bytes; new databases only busy_timeout = 0, # ms; 0 = fail immediately wal_size_limit= 0, # auto-checkpoint every N commits; 0 = off read_only = 0 # 1 = read-only )
# Minimal
db = KVStore("mydb.db")

# In-memory
db = KVStore()    # or KVStore(None)

# Fully configured
db = KVStore(
    "mydb.db",
    journal_mode=JOURNAL_WAL,
    sync_level=SYNC_NORMAL,
    cache_size=4000,
    busy_timeout=5000,
    wal_size_limit=200,
)

# Context manager (recommended)
with KVStore("mydb.db") as db:
    db["key"] = "value"

Core Operations

put(key, value, ttl=None) → None

Insert or overwrite a key-value pair. ttl is seconds until expiry (int or float); None means no expiry. Both key and value accept str, bytes, bytearray, or memoryview.

db.put("user:1", b"\x01\x02\x03")
db.put("session", "tok", ttl=3600)  # expires in 1 hour
db.put("cache",   data,   ttl=0.5)  # half a second

get(key, default=None) → bytes | None

Return the stored value as bytes, or default if the key does not exist or has expired. Never raises NotFoundError — use db[key] for that.

val = db.get("user:1")         # bytes or None
val = db.get("missing", b"") # b'' if not found

delete(key) → None

Delete a key. Raises NotFoundError (KeyError) if the key does not exist.

exists(key) → bool

Return True if the key exists, without fetching the value.

TTL / Key Expiry

ttl(key) → float | None

Return valueMeaning
Positive floatSeconds remaining.
0.0Key just expired (lazy delete performed).
NoneKey exists but has no expiry.
raises NotFoundErrorKey does not exist.
db.put("session", "tok", ttl=3600)
print(db.ttl("session"))   # e.g. 3599.97
print(db.ttl("perm"))      # None (no TTL)

purge_expired() → int

Scan the TTL index and delete all expired keys in a single transaction. Returns the count of keys deleted. O(expired keys).

n = db.purge_expired()
print(f"Removed {n} expired keys")

Conditional Insert / Bulk Clear / Key Count

put_if_absent(key, value, ttl=None) → bool

Atomically insert key only when it is absent (or has expired). Returns True if inserted, False if already present. Safe for distributed locks and deduplication.

# Distributed lock with auto-release
inserted = db.put_if_absent(b"lock:job-1", b"worker-A", ttl=30)
if inserted:
    run_job()   # only one worker reaches here

# Deduplication
if db.put_if_absent(b"msg:001", b"payload"):
    process()   # first write wins

clear() → None

Remove every key-value pair from the default column family in a single atomic write transaction. TTL index entries are cleared atomically. Runs in O(pages) — much faster than iterating and deleting individually. Close all iterators before calling.

db.clear()   # truncate default CF; all keys gone
db.put(b"fresh", b"start")   # normal inserts work immediately after

count() → int

Return the number of entries in the default column family. Reads page-level nCell headers via sqlite3BtreeCount — O(pages), not O(rows). Expired-but-not-yet-purged keys are included; call purge_expired() first for an exact live count.

n = db.count()   # e.g. 10000
db.purge_expired()
n = db.count()   # exact live count

Dict-like Interface

SyntaxEquivalentNotes
db["key"]getRaises NotFoundError on miss/expiry.
db["key"] = "val"putNo TTL.
db["key", ttl] = "val"put(ttl=...)TTL in seconds.
del db["key"]deleteRaises NotFoundError on miss.
"key" in dbexists
for k, v in dbiterator()Yields (bytes, bytes).
db["session:abc"] = "active"
db["token:xyz", 60] = "bearer-abc"   # expires in 60 s
print(db["session:abc"])               # b'active'
del db["session:abc"]

Transactions

begin(write=False) / commit() / rollback()

ACID transactions for batching operations atomically. Without an explicit transaction, each individual put/delete auto-commits.

db.begin(write=True)
try:
    db["a"] = "1"
    db["b"] = "2"
    db.commit()
except Exception:
    db.rollback()
    raise

Column Families

db.create_column_family(name) → ColumnFamily db.open_column_family(name) → ColumnFamily db.default_column_family() → ColumnFamily db.list_column_families() → list[str] db.drop_column_family(name)
with db.create_column_family("users") as users:
    users["alice"] = b"admin"

with db.open_column_family("users") as users:
    print(users["alice"])  # b'admin'

names = db.list_column_families()  # ["users", ...]
db.drop_column_family("users")

Iterators

db.iterator(*, reverse=False, prefix=None) → Iterator db.prefix_iterator(prefix) → Iterator db.reverse_iterator() → Iterator db.reverse_prefix_iterator(prefix) → Iterator
# Forward scan
for key, value in db.iterator():
    print(key.decode(), value.decode())

# Reverse scan
for key, value in db.iterator(reverse=True):
    print(key.decode(), value.decode())

# Prefix, ascending
for key, value in db.iterator(prefix="user:"):
    print(key, value)

# Prefix, descending
for key, value in db.iterator(prefix="user:", reverse=True):
    print(key, value)

Maintenance

sync() / vacuum(n_pages=0) / integrity_check() / checkpoint(mode) / stats() / stats_reset()

db.sync() # flush pending writes to disk db.vacuum(n_pages=0) # reclaim unused pages (0 = all) db.integrity_check() # raises CorruptError on failure nlog, nckpt = db.checkpoint(mode) # returns (nLog, nCkpt) tuple s = db.stats() # dict with 12 counters (see below) db.stats_reset() # zero all cumulative counters; db_pages is unaffected print(db.errmsg) # last error message string (property)

stats() returns a dict with these keys:

KeyDescription
putsTotal put operations (includes TTL puts and put_if_absent).
getsTotal get operations (includes TTL gets).
deletesTotal delete operations.
iterationsTotal iterators created (each iterator(), prefix_iterator(), etc. call counts as one).
errorsTotal non-NOTFOUND error returns.
bytes_readTotal value bytes returned by get operations.
bytes_writtenTotal key+value bytes written by put operations.
wal_commitsTotal WAL write transaction commits.
checkpointsTotal WAL checkpoint operations.
ttl_expiredTotal keys lazily deleted on access due to TTL expiry.
ttl_purgedTotal keys deleted by purge_expired().
db_pagesCurrent live database page count (always up-to-date; not reset by stats_reset()).
import snkv
from snkv import CHECKPOINT_TRUNCATE, CorruptError

# Vacuum after bulk deletes
db.vacuum()

# Checkpoint and reclaim WAL space
nlog, nckpt = db.checkpoint(CHECKPOINT_TRUNCATE)
print(f"WAL: {nlog} frames, {nckpt} checkpointed")

try:
    db.integrity_check()
except CorruptError as e:
    print(f"Corruption: {e}")

st = db.stats()
print(st["puts"], st["gets"], st["bytes_written"], st["db_pages"])
db.stats_reset()   # counters zeroed; db_pages still live

Lifecycle

close() / context manager

Close the store and release all resources. All column family and iterator handles must be closed first. Always prefer the context manager form.

with KVStore("mydb.db") as db:
    ...
# db.close() called automatically

ColumnFamily

A logical namespace within a KVStore. Obtained via db.create_column_family(), db.open_column_family(), or db.default_column_family(). Always use as a context manager.

Core Operations

cf.put(key, value, ttl=None) # insert/overwrite; ttl in seconds cf.get(key, default=None) # returns bytes or default cf.delete(key) # raises NotFoundError on miss cf.exists(key) → bool
with db.open_column_family("users") as cf:
    cf.put("alice", b"admin")
    cf.put("token", b"xyz", ttl=300)   # 5 minutes
    role = cf.get("alice")            # b'admin'
    cf.delete("alice")

CF TTL

cf.ttl(key) → float | None # remaining seconds or None (no expiry) cf.purge_expired() → int # delete expired keys in this CF only
remaining = cf.ttl("token")   # e.g. 284.3
n = cf.purge_expired()
print(f"Cleaned {n} entries")

CF Conditional Insert / Bulk Clear / Key Count

cf.put_if_absent(key, value, ttl=None) → bool # True if inserted cf.clear() # truncate this CF only; O(pages) cf.count() → int # entry count; O(pages)
with db.open_column_family("dedup") as cf:
    # First write wins — safe deduplication
    if cf.put_if_absent(b"msg:001", b"hello"):
        process()

    print(cf.count())   # entry count for this CF only
    cf.clear()          # other CFs unaffected
    print(cf.count())   # 0

CF Dict Interface

SyntaxNotes
cf["key"]Raises NotFoundError on miss/expiry.
cf["key"] = "val"No TTL.
cf["key", ttl] = "val"TTL in seconds.
del cf["key"]Raises NotFoundError on miss.
"key" in cfExistence check.
cf["alice"] = "admin"
cf["session:u42", 300] = "logged-in"
print(cf["alice"])           # b'admin'
del cf["alice"]

CF Iterators

cf.iterator(*, reverse=False, prefix=None) → Iterator cf.prefix_iterator(prefix) → Iterator cf.reverse_iterator() → Iterator cf.reverse_prefix_iterator(prefix) → Iterator
for key, value in cf.iterator(prefix="user:", reverse=True):
    print(key, value)

Iterator

Ordered cursor over key-value pairs. Forward iterators yield keys ascending; reverse iterators yield keys descending. Each iteration step yields a (bytes, bytes) tuple.

Python Iterator Protocol

The most common usage — iterate with a for loop or context manager:

# Simple forward
for key, value in db.iterator():
    print(key.decode(), "->", value.decode())

# Reverse with context manager (auto-closes)
with db.iterator(reverse=True) as it:
    for key, value in it:
        print(key, value)

Manual Control

it.first() # seek to first key (forward iterator) it.last() # seek to last key (reverse iterator) it.seek(key) → self # O(log N) positional seek; returns self for chaining it.next() # advance forward it.prev() # advance backward it.eof # True when no more keys it.key # current key as bytes it.value # current value as bytes it.item() # (key, value) tuple it.close()
# Manual forward
it = db.iterator()
it.first()
while not it.eof:
    print(it.key.decode(), it.value.decode())
    it.next()
it.close()

# Manual reverse
it = db.reverse_iterator()
it.last()
while not it.eof:
    k, v = it.item()
    print(k, v)
    it.prev()
it.close()

# seek — jump to any position in O(log N)
with db.iterator() as it:
    it.seek(b"user:carol")   # forward: first key >= target
    while not it.eof:
        print(it.key)
        it.next()

# seek on a prefix iterator — boundary still enforced
with db.iterator(prefix=b"user:") as it:
    it.seek(b"user:m")   # skips to "user:m..." within prefix
    while not it.eof:
        print(it.key)
        it.next()

# seek returns self — chainable
key = db.iterator().seek(b"target").key

Type Notes

Input typeBehaviour
strEncoded to UTF-8 bytes automatically.
bytesPassed through as-is.
bytearrayConverted to bytes.
memoryviewConverted to bytes.

Return values are always bytes.

Thread safety: The underlying C store is mutex-protected. Each Python method acquires a mutex and releases the GIL for blocking I/O — safe to share a KVStore across threads. Column family and iterator handles are not independently thread-safe.