Library API

Toolsmith functions return typed results. File handling, readers, search, journals and HTTP downloads can be used with your own data store. The optional text archive combines these functions with versioned storage and citations.

The tables below list public modules. Use their functions rather than querying Toolsmith’s private index tables.

When correcting indexed text, update the original text and invalidate both indexes in the same application transaction. Use unwrap inside the transaction so a failed invalidation rolls back the text change. Compute embeddings outside the transaction; recheck the text and previous digest before publishing them. Toolsmith leaves the caller’s schema version and row factory unchanged.

lexical.find_text preserves punctuation and caller-owned regions. lexical.search performs explicit indexed FTS matching over authorized database scopes. These are distinct query semantics, never fallback paths. Semantic inspect_index is cheap bulk metadata; full vector/source validation is a separate explicit audit.

Generic journal payloads are bounded arbitrary JSON; event schemas, authority and retention stay with the caller. One ASCII-escaped canonical hash format preserves the reviewed generic record framing. No record is automatically discarded, and an incomplete tail fails for explicit operator review rather than silently deleting it.

Distribution: ctlflow-toolsmith; import: toolsmith; command: toolsmith. Public modules/functions are named exports. There is no dynamic plugin registry, HTTP client for a Toolsmith service, skill installation or fleet integration.

The following modules compose the optional text archive, not a generic storage backend for consumer applications. Its parsing and indexing still use the same libraries listed above.

Operations validate boundaries and return typed Result[T]. unwrap and internal ToolsmithError are composition helpers used by the CLI/implementation, not an exception-based alternative public API. Do not bypass invariants with internal Store/connection helpers or direct SQL/file writes.

from pathlib import Path
from toolsmith.result import Failure
from toolsmith.storage.database import initialize_store
from toolsmith.readers.models import Document, TextPart
from toolsmith.storage.text import put_text

root = Path.cwd() / "local-store"
initialized = initialize_store(root, "sample")
if isinstance(initialized, Failure):
    raise RuntimeError(initialized.message)
document = Document("Report", (TextPart("Physical page 1", "Retained source words", 1),))
result = put_text(root, "sample", "report", document, "import-report-1")
if isinstance(result, Failure):
    print(result.code, result.message)
else:
    version = result.value.version

Stored physical pages and warnings describe supplied extraction, not certification that the original was complete. Citation offsets count Unicode characters, not UTF-8 bytes. Old versions remain exact; searches intentionally cover current versions only. Request identities are collection-wide across text, files and HTTP.

The optional HTTP boundary accepts Clock, Resolver and Transport protocols for deterministic integration/tests. The shipping composition uses bounded native DNS/TLS/HTTPS. These interfaces allow deterministic tests and custom composition. DNS approval and byte/deadline checks remain in acquisition; custom callers remain responsible for their own transport’s actual network behavior.

Importing the HTTP module does not import its optional DNS dependency. Missing dependency fails on selected network acquisition, not on text/storage import.

Caller-owned lexical index#

import hashlib
import sqlite3
from toolsmith import lexical
from toolsmith.result import unwrap

with sqlite3.connect(":memory:") as connection:
    connection.execute("PRAGMA foreign_keys = ON")
    unwrap(lexical.initialize_index(connection))
    source = lexical.Source("opaque-reference", "Exact source words", "Source")
    unwrap(lexical.index_source(connection, source, expected_digest=None))
    connection.execute("CREATE TABLE permitted_text (reference TEXT PRIMARY KEY, text_digest TEXT NOT NULL)")
    connection.execute("INSERT INTO permitted_text VALUES (?, ?)", (source.reference, hashlib.sha256(source.text.encode()).hexdigest()))
    matches = unwrap(lexical.search(connection, '"source words"', lexical.Scope("permitted_text")))

Initialization owns only derivative relations, not the application’s canonical tables or schema version. Both index initializers reject existing or partial schemas rather than silently adopting or repairing them. The selected database table/view is caller-owned authorization; digests exclude stale derivatives before ranking and limits. BM25 uses DB-wide statistics, so separate room/tenant databases remain necessary for statistical isolation. Cheap bulk status does not perform a full integrity audit.

Low-level descriptor/lock and SQLite transaction context managers are composition primitives: they raise ToolsmithError on expected failures within an enclosing public Result boundary. They do not start a process.

toolsmith.semantic exports the optional encoder and generic SQLite API; toolsmith.semantic.store composes those functions with exact retained citations. Importing either loads no optional native package or model. See semantic contracts and fleet integration .