Toolsmith
Optional Python libraries and commands for HTTP downloads, files, text readers and local search.
Toolsmith provides Python functions and command-line tools for HTTP downloads, file handling, text reading, journals and search. A fleet can use them with its own files and SQLite databases. Your application supplies source permissions, summaries, interpretation and domain-specific workflows.
It is a separate project from ctlflow and is not required to run a fleet. It runs no server or background worker; a fleet may use it or omit it completely.
For example, an agent can download an approved article, extract its text, and add that text to an index in the fleet’s own SQLite database. Lexical search finds words and phrases. Semantic search uses a local model to convert text into numeric vectors and compare them for similarity. It does not generate answers.
Capabilities#
| Task | What Toolsmith does | Interface |
|---|---|---|
| Download a URL | Fetch approved HTTPS sources with DNS, redirect and quota checks | Python / CLI |
| Handle files | Hash, copy and inventory files; pack or extract selected ZIP entries | Python / CLI |
| Track request allowances | Reserve and settle shared request and byte budgets | Python / CLI |
| Keep a JSON journal | Append and verify caller-defined records | Python |
| Read text | Decode UTF-8, JSON, static HTML, RSS, Atom and CSV | Python / CLI |
| Find words | Search literal passages or an SQLite FTS5 index | Python / CLI |
| Find similar passages | Encode English text locally and query vectors in SQLite | Python / CLI; optional model dependencies |
| Keep a text archive | Retain versions of text and files with references to source passages | Python / CLI; optional archive |
| Maintain an archive | Back up, restore, audit and rebuild its indexes | Python / CLI; optional archive |
HTML readers preserve table markup for a model or caller to interpret. They do not infer a table’s meaning. Toolsmith does not run a browser, extract PDFs or perform OCR.
The base installation has no third-party runtime dependencies. HTTP and semantic search have optional dependencies. Semantic search compares the query vector with every vector in the requested corpus using a native SQLite extension.
The optional archive stores immutable imports. A citation identifies the retained version, text part and character range. Physical page numbers are supplied by the caller; they are not inferred by the archive. Applications with their own stores can use the file and search libraries without adopting this archive.
Use from any controller implementation#
A fleet can invoke Toolsmith commands regardless of which controller implementation it uses. Python programs can import the library directly. Installing it for a fleet does not add Python to the crossfire server.
Installation
Build and install Toolsmith, then select HTTP or semantic search dependencies.
Library API
Python modules, result handling and examples for archives and caller-owned indexes.
CLI
Read files, query indexes, download approved URLs and maintain an archive.
Capability reference
Available functions, their inputs, outputs and resource limits.
Semantic retrieval
Prepare a local MiniLM model, index text in SQLite and search by similarity.
Capacity and limits
Search corpus size, result limits and budgets for individual operations.
Archive operations
Back up, restore, inspect and migrate the optional text archive.
Security boundaries
Filesystem checks, HTTPS restrictions, local model validation and isolation limits.
Toolsmith behavior
Library and CLI behavior, persistence, search and supported operations.
Library architecture
Library modules, optional archive composition and persistent state.