Engine workflow

Hydra Forge is not a one-shot generator. It is an engine designed to continuously enforce structural and semantic consistency throughout the lifetime of a scientific Python library.

This page describes the intentional, developer-centric execution workflow that governs how a Hydra Forge–generated library is evolved, rebuilt, and verified over time.

The workflow is designed to feel like a natural extension of how developers already work: write Python, reconcile the namespace, test and iterate; introduce native code when needed, rebuild and expose it explicitly, switch implementations freely, and continue evolving the system without breaking structure or visibility.

You write Python, run forge.namespace, import and test. When performance matters, you drop native code into _native, run Smithery, expose it, and keep going. When it breaks, you replace it — not the workflow.

The problem Hydra Forge solves

In long-lived codebases, most failures do not originate in algorithms. They emerge from structural drift.

• Namespaces that no longer reflect the filesystem
• Partially exposed or inconsistently wired functionality
• Native code that compiles but is misintegrated
• Documentation that no longer reflects behavior
• Implicit assumptions known only to original authors

Hydra Forge addresses this by enforcing an explicit and repeatable workflow that keeps structure, native code, interfaces, and documentation in sync.

These failures rarely surface immediately — they accumulate silently until maintenance, onboarding, or extension becomes risky.

The Hydra Forge engine model

Hydra Forge operates as a reconciliation engine. Each engine command is responsible for enforcing correctness in a single, well-defined dimension of the system.

Rather than hiding behavior behind implicit build steps, Hydra Forge exposes explicit entry points for each responsibility.

The engine is composed of four primary components:

• Namespace — Python API surface and visibility
• Smithery — native build, exposure, and inventory
• Tree — materialized structure and exposure layout
• Docs — documentation parity and API truth

Each component can be executed independently, but all are designed to converge on a single, verifiable system state.

Namespace reconciliation: forge.namespace

Hydra Forge treats the Python namespace as a generated artifact, not a manually curated structure.

The namespace phase reconciles filesystem layout with the public API surface.

• Scans project modules and packages
• Identifies public symbols
• Regenerates __init__.py and __all__
• Ensures import paths reflect actual structure

This eliminates a common failure mode where functionality exists on disk but is invisible or inconsistently exposed to users.

Manual namespace maintenance is intentionally avoided.

Native evolution: forge.smithery

Native Smithery is the engine responsible for the full lifecycle of native code inside a Hydra Forge project.

It governs compilation, exposure, inventory, and verification as a single, explicit operation.

• Scans native source trees
• Generates and updates build configuration
• Compiles native extensions deterministically
• Inspects compiled artifacts
• Materializes the exposed native API surface
• Updates native inventories and manifests

Native code is never compiled implicitly. All native evolution occurs through this explicit phase.

If Smithery fails, native code is not partially integrated or exposed.

Exposure and structure materialization: forge.tree

The tree phase materializes the structural truth of the project.

It generates and verifies the explicit filesystem representation of exposed Python and native components.

• Materializes the exposed native tree
• Ensures portal imports match exposed artifacts
• Verifies structural consistency
• Provides a human-inspectable representation of exposure

The generated tree is not a visualization — it is a verified artifact used by other engine components.

Tree generation never performs compilation or imports.

Documentation parity: forge.docs

Documentation is treated as a generated view of the system, not a parallel source of truth.

The documentation phase derives its content directly from Python code, native bindings, and exposure manifests.

• Scans Python modules statically
• Extracts native runtime docstrings from compiled artifacts
• Generates API references with stable import paths
• Builds documentation in an isolated environment

Documentation always reflects what is actually importable and executable.

No hand-written documentation layer exists alongside the generated output.

System diagnostics: forge.doctor

The doctor command performs deep diagnostics of the project environment.

• Validates Python interpreter and virtual environments
• Detects import path inconsistencies
• Checks native artifact availability
• Identifies common misconfigurations

Doctor is designed for investigation and validation, not mutation.

Structural inspection: forge.structure

The structure command reports the enforced project layout as a verified artifact.

It reflects the current, validated structure of the project — not a template or recommendation.

Cache hygiene

To keep execution deterministic across rebuilds and environments, Hydra Forge treats Python cache artifacts as a source of nondeterminism.

Cache cleanup is handled explicitly to ensure that rebuilds, imports, and documentation generation reflect actual code state.

No engine operation relies on stale bytecode.

Why this workflow matters

Together, these operations enforce a system-wide invariant:

• The filesystem reflects intent
• The namespace reflects the filesystem
• Native artifacts reflect native sources
• Documentation reflects exposed behavior

Breakage in this chain is detected early and explicitly.

Relationship to verification

The engine workflow is inseparable from Hydra Forge verification.

After each major operation, the system validates:

• Structural integrity
• Inventory consistency
• Interface visibility
• Documentation availability
• Test correctness

Verification is the mechanism that keeps the workflow trustworthy over time.

Summary

The Hydra Forge engine is a reconciliation system, not a generator.

Each engine command enforces correctness in one dimension of the project: namespace, native integration, structural exposure, and documentation.

Together, these components allow scientific Python libraries to evolve without accumulating structural drift, hidden dependencies, or undocumented behavior.