API Reference

This document is a task-oriented map of the API surface. It is intentionally not a generated endpoint inventory.

Source Of Truth

Use these sources in this order:

1. Running OpenAPI UI at http://localhost:8001/docs
2. Route registry in src/web/routes/__init__.py
3. The individual route modules under src/web/routes/

If this document disagrees with code or OpenAPI, trust the runtime.

Core Endpoint Groups

Health

• GET /health
• GET /api/health
• GET /api/health/database
• GET /api/health/services
• GET /api/health/celery

Use these first when verifying the stack.

Sources And Ingestion

• GET /api/sources
• GET /api/sources/{source_id}
• POST /api/sources/{source_id}/collect
• PUT /api/sources/{source_id}/image_ocr — Set the per-source OCR override. Body: { "image_ocr_enabled": true | false | null }. true forces OCR on regardless of the global OCR_INGEST_ENABLED env; false forces it off; null clears the override and inherits the global default. Protected sources (eval/manual) reject this call with HTTP 400.
• POST /api/scrape-url

These endpoints control source state and manual collection.

Articles

• GET /api/articles
• GET /api/articles/{article_id}
• GET /api/articles/{article_id}/similar
• DELETE /api/articles/{article_id}
• GET /api/articles/{article_id}/chunk-debug — Chunk-level breakdown of the junk filter's keep/remove decisions, powering the Junk Filter Tuning modal. Local sklearn inference only; no LLM provider calls.
• GET /api/articles/{article_id}/chunk-debug/progress — Progress snapshot for an in-flight chunk-debug run. Returns in_progress: false when nothing is running; otherwise phase, processed_chunks, total_chunks, article_total_chunks and chunk_limit_applied. phase is filtering (full-article pass, no per-chunk counter yet) → analyzing (processed_chunks advancing toward total_chunks) → finalizing (assembling the response); only analyzing counts chunks, so a client must not render a counter during the other two. total_chunks is what this pass will analyse, article_total_chunks is what the article holds — they differ whenever chunk_limit_applied is true. Progress lives in Redis, not process memory, because a poll can land on a different uvicorn worker than the analysis.

These are the main article browsing and maintenance endpoints.

Article AI Endpoints

• POST /api/articles/{article_id}/detect-os — Detect operating system from article content using deterministic entity/keyword-registry classification (no embedding model). Applies the content filter first. Returns HTTP 422 with { "error": "no_huntable_content" } when the content filter finds no huntable chunks above the confidence threshold.
• POST /api/articles/{article_id}/rank-with-gpt4o — Rank article huntability using the active workflow config's RankAgent prompt and model.

Route module: src/web/routes/ai.py.

Search

• POST /api/search/semantic
• GET /api/search/help

These power the semantic search workflow. For conversational retrieval, use the Huntable MCP server (see MCP tools reference).

Embeddings And Vector Coverage

• GET /api/embeddings/stats — Embedding coverage summary. Response includes a sigma_corpus block (SigmaHQ sigma_rules row counts vs rows with embeddings), distinct from the AI sigma_rule_queue. Used by CLI embed stats and MCP get_stats.

Workflow Execution

• GET /api/workflow/executions — List executions with pagination. Query params: page (default 1), limit (default 50, max 200), status, step, article_id, sort_by, sort_order. Response: executions, total, page, total_pages, limit, running, completed, failed, pending.
• GET /api/workflow/executions/{execution_id}
• POST /api/workflow/articles/{article_id}/trigger — Query: force (bool, default false). When true, skips the RegexHunt auto-trigger threshold; use for explicit manual runs. Ingestion auto-trigger still uses the threshold.
• POST /api/workflow/executions/{execution_id}/retry
• POST /api/workflow/executions/{execution_id}/cancel
• POST /api/workflow/executions/cleanup-stale
• POST /api/workflow/executions/trigger-stuck — Re-dispatches stuck pending executions onto the Celery workflow queue (one trigger_agentic_workflow task per row). Use it when a Celery task was lost — worker down at dispatch time, broker restart. Only rows pending longer than STUCK_PENDING_AFTER (5 min) and never started qualify; fresher rows are reported under skipped because their task is probably still queued. Eval rows (eval_run in the execution snapshot) get a longer window, eval_pending_stuck_after(): the 5-minute base plus the latest countdown the eval launcher can schedule (MAX_EVAL_EXECUTIONS_PER_LAUNCH − 1 steps of EVAL_STAGGER_SECONDS + the 60 s throttle ceiling, about 1 h 44 min at the default cap of 100), because a staggered launch commits every row up front and leaves its tail pending with live tasks. Double runs are prevented in the worker, not here: run_workflow claims its row with a conditional UPDATE … WHERE status = 'pending' and a duplicate dispatch for an already-claimed row logs a skip and returns without invoking the graph. The age filter keeps the endpoint from queueing noise and from counting such no-op dispatches as successful; count / successful / failed describe tasks queued, not runs performed. is_stuck_pending_execution in workflow_trigger_service.py is the single shared definition — the article-trigger path fails these same rows before starting a fresh execution, and blocks the trigger ("already has an active workflow execution") for a row still inside its window. It queues work; it does not run the LangGraph pipeline in the web process, which cannot import langgraph.

The workflow engine writes its state into agentic_workflow_executions and exposes it through these endpoints.

Workflow Configuration

• GET /api/workflow/config

• PUT /api/workflow/config — Rejects a config whose enabled extractors have no prompt (HTTP 400), because such a config produces runs that complete with zero observables and no error. Send allow_prompt_warnings: true to override deliberately ("Save Anyway"). Settings-only autosaves and disabled agents are exempt. See src/web/routes/workflow_config.py.

  Also rejects (HTTP 400, no override) an agent whose provider is paired with a model the catalog attributes to a different provider — for example lmstudio with gpt-5.6-sol. Such a pair fails later at call time with a confusing "model not found" from the wrong provider. openai and codex share one model namespace, so codex + gpt-5.6-* is valid. Only pairs this request changes are checked, so a config carrying a pre-existing mismatch stays saveable and can be repaired. Models absent from config/provider_model_catalog.json (LMStudio local models, newly released models) are never rejected. - GET /api/workflow/config/prompts - GET /api/workflow/config/prompts/{agent_name} - GET /api/workflow/config/prompts/defaults/{agent_name} -- on-disk user template the runtime falls back to (SigmaAgent: sigma_generate_multi.txt) plus the code-default system prompt; used by the effective-prompt preview - PUT /api/workflow/config/prompts - GET /api/workflow/config/prompts/{agent_name}/versions - GET /api/workflow/config/prompts/{agent_name}/by-config-version/{config_version} - POST /api/workflow/config/prompts/{agent_name}/rollback - POST /api/workflow/config/prompts/bootstrap - POST /api/workflow/config/prompts/reset-to-defaults — Reset individual agent prompts to seed defaults. Unlike bootstrap (which wholesale replaces all prompts), this selectively resets only the requested agents while preserving others. Body: { "agent_names": ["RankAgent", ...] } (omit agent_names to reset all). See src/web/routes/workflow_config.py. - GET /api/workflow/config/versions — List config versions with pagination. Query params: page (default 1), limit (default 20, max 100), version (optional, exact integer match). Response: versions, total, page, total_pages. - GET /api/workflow/config/preset/list - POST /api/workflow/config/preset/save - GET /api/workflow/provider-options — Server-owned availability and model list for OpenAI, Anthropic, LM Studio, and the optional Codex subscription provider. Each provider block also carries model_capabilities (model id → supports_temperature, supports_top_p, effort_levels, default_effort, source), read from config/model_capabilities.json for OpenAI/Anthropic and discovered live from model/list for Codex (which adds effort_descriptions). The Agents page uses it to disable unsupported Temperature/Top_P sliders and populate the per-agent Effort select; PUT /api/workflow/config rejects an {Agent}_effort outside the resolved model's tiers with HTTP 400. - PATCH /api/workflow/config/auto-trigger-threshold — Update the auto-trigger hunt score threshold (0–100). Body: { "auto_trigger_hunt_score_threshold": <float> }. This is the only endpoint that changes this value. It mutates the active config row in-place and is intentionally excluded from the main PUT /api/workflow/config endpoint and from all preset import/export paths. Manage this setting only through the Settings UI.

The v2 configuration contract's prompt-bearing agent names are RankAgent, SigmaAgent, CmdlineExtract, ProcTreeExtract, HuntQueriesExtract, RegistryExtract, ServicesExtract, ScheduledTasksExtract, and NetworkIndicatorExtract. ExtractAgent supplies model/provider fallback configuration but is not prompt-bearing. The legacy prompt endpoints do not enforce that schema allowlist and may expose established auxiliary database keys such as OSDetectionAgent or SigmaRepair; clients should not invent new names. QA agents (RankAgentQA and all extractor QA agents) were fully removed in v7.2.0 (commit b9645305, 2026-05-22; released 2026-05-29) and are no longer valid agent names.

Each prompt object is a JSON dict with these fields:

Field	Type	Purpose
role (or system)	string	Agent persona — identity and expertise statement
task (or objective)	string	What the agent does this run
instructions	string	Rules, constraints, output format, JSON enforcement
json_example	string or dict	Concrete output schema example
output_format	dict	Legacy schema description (used if json_example absent)

The user message scaffold (Title/URL/Content headers, instructions footer) is assembled by the runtime and is not part of the prompt config. See Prompt Architecture for how prompts are assembled at runtime.

The strict configuration contract is defined in src/config/workflow_config_schema.py.

Settings And Integrations

• GET /api/settings/* — Reads never return a stored credential. For a key matching _is_sensitive_setting the response carries value: null plus configured and a short hint; non-sensitive settings round-trip their value as before. Both read routes send Cache-Control: no-store.
• POST /api/settings/codex/test — Verify the deployment-managed Codex login without invoking a workflow model.
• POST /api/settings/github/test — Verify the stored GitHub PAT against the configured repository, server-side. Accepts optional token / repo overrides so a freshly typed credential can be checked before it is saved.
• POST /api/test-openai-key
• POST /api/test-anthropic-key
• POST /api/test-lmstudio-connection

These endpoints manage runtime settings and provider connectivity. The provider key tests accept an empty api_key and fall back to the stored credential, since the Settings page cannot read one back to resend it.

Models And MLOps

• GET /api/model/retrain-status — Poll retraining progress (idle / starting / loading / complete / error)
• POST /api/model/retrain — Trigger model retraining from user feedback and annotations. The script trains to a staging path first; a quality gate (recall_huntable ≥ 0.30 and f1_huntable ≥ 0.30) must pass before the staged model is promoted to the live path. If the gate fails, the status file is set to error with message RETRAIN REJECTED and the live model is left untouched.
• GET /api/model/versions — List model versions with metrics. Query params: page (optional; omit for unpaginated), limit (default 10, max 100), version (exact version number search)
• POST /api/model/evaluate — Run evaluation of the current model on the annotated test set
• GET /api/model/eval-chunk-count — Count of chunks in the evaluation dataset
• GET /api/feedback/chunk-classification/{article_id} — All stored chunk feedback for an article, keyed by chunk id (newest per chunk). One request replaces the per-chunk lookups the Junk Filter Tuning modal used to issue while rendering.
• GET /api/feedback/chunk-classification/{article_id}/{chunk_id} — Feedback for a single chunk, or feedback: null. Used to re-read one chunk after a write.
• POST /api/feedback/chunk-classification — Record user feedback on a chunk classification (writes into the ML training corpus).
• GET /api/model/feedback-count — Count of available feedback and annotation samples for retraining
• POST /api/model/rollback/{version_id} — Roll back to a specific model version. Copies the saved artifact to the live path, flips is_current, clears the ContentFilter cache, and starts a background chunk re-scoring backfill
• GET /api/model/compare/{version_id} — Get or generate comparison results between a version and its predecessor
• GET /api/model/feedback-comparison — Before/after confidence levels for chunks that received user feedback
• GET /api/model/classification-timeline — Classification breakdown across model versions for time series charting

Route module: src/web/routes/models.py. Version data is stored in the ml_model_versions table (see src/database/models.py).

AI / Inline LLM Endpoints

• POST /api/articles/{article_id}/detect-os — Run OS detection on a single article. Returns OS label, confidence, and method. HTTP 422 when content filter finds no huntable chunks ({ "error": "no_huntable_content" }); client should check is_huntable before calling if this matters.

Sigma Queue And Evaluation

• GET /workflow/queue — HTML page for the Sigma queue console (redirects to /workflow#queue; uses /api/sigma-queue/* for data).
• GET /api/sigma-queue/list — List queued Sigma rules with pagination. Query params: status (optional, values: pending, needs_review, approved, rejected, submitted), workflow_execution_id (optional, filter to one workflow job's rules), limit (default 50, max 500), offset (default 0). Response: { "items": [...], "total": N, "limit": L, "offset": O }.
• POST /api/sigma-queue/{queue_id}/enrich — Enrich a queued rule with OpenAI, Anthropic, LM Studio, or the optional Codex subscription provider. The configured provider credential stays server-side. A successful response includes the enriched YAML and conversation_id for a persisted, rule-scoped follow-up conversation.
• GET /api/sigma-queue/{queue_id}/enrichment-chat — Return the latest persisted enrichment conversation for the rule. System prompts are excluded from the browser response.
• POST /api/sigma-queue/{queue_id}/enrichment-chat/{conversation_id}/turn — Continue that conversation through its original provider and model. Follow-up turns are limited to revising the current Sigma rule or discussing earlier enrichment turns.
• POST /api/sigma-queue/{queue_id}/validate — Validate and optionally LLM-enrich a queued rule. Returns { "validated_yaml": ... }.
• GET /api/sigma-queue/* (other endpoints)
• GET /api/eval/os-detection-manual-results and GET /api/eval/observables-count-results — remaining legacy evaluation result routes (src/web/routes/evaluation.py)
• /mlops/agent-evals and /mlops/agent-evals2 — HTML evaluation pages registered in src/web/routes/pages.py

Subagent Evaluation Endpoints

These support per-subagent extraction evals (CmdlineExtract, ProcTreeExtract, HuntQueriesExtract, RegistryExtract, ServicesExtract, ScheduledTasksExtract, NetworkIndicatorExtract):

• GET /api/evaluations/subagent-eval-articles — List seeded eval articles for a given subagent.
• POST /api/evaluations/run-subagent-eval — Trigger a subagent eval run against the active config (the legacy use_active_config field is ignored). Rejected with 422 before any write when a URL has no committed fixture or the run exceeds MAX_EVAL_EXECUTIONS_PER_LAUNCH (default 100). Shares its planner with the run_subagent_eval MCP tool.
• GET /api/evaluations/subagent-eval-results — Get results for completed subagent eval runs (includes expected_items, actual_items, matched_count, missed_count, extra_count when item-level ground truth is set).
• GET /api/evaluations/subagent-eval-status/{eval_record_id} — Poll status of a single eval record.
• DELETE /api/evaluations/subagent-eval-clear-pending — Clear pending/stuck eval records.
• POST /api/evaluations/subagent-eval-backfill — Backfill pending eval records from existing workflow executions.
• POST /api/evaluations/subagent-eval-rescore — Repair completed count-only records that have item-level ground truth but no score, using each record's retained extractor output (no paid LLM). ?subagent=<name|all>; ?apply=false (default) is a dry-run reporting candidates/scorable/unrepairable_no_output per agent, ?apply=true persists. Scores against the ground truth stored on the record (no disk reload) and is idempotent.
• GET /api/evaluations/subagent-eval-aggregate — Aggregated metrics per config_version. Includes count-based fields (mean_score, raw_mae, score_distribution) and item-level fields (mean_precision, mean_recall, mean_f1, scored_articles). Top-level eval_set_total returns the canonical eval-article count from config/eval_articles.yaml (used by the MAE chart to flag subset runs in amber). Optional ?model= query param filters to versions where the subagent used the given model (powers the SYS.04 trend chart).
• GET /api/evaluations/subagent-eval-models?subagent=... — List models that have eval data for the given subagent, sorted by usage frequency. Powers the model dropdown on the /mlops/agent-evals2 SYS.04 chart.
• GET /api/evaluations/config-versions-models — List config versions with model info for each agent.

Route module: src/web/routes/evaluation_api.py.

Finding The Right Route Module

Start in src/web/routes/__init__.py, then open the matching module:

• workflow_executions.py
• workflow_config.py
• articles.py
• sources.py
• settings.py
• sigma_queue.py
• models.py
• evaluation_api.py — subagent eval runs, export bundles, config-version model lookup

Verification Guidance

• API behavior changes: run python3 run_tests.py api
• Workflow API changes: run python3 run_tests.py integration
• UI flows that call the API: run python3 run_tests.py ui or python3 run_tests.py e2e

Last updated: 2026-08-13 Last reviewed: 2026-09-01