Workflows¶
Workflows turn AtDork from a single-search CLI into an orchestration tool: a workflow is a named, structured YAML definition that runs a series of dork steps strictly in order — AtDork finishes one command before moving to the next, exactly like chaining commands by hand, but with threshold retries, worker concurrency, guarded outputs, and resumable state.
workflows:
recon-acme:
target: "example.com"
workers: 2
steps:
- name: exposed-files
query: 'site:{target} ext:(pdf|docx) (intitle:index OR intitle:download)'
when_results_below: 5
retry_with:
- 'site:{target} ext:pdf'
- name: login-portals
query: 'site:{target} inurl:(login OR signin)'
atdork workflow list # what's defined?
atdork workflow validate # schema check, zero API credits
atdork workflow run recon-acme # execute
atdork --run-workflow . # one-shot: run the single workflow in the root
Where definitions live¶
Workflow definitions are merged by name from (later wins):
workflows:in any configuration layer (global~/.config/atdork/config.yml, working-diratdork.yaml, the active environment config, or--config);- explicit files via
--file/--workflow-file(repeatable); - auto-discovered files named
workflow.yaml,workflow.yml,workflows.yaml,workflows.yml,atdork-workflow.yamloratdork-workflow.ymlin the active environment root and the current working directory ("root"). The environment root wins on name collisions, mirroring the config layer order (env > CWD).
A standalone file uses the same shape as the config section:
version: 2
workflows:
my-campaign:
description: "..." # shown by `workflow list`
target: "example.com"
steps:
- name: first
query: 'site:{target} inurl:login'
See examples/workflows/recon.yml
for a fully commented example and the repository-root workflow.yaml demo.
CLI¶
atdork workflow list¶
Every workflow from the merged sources with its step count, source
(config or file path) and description.
atdork workflow validate [name]¶
Deep validation without spending a single request: schema, unknown keys,
step names, substitution tokens, threshold/retry pairing, output formats,
and whether every {target} usage has a target available. Exit code 1 on
errors — safe for CI.
✗ broken — valid? no:
workflows.broken.steps[0]: requires 'query' or 'queries'
workflows.broken.steps[1].outputs[0]: format must be one of: csv, json, txt
atdork workflow run <name>¶
<name> accepts three forms:
| Form | Meaning |
|---|---|
recon-acme |
Workflow name from config or a discovered file. |
./my-workflow.yml |
File path — runs when the file holds exactly one workflow. |
. |
The single workflow discovered in the current root (error listing choices if there are several). |
| Flag | Description |
|---|---|
--file FILE |
Additional workflow YAML file(s), repeatable. |
--target TARGET |
Override the workflow-level target. |
--workers N |
Override the worker (thread) count of every step. |
--step NAME |
Run only this step. |
--resume |
Skip steps already recorded as completed. |
--dry-run |
Print the fully resolved queries; no search, no state written. |
One-shot main flags¶
The same engine is reachable without the subcommand:
atdork --run-workflow recon-acme --workflow-target example.org --workflow-workers 2
atdork --run-workflow . --workflow-dry-run
atdork --run-workflow recon-acme --workflow-resume
| Flag | Description |
|---|---|
--run-workflow NAME |
Workflow name, file path, or . (single workflow in the root). |
--workflow-file FILE |
Read definitions from this file. |
--workflow-target TARGET |
Override the target. |
--workflow-workers N |
Override the worker count. |
--workflow-resume |
Resume a previous run. |
--workflow-dry-run |
Resolve and print without searching. |
Reference¶
Workflow-level keys¶
| Key | Type | Description |
|---|---|---|
description |
string | Shown by workflow list. |
target |
string | Default target for {target} substitution (overridden by --target). |
workers |
int 1–64 | Threads inside each step; steps themselves stay sequential. Default 1. |
steps |
list | The step definitions, executed in order. |
on_error |
stop | continue |
Default failure policy for all steps. Default stop. |
worker_overrides |
mapping | Search kwargs merged on top of every step's kwargs (see below). |
| search keys | Workflow-wide defaults for any step search kwarg: backend, max_results, region, safesearch, timelimit, timeout, retries, delay, user_agent, verify, fallback_backends, fallback_chain, forced_proxy. |
Step keys¶
| Key | Type | Description |
|---|---|---|
name |
string | Unique per workflow; defaults to step-1, step-2, ... |
query |
string | The dork to run (mutually exclusive with queries). |
queries |
list | Several dorks in one step — distributed across workers. |
when_results_below |
int | Threshold: fewer unique results → run retry_with. |
retry_with |
list | Alternate queries executed when the threshold is missed; their results are merged and deduplicated into the step results. |
on_error |
stop | continue |
Overrides the workflow-level policy for this step. |
outputs |
list | Result files: - file: path.json + optional format: txt\|json\|csv. |
worker_overrides |
mapping | Search kwargs applied on top for this step. |
| search keys | Any workflow-level search key, overridden for this step. |
Search kwargs precedence: workflow defaults → step keys →
worker_overrides (workflow-level, then step-level) — later wins. This is how
one step can force an API backend while the rest of the campaign scrapes:
steps:
- name: portals
query: 'site:{target} inurl:login'
worker_overrides:
backend: serper # requires engines.serper.api_key (see Engines)
Substitution¶
| Token | Value |
|---|---|
{target} |
The resolved target (--target > workflow target). |
{step} |
Current step name. |
{date} |
Run date, YYYYMMDD. |
{run_id} |
Short id of this run (stable across --resume). |
Available in queries, retry_with, output paths and description-adjacent
strings. Unknown tokens fail validation with a typo-friendly error; literal
braces such as ext:{pdf,docx} are never treated as tokens.
Semantics¶
Sequential steps, concurrent queries¶
Steps always run in order: step N+1 starts only after N finished.
Inside a step, workers: N runs that step's queries (queries: lists or
retry_with batches) on N threads — the same thread pool machinery as
--concurrency, so caching, dedup and the circuit breaker all apply.
Threshold retries (when_results_below + retry_with)¶
After the step's primary queries finish, unique results (deduplicated by
URL) are counted. Below the threshold → each retry_with query runs in
order, and all results merge into the step's output. The step is then
marked:
| Status | Meaning |
|---|---|
done |
Threshold met (or no threshold configured). |
below_threshold |
Finished, but still fewer unique results than requested. Recorded; the run continues. |
failed |
All queries of the step failed (backend errors exhausted) — on_error decides whether the run stops. |
A below_threshold step is not an error: campaigns against small targets
legitimately return little.
Failure policy (on_error)¶
stop(default): the firstfailedstep aborts the remaining steps — fast, deterministic, and re-runnable with--resume.continue: later steps still run; the summary lists the failure.
Step-level errors (including output paths rejected by the environment write guard) are recorded the same way — a workflow never crashes mid-run with a half-written state.
State and --resume¶
Every run persists a JSON state file:
<environment root>/state/workflows/<name>.json # env active
~/.atdork/state/workflows/<name>.json # no env
It records the run_id, per-step status, result counts and written outputs.
--resume (or --workflow-resume) skips steps whose status is done or
below_threshold and re-runs only failed/missing steps under the same
run_id. Without --resume a run starts fresh and overwrites the state.
--dry-run never writes state and never calls a search backend — use it to
preview the exact queries a campaign would execute.
Outputs and the environment guard¶
Each step may write its merged results:
- Formats:
txt,json,csv(default: inferred from the extension). - Relative paths resolve against
<env>/results/when an environment is active, else the working directory. - With an active environment every output goes through the strict write guard: paths escaping the root fail the step — results cannot leak outside the workspace.
Backends, API providers and credit caution¶
Workflows use the normal search machinery: backend accepts scraping
backends, API engines (serper, serpapi, brave) and profiles: names;
fallback_chain/search.fallback and the --resilient circuit breaker
apply per query. Each step query counts against your API quota — run
--dry-run first, keep max_results sane, and prefer scraping backends for
exploratory steps with API backends reserved for worker_overrides on the
steps that need them.
Complete example¶
version: 2
workflows:
recon-acme:
description: "Full recon sweep for one target"
target: "example.com"
workers: 2
max_results: 50
region: "us-en"
steps:
- name: exposed-files
query: 'site:{target} ext:(pdf|docx) (intitle:index OR intitle:download)'
when_results_below: 10
retry_with:
- 'site:{target} ext:pdf'
- 'site:{target} (filetype:doc OR filetype:xls)'
outputs:
- file: recon/files.json
format: json
- name: login-portals
queries:
- 'site:{target} inurl:(login OR signin OR admin)'
- 'site:{target} intitle:"index of" (.env OR .git)'
worker_overrides:
backend: serper
on_error: continue
- name: fingerprints
query: 'site:{target} ("wordpress" OR "drupal") inurl:wp-admin'
outputs:
- file: recon/fingerprints.csv
format: csv
atdork workflow validate recon-acme
atdork workflow run recon-acme --dry-run
atdork workflow run recon-acme # fresh run
atdork workflow run recon-acme --resume # after a stop/failure
Tips¶
- Validate in CI:
atdork workflow validateexits non-zero on bad definitions — catch typos before they burn API credits. - Environments: pair workflows with
atdork env createso every result file, cache entry and state file stays inside one portable folder. - Combine with packages:
queries:can reference installed dork files via--database-dork-style content only in plain text — copy the dorks intoqueries:or generate a workflow with a small script. - Debug a step in isolation:
--step <name>runs a single step under the workflow's own configuration.