Call Chain

---

V2 — FastAPI bot

Shortcut webhook POST /webhook
  └── _verify_hmac()                          server.py
  └── _handle_comment(sc_id, text)            server.py
        ├── db.get_session(sc_id)             adapters/db.py
        │
        ├── [no session + "@agentbot clarify"]
        │     └── _run_clarify(sc_id)         server.py
        │           ├── db.create_session()
        │           ├── shortcut.fetch_story()         adapters/shortcut.py
        │           ├── call_embed([story_text])        adapters/llm.py
        │           ├── pgvector_client.search()        adapters/pgvector_client.py
        │           ├── build_clarify_messages()        prompt_builder.py
        │           │     ├── _read(clarify.md)
        │           │     ├── _read(rules.md, templates.md, glossary.md)
        │           │     └── _schema_section()  ← detect_schema_tables()
        │           ├── call_chat(Sonnet)               adapters/llm.py
        │           ├── parse_clarify_result()          prompt_builder.py
        │           ├── db.append_clarify_result()
        │           └── shortcut.post_comment(01_extract.sql)
        │
        └── [session exists + status=awaiting_pm]
              └── db.update_status("drafting", "awaiting_pm")  ← atomic lock
              └── _preprocess_and_advance(sc_id)  server.py
                    ├── db.get_session()
                    ├── shortcut.fetch_story()     ← fresh fetch, gets all comments
                    ├── build_preprocess_messages()       prompt_builder.py
                    ├── call_chat(Haiku)
                    ├── parse_session_context()    → SessionContext
                    │
                    ├── [needs_reclarify]
                    │     ├── build_clarify_messages()
                    │     ├── call_chat(Sonnet)
                    │     ├── db.append_clarify_result()
                    │     ├── db.update_status("awaiting_extraction")
                    │     └── shortcut.post_comment(updated 01_extract.sql)
                    │
                    ├── [still_open questions]
                    │     ├── shortcut.post_comment(remaining questions)
                    │     └── db.update_status("awaiting_pm")
                    │
                    └── [all resolved]
                          ├── build_draft_messages()    prompt_builder.py
                          ├── call_chat(Sonnet)
                          ├── parse_draft_result()
                          ├── db.update_draft()
                          ├── db.update_status("done")
                          └── shortcut.post_comment(02_patch.sql + 03_revert.sql)

GitLab CI POST /submitextraction
  └── auth: x_api_key check                  server.py
  └── db.update_status("drafting", "awaiting_extraction")  ← atomic lock
  └── db.append_extraction_round()
  └── db.get_session()
  └── build_preprocess_messages(clarify_result, [], extraction_rounds)
  └── call_chat(Haiku)  → SessionContext
        ├── [still_open] → shortcut.post_comment(questions) + status=awaiting_pm
        └── [resolved]   → build_draft_messages → call_chat(Sonnet) → post draft

---

V1 — Local CLI

python -m dp_agent clarify SC-XXXXX
  └── cli.main()                             dp_agent/cli.py
  └── pipeline.clarify(sc_id)               dp_agent/pipeline.py
  └── scripts.test_clarify.main(sc_id)      scripts/test_clarify.py
        ├── corpus check (data/indexed.json) — skip if already deployed
        ├── staleness check (examples/raw/ vs indexed.json mtime)
        ├── shortcut.fetch_story()
        ├── call_embed([title+labels+desc])
        ├── qdrant_client.search(top_k=5+holdout)   ← V1 still uses Qdrant
        ├── detect_schema_tables(story_text, hits_sql)
        ├── build_clarify_messages()
        ├── call_chat(Sonnet)
        ├── parse_clarify_result()           ← retry at temp=0.3 on bad JSON
        └── writes: clarifications.md, 01_extract.sql, hits.json

[engineer runs 01_extract.sql, pastes results into clarifications.md]

python -m dp_agent auto-answer SC-XXXXX
  └── scripts.test_auto_answer.main()
        ├── reads clarifications.md
        ├── checks ## Extraction results is non-empty
        ├── build_auto_answer_messages(clarifications_md)
        ├── call_chat(Haiku)
        ├── parse_auto_answer_result()       ← retry on bad JSON
        └── apply_auto_answers() → writes clarifications.md in-place

python -m dp_agent validate SC-XXXXX
  └── scripts.test_validate.main()
        ├── checks at least one **Answer:** field filled
        ├── build_validate_messages(clarifications_md)
        ├── call_chat(Haiku)
        ├── parse_validate_result()
        └── either stamps RESOLVED or injects ⚠️ annotation blocks inline

python -m dp_agent draft SC-XXXXX
  └── scripts.test_draft.main()
        ├── gate: RESOLVED + extraction present
        ├── shortcut.fetch_story()           ← fresh fetch for latest comments
        ├── build_draft_messages(story, clarifications_md, hits)
        ├── call_chat(Sonnet)
        ├── parse_draft_result()
        └── writes: 02_patch.sql, 03_revert.sql, rationale.md

---

Known issues at key nodes

Node	Current approach	Problem / opportunity
Corpus embedding (call_embed in both flows)	Embeds title + labels + description only	SQL structure (tables, operations) not embedded — misses strongest retrieval signal
Corpus payload (pgvector_client.upsert_entry)	Stores raw SQL in payload but no template label	Clarify LLM has to infer template from SQL instead of reading it directly
Clarify → template	LLM infers template from retrieved hits + templates.md rubric	Would be more reliable if hits carried explicit template labels
max_tokens=4096 (call_chat)	Hard cap on all LLM output	Long draft SQL can get truncated mid-JSON, causing a parse error with no retry in V2
Retry logic	V1 scripts retry on bad JSON at temp=0.3; V2 server does not	A parse failure in V2 rolls back status with no user feedback
Two identical LLM clients (adapters/llm.py)	_chat_client and _embed_client are the same config	One client is enough
SSL_VERIFY=False (config.py)	Hardcoded false	Must be fixed before airbase deployment
V1 uses Qdrant, V2 uses pgvector	Two separate vector stores in parallel	One-line import swap unifies them; Qdrant + Docker container can then be removed
Dead classify phase	build_classify_messages() + classify.md built, never called	Remove entirely