M14: API runner contract test (verification milestone)
- Add test_api_runner_contract.py: proves API txt2img path invokes ProcessingRunner - No routing changes: API continues to call process_images (orchestration boundary) - Monkeypatch CI env + runner.run; call API method directly - M14_plan.md, M14_toolcalls.md: governance docs Phase III — Runner & Service Boundary Made-with: Cursorpull/17338/head
parent
a12028b148
commit
961297f09d
|
|
@ -0,0 +1,274 @@
|
||||||
|
# M14_plan — API Integration Through ProcessingRunner
|
||||||
|
|
||||||
|
## 1. Intent / Target
|
||||||
|
|
||||||
|
**Primary objective:**
|
||||||
|
Ensure API generation paths (`/sdapi/v1/txt2img`, `/sdapi/v1/img2img`) **continue to route through** `process_images` → `ProcessingRunner`, and **lock that behavior with a contract test**.
|
||||||
|
|
||||||
|
This is a **governance milestone** — verification + contract expansion, **not** a routing change.
|
||||||
|
|
||||||
|
### Why this matters
|
||||||
|
|
||||||
|
* M10–M13 established the runner as a **safe execution boundary**
|
||||||
|
* M13 proved txt2img UI already flows through it and is contract-protected
|
||||||
|
* API is the next highest-value surface (external contract)
|
||||||
|
* M14 proves API uses the same boundary and protects it from regression
|
||||||
|
|
||||||
|
**Outcome:**
|
||||||
|
API → runner routing is **provable and protected** by contract test.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Scope Boundaries
|
||||||
|
|
||||||
|
### In Scope
|
||||||
|
|
||||||
|
* `modules/api/api.py`
|
||||||
|
|
||||||
|
* `text2imgapi`
|
||||||
|
* `img2imgapi`
|
||||||
|
* `modules/runtime/runner.py`
|
||||||
|
* `modules/processing.py` (only if minimal adapter needed)
|
||||||
|
* New contract tests
|
||||||
|
|
||||||
|
### Out of Scope
|
||||||
|
|
||||||
|
* Queue/background execution (M15)
|
||||||
|
* Runtime extraction (M16+)
|
||||||
|
* UI refactor (Phase V)
|
||||||
|
* Extension behavior changes
|
||||||
|
* Any change to request/response schemas
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Invariants (Must Not Change)
|
||||||
|
|
||||||
|
From Serena invariant registry:
|
||||||
|
|
||||||
|
### API Contract Invariants
|
||||||
|
|
||||||
|
* JSON request/response schemas unchanged
|
||||||
|
* Status codes unchanged
|
||||||
|
* Error behavior unchanged
|
||||||
|
|
||||||
|
### Runtime Invariants
|
||||||
|
|
||||||
|
* Output images identical (same seed → same result)
|
||||||
|
* Metadata / infotext unchanged
|
||||||
|
* File save paths unchanged
|
||||||
|
|
||||||
|
### System Invariants
|
||||||
|
|
||||||
|
* Extensions behave identically
|
||||||
|
* CLI/UI unaffected
|
||||||
|
* Coverage ≥ 40% maintained
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Verification Plan
|
||||||
|
|
||||||
|
### Tests (Required)
|
||||||
|
|
||||||
|
#### 1. API → Runner Contract Test (NEW)
|
||||||
|
|
||||||
|
* Monkeypatch `ProcessingRunner.execute`
|
||||||
|
* Call `/sdapi/v1/txt2img`
|
||||||
|
* Assert runner is invoked
|
||||||
|
|
||||||
|
#### 2. Existing API Tests
|
||||||
|
|
||||||
|
* Must pass unchanged
|
||||||
|
* Ensures no schema drift
|
||||||
|
|
||||||
|
#### 3. Smoke Tests
|
||||||
|
|
||||||
|
* Full API roundtrip
|
||||||
|
* Ensures server boot + endpoint works
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### CI Signals (Required)
|
||||||
|
|
||||||
|
* Linter ✓
|
||||||
|
* Smoke Tests ✓
|
||||||
|
* Quality Tests ✓
|
||||||
|
* Coverage gate ≥ 40% ✓
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Evidence
|
||||||
|
|
||||||
|
* `M14_run1.md` (PR CI)
|
||||||
|
* `M14_run2.md` (post-merge CI)
|
||||||
|
* Contract test proves routing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Implementation Steps (Small, Reversible)
|
||||||
|
|
||||||
|
### Step 1 — No Routing Changes
|
||||||
|
|
||||||
|
**KEEP existing API code:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
processed = process_images(p) # ✅ DO NOT CHANGE
|
||||||
|
```
|
||||||
|
|
||||||
|
**DO NOT** replace with `runner.run()`. `process_images` is the orchestration boundary; bypassing it would break override_settings, model reload, script callbacks, and extension behavior.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 2 — Add Contract Test
|
||||||
|
|
||||||
|
Create:
|
||||||
|
|
||||||
|
```
|
||||||
|
test/quality/test_api_runner_contract.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Test:
|
||||||
|
|
||||||
|
* Monkeypatch `os.getenv("CI")` to `"false"` so real execution path runs
|
||||||
|
* Monkeypatch `ProcessingRunner.run` to track invocation
|
||||||
|
* Call API method directly (not HTTP) to avoid server startup
|
||||||
|
* Assert runner is invoked when API executes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 3 — Validate Both Paths
|
||||||
|
|
||||||
|
Confirm:
|
||||||
|
|
||||||
|
* `text2imgapi` → `process_images` → runner ✓
|
||||||
|
* `img2imgapi` → `process_images` → runner ✓
|
||||||
|
|
||||||
|
Both already flow through `process_images`; contract test locks txt2img; img2img uses identical pattern.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 4 — Run CI + Verify
|
||||||
|
|
||||||
|
* All tests pass
|
||||||
|
* No diff in outputs
|
||||||
|
* Coverage unchanged or improved
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Risk & Rollback Plan
|
||||||
|
|
||||||
|
### Risk Level: LOW
|
||||||
|
|
||||||
|
Why:
|
||||||
|
|
||||||
|
* `process_images` already delegates to runner (M10)
|
||||||
|
* This is a **routing normalization**, not new logic
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Potential Risks
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
| --------------------------------- | ------------------ |
|
||||||
|
| API bypasses runner accidentally | Contract test |
|
||||||
|
| Subtle response formatting change | Existing API tests |
|
||||||
|
| Extension interaction edge case | Smoke tests |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Rollback Plan
|
||||||
|
|
||||||
|
* Revert API call site change only
|
||||||
|
* No data/model changes required
|
||||||
|
* Single-file rollback (`api/api.py`)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Deliverables
|
||||||
|
|
||||||
|
### Code
|
||||||
|
|
||||||
|
* No API routing changes
|
||||||
|
* New contract test only
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
|
||||||
|
* `test_api_runner_contract.py`
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
* `docs/milestones/M14/M14_plan.md`
|
||||||
|
* `M14_run1.md`, `M14_run2.md`
|
||||||
|
* `M14_summary.md`
|
||||||
|
* `M14_audit.md`
|
||||||
|
|
||||||
|
### Ledger
|
||||||
|
|
||||||
|
* Add M14 row to `docs/serena.md` with:
|
||||||
|
|
||||||
|
* commit SHA
|
||||||
|
* CI run IDs
|
||||||
|
* audit score
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Acceptance Criteria
|
||||||
|
|
||||||
|
### Functional
|
||||||
|
|
||||||
|
* API endpoints produce identical outputs
|
||||||
|
* No schema or contract changes
|
||||||
|
|
||||||
|
### Structural
|
||||||
|
|
||||||
|
* API continues to call `process_images` (orchestration boundary)
|
||||||
|
* All generation flows through process_images → runner
|
||||||
|
* Contract test proves API path invokes runner
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
* Contract test passes
|
||||||
|
* CI fully green
|
||||||
|
* Coverage ≥ 40%
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Architectural Outcome
|
||||||
|
|
||||||
|
After M14:
|
||||||
|
|
||||||
|
### Unchanged (Verified)
|
||||||
|
|
||||||
|
```
|
||||||
|
API → process_images → ProcessingRunner → process_images_inner
|
||||||
|
UI → process_images → ProcessingRunner → process_images_inner
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result:**
|
||||||
|
|
||||||
|
* API → runner flow **provably true** and regression-protected
|
||||||
|
* Contract test locks API path
|
||||||
|
* No behavior change; governance milestone only
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Next Milestone Preview
|
||||||
|
|
||||||
|
### M15 — Background / Queue Runner Preparation
|
||||||
|
|
||||||
|
Will build on M14 by:
|
||||||
|
|
||||||
|
* Introducing async/queued execution
|
||||||
|
* Adding cancellation + lifecycle control
|
||||||
|
* Enabling multi-request orchestration
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ✅ Final Instruction for Cursor
|
||||||
|
|
||||||
|
Implement M14 exactly as specified:
|
||||||
|
|
||||||
|
* Minimal diff
|
||||||
|
* No behavior change
|
||||||
|
* Add contract test
|
||||||
|
* Verify CI
|
||||||
|
* Produce run, summary, audit, and ledger update
|
||||||
|
|
@ -0,0 +1,21 @@
|
||||||
|
# M14 Toolcalls
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Milestone: M14 — API integration (runner contract enforcement)
|
||||||
|
Phase: Phase III — Runner & Service Boundary
|
||||||
|
|
||||||
|
## Actions
|
||||||
|
|
||||||
|
| Timestamp | Tool | Purpose | Files/Target | Status |
|
||||||
|
|-----------|------|---------|--------------|--------|
|
||||||
|
| (start) | write | Create M14_toolcalls.md | docs/milestones/M14/ | done |
|
||||||
|
| | write | Create M14_plan.md | docs/milestones/M14/ | done |
|
||||||
|
| | search_replace | Update M14_plan (verification-only scope) | docs/milestones/M14/M14_plan.md | done |
|
||||||
|
| | write | Create test_api_runner_contract.py | test/quality/ | done |
|
||||||
|
| | run | Create m14 branch, push, open PR | git | pending |
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- No routing changes applied (behavior preserved)
|
||||||
|
- M14 is verification + contract milestone
|
||||||
|
|
@ -0,0 +1,64 @@
|
||||||
|
"""M14 contract test: API txt2img path uses ProcessingRunner.
|
||||||
|
|
||||||
|
Verifies that the API execution path flows through process_images → runner,
|
||||||
|
not direct process_images_inner calls. No routing changes; verification only.
|
||||||
|
"""
|
||||||
|
from threading import Lock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
def test_api_txt2img_uses_runner(monkeypatch, initialize):
|
||||||
|
"""API txt2img path invokes ProcessingRunner when process_images is called."""
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from modules.api.api import Api
|
||||||
|
from modules.api import models
|
||||||
|
from modules.processing import Processed
|
||||||
|
from modules.runtime.runner import ProcessingRunner
|
||||||
|
|
||||||
|
called = {"run": False}
|
||||||
|
|
||||||
|
# Force real execution path (bypass CI early return)
|
||||||
|
monkeypatch.setenv("CI", "false")
|
||||||
|
|
||||||
|
# Patch runner to track invocation
|
||||||
|
original_run = ProcessingRunner.run
|
||||||
|
|
||||||
|
def tracking_run(self, request):
|
||||||
|
called["run"] = True
|
||||||
|
return original_run(self, request)
|
||||||
|
|
||||||
|
monkeypatch.setattr(ProcessingRunner, "run", tracking_run)
|
||||||
|
|
||||||
|
# Mock process_images_inner to avoid full pipeline
|
||||||
|
def fake_inner(proc):
|
||||||
|
return Processed(proc, [], seed=-1, info="", comments="")
|
||||||
|
|
||||||
|
import modules.processing as proc_mod
|
||||||
|
|
||||||
|
monkeypatch.setattr(proc_mod, "process_images_inner", fake_inner)
|
||||||
|
|
||||||
|
# Mock model reload and token merging to avoid model/device ops
|
||||||
|
import modules.sd_models as sd_models_mod
|
||||||
|
|
||||||
|
monkeypatch.setattr(sd_models_mod, "reload_model_weights", lambda: None)
|
||||||
|
monkeypatch.setattr(sd_models_mod, "apply_token_merging", lambda m, r: None)
|
||||||
|
|
||||||
|
# Call API method directly (no HTTP)
|
||||||
|
app = FastAPI()
|
||||||
|
api = Api(app, Lock())
|
||||||
|
|
||||||
|
req = models.StableDiffusionTxt2ImgProcessingAPI(
|
||||||
|
prompt="test",
|
||||||
|
steps=1,
|
||||||
|
width=64,
|
||||||
|
height=64,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = api.text2imgapi(req)
|
||||||
|
|
||||||
|
assert called["run"] is True
|
||||||
|
assert result is not None
|
||||||
|
assert hasattr(result, "images")
|
||||||
|
assert hasattr(result, "info")
|
||||||
Loading…
Reference in New Issue