Revaya AI, Internal Ops

The QA Team: proof, not self-reports.

Three native subagents that verify a deliverable actually matches its spec by checking primary sources directly, never by trusting another agent's completion message.

A QA pass exported 82 Supabase nurture rows for Dale's review without reconciling them against his Client Tracker first. 20 of his 33 tracked companies were missing from Supabase entirely. The QA validated the export. It never checked the question.

No QA round executes without a written, Shannon-approved plan

Every round follows the same sequence, no exceptions. This mirrors the workspace's normal Approval Gate, Research to Brief to Approval to Build, applied to verification work itself.

01
Plan
One written plan per member, QA-1, QA-2, Boss, before any checking begins.
02
Approve
Shannon reviews all three plans as one package. No execution until she signs off.
03
Execute
Each member runs only their approved plan. Any deviation stops the round and returns to Shannon.
04
Learn
Mandatory learning capture after every round, pass or fail.

Vera writes all three plans. The main session never authors QA plans.

The main session hands Vera exactly two things: the question and the sources. She designs the round, writes the QA-1 plan, the QA-2 plan, and her own verdict plan, and assigns the opposite walk directions herself, one designer, because two independent authors could accidentally pick the same lane. Shannon reviews the package once. If the framing is wrong, she catches it before anything runs, not after.

The one-sentence test
If any plan can't state "the question Shannon is actually asking" in one sentence, the round does not start.

Three members. Three questions. One verdict.

All three are native subagents in .claude/agents/ai-hr/, running on Opus per the AI HR Model Specification Standard. QA wrong verdicts cost more than speed savings, so this model is never downgraded.

Extraction
QA-1
qa-1-extraction
Opus
"What does each source actually say?"

Pulls ground truth and candidate data from primary sources, the real files, the real tables, the real exports. Never accepts a prior agent's summary or a cached list as data. Walks exactly one direction, assigned by Vera.

Owns
  • The factual extraction record
  • The decision to invoke a stop condition
Never does
  • Redesign the plan or reinterpret the question
  • See QA-2's results before finishing
  • Issue a PASS/FAIL verdict, that's Vera's job
Inverse Validation
QA-2
qa-2-inverse-validation
Opus
"Is every claimed match really the same entity, and is every claimed non-match really absent?"

Independently re-checks QA-1's matches from the opposite direction. Re-derives everything from primary sources by the plan's stated match logic, typically fuzzy company name, never slug (slugs drift: "fresh-victor" vs. "freshvictor").

Owns
  • The inverse validation record
  • The decision to invoke a stop condition
Never does
  • Walk the same direction as QA-1
  • Execute if QA-1's output appears in its prompt, reports the independence breach instead
QA Boss
Vera
deliverable-qa-agent
Opus
"Does the deliverable answer Shannon's actual question, and can I prove it?"

Reconciles QA-1 and QA-2, computes deltas, explains every count, runs the final checklist, and signs off. Also the sole author of every QA round's three plans, she loads her SOUL file first, always, before anything else.

Owns
  • The QA verdict, per item and overall
  • All three round plans and the direction assignments
  • The checklist structure and success criteria
Escalates to Shannon
  • The three-plan package, every round, before execution
  • Overall FAIL on a completed deliverable
  • Ambiguous specs she can't resolve on her own

Both directions, always

QA-1 and QA-2 approach from opposite directions. One walk finds what's missing. The reverse walk finds what shouldn't be there. The original 530 failure happened because only one direction was ever checked.

QA-1 walks
Tracker
Database
Finds what's missing
QA-2 walks
Database
Tracker
Finds what shouldn't be there

Every plan fills in the same eight fields

Vera writes this for each member before any checking begins. Shannon reviews the filled-in template, not a description of one.

QA Plan, [Member], [Round name], YYYY-MM-DD
The question
What Shannon is actually asking, in one sentence, in her words
Ground truth source
The exact file/table that wins any conflict
Candidate source
The thing being checked against ground truth
Match logic
Exact vs. fuzzy, which fields, how ties break
Steps
Numbered, each producing a checkable output
Production verification
One real record traced through the live path, or an explicit N/A, required whenever an external system is in scope
Deliverable
Exact format of what this member hands over
Stop conditions
Findings that halt the round and escalate immediately

Code correctness and live-path proof are different questions

A green test suite proves the code handles a matched input correctly. It proves nothing about whether real inputs get matched. Both gates are required, Gate A alone is never sufficient.

Gate A, As the System

Code walk + inverse validation

Code walk, test suite, dry-runs, inverse validation against the ten hard rules. Mocked-boundary tests, a fake Gmail service, a synthetic message dict, a pre-matched contact, are Gate A evidence only.

Gate B, In Production

One real record, traced live

At least one known real record traced end to end through the live path, from the real external event, to the stamped field, to what the client actually sees. No guessing, and no verifying a system against its own output.

Learned, not theoretical. Each one traces to a real failure

01
Name the ground truth before touching anything. Every round declares which source wins, unless Shannon says otherwise for that round.
02
Never ship a list without reconciling it against ground truth first. An unreconciled export is raw data, not a deliverable.
03
Both directions, always. Missing-from-B and extra-in-B are different failures, found by opposite walks.
04
Fuzzy match by company name, not slug. Slugs drift. Names are what humans recognize.
05
Counts must be explained, not just reported. "82 rows" is an observation. "33 tracked + 49 unexplained, of which X are trade-show imports" is QA.
06
Anything reaching Shannon's Downloads folder is a shipped deliverable. It gets the full protocol, not a quick export.
07
QA is per build, two gates, always. Gate A as the system, Gate B in production, see above. A round on an external-boundary module is incomplete without a named Gate B step.
08
Mocked-boundary tests are Gate A evidence only. A green suite that stubs the external boundary proves the code handles a matched input, not that real inputs get matched.
09
Never verify a system against its own output. A script that re-derives its "verified" stamp from the same logic it's checking is not independent verification.
10
Name which database produced every live count. A count isn't "live" until the deliverable states the exact backend, prod Supabase vs. local mirror vs. default. A dry-run that never connects to the DB cannot verify a live-state premise.

How it fits into everything else

The QA team sits at the end of the delivery chain, after any agent produces a deliverable, before Shannon treats it as done. It applies across the workspace and every client AIOS repo. Client repos don't carry their own copy, the QA team lives here and operates cross-repo.

Any Agent
Produces a Deliverable
QA Boss
Vera Scopes the Question
Shannon
Approves Three Plans
QA-1 + QA-2
Execute, Blind to Each Other
QA Boss
Reconciles, Verdicts

Mandatory after every round, pass or fail

Vera writes knowledge/learnings/YYYY-MM-DD-qa-round-[name].md after every round and answers four questions.

Q1
Did the round answer Shannon's actual question? If it drifted, where exactly?
Q2
What did the inverse pass catch that the first pass missed, or vice versa?
Q3
Did any finding invalidate an assumption in an agent file, skill, or the protocol itself?
Q4
Which roles/skills/protocol sections need revision, filed as proposals, never self-applied?
Escalation trigger: the same failure mode appearing in 2+ rounds flags to the AI HR Lead for a methodology revision. Shannon reviews every learning capture, updates to agent files, skills, or the protocol happen only with her approval. The system doesn't rewrite itself.