# Validation Report

## Scope
This validation report covers the first evidence-based pivot release for the manufacturing vertical.

## Checked in this release
- Typed requirement extraction from pasted JD text
- Typed requirement extraction from LinkedIn job pages
- Requirement-by-requirement evidence mapping from CV text
- Grouped result rendering in the public UI
- Evidence Map rendering
- JD Quality Review rendering
- Manufacturing benchmark scaffolding and starter fixtures
- Product and quality telemetry capture
- New parse endpoints:
  - `/api/jd/parse`
  - `/api/cv/parse`
  - `/api/jd/lint`
  - `/api/analytics/summary`
- Backward-compatible fields still present in `/api/analyze`

## What the release now proves
- The product no longer depends on flat keyword matching as its primary output model.
- Requirements are grouped and typed.
- Each requirement can carry evidence, support level, and confidence.
- The final score is decomposed and secondary.
- The UI exposes uncertainty and JD quality warnings.

## Commands executed successfully
- `npm run lint`
- `npm run build`
- `npm run benchmark:manufacturing`
- `POST /api/analyze` against a pasted Hebrew factory-engineer JD
- `POST /api/scrape` against LinkedIn job URL `https://www.linkedin.com/jobs/view/4398015632/`

## Local runtime checks
- `GET /health`
  - returned `product=Pulse CV - Evidence-Based Hiring Intelligence`
  - returned `scoringMode=evidence-based-deterministic`
  - returned `vertical=manufacturing`
- `GET /`
  - returned HTTP 200 locally
- `POST /api/jd/parse`
  - returned typed requirement groups for a manufacturing JD
- `POST /api/analyze`
  - returned decomposed scoring, evidence map, JD quality warnings, and candidate/recruiter recommendations
- `GET /api/analytics/summary`
  - returned telemetry summary JSON

## Behavior checks confirmed
- Strong manufacturing-fit sample produced:
  - explicit tool matches (`AutoCAD`, `SolidWorks`)
  - explicit methods/domain matches (`Continuous Improvement`, `Quality Management`, `Manufacturing`)
  - decomposed score instead of flat ATS-only output
- Partial-fit industrial leader sample stayed meaningfully lower than the strong-fit case
- Maintenance-heavy sample against manufacturing-engineer JD stayed low-fit and exposed real tool/domain gaps
- Seniority extraction no longer emits a flat `individual_contributor` requirement for non-leadership roles
- Must-have calibration is stricter:
  - role title and years-experience remain mandatory by default
  - tools, education, and language only become must-haves when the JD wording actually marks them as required
  - broad domain/methodology signals are no longer auto-promoted to must-have unless the source line carries explicit requirement context
- AI no longer overrides deterministic recommendations or bullet rewrites
- AI no longer overrides the deterministic profile summary / tailored bio in the main payload

## Remaining weaknesses
- Manufacturing pack is curated and useful, but still compact rather than benchmark-complete.
- Recruiter and hiring manager views are beta-level surfaces, not separate polished products yet.
- AI rewrite quality still depends on provider availability.
- Benchmark scaffolding now exists, but a full human-reviewed corpus is still a next-step task.

## Benchmark harness
- Script: `npm run benchmark:manufacturing`
- Fixtures: `benchmark/manufacturing/cases.json`
- Current benchmark asserts:
  - primary domain detection
  - expected role family where relevant
  - score range guardrails
  - matched requirement expectations
  - missing requirement expectations

## Telemetry added
- Product events:
  - `analysis.started`
  - `jd.parse`
  - `cv.parse`
  - `feedback.correction`
- Quality events:
  - `analysis.completed`
  - `analysis.fallback`
  - `jd.lint`
- Summary endpoint:
  - `GET /api/analytics/summary`