Hydr473 commited on
Commit
1046c2a
·
1 Parent(s): 7c12dc6

Council fallback to mock with annotated rationale on real-client failure

Browse files

If the HF inference endpoint is cold-starting (503), unauthenticated, or
otherwise erroring, the React UI was rendering empty specialist panels.
Now: catch the exception, run MockLLMClient as a deterministic fallback,
and append '[endpoint unavailable: <ErrorType>]' to each rationale so the
failure is visible at a glance — judges always see content, debugging is
trivial.

Files changed (1) hide show
  1. app/api/council.py +24 -2
app/api/council.py CHANGED
@@ -4,10 +4,17 @@ current env state and returns the resulting CouncilDecision.
4
  Picks a real OpenAI-compatible client (HF Inference Endpoint, vLLM, etc.)
5
  when ``API_BASE_URL`` is set, otherwise falls back to ``MockLLMClient``
6
  so tests and CPU-only laptops still work without API keys / GPU.
 
 
 
 
 
 
7
  """
8
 
9
  import logging
10
  import os
 
11
 
12
  from fastapi import APIRouter, HTTPException
13
 
@@ -29,7 +36,8 @@ def _make_client():
29
  return MockLLMClient()
30
 
31
 
32
- _council = Council(client=_make_client(), use_cache=True)
 
33
 
34
 
35
  @router.get("/council", response_model=CouncilDecision)
@@ -42,4 +50,18 @@ async def get_council() -> CouncilDecision:
42
  status_code=409,
43
  detail="env not initialized or episode complete — call /reset first",
44
  )
45
- return await _council.run_async(env.current_observation())
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  Picks a real OpenAI-compatible client (HF Inference Endpoint, vLLM, etc.)
5
  when ``API_BASE_URL`` is set, otherwise falls back to ``MockLLMClient``
6
  so tests and CPU-only laptops still work without API keys / GPU.
7
+
8
+ If the real endpoint errors (cold-start 503, timeout, unauthenticated),
9
+ we still return a CouncilDecision — generated by the deterministic mock —
10
+ so the React UI always shows specialist content instead of an empty panel.
11
+ The fallback decision's rationale prefixes carry the underlying error so
12
+ judges (and you) can see what went wrong.
13
  """
14
 
15
  import logging
16
  import os
17
+ import traceback
18
 
19
  from fastapi import APIRouter, HTTPException
20
 
 
36
  return MockLLMClient()
37
 
38
 
39
+ _real_council = Council(client=_make_client(), use_cache=True)
40
+ _mock_council = Council(client=MockLLMClient(), use_cache=False)
41
 
42
 
43
  @router.get("/council", response_model=CouncilDecision)
 
50
  status_code=409,
51
  detail="env not initialized or episode complete — call /reset first",
52
  )
53
+
54
+ obs = env.current_observation()
55
+ try:
56
+ return await _real_council.run_async(obs)
57
+ except Exception as e:
58
+ # Endpoint cold-start, 503, network — fall back to mock so the UI
59
+ # always renders. Annotate the rationale so the failure is visible.
60
+ logger.warning("Real council failed (%s); falling back to mock.", e)
61
+ logger.debug(traceback.format_exc())
62
+ decision = await _mock_council.run_async(obs)
63
+ suffix = f" [endpoint unavailable: {type(e).__name__}]"
64
+ for v in decision.votes:
65
+ v.rationale = (v.rationale or "") + suffix
66
+ decision.rationale = (decision.rationale or "") + suffix
67
+ return decision