# Babelbox: Edge Vision Translator

Babelbox is a real-time, client-side AI application that detects objects via a webcam feed and translates their labels into multiple languages. It leverages `transformers.js` for on-device inference, prioritizing WebGPU for high performance on modern hardware.

## Architecture

The project is designed as a lightweight, zero-build web application:

- **Frontend (`index.html`)**: Manages the UI, camera stream (WebRTC), and rendering detection boxes on an overlay canvas.
- **Worker (`worker.js`)**: Handles heavy AI inference in a background thread to maintain 60 FPS in the UI.
- **AI Models**:
  - **Vision**: `Xenova/detr-resnet-50` (Object Detection)
  - **Translation**: `Xenova/nllb-200-distilled-600M` (NLLB - No Language Left Behind)
- **Acceleration**: WebGPU (via ONNX Runtime Web) with WASM fallback.

## Building and Running

Since the project uses ES Modules and CDN imports, no build step or package manager is required.

### Local Development
To run the project locally, serve the `src` directory using a static file server to satisfy WebRTC and Web Worker security requirements:

```bash
# Using Python
python3 -m http.server 8000

# Using Node.js (npx)
npx serve src
```

Access the app at `http://localhost:8000`.

## Key Technologies
- **transformers.js**: Hugging Face's library for running transformers in the browser.
- **WebGPU**: Modern API for hardware-accelerated graphics and compute.
- **WebRTC**: Real-time camera access.
- **OffscreenCanvas**: Efficient frame processing between threads.

## Development Conventions
- **ES Modules**: Use `import/export` and CDN-based dependencies.
- **Worker Communication**: All inference MUST happen in the worker to prevent UI lag.
- **Color Palette**: Strictly professional slate (#0f1115), surface (#1e2128), and high-contrast blue (#007bff) for bounding boxes. Avoid purple.

## Known Issues / TODOs
- [ ] **Model Persistence**: Consider implementing Cache API or IndexedDB for faster subsequent model loads.
- [x] **UI Polish**: Add a more robust loading progress indicator. (Implemented visual progress bar with real-time worker feedback)

## Memory & Performance Optimizations (Applied)
- **4-bit Quantization (q4)**: Used for both Vision and Translation models to reduce total download size to ~270MB.
- **Transferable Objects**: Zero-copy pixel data transfer to the worker to prevent OOM errors.
- **Canvas Pooling**: Reusable OffscreenCanvas to prevent GC thrashing.
- **Conservative Threading**: WASM worker threads optimized for low memory overhead.