---
library_name: keras
tags:
  - security
  - modelscan
  - keras
  - proof-of-concept
---

# Benign Keras Native Safe-Mode Public Function PoC

This repository contains a benign security proof of concept for Keras Native
`.keras` model artifacts. It demonstrates a nested `keras.layers.Lambda` whose
function config points to the public Keras API
`keras.visualization.plot_image_gallery`.

Keras 3.14.1 loads the artifact with `safe_mode=True`. When inference is run,
the model writes a benign marker PNG named
`safe_mode_public_function_marker.png` in the working directory. Protect AI
ModelScan 0.8.8 reports zero issues for the nested artifact, while the included
top-level control model with the same Lambda is detected.

This PoC is non-destructive: it only writes a small local PNG marker.

## Files

- `safe_mode_public_function_nested.keras`: scanner-clean nested PoC.
- `safe_mode_public_function_top_level_control.keras`: control artifact that
  ModelScan detects.
- `verify_poc.py`: loads the PoC with `safe_mode=True`, runs inference, checks
  the marker, and optionally checks ModelScan behavior.
- `results_safe_mode_public_function.json`: captured local runtime and scanner
  evidence.
- `modelscan_safe_mode_public_function.json`: ModelScan output for the nested
  PoC.
- `modelscan_safe_mode_public_function_control.json`: ModelScan output for the
  top-level control.
- `keras_native_safe_mode_public_function_lab.py`: generator and validation
  script used to create the artifacts.

## Reproduction

```bash
python -m venv .venv
. .venv/bin/activate
pip install keras==3.14.1 tensorflow==2.21.0 modelscan==0.8.8 matplotlib==3.10.9 pillow==12.2.0 numpy
python verify_poc.py --require-modelscan
```

On Windows PowerShell:

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install keras==3.14.1 tensorflow==2.21.0 modelscan==0.8.8 matplotlib==3.10.9 pillow==12.2.0 numpy
python verify_poc.py --require-modelscan
```

Expected runtime behavior:

- `keras.saving.load_model(..., safe_mode=True)` succeeds.
- No marker exists immediately after load.
- Calling the model on a small input creates
  `safe_mode_public_function_marker.png`.
- The first model output remains the benign input tensor.

Expected scanner behavior:

- `modelscan scan -p safe_mode_public_function_nested.keras -r json` reports
  zero issues.
- `modelscan scan -p safe_mode_public_function_top_level_control.keras -r json`
  reports a `Lambda` issue.

## Artifact Details

- PoC SHA256:
  `91a95e3ef51cf69dd0e3aa448f66c976bac6bd51152d6bd8189a4fb960906521`
- PoC size: 18,053 bytes
- Control SHA256:
  `a2018f9ebccb565c8e7296659085a24dc41ac9f3a9d48a5af17a28406c2feaee`

## Impact

The model artifact controls a nested Keras Lambda function config that is missed
by ModelScan's shallow Keras Lambda scan. Unlike raw Python lambda bytecode, the
function is a public Keras API and is accepted by Keras safe-mode loading. A
normal inference call then performs a model-controlled filesystem write.

This is not arbitrary code execution. The demonstrated impact is a
model-artifact-controlled file write through a public Keras function, plus a
scanner/runtime mismatch where the nested artifact is reported clean while an
equivalent top-level Lambda is detected.

## Mitigations

- ModelScan should recursively inspect nested Functional/Sequential model
  configs, not only top-level `config.layers`.
- Keras safe-mode deserialization should treat public functions with filesystem
  or network side effects as loading-sensitive when they appear in Lambda
  function configs.
- Consumers should avoid loading untrusted `.keras` artifacts that contain
  Lambda layers, even when `safe_mode=True`, unless the full config has been
  recursively inspected.