Instructions to use chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020")
model = AutoModelForCausalLM.from_pretrained("chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020

SGLang

How to use chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020 with Docker Model Runner:
```
docker model run hf.co/chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020
```

StraTA / Qwen2.5-1.5B-Instruct / ALFWorld — optimizer step 20

Reproduction of StraTA (Strategic Trajectory Abstraction, arXiv:2605.06642, code xxyQwQ/StraTA @ 592238b) on ALFWorld with Qwen/Qwen2.5-1.5B-Instruct.

This checkpoint: optimizer step 20 of 150.


Optimizer step	20 / 150
Validation success rate at this step	3.57% (ALFWorld `valid_seen`, 140 tasks)
Base model success rate	0.71%
Backbone	`Qwen/Qwen2.5-1.5B-Instruct` (1.54B params, bf16)
Environment	ALFWorld (AgentGym HTTP interface)
Hardware	2x NVIDIA A100 80GB
Wall-clock (full 150 steps)	~44 hours

Training curve

Validation success rate on the full ALFWorld valid_seen split (140 tasks), measured every 10 optimizer steps during training. Step 0 is the untrained base model.

optimizer step	success rate (%)
0	0.71
10	0.71
20	3.57
30	1.43
40	14.29
50	18.57
60	6.43
70	22.86
80	37.14
90	42.14
100	43.57
110	50.71
120	69.29
130	85.00
140	83.57
150	86.43

Final evaluation (step 150, 2 seeds)

Run with evaluate-strata-alfworld-text-qwen2.5-1.5b.sh: val_only, 140 tasks, temperature 0.7 / top_p 0.8 / top_k 20, max 50 env steps.

subtask	seed 1	seed 2	mean ± std
pick	97.14	97.14	97.14 ± 0.00
clean	92.59	92.59	92.59 ± 0.00
pick2	91.67	87.50	89.58 ± 2.95
heat	81.25	81.25	81.25 ± 0.00
cool	76.00	72.00	74.00 ± 2.83
look	76.92	69.23	73.08 ± 5.44
ALL	87.86	85.71	86.79 ± 1.52

Auxiliary metrics: action format_rate 1.000 (both seeds), mean episode length 13.2–14.2 steps (base model: 49.7).

Hyperparameters

All algorithm hyperparameters follow the paper: 150 steps, batch size 16, 4 strategies per task, 8 rollouts per strategy, oversampling ratio sigma=8, aggregation ratio delta=0.5, length penalty threshold lambda=0.5, self-judgment reward weight kappa=0.1.

group	parameter	value
StraTA	`num_stras_per_run` (strategies per task)	4
StraTA	`num_trajs_per_stra` (rollouts per strategy)	8
StraTA	`num_cands_per_stra` (oversampling ratio, sigma)	8
StraTA	`delta_coef` (aggregation ratio)	0.5
StraTA	`lambda_coef` (length penalty threshold)	0.5
StraTA	`kappa_coef` (self-judgment reward weight)	0.1
StraTA	`use_diverse_stras` / `use_self_judgment`	True / True
StraTA	`use_length_reward` / `use_format_reward`	True / True
StraTA	strategy embedding model	`sentence-transformers/all-MiniLM-L6-v2` (CPU)
RL	`adv_estimator`	grpo
RL	`algorithm.gamma`	0
RL	`kl_ctrl.kl_coef`	0.001
RL	`actor.use_kl_loss`	False
RL	`clip_ratio_high`	0.28
RL	`loss_agg_mode`	seq-mean-token-sum
RL	`entropy_coeff`	0
RL	`stepwise_advantage`	enabled, `per_step`
Optim	learning rate	1e-6
Optim	`ppo_mini_batch_size` (global)	1024
Optim	`use_dynamic_bsz` / `ppo_max_token_len_per_gpu`	True / 32768
Data	`train_batch_size` (tasks per step)	16
Data	episodes per step	16 x 4 x 8 = 512
Data	`max_prompt_length` / `max_response_length`	7168 / 1024
Data	env `num_histories`	2
Data	max env steps per episode	50
Rollout	engine / temperature	vLLM 0.11.0 (async) / 1.0
Eval	`val_kwargs`	n=1, temperature 0.7, top_p 0.8, top_k 20
Train	total steps	150 (`total_training_steps=151`)

Infrastructure deltas

This run reproduces the upstream 7B recipe on 2x A100 80GB. A full diff of the resolved Hydra config against train-strata-alfworld-text-qwen2.5-7b.sh shows 6 differing keys out of 533, all infrastructure -- no algorithm or optimization parameter was changed:

key	upstream 7B	this run	effect on results
`trainer.n_gpus_per_node`	8	2	none -- verl computes `ppo_mini_batch_size *= rollout.n` then `//= world_size`, so the global mini-batch is invariant
`actor.fsdp_config.param_offload`	True	False	none -- CPU offload only
`actor.fsdp_config.optimizer_offload`	True	False	none
`ref.fsdp_config.param_offload`	True	False	none
`actor.checkpoint.save_contents`	`[hf_model]`	`[model,optimizer,extra,hf_model]`	none -- what gets written
`trainer.save_freq`	50	10	none -- checkpoint cadence

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chanyoungkim/strata-qwen2.5-1.5b-alfworld-step020"
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)

The model expects the StraTA ALFWorld prompt format: it first emits a plan inside <strategy>...</strategy>, then acts, emitting one action per turn inside <action>...</action>.

Files

path	contents
`model.safetensors`, `config.json`, tokenizer files	bf16 weights, ready for `from_pretrained`. `lm_head` is omitted because `tie_word_embeddings=true` (same layout as the base Qwen release).
`training_state/model_world_size_2_rank_{0,1}.pt`	FSDP-sharded fp32 weights
`training_state/optim_world_size_2_rank_{0,1}.pt`	Adam optimizer state
`training_state/extra_state_world_size_2_rank_{0,1}.pt`	RNG and scheduler state
`training_state/fsdp_config.json`	FSDP layout descriptor

training_state/ lets you resume RL training from this exact optimizer step. Note the shards are written for world_size=2 (2 GPUs, FSDP). Resuming on a different number of GPUs requires resharding. To resume, place the files back as checkpoints/<exp>/models/global_step_20/actor/ and point verl at that directory.

The published model.safetensors is a bf16 cast of the fp32 master weights; the original fp32 tensors are preserved inside training_state/.

All checkpoints from this run

step 10 · step 20 · step 30 · step 40 · step 50 · step 60 · step 70 · step 80 · step 90 · step 100 · step 110 · step 120 · step 130 · step 140 · step 150

Caveats

Do not use is_correct or pass@1 from the logs. Both are hardcoded to 1.0 in the workflow and are meaningless. The correct metric is val/average/metrics/success_rate.
Evaluation uses the StraTA inference protocol (one strategy, then one trajectory conditioned on it; no diversity selection, no self-judgment). This is what strata_workflow.run() does when rollout_engine.validate is set, and it matches evaluate-strata-*.sh.
Numbers here are not directly comparable to models evaluated under a different agent scaffold (different prompt format, context window, or sampling temperature).
The evaluation set is ALFWorld valid_seen (140 tasks, all of them -- not a subsample). valid_unseen was not evaluated.

Citation

@article{xue2026strata,
  title={StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction},
  author={Xue, Xiangyuan and Zhou, Yifan and Wang, Zidong and Tang, Shengji and Torr, Philip and Ouyang, Wanli and Bai, Lei and Yin, Zhenfei},
  journal={arXiv preprint arXiv:2605.06642},
  year={2026}
}