proofread-demo / assets /pipelines /pipeline_dev_v34.yaml
dev-strender's picture
Replace v24-era demo with v34 pipeline demo (engine-vendored bundle)
9c84f9d verified
Raw History Blame
1.68 kB
name: pipeline_dev_v34
description: "v33 + yul_ryul_normalize (์œจ/๋ฅ  ๋ฐ›์นจ ๊ทœ์น™์˜ ๊ฒฐ์ •๋ก ์  ๊ฐ•์ œ, ๋งž์ถค๋ฒ• ์ œ11ํ•ญ). ์–ด์ ˆ ๊ผฌ๋ฆฌ๊ฐ€ ์กฐ์‚ฌ์ด๊ฑฐ๋‚˜ ๋น„์–ด ์žˆ์„ ๋•Œ๋งŒ ๋ฐœ๋™ํ•˜๋Š” ์ธ๋ช… ๊ฐ€๋“œ + exceptions ํ›… ํฌํ•จ โ€” ์žฌ์ƒ ์‹ค์ธก(v33 ์ถœ๋ ฅ ์œ„)์—์„œ ๋ณ€๊ฒฝ 4๊ฑด ์ „๋ถ€ ๊ณจ๋“  ์ผ์น˜ยท๋ถ€์ˆ˜ ํ”ผํ•ด 0 (yul-ryul +1.49, ci-v2 +0.59, ci-v1 +0.58, paragraphยทci-v0 ๋ฌด๋ณ€ํ™”), ์œ ์ผ ์˜คํƒ์ด๋˜ '๊น€์œจํฌ๊ฐ€' ๋Š” ์กฐ์‚ฌ ๊ฐ€๋“œ๊ฐ€ ์ฐจ๋‹จ. rule ์Šคํ…์ด๋ผ latency ๋น„์šฉ 0. ์Šคํ… ์ถ”๊ฐ€๋ผ extends ๋ถˆ๊ฐ€, ์ „์ฒด ํŒŒ์ผ."
text_split_size: 500
steps:
- name: vocabulary_sub_pre
type: rule
rule_type: vocabulary_sub
config:
vocab_dataset: resources/vocabulary.csv
- name: proofread
type: llm
model: ft:solar-news-correction
prompt_key: proofread
max_tokens: 2000
- name: basic_correction
type: llm
prompt_key: step1
user_template: |
<์›๋ฌธ>
{steps.vocabulary_sub_pre}
</์›๋ฌธ>
<๊ต์—ด_๋ชจ๋ธ_์ˆ˜์ •๊ฒฐ๊ณผ>
{current}
</๊ต์—ด_๋ชจ๋ธ_์ˆ˜์ •๊ฒฐ๊ณผ>
- name: context_correction
type: llm
prompt_key: step2
- name: style_correction
type: llm
prompt_key: step3
- name: vocabulary_reapply
type: rule
config:
vocab_dataset: resources/vocabulary.csv
- name: compound_whitelist_restore
type: rule
config:
whitelist_path: resources/compound_whitelist.txt
- name: yul_ryul_normalize
type: rule
config: {}
- name: post_process
type: rule
config:
pronoun_replacements:
์ด์žฌ๋ช… ๋Œ€ํ‘œ: ์ด์žฌ๋ช… ๋Œ€ํ†ต๋ น
currency_compact: true
comma_removal: true
unit_unicode: true