I did a very quick scan of the repo and I don't see any logic for RYS. It's wasteful to load the duplicated layers into vram, can you make it so that the fork repeats the weights programmatically and thus we only use more kv cache usage + compute?
from my testing it quantises alot better. This combination across a probably over a hundred tests was the one that quantises the best. Do I know why? not really., but it seems to work and this model has been pretty trouble free from my personal tests. I have tested to use the same quatisation method without the rys, and the results were alot worse.
What the f*ck are you talking about? This has literally nothing to do with the fact that identical weights are unnecessarily loaded twice in memory. Sorry but you instantly lost any semblance of trust anyone could've had towards your project. What is wrong with you?
ok, I reread your comment and I answered the wrong part before.
You were asking about runtime memory representation, not whether the RYS quantization result was useful.
This release is materialized RYS: the copied window is exported as normal layer tensors, so output layers 20..24 have their own weights copied from source layers 15..19. That keeps the HF checkpoint, GGUF conversion, quantization path, and later fine-tune workflow explicit and stable, but yes, it means those copied weights are loaded like normal weights.
Your suggestion is a procedural/aliased RYS runtime: keep the extra logical layers, but reuse the source-layer weight buffers at runtime. That is a fair optimization idea, and I’ll keep note of it for future models/runtime work.
For the current release I don’t plan to change the representation, because changing the runtime weight layout would require a new implementation path and a fresh verification pass. The current tested release remains the materialized Q4_NL GGUF path.
