AI & ML
I built an RWKV-7 + KAN adapter that rewrites prose without changing the facts
Mhmad hassn Dev.to (EN Zone)
1 views
What if a rewriting model could make dense prose feel natural without quietly changing the numbers, identifiers, units, or negation that make the text correct?
I built RWKV-7 KAN Humanizer v7.2 to explore that question. It is an Apache-2.0 release built around the RWKV-7 World3 1.5B language model, but the interesting part is not another full fine-tune: it is a small, inspectable adapter that changes how the model rewrites while leaving the recurrent backbone frozen.
To my knowledge, this is among the earliest public open implementations combining a KAN adapter with an RWKV language model. I have not seen many public RWKV + KAN adapter releases, which is exactly why I am publishing the code, weights, configuration, and evaluation notes for people to inspect and challenge.
The problem: “natural” can still be wrong
A rewrite can sound better and still fail its job if it changes:
0.05 into 0.5
p < 0.05 into a different statistical claim
an API identifier, acronym, or model name
a unit or measurement
“not” into the opposite meaning
For technical, research, and operational writing, those are not cosmetic errors. They are correctness failures.
The project therefore treats rewriting as a constrained transformation: improve readability, but preserve the details that carry meaning.
Why KAN inside RWKV?
The upstream RWKV-7 recurrent backbone remains frozen. V7.2 adds a residual Kolmogorov–Arnold Network (KAN) branch inside every feed-forward block:
x ───────────────► frozen RWKV-7 FFN ───────────┐
\► down projection ► KAN B-splines ► up projection ─┤► residual sum
┘
Only the adapter parameters are trained. That gives the model a focused place to learn rewriting behavior without replacing the base model or publishing a second copy of a 1.5B checkpoint. The released adapter is 78.14 MB, and the repository pins the upstream base-model revision used by the quick start.
What I trained and checked
V7.2 was built from a training line whose first-stage foundation contained approximately 100,000 custom text pairs. The release itself specializes the adapter with a smaller, stricter technical corpus focused on long context, numbers, identifiers, units, and negation.
The V7.2 snapshot contains:
3,233 accepted training pairs
321 validation examples
a 1,000-example replay pool
48 old-validation examples
442 quarantined rows removed by conservative quality gates
The recorded eight-case generation gate reported:
Check
Result
Number recall
1.00
Identifier recall
1.00
Negation preservation
1.00
Completed generations
1.00
4-gram copy score
0.462
These are release-run diagnostics, not a guarantee of semantic equivalence for every input. When facts matter, the output should still be compared with the source and reviewed by a human.
What is in the repository?
The GitHub repo includes:
adapter-only weights: model/kan_rwkv7_v7.2_best_adapter.pt
adapter configuration and pinned base-model revision
the KAN B-spline layers and RWKV-7 FFN wrapper in src/kan_rwkv7_adapter.py
architecture and training visuals
an Apache-2.0 license for the release files
a PyTorch + Transformers quick-start example
The upstream RWKV-7 base model is downloaded separately under its own license; base-model weights are not redistributed here.
Quick start
The intended runtime is deliberately close to the training/evaluation setup:
pip install torch transformers
Then load the pinned RWKV-7 base model, attach the KAN adapters, load model/kan_rwkv7_v7.2_best_adapter.pt, and generate with the prompt format and sampling settings shown in the README. Keeping the prompt prefix, EOS handling, tokenizer revision, and sampling configuration aligned matters if you want the closest match to the recorded diagnostics.
Responsible use
This project is for readable, faithful rewriting and research. It should not be used to misrepresent authorship, fabricate evidence, alter safety-critical instructions, or conceal material changes. A humanizer should not become a way to hide what changed.
Try it, inspect it, break it
Repository: rwkv7-kan-humanizer-v7-2 on GitHub
I would especially value feedback from people working on local inference, PEFT, KANs, or evaluation: which preservation failure is hardest in your own rewriting pipeline—numbers, identifiers, negation, units, or something else?
Read original: https://dev.to/moe110/i-built-an-rwkv-7-kan-adapter-that-rewrites-prose-without-changing-the-facts-5e09
Related
Ich habe mein eigenes Repo angegriffen — mein PR-Bot hat den Angriff selbst blockiert
AI & ML
3
DEV Community
A beginner's guide to the Qwen3.8-Flash-Next model by Qwen on Huggingface
AI & ML
1
DEV Community
Translating 300-Page Books with Claude: Taming Token Limits and Context Windows
AI & ML
3
DEV Community
Slop-Creep: When Building Gets Cheaper Than Thinking
AI & ML
2
Dev.to (EN Zone)
Comments0
No comments yet — be the first