ProCreations commited on
Commit
1c3912e
·
verified ·
1 Parent(s): 0fdad59

Remove duplicate claim-6 page (merged into the canonical one)

Browse files
pages/claim-6-alpha-tradeoff/page.md DELETED
@@ -1,58 +0,0 @@
1
- # Claim 6: the alpha-dependent tradeoff between drift and stability, with low alpha producing drift and high alpha producing stability
2
-
3
- **Outcome: the 10-D Gaussian-mixture half is now run independently. The previous
4
- revision hash-pinned the paper's Figure 1 panels and stated explicitly that these
5
- were primary-source results, not a rerun.**
6
-
7
- ## Setup — a self-consuming training loop
8
-
9
- `D = 10`, a `K = 4` Gaussian mixture as the true distribution. Generation 0 fits
10
- a GMM to real samples. Each later generation trains on a mixture of **`alpha`
11
- real data** and `1 - alpha` of its own previous output — the setting in which
12
- model collapse is defined. 80 generations, 5 seeds, `n = 150` samples per
13
- generation.
14
-
15
- Error is the distance to the **true** distribution in first and second moments
16
- (both available in closed form for a GMM), not to the previous generation.
17
-
18
- ## Result
19
-
20
- | alpha | moment error, gen 0 | gen 80 | growth | variance retained vs true |
21
- | --- | --- | --- | --- | --- |
22
- | 0.00 | 7.4861 | 70.2140 | **9.38x** | 171.5% |
23
- | 0.10 | 7.4861 | 23.4685 | **3.13x** | 112.3% |
24
- | 0.25 | 7.4861 | 20.9311 | **2.80x** | 107.5% |
25
- | 0.50 | 7.4861 | 21.6991 | **2.90x** | 114.5% |
26
- | 1.00 | 7.4861 | 9.2051 | **1.23x** | 127.1% |
27
-
28
- **The alpha tradeoff reproduces.** With no real data re-injected (`alpha = 0`)
29
- the moment error grows **9.379x** over 80 generations; with full re-injection
30
- (`alpha = 1`) it grows only **1.23x**. Every intermediate `alpha` sits between,
31
- and the ordering is monotone apart from the `0.25` / `0.50` pair
32
- (2.796x vs 2.899x), which is inside the seed-to-seed spread. That is
33
- the qualitative claim: low `alpha` drifts, high `alpha` stabilises.
34
-
35
- ## Where this instantiation differs from the usual collapse picture
36
-
37
- Model collapse is often described as **variance shrinkage**. That is not what a
38
- GMM self-consuming loop does here: variance *inflates*, reaching 171% of the
39
- true trace at `alpha = 0`. The mechanism is component drift — EM on a small
40
- sample spreads the mixture components, and refitting on samples from a spread
41
- model spreads them further.
42
-
43
- This is reported rather than suppressed because it locates the effect precisely:
44
- the degradation is real and strongly `alpha`-dependent, but in this model class
45
- it is driven by drift of the mixture components, not by the variance contraction
46
- familiar from single-Gaussian MLE (where the `(n-1)/n` bias compounds).
47
-
48
- A first attempt used `n = 2000` per generation and saw **no degradation at all**
49
- (variance stayed at 116% of true, errors non-monotone in `alpha`). With that many
50
- samples the refit is a low-variance consistent estimator and per-generation loss
51
- is `~1/n`, so nothing compounds over 25 generations. Collapse experiments need
52
- small `n` and many generations, and that is why the parameters above were chosen.
53
-
54
- ## Scope
55
-
56
- The 10-D Gaussian mixture only. The Fashion-MNIST and CIFAR-10 halves of Claim 6
57
- are not reproduced — they need diffusion-model training runs that are outside
58
- what this reproduction executes, and no image-dataset numbers are claimed here.