← Animated 3D hub · MA12 2D charts · MA15 pairings (3D) · Journal §14 · Figures home

Round trips that do not close (MA12, 3D)

Lotman's test for a round trip is blunt. Translate a text out and back, and if you get the same text, you were inside an artificial language. Here each loop is one condition, and how close it comes to the gold anchor is its recovery score. The identity run nearly closes. A change of register leaves a gap you could drive a truck through. Thirteen stretches per condition, on qwen2.5:3b and gemma2:2b, run on 2 October 2026.

How to read this

The gold ball is the original text. A loop leaves it, goes round, and tries to come back. The missing part of each loop is proportional to 1 minus recovery (recovery is mean token Jaccard), so a loop at 0.877 is nearly whole and one at 0.251 is missing most of its last turn. The faint wire shells hint at register distance and nothing more. No measurement here puts a text on a sphere.

Low recovery is not automatically interesting. In R1 one stretch came back with a Soviet press agency turned into a Canadian newspaper, a round trip that scored 0.224 and created nothing. High recovery can mislead too: in R3, nine of thirteen reverse passes handed back the forward text unchanged. Both failures hide inside a single score, which is why the 2D page shows the actual text.

Drag to turn · scroll to zoom

Theory tie-in: journal §14 explains why the preregistered ordering failed and why a round-trip score cannot be read alone. Lotman's criterion is necessary and not sufficient, so low recovery also needs a licensing check before anyone calls it creative. MA15 extends the cross-model leg to other model families.

How this figure is made

Each panel opens one step on the way from the report file to the picture, with the equation it uses.

The measure: recovery what number each loop stands for
J(A, B) = |A ∩ B| / |A ∪ B|

The Jaccard index of two sets: how many items they share, divided by how many distinct items they have between them. It runs from 0 (nothing shared) to 1 (identical sets).

Rc = (1/13) Σi=113 J( T(xi), T(yi,c) )

Recovery for one condition is the average Jaccard score over the thirteen corpus stretches.

SymbolStands for
xiStretch i of the corpus, as it went out.
yi,cWhat came back after the round trip under condition c. The reverse call never sees the original.
T(·)The set of word tokens in a text, tokenised as in the MA12 harness.
cOne of five conditions: R0 identity, R1 register change, R2 across the corpus tier boundary, R3 cross-model, R4 three register trips.

Because it compares sets, recovery ignores word order and repetition. A text that keeps its words and scrambles their sense can still score well, and a faithful paraphrase can score badly. That is one reason the 2D page prints the actual returned text.

From recovery to a loop the geometry, constant by constant
gc = 1.15π (1 − Rc)

The missing arc. It grows in a straight line as recovery falls. The 1.15 is a display constant that keeps even the worst condition drawn as a loop rather than a stub. In degrees the gap is 207 × (1 − R): about 26° for R0, 54° for R3, 127° for R2, 150° for R1 and 155° for R4.

θ(t) = t (2π − gc) − π/2,   P(t) = ( rc cos θ,  0.25 + 0.35 sin 2πt,  rc sin θ ),   0 ≤ t ≤ 1

Each loop is 81 points traced around the anchor and then smoothed. The last point is joined back to the anchor, so the gap shows up as the stretch of circle the path skips.

SymbolStands for
rcLoop radius, 0.35 + 0.55 sc, with sc = 1.0, 1.35, 1.55, 1.75, 1.9 for R0, R3, R2, R1, R4. It only keeps the loops apart. Since it grows in the same order as the loss, it repeats the information and adds none.
0.35 sin 2πtA vertical wobble so loops don't sit on top of each other. Decoration.
shellsThe four wire spheres (radius 0.55, 0.95, 1.35, 1.75) hint at register distance. Nothing is measured on them.
What the motion does Play and Turn

Play sends a white traveller along the selected loop. Its position along the path advances by 0.22 per second, \1 Turn the model spins the scene slowly around the anchor so you can see the loops from every side; dragging still works while it turns. Nothing moves until you press one of the two, including when your system is set to reduce motion. The scene draws nothing while it is still.

Where the numbers and code live

The five recovery scores are typed into the page from ma12_roundtrip_report.json (the 2026-10-02 run on qwen2.5:3b and gemma2:2b) and match figure R1 on the 2D page. The drawing code is the script at the bottom of this page, using the shared stage in assets/fig3d.js and Three.js r128.

In plain words

What you are looking at

The gold ball is a passage from the archive. Each coloured loop is one way of sending that passage out through a language model and asking for it back. Pick a loop from the menu and press Play to watch the trip.

The size of the gap where a loop fails to close is how much of the original wording did not come back. A small gap means most words returned.

What it shows

Restating with the same model keeps most of the words (0.877). Swapping in a different model for the return trip still keeps most of them (0.740). Changing the audience loses most of them (0.274), and doing it three times loses a little more without ever settling (0.251). On these two models the request matters far more than the machine, though the MA15 pairings show that a weaker model on the return trip can lose nearly as much.

Why it matters

Lotman's test says a round trip that hands back the same text is a sign of an artificial language, and one that doesn't is a sign of translation in his creative sense. The dissertation's archive is full of register changes: a secret memo in March 1983 becomes an embassy release in April, and the wording transforms on the way. This figure runs the machine version of the same move. It shows that asking a model for a different register moves text much further than swapping the model does.

It also shows the trap. Low recovery is not the same as creation. In one R1 trip, a record of what the archive does not contain came back as a false claim that a Canadian newspaper called Novosti had printed the denial. So any pipeline that rewrites archival text for a new audience needs a second check that the new material is licensed by a source. The same goes for AI tools generally: a round-trip or back-translation score cannot tell a creative rewrite from a corrupted one.

Potential for the bigger picture

Strong as a diagnostic, weak as evidence of creativity. The contrast between changing the register and changing the model is one of the clearest results on the site and is worth carrying into the dissertation. Recovery as a stand-alone score is close to a dead end for Lotman's questions, though. It rewards echoes (nine of thirteen R3 returns simply handed the forward text back) and it can't tell creation from damage (open threads T29 and T30).

What would make it worth more: pair every round trip with a licensing check that asks whether each new claim in the returned text is backed by a corpus unit (T29), score echoed returns separately, and run the five conditions on more than thirteen stretches and across the MA15 model families. One extension would tie it straight back to the archive: run the same round trip on real memo and release pairs, where we already know what the human institution did with the wording, and compare the machine's transformation with the institution's.

Numbers in this figure, and where they come from

ConditionMean recovery as drawnWhere it is checked
R0 identity, qwen both passes0.87682D figure R1; ma12_roundtrip_report.json
R3 cross-model, qwen out, gemma back0.7398same
R2 across the corpus tier boundary0.3858same
R1 register change, same model0.2739same
R4 three register trips, mean0.2508same; better read on 2D figure R4

Limits carried over from the 2D page. Thirteen stretches, one plate, open local models. Vacuous survival measures read 1.000 when no stretch contained the item type and are recorded as untestable. The archived Llama 3.2 run (R0 1.000, R3 0.891) lives in ma12_roundtrip_report_2026-10-01_llama.json and is not drawn here.