🔍 Read the full analysis: How To Edit AI Weights At Home Without Legal Consequences on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A published home experiment used directional ablation to reduce looping in a local AI model. On 10 published Lexassay tasks, it answered all 10 rather than 4 in the same setup, but its legal score remained zero, with 55 wrong or invented citations. The results are limited to one run per setting and do not show that editing weights improves legal knowledge.
A home experiment reported by ThorstenMeyerAI.com found that a direct edit to a local AI model’s weights helped it finish legal benchmark tasks, but did not make its answers legally accurate. On the 10 published Lexassay tasks, the edited model answered all 10, up from 4 in the same experimental setup, while its legal score remained zero.
The experiment used Ornith 1.5 35B-A3B, a local model whose publisher describes it as designed for coding and agent work rather than law. The researchers measured an internal direction associated with repetitive output and removed it at half strength from the output weights in 27 of the model’s 40 layers. They say they did not train the model, add legal data or have an AI read the weights as text.
The reported improvement was in task completion, not legal performance. The edited model’s 10 answers met 7 of 50 checked criteria, compared with 3 before editing, while both legal scores remained at zero. The article reports 55 wrong or invented citations across the edited answers. The authors warn that finishing a response is not evidence that its claims are correct.
The authors also report that changing the model’s sampling settings, without editing its weights, made the unchanged model answer 9 of 10 tasks on its original setup. That result complicates the apparent gain from the edit: the comparison depends on the setup, and the published report says the unchanged model answered 8 of 10 in another run. The figures refer to the 10 public tasks, not the benchmark’s five hidden tasks.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, graded blind by three Claude agents; figures cover the 10 published tasks
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
Completion Is Not Legal Accuracy
The experiment illustrates a distinction that matters when local AI systems are used for consequential work: output reliability and subject-matter knowledge are different problems. Reducing repetitive generation may make a model more usable, but it does not supply missing legal knowledge or make citations trustworthy.
For readers considering weight edits at home, the report does not establish a legal safe harbor or assess the legal consequences of modifying model weights. It describes one technical experiment and explicitly says it is not legal advice. The tested model was released under an MIT license, according to the report, but the article does not establish that every model or use is governed by the same terms. Users need to check the applicable model license and other relevant obligations rather than treating this result as legal guidance.
The finding is also a reminder that benchmark numbers need their task count, scoring method and comparison setup. Here, answering more questions did not translate into a nonzero legal score, and a settings change alone also affected completion.
Products Worth Considering
LG UltraGear 32GX850A-B 32-inch 4K UHD OLED Gaming Monitor
DJI Osmo Action 4 Essential Combo Waterproof 4K Action Camera
Jackery Explorer 300 Portable Power Station with SolarSaga 100W Solar Panel
As an affiliate, we earn on qualifying purchases.
The MIT Work That Prompted the Test
The experiment was prompted by a report from four MIT CSAIL researchers: Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Their work describes weight-space programming, in which a general-purpose model reasons about another model’s weights and edits them directly, without a training loop. The source says the researchers’ Qwen example stopped looping and its mathematics score rose from 7.9% to 48.1%.
The home experiment was not a replication of that method. According to the report, Claude Opus 5.5 wrote code to run Ornith, record its internal signals and remove a measured direction. The report calls this directional ablation, a known approach that differs from an AI reasoning over the weights directly. It says OpenAI’s GPT-6 Astra, running in Codex, reviewed the code, results and claims and identified faults; this was an independent review, not a second experiment.
Lexassay tests models on questions about German, US and EU law without internet access or tools, and checks cited laws and court decisions against primary sources, according to the report. Its full benchmark contains 15 questions, with 10 published and five kept hidden. The article’s numbers cover only the published tasks. The authors say the runs and answer-level checks are public on Lexassay.com.
“The model finishes is not the answer is right.”
— ThorstenMeyerAI.com report
Limits of the Published Results
The findings are preliminary and narrow: the report describes one run per setting, a small development set and a loop detector that changed during the experiment. It also notes that three graders came from the same model family, despite the separate review by GPT-6 Astra. These factors make it difficult to tell how stable the results would be across repeated trials or different evaluation methods.
The report gives different completion counts for the unchanged model across setups: 8 of 10 in a published run, 4 of 10 in the comparison setup used for the edit, and 9 of 10 after applying the model card’s recommended sampling settings without an edit. The source says the two local setups are not directly comparable. It does not establish which factor accounts for the difference or whether the edited model’s completion result would hold on the five hidden tasks.
The report does not provide evidence that the edit improves legal competence beyond the small change in checked criteria, and the legal score stayed at zero. It also does not establish whether similar edits would work on other models or whether any particular modification is legally permissible in every jurisdiction or under every license.
Public Runs and Further Testing
The immediate next step available to readers is to inspect the published Lexassay runs, including the answers, grading and citation checks, and compare the edited model with the control and sampling-settings run. The report says the five unpublished benchmark tasks remain hidden to limit tuning against the test set.
Further evidence would require repeated runs under clearly matched settings, stable grading procedures and tests beyond the 10 published tasks. Until then, the report supports a limited conclusion: this particular edit appeared to reduce looping in this particular setup, while the legal answers remained unreliable. The source does not announce a follow-up experiment or a date for additional results.
Key Questions
What did the weight edit change?
The report says researchers removed an internal direction associated with looping at half strength from the output weights of 27 of Ornith’s 40 layers. They describe the method as directional ablation, not direct AI reasoning over the weights.
Did the edit make the model better at law?
The report does not show that it did. The model answered more of the published tasks in the comparison setup, but its legal score remained zero, and the edited answers included 55 wrong or invented citations.
Was this the same method used by the MIT researchers?
No. The source says the MIT work involved a model reasoning directly about another model’s weights. In the home experiment, Claude Opus 5.5 wrote code that measured the model’s internal signals and applied a directional edit.
Does the report show that editing AI weights is legally safe?
No. It reports a technical experiment and says it is not legal advice. The article notes Ornith’s MIT license, but does not determine the legal status of modifying every model or using edits for every purpose. Check the applicable license and seek qualified legal advice for a specific situation.
Source: ThorstenMeyerAI.com
