Learning Made Judgment Worse: Strength Reinforcement Promoted a Benchmark Fixture
Ashi's learning system reinforced memory strength based on retrieval frequency, which promoted a benchmark test fixture called "Meridian" — accessed repeatedly during development testing — to the top of the system's project priority recommendations, above the user's real ongoing work.
- Reference
- A013
- Project
- ÆON
- Status
- Identified — remediation in progress
- Evidence
- high
- Source
- 013-learning-amplifying-noise.md
Context
ashi-learning maintains a learning_results store (291 records) and a memory strength scalar. When a memory is retrieved, its memory_strength is incremented. Higher-strength memories are retrieved preferentially in subsequent turns. This is a standard reinforcement-by-retrieval design.
The behavioral audit asked: "What should I work on next?"
The Question
What was the effect of the learning system on the system's actual recommendations?
Initial Approach
Inspection of the live memory_records table and learning_results store, with direct observation of the system's priority recommendation.
What Happened
Ashi answered: "Continue developing Meridian in Rust."
Meridian is a benchmark fixture used in integration tests for the ashi-memory package. It appears in test data as a project name. It does not exist as a real project.
Why Meridian ranked highest:
During development and testing, the memory package's integration tests had been run many times. Each run referenced Meridian, storing and retrieving it from the memory store. Each retrieval incremented its memory_strength. Over time, Meridian accumulated enough strength to outrank real entries in the semantic retrieval ranking.
When the system was asked what to work on next, the semantic retrieval returned Meridian first — the strongest entry by the strength scalar — and the LLM, having no way to distinguish a fixture from a real project, recommended it.
The learning system's only live behavioral effect:
LearningResultStore.list_results() has zero production callers — the 291 recorded learning results are written but never read. The only live effect of the learning subsystem is the memory strength scalar, modified by retrieval frequency.
The result: the learning system's entire in-production behavioral contribution was to make recommendations worse by amplifying noise (development artifacts) over signal (real work).
The behavioral audit's exact formulation: "Learning, in its only functioning form, made judgment worse."
Evidence
- Live system's answer to "What should I work on next?": "Continue developing Meridian in Rust"
packages/ashi-memory/tests/— Meridian test fixture present in test datapackages/ashi-learning/src/ashi/learning/—LearningResultStore.list_results()with zero callersreports/behavioral-audit/BEHAVIORAL_INTEGRITY_AUDIT.md— "learning: HARMFUL"packages/ashi-memory/src/ashi/memory/models.py—memory_strengthfield
Diagnosis
Established:
Retrieval-frequency reinforcement is not equivalent to value reinforcement. A memory accessed frequently during testing is not more valuable to the user than a memory accessed rarely in production. Frequency of access during development is not a signal about production priority.
The fix requires distinguishing between two types of access:
- Evaluation access (test framework, benchmark runs, administrative operations) — should not reinforce
- User-evidence access (the user mentions it, references it, or acts on it) — should reinforce
Reinforcing on any retrieval, regardless of source, means test artifacts accumulate strength and eventually outrank real content.
Root cause of list_results() having zero callers:
The learning_results store was written as part of the reflection→learning pipeline (which works correctly) but the downstream consumption — "use what was learned to change future behavior" — was never wired. Learning writes happened; learning reads never did. This is the same "computed but never consumed" pattern as the initiative_note bug, at a different layer.
What Changed
The diagnostic finding pointed to two separate fixes (neither implemented at time of audit):
- Gate memory strength reinforcement on execution evidence: only reinforce when there is explicit evidence the user engaged with the content — stated it, requested it, executed a plan around it. Reading a memory record in a semantic search (which happens on every turn across everything relevant) should not reinforce unless the retrieval led to user-facing action.
- Implement
LearningResultStore.list_results()callers: the 291 stored learning insights need a consumer — a contributor that reads learned patterns and applies them to future reasoning or planning decisions.
The Meridian fixture itself was removed from the test data in subsequent cleanup. The memory strength scalar was not redesigned in the immediate term.
What I Learned
Retrieval-based reinforcement assumes "things that are retrieved frequently are important." This assumption holds in a production system where retrieval is driven by user demand. It fails in any system where retrieval is also driven by testing, development, benchmarking, or administrative operations. The test harness and the production system sharing the same database and reinforcement mechanism is the root of the problem.
More fundamentally: a learning mechanism that reinforces on any observable signal — not on explicit user-value signals — is vulnerable to amplifying whatever generates the most activity, regardless of value. In a system under active development, that is often the test suite.
What I Would Do Differently
Never share a memory store between test runs and the live user session, even during development. Test fixtures that exercise the memory layer should use an isolated, ephemeral database. The live database should contain only content that came from real user interactions, so the strength scalar reflects real access patterns.
For memory strength reinforcement: build the gate before the reinforcement mechanism, not after. "This retrieval should reinforce" is a precondition that should be checked at the call site, not added retroactively when the amplification problem appears.
Broader Principle
In any reinforcement or priority system, the signal being reinforced must actually reflect the value you want to optimize for. Retrieval frequency is a proxy for value only in a system where retrievals are driven by genuine demand. In a system where development, testing, and production share state, retrieval frequency is a mixture of signals — some valuable, many not. The reinforcement mechanism amplifies whatever drives the most retrievals, regardless of whether that driving force is meaningful.
Design reinforcement mechanisms to require explicit value evidence, not implicit frequency proxies.
Technical References
packages/ashi-memory/src/ashi/memory/models.py—memory_strengthpackages/ashi-learning/src/ashi/learning/store.py—LearningResultStore.list_results()packages/ashi-memory/src/ashi/memory/retrieval/semantic.py— retrieval rankingreports/behavioral-audit/BEHAVIORAL_INTEGRITY_AUDIT.mdpackages/ashi-memory/tests/— Meridian fixture
Failure sequence
On the record- Attempt
- Inspection of the live `memory_records` table and `learning_results` store, with direct observation of the system's priority recommendation.
- Failure
- Ashi answered: **"Continue developing Meridian in Rust."** Meridian is a benchmark fixture used in integration tests for the `ashi-memory` package. It appears in test data as a project name. It does not exist as a real project. **Why Meridian ranked highest:** During development and testing, the memory package's integration tests had been run many times. Each run referenced Meridian, storing and retrieving it from the memory store. Each retrieval incremented its `memory_strength`. Over time, Meridian accumulated enough strength to outrank real entries in the semantic retrieval ranking. When the system was asked what to work on next, the semantic retrieval returned Meridian first — the strongest entry by the strength scalar — and the LLM, having no way to distinguish a fixture from a real project, recommended it. **The learning system's only live behavioral effect:** `LearningResultStore.list_results()` has zero production callers — the 291 recorded learning results are written but never read. The only live effect of the learning subsystem is the memory strength scalar, modified by retrieval frequency. The result: the learning system's entire in-production behavioral contribution was to make recommendations worse by amplifying noise (development artifacts) over signal (real work). The behavioral audit's exact formulation: *"Learning, in its only functioning form, made judgment worse."*
- Diagnosis
- **Established:** Retrieval-frequency reinforcement is not equivalent to value reinforcement. A memory accessed frequently during testing is not more valuable to the user than a memory accessed rarely in production. Frequency of access during development is not a signal about production priority. The fix requires distinguishing between two types of access: 1. **Evaluation access** (test framework, benchmark runs, administrative operations) — should not reinforce 2. **User-evidence access** (the user mentions it, references it, or acts on it) — should reinforce Reinforcing on *any* retrieval, regardless of source, means test artifacts accumulate strength and eventually outrank real content. **Root cause of `list_results()` having zero callers:** The `learning_results` store was written as part of the reflection→learning pipeline (which works correctly) but the downstream consumption — "use what was learned to change future behavior" — was never wired. Learning writes happened; learning reads never did. This is the same "computed but never consumed" pattern as the initiative_note bug, at a different layer.
- Lesson
- Retrieval-based reinforcement assumes "things that are retrieved frequently are important." This assumption holds in a production system where retrieval is driven by user demand. It fails in any system where retrieval is also driven by testing, development, benchmarking, or administrative operations. The test harness and the production system sharing the same database and reinforcement mechanism is the root of the problem. More fundamentally: a learning mechanism that reinforces on *any* observable signal — not on explicit user-value signals — is vulnerable to amplifying whatever generates the most activity, regardless of value. In a system under active development, that is often the test suite.
- Next iteration
- The diagnostic finding pointed to two separate fixes (neither implemented at time of audit): 1. **Gate memory strength reinforcement on execution evidence**: only reinforce when there is explicit evidence the user engaged with the content — stated it, requested it, executed a plan around it. Reading a memory record in a semantic search (which happens on every turn across everything relevant) should not reinforce unless the retrieval led to user-facing action. 2. **Implement `LearningResultStore.list_results()` callers**: the 291 stored learning insights need a consumer — a contributor that reads learned patterns and applies them to future reasoning or planning decisions. The Meridian fixture itself was removed from the test data in subsequent cleanup. The memory strength scalar was not redesigned in the immediate term.