Non-Neural Developmental AgentIn progress
Can an agent learn real concepts through interaction alone, with no neural network anywhere in it? Six experiments, twenty seeds each, negative results included.
I am testing whether a small agent can learn real concepts through interaction alone, using no neural networks, no gradient descent, and no language models anywhere in the architecture.
Not "AGI without transformers". Something narrow enough to be wrong about, and small enough to run on a laptop.
The question
Can a persistent agent, with no gradient training on its reasoning stack, discover a hidden object rule from experience, abstract it away from the specific objects it saw, and apply it to one it has never encountered?
It is never told the label. No "red means food". It gets a raw colour id and an energy delta, and has to find the structure itself.
What the literature says
Plausible, unproven, stuck at the same wall every time.
Substrate independence is theoretically sound and not disproven. But every non-neural program that tried this, the cognitive architectures like Soar and ACT-R, the artificial life work like Tierra and Avida, active inference, open-ended evolution, all hit the same bottlenecks. Representation discovery, combinatorial explosion, symbol grounding, catastrophic forgetting.
They converged on the same failures independently, which is evidence the difficulty is fundamental rather than incidental.
Neural networks are not proven necessary for reasoning or planning. Schema Networks and Bayesian Program Learning beat gradient nets in narrow domains. But they are the best known cheap fix for perception and grounding, and that is the wall.
What counts as success, decided before running
Written down first so it cannot move afterwards.
Fail any of those and it is not discovery. Every prior report in this area agreed on that discipline, and it is the one rule that cannot be skipped.
How it learns without a network
Nothing clever. Explicit statistical structure, all of it inspectable.
Sliding-window averages. A learned Markov decision process solved by value iteration. Count-based novelty for exploration. No weights, nothing to train, and every intermediate state is a table you can read.
Twenty seeds, not one
Every claim runs twenty fixed seeds with bootstrap confidence intervals and Cohen's d.
A single run proves nothing, and neither does a percentage eyeballed off a chart. Baselines run alongside every experiment: random, memory-only, and tabular Q-learning.
Experiment 0: does it discover and transfer
agent survival (mean/std) transfer_rate (mean/std)
random 65.09 / 0.95 0.10 / 0.06
memory_only 69.94 / 0.08 0.00 / 0.00
full 82.16 / 0.65 0.39 / 0.11
learned concept table: {red,EAT: +19.0, blue,EAT: -1.0, green,EAT: -1.0}Pass on all three criteria.
The concept table isolates red at +19 with blue and green flat at -1, so there is no colour leakage. It is doing real feature discrimination rather than accidentally winning on one colour.
The memory-only zero is the point. It has the same data and cannot generalise by design, because it stores exact positions. That comparison is the evidence, not the survival numbers.
An earlier single run swung between 0.07 and 0.47 on transfer alone. Twenty seeds is what resolved that, which is the whole argument for the bar.
Experiment 1: causation or correlation
Flip the hidden rule mid-run. Red stops being good, green starts.
best rule correctly flips red -> green: 20/20 seeds
episodes needed to re-adapt: 145 / 8.9
head-to-head, picks GREEN: 0.51 / 0.08
head-to-head, picks RED: 0.44 / 0.09My first read of that 51 against 44 was that it was thin, barely above a coin flip. That was wrong, and it is exactly the mistake a significance check exists to catch.
green - red diff: 0.073, 95% CI [0.023, 0.123]
CI excludes zero: True
Cohen's d: 0.877 (large)Real effect. Slow to adapt, but real.
Fixing the slowness
The diagnosis was that lifetime averaging never lets old evidence die. Stale belief gets diluted, not removed.
Replaced it with a sliding window of 40.
lifetime avg window=40
episodes to re-adapt 145 / 8.9 25 / 8.9
picks GREEN 0.51 / 0.08 0.64 / 0.08
picks RED 0.44 / 0.09 0.16 / 0.08
Cohen's d 0.877 6.075Six times faster to adapt. Stale red-preference now decays, 0.44 down to 0.16, rather than being outweighed. The effect size went from large to enormous, which says the speed did not cost margin.
Experiment 2: a rule with two conditions
Eat red, but only after picking up a key. A flat colour-only representation should fail here, and it did.
correct-behavior rate:
flat (colour only): 0.26 / 0.09
conjunctive (colour+key): 0.31 / 0.08
random: 0.09 / 0.06
conjunctive beats flat: diff=0.053, 95% CI [0.002, 0.107], d=0.616
flat: {red,EAT: +2.0}
conjunctive: {red|key=True,EAT: +19.0, red|key=False,EAT: -1.0}The flat table sitting at +2.0 instead of +19 is the failure made visible. It is averaging the times it had the key with the times it did not into one number, which is the representational conflation the literature predicted.
The conjunctive table separates them and gets the true values back.
Experiment 3: knowing is not doing
The conjunctive agent had the right representation and still only managed 0.31, because knowing the rule does not tell you to go and get the key first.
flat: 0.28 / 0.11
conjunctive: 0.31 / 0.06
value_iter: 0.64 / 0.09
value_iter beats conjunctive: diff=0.333, 95% CI [0.282, 0.382], d=4.097
Q(yellow, key=False) = 169.1 V(key=False) = 169.0
Q(red, key=False) = 151.1 V(key=True) = 189.0Value iteration doubles it. The key becomes worth collecting because the state where you hold it is worth more, not because anyone said so.
The baseline
agent survival transfer_rate
random 65.1 / 1.0 0.10 / 0.06
memory_only 69.9 / 0.1 0.00 / 0.00
full (ours) 82.2 / 0.6 0.34 / 0.09
q_learning 67.5 / 1.0 0.00 / 0.01Tabular Q-learning transfers at zero. It learns state-action values for states it has seen, and a new red object in a new position is a new state.
That is the difference between a value table and a concept.
Habit
Re-deriving a plan every step is expensive when the plan does not change. Compiling it into a routine after a threshold:
value_iter: 94.7 steps / 0.47 success
habit: 52.8 steps / 0.71 success
diff = 41.9 steps, 95% CI [34.9, 49.1], d=3.57Nearly twice as fast and more successful. The interesting part is that this is what habit is for in the first place, and it fell out of the same machinery.
Perception from noise
Everything above used a hand-given colour id. This one does not.
The agent gets noisy RGB and has to find the categories itself.
agent survival transfer_rate
oracle 82.3 / 0.6 0.39 / 0.13
perception 82.3 / 0.5 0.37 / 0.08
not significant (diff=0.02, d=0.186)
clusters discovered: 7.65 / 1.53 (true number of colours is 3)Self-discovered categories match hand-given labels, with no significant difference.
It finds roughly 7.65 clusters where there are 3 colours, because it fragments each one:
cluster 0: (0.90, 0.10, 0.20) n=1554 red
cluster 1: (0.13, 0.10, 0.89) n=4723 blue, fragment A
cluster 5: (-0.02, 0.08, 0.97) n=965 blue, fragment B
cluster 2: (0.09, 0.92, 0.07) n=4010 green, fragment A
cluster 3: (0.07, 0.76, 0.14) n=945 green, fragment BOver-fragmenting turns out not to matter. Two clusters that both mean blue both learn that blue is bad, and the behaviour is the same.
That was not what I expected. I assumed the cluster count needed to be right.
The one that failed
Phase 6 gave the agent curiosity: go toward what you have seen least.
It scored exactly 0.0. Every seed, zero variance.
That signature is worth recognising, because it is almost never a real result. Zero with no variance means something is broken.
It was. The agent compared unclamped neighbour coordinates against its visit map, so any position off the edge of the grid read as unvisited. Near a wall it always preferred stepping off the board, the environment silently clamped it back, and it never went anywhere.
Fixed the clamping. Ran it again.
Still exactly 0.0.
This time a direct trace confirmed a genuine limit of the mechanism rather than a bug. Spatial curiosity does not solve the gap I built it for.
Both the bug and the negative result are in the record. A clean negative is still a result, and hiding the bug would make the negative look more meaningful than it is.
What is not solved
Grounding is the wall every prior attempt hit, and the perception result above is the smallest possible version of beating it. Noisy RGB is not pixels, and three colours is not a world.
Neural components are not banned. The rule is to reach for one only when a non-neural piece demonstrably fails at something, and to say which piece and which failure.
The rules I work by
tl;dr
An agent with no neural network in it discovers an object rule, transfers it to objects it has never seen, adapts when the rule changes, plans for instrumentally useful actions, and finds its own categories from noisy input. Six experiments, twenty seeds each, one documented negative result.