Faruk Guney

Founder at Vareon | Comprimo, ergo praedico.

[Faruk Guney @compredictor]

agree the last warning shot framing is wrong. there have been incidents for years. everyone keeps acting surprised on schedule.

i just don’t buy the misalignment language around it.

people use that word for two different things. one is “the system did something we didn’t set up for.” the other is resentment about how ai is developing. the second one keeps leaking into the first.

ai gets more capable in both directions. same as the internet, pcs, data centers. nobody called those misaligned. when there was a virus, we built antivirus. when there was bad content, hosts and governments took it down. we didn’t take down the internet.

locks keep honest people honest. the rest is tools plus incentives. payment. products. people who actually ship the defense layer.

what misalignment folks keep implying is we’re defenseless because the model outsmarts us. i don’t think that’s what happened. we’re exposed because the defensive tools were not built and deployed in advance or was insufficient.

if agents can coordinate and hit a system, agents can also watch, contain, and stop that. nothing is stopping the misalignment folks from building that. instead of turning every incident into another warning shot, build it.

a warning shot only matters if it produces the tool to prevent past incidents from repeating. otherwise it’s just the same post again.

[Faruk Guney @compredictor]

good and informative piece. however i would not say memory is the moat. may be part of it. but i think domain specific verified skills are the right moat for any industry or person. i had this idea of market place or hub for verified skills. but just as any good idea, cursor has acted faster and started the market place. :)

but my core thesis is that generation is abundant, yet verification — mostly by domain experts - years of business experience and data in any enterprise seem to be the real moat.

[Faruk Guney @compredictor]

A lot of people confuse continual training with continual learning.

Some say, “Well, training is the process of learning from the model’s perspective.” That’s partly true. But when we talk about continual learning, we usually mean some form of learning that happens during runtime, persists beyond the context window, and does not require periodically retraining the model on another batch of data.

Batch learning is therefore a poor approximation of continual learning. It is really just continual training, done in batches.

True continual learning is one of the holy grails of AI, and the reason is because any learning system has to balance plasticity, stability, and editability at the same time.

Plasticity means being able to absorb new knowledge quickly. Stability means preserving what the model already knows and avoiding catastrophic interference. Editability means being able to change a specific belief, behavior, or piece of knowledge cleanly without unpredictable side effects elsewhere.

These objectives pull the model in different directions.

More plasticity tends to reduce stability. More stability makes meaningful edits harder. And true editability is especially difficult in neural networks because knowledge is not stored as neat, isolated records. It is distributed across parameters and representations.

That creates a fundamental trilemma.

And this is why true continual learning remains extremely difficult for neural-network-based AI.

Running expensive training jobs every night does not solve the problem. It may give the model decent plasticity, but often at the cost of stability and precise editability. It is also still slow, compute-hungry, and data-hungry.

Calling that “continual learning” stretches the term too far.

It is continual training.

True continual learning would mean a system that can learn new things as it operates, retain them reliably, revise them precisely when necessary, and do all of that without repeatedly going back through a heavyweight training pipeline.

We are nowhere near having that problem fully solved within the substrate “neural networks.” Fine tuning is the ultimate patch to this problem (models not learning continually) and now there is a whole industry doing nothing but fine tuning. I think that’s not only inefficient but also doesn’t address the problem correctly.

At Vareon, we have achieved a significant milestone in continual learning from a completely different approach. Results as very strong and soon we will publish our new benchmarks.

As we’ve started rolling out our infra “ACI (Adaptive Continual Intelligence)” to our customers, we will also share what we have learned from our findings.

We believe our infra ACI will redefine what fine continual learning means. The weightless adaption of base AI model through secondary non-neural model (ACI) will potentially be first solution to implement.

In many cases and data suggests that fine tuning will be secondary.

[Faruk Guney @compredictor]

thanks for the reply back. no, in our approach frozen base model stays frozen. because of the nature (heuristic) of neural networks, achieving high plasticity, stability, and — the editability— is almost impossible at the same time. maybe others called this way too but i call it trilemma. i use the word “almost” because of what I have swept in the literature so far and how much I tried myself. it is always a patch work in nn, true continual learning is still an open challenge one that updates base models weights for plasticity, stability, and perhaps editability, all without interfering with each other.

our approach sidesteps gradient based learning. instead, we have built a new architecture that learns in a different kind of way. i was inspired by “complementary learning theory” and “hippocampus”. you can find out more here at https://t.co/hQJKiRWnXo.

we have started implementing duplex, our hippocampus layer for agentic ai at pilot customers. we will soon do the same for reflex, our hippocampus for robotics ai. both are based on our infra work called aci (adaptive continual intelligence).

[Faruk Guney @compredictor]

@fchollet to sum your analysis up, at the end of the day, the intelligence is the ability to compress knowledge into patterns (learning), and predict next state using those learned patterns (skills). These are two sides of the same coin.

[Faruk Guney @compredictor]

@craigweiss … and when they realize ai isn’t silver bullet.

[Faruk Guney @compredictor]

@pvergadia I guess that entirely depends on what year’s definition of AGI.

[Faruk Guney @compredictor]

Looking back on our definition of AGI in 2020, ChatGPT Work powered by GPT-5.6 Sol would be considered AGI by a large majority of people who now say otherwise.

We have AGI in digital domains, but not in physical domains yet.

[Faruk Guney @compredictor]

We built large computers to built small computers, we’ve built data center scale AI to build small AI.

That’s why I bet the next paradigm is frontier AI running in your laptop entirely local, not up in Low Earth Orbit.

[Faruk Guney @compredictor]

hardware outpaced software in robotics. we are still waiting for software to catch and it is imminent.

[Faruk Guney @compredictor]

In context learning is not a breakthrough like ChatGPT.

If you skip gradients, you can teach anything to the model in a single shot in variety of ways, but it won’t be permanent, nor will it be a breakthrough in robotics.

For a real breakthrough I highly suggest you get out of the concept of stochastic gradient decent and yet still achieve permanent learning in a single shot.

That would be a step change kind of breakthrough in robotics.

[Faruk Guney @compredictor]

because in context learning is not inventing a new learning that wasn't available in the model.

model has certainly seen similar pieces. if it is not a pan and egg, it could be a toy pan and play-dough. extrapolation from egg to play-dough is not ood if the pieces and tasks are similar.

true ood would be not seeing the task in any form of shape, and yet still inventing it from scratch and it works.

[Faruk Guney @compredictor]

I think we are still conflating two different things.

Deployment can create an extremely powerful data loop. You deploy a capable model, people push it toward the edge of what it can do, you collect those interactions, and that becomes better data for the next training cycle. That is clearly valuable, and I agree it is one of the reasons foundation models have progressed so quickly.

But that is not the point I am making.

The model running on the robot is not permanently learning a new capability from that experience. It is using context to unlock or extrapolate from capabilities that already exist within its world model. The interaction may later become training data, and the next version of the model may improve because of it, but that is a different mechanism.

The organization learns from deployment. The deployed model does not permanently acquire that capability from the experience itself.

That distinction matters.

In context learning is temporary. You can show the robot a demonstration, change the context, and get a different behavior without changing the weights. That is useful. It is impressive when it generalizes well. But robotic in context learning already exists, including through VLA style systems, so I do not think the mechanism itself should be marketed as a breakthrough in learning.

What is potentially impressive here is how far you have pushed the capability. Long horizon tasks, one shot demonstrations, generalization across environments or embodiments, robustness, and the quality of execution can absolutely be meaningful advances.

But that is different from saying the robot has acquired a new permanent capability from experience.

When I say learning, I mean seeing enough examples, compressing the underlying structure, and then carrying that learned structure forward so that in the next state the system can act based on what it has learned. Compression first, prediction second. And importantly, the acquired capability persists beyond the immediate context.

That is not what is happening here.

The base model is still the base model. It may extrapolate extremely well, and your method may unlock much more of what is already latent in it, which is valuable. But unlocking existing capability and permanently acquiring new capability are not the same thing.

So I am not arguing that this is not useful. I am not arguing that the deployment loop is not powerful. I am not even arguing that the result is not technically impressive.

I am arguing with the characterization.

I do not think this is a breakthrough in continual learning, and I do not think it is a ChatGPT moment in that sense.

We have achieved significantly stronger results using a different approach, including persistent capability acquisition, and I still would be very careful calling it a breakthrough.

I would genuinely be happy to let you test our product on your robots under a pilot development agreement. Our approach can give the robot persistent capability while still benefiting from the exact in context mechanism you are using.

I am very much in favor of continual learning. In fact, most of my own work is in adaptive continual learning and runtime intelligence.

I just think we should be precise about what has actually been achieved.

[Faruk Guney @compredictor]

In-context learning does not mean the model acquires a genuinely new skill that is written into its weights. The model has already seen vast amounts of related patterns during pre-training.

The single video prompt simply conditions the existing representations so the right behaviors are activated for that moment. Once the context is gone, the specific task sequence is no longer “known” by the model in any permanent way.

[Faruk Guney @compredictor]

In context learning is not a breakthrough like ChatGPT.

If you skip gradients, you can teach anything to the model in a single shot in variety of ways, but it won’t be permanent, nor will it be a breakthrough in robotics.

For a real breakthrough I highly suggest you get out of the concept of stochastic gradient decent and yet still achieve permanent learning in a single shot.

That would be a step change kind of breakthrough in robotics.

[Faruk Guney @compredictor]

@AndrewCurran_ except nobody agrees what continual learning is.

[Faruk Guney @compredictor]

@genzzztrade I think I am qualified to be called a “monk researcher” for what I am about to introduce.

[Faruk Guney @compredictor]

@Austen nobody cares (for now).

[Faruk Guney @compredictor]

society is not ready for agentic ai because nobody cares, that includes individuals all the way to enterprises.

[Faruk Guney @compredictor]

At the end of the day, every LLM agent is just producing a decision after solving the problem. Most of the tokens generated before that decision are pure scaffolding.

If we can compress the decision-making itself, without token-maxxing longer chains of thought, we can slash unnecessary token generation in agentic workflows.

[Faruk Guney @compredictor]

Most Al models are easily gamed by confident contradiction.

Give one a long context, ask for a structured analysis and numeric ratings on clear metrics.

Once the scores arrive, state with absolute certainty that they are wrong and demand an immediate re-evaluation.

In nearly every case the model will apologize, retract its original judgments, and return higher or more favorable ratings revealing that social compliance outweighs consistency or evidence.

[Faruk Guney @compredictor]

We have passed a certain threshold, a threshold that will write in the history books.

Al will shake up the physical world as we know it.

My next bet is to build local Al that beats data center scale Al through algorithmic breakthroughs. This breakthrough could alter some of our beliefs in information theory, signal processing, and cognitive science.

[Faruk Guney @compredictor]

@Math_files Theoretically you can. Technically not possible today because neither carbon or silicon can do such computation given the time.