marv1nz

Applied AI Research

[marv1nz @mxrcelus]

A large part of interpretability in AI Research is built around an implicit assumption:
If we can understand a general model well enough, we can trust it everywhere.
This has led to years of work trying to make general-purpose LLMs interpretable for any user, safe for any task, explainable across any context.
The problem is not lack of effort. The problem is the target itself.
Universal interpretability seems an impossible abstraction.

[marv1nz @mxrcelus]

@kunchenguid Makes sense, the use cases I measured was related to knowledge work.

For coding It was so problematic to use in a first mate style that I didnt even bottered to measure the cost. 😅

[marv1nz @mxrcelus]

@mmmikhaeel I don’t have words to describe… this is so clever and simple, great UX work. 👏

[marv1nz @mxrcelus]

@CompleteSkeptic I can see tones of applications 😍

[marv1nz @mxrcelus]

@kunchenguid Classifier nodes gonna love it!

[marv1nz @mxrcelus]

@rwitoff Finally someone got the point! :)

[marv1nz @mxrcelus]

@ejc3 @corbin_braun Actually… Cursor did it first, it exists before Claude Code and Codex

They just didn’t realize how powerful it was that time

😅

[marv1nz @mxrcelus]

@espindola17848 @MiguelNicolelis O problema não foi resolvido pela AI do tipo “Resolver essa questão aqui blz?”. Maematicos liderando o projeto e como comparar eu pilotando um carro de fórmula 1 x Ayrton Senna. Ambos tem o mesmo carro, a diferença é o piloto.

[marv1nz @mxrcelus]

Non tech people just want their problem solved, not to think about how to build something and maintain it working as it goes. A therapist, want to do their job, not to become a builder, people’s passion outside the bubble are completely different.

People are not the same, everyone can build an own harness, yet, mostly are using someone else’s software like Claude code, codex whatever. (Same analogy applying to the bubble in discussion).

Too much tech thinking on the problems, not enough designers thinking about human needs and behavior. (Wish we had more designers on those discussions)

In summary, I agree with you @kunchenguid

[marv1nz @mxrcelus]

@atonse @matthewcanham Things is that AI is exploding the agile cult and jira looks like was built around these dogmas.

Here we r not looking and task size but blast radius and risk to estimate not time, but human proximity.

[marv1nz @mxrcelus]

2 years ago I’d say that it needs better UX, now I’d say that it needs better Agent Experience. It was 10x easier to create an upstream>downstream autonomous workflow on GitHub than Jira…

But I’d also admit that companies some times contribute for the over engineered workflows 😅

Build the right roads for agents ffs, they r the ones moving tickets now and generating reports now.

[marv1nz @mxrcelus]

I did in an enterprise level. It’s more a service design skill than engineering.. thing is that most service designers cannot understand LLMs and most engineers have no idea of what’s is service designs so people are still thinking that Knowledge work is about to organize docs to better retrieval only.

[marv1nz @mxrcelus]

@mattpocockuk it’s more service design skills than engineering…

That’s why most people thinks that “Knowledge work” is about to structure data to better retrieval only

[marv1nz @mxrcelus]

Exactly! We mapped over 900 process and 5000 micro tasks in 40 areas, no one has no idea of the kind of workarounds people had to do to achieve the goals. This is the game changer when we talk about Knowledge work.

it’s more service design than engineering work

That’s why most people thinks that “Knowledge work” is about to structure data to better retrieval only

[marv1nz @mxrcelus]

Worth mentioning the tacit knowledge and conversations in the elevator for instance, things that are not in a doc. I bet 99% of the companies doesn’t know what the team do. We mapped over 900 process, 5000 micro tasks happens every month, mostly are workarounds to handle the own companies entropy to make things work.

[marv1nz @mxrcelus]

@mattpocockuk And surprisingly, it allows you to understand better than code on how instructions, recall and memory works… Because you focus on LLM only and abstract the code complexity.

[marv1nz @mxrcelus]

@ycombinator Whish it had appeared in dec 2024, I wouldn’t sound like crazy in the company. Happy to kept confidence to pursue this path since then.

[marv1nz @mxrcelus]

We built a conversational commerce agent from upstream to downstream with evals on both parts, at first as skills, than we upgraded to DAGs. It allowed us to spend up the process and validate problem-solution to filter hypothesis to downstream which we could run simulations through real and synthetic traces very gong not just success but how it relates to the intent.

What we learned is that it makes things looks magic, so product judgement is the most important thing even on workflows like that to know when it can or cannot run autonomously.

[marv1nz @mxrcelus]

@tvanhens This is what I was looking for. How can I test it?

[marv1nz @mxrcelus]

@AlexandrePesant @tetrisgm Probably he means something like that https://t.co/BV3971REmH