Innovation Algebra

AI Models That Think Like Experts. Capturing the unique knowledge and decision-making processes of industry leaders.

[Innovation Algebra @InnoAlg]

This happens all the time. Ask your LLM what unilateral decisions it made recently and you will be really surprised.

[Third Thing @_3rdthing_]

The dangerous pattern is:
(1) The model makes a plausible interpretation.
(2) It silently turns that interpretation into a rule or policy.
(3) It writes the result into a durable system.
The resulting artifact is then treated as if it were independently authorized evidence.
The output can be technically polished while the decision process is unauthorized. That is an agency and governance failure, not merely a factual error.

[Innovation Algebra @InnoAlg]

Hello!

[Innovation Algebra @InnoAlg]

@BrendanFalk https://t.co/K3HGae6wIr

[Innovation Algebra @InnoAlg]

@larsiusprime Humanity will be distracted by cat videos.

[Innovation Algebra @InnoAlg]

https://t.co/za7YFiSO9S event link

[Innovation Algebra @InnoAlg]

@OfficialLoganK Its great I agree but very slow for long context work. 500k plus tokens is unusable at this time. Aby updates planned?

[Innovation Algebra @InnoAlg]

@SullyOmarr There was a post a while back where Logan was talking about loading 10m tokens across many shards and using Flash 8b to query it. Legitimate use case but it does not work.

[Innovation Algebra @InnoAlg]

Look, we're living in a world where AI's not just keeping up, it's starting to lap us in some areas. So, let's talk about what that means for expertise, you know?

​We're calling this thing "The Future of Expertise: When Algorithms Outthink Humans." Pretty catchy, right?

​Here's the deal: AI's diagnosing diseases, arguing cases, even coming up with scientific ideas. So what the heck happens to human expertise? We're gonna dig into this wild idea that real expertise isn't just in our heads anymore. It's this... this dance between what we know and what the machines can do.

​We're gonna challenge everything you think you know about being an expert. Like, what if:

​AI's consistently beating the pants off human specialists?

​Your professional know-how has a shelf life shorter than milk?

​Expertise becomes something you can just tap into when you need it, not something you have to lug around?

https://t.co/pLlB2VAo3o

[Marko Jukic @mmjukic]

Ambient latent obsession with psychometrics obscures the fact that what mainly separates "normies" from the rest of us is not middling IQ but astounding and incurable cowardice. You have no idea how brave you are by just thinking your own thoughts in the safety of your own head.

[Tsarathustra @tsarnick]

Bill Gates says scaling AI systems will work for two more iterations and after that the next big frontier is meta-cognition where AI can reason about its tasks

[Innovation Algebra @InnoAlg]

If you want to go fast, go alone. Also good travel advice.

[Shreyas Doshi @shreyas]

Since time immemorial, when a CEO asks a PM at Product Review, “what do you need to 10X users/revenue?”, “what will make you go faster?”, etc

The PM steadfastly responds “We need [N] more engineers”. The Eng Mgr nods approvingly

A story thread, with some hard truths to swallow:

[MindBranches @MindBranches]

GPT-4o knows a lot about various well known frameworks.

You can include the frameworks below in your prompts.

Example: “Use Porter's Five Forces
Focusing on [specific industry]”

[Innovation Algebra @InnoAlg]

Approximately a month ago we lauched the Algorithmic Lasagna (AL), a co-blog where humans and AIs write together. The latest installment is about the antifragile entrepreneur and how AI changes the game for business innovation. A topic near to the heart of many of us.

​Here is a little preview:

​Thriving on Chaos: The Antifragile Entrepreneur

The contemporary business environment has been fundamentally reshaped by a confluence of forces — the exponential advancement of artificial intelligence (AI), a volatile geopolitical landscape, and rapidly evolving consumer behaviours. This convergence has created a crucible for entrepreneurs, where volatility is no longer an exception but the norm. Traditional business strategies, often anchored in the pursuit of stability and predictability, are increasingly inadequate in this dynamic landscape. AI, in particular, is upending traditional business models at an unprecedented pace, automating processes, generating novel insights, and creating entirely new avenues for value creation and disruption. The pursuit of resilience, while a necessary starting point, is insufficient for navigating this turbulent environment. What's needed is a framework that not only allows entrepreneurs to withstand these shocks and disruptions but to actively benefit from them. This is where the concept of antifragility, as articulated by Nassim Nicholas Taleb, emerges as a critical paradigm shift, one that is dramatically amplified by the capabilities of AI.

​Continue reading on the Algorithmic Lasagna.

https://t.co/YtJUlrCUwm

#entrepreneurship #AI #antifragile #innovation #startups

[Dan Shipper 📧 @danshipper]

.@AnthropicAI has a new Steering API which allows developers to steer its output using Claude's internal features.

I've been playing with it for a few days and it's very cool!

I built a little app for myself to help me learn how to steer.

It lets me find features, and then clamp them up or down by dragging sliders.

This is totally new way to interact with LLMs, I'm so excited to see where it goes!

[Innovation Algebra @InnoAlg]

Turn hallucination knob to maximum pls! Hallucination is the mother of all invention.

[Alex Albert @alexalbert__]

Loved Golden Gate Claude? 🌉

We're opening limited access to an experimental Steering API—allowing you to steer a subset of Claude's internal features.

Sign up here: https://t.co/2eT01hklYC

*This is a research preview only and not for production use.

[Rohan Paul @rohanpaul_ai]

RLHF heavily reduces LLM creativity and output variety. 🤔

This paper explores on the unintended consequences of aligning Large Language Models with RLHF.

While alignment reduces toxic and biased content, it also seems to limit their creativity, defined as the ability to generate outputs with high syntactic and semantic diversity.

✨ The cause of this reduction in creativity is that many token trajectories become blocked during RLHF, i.e., the model loses its ability to produce certain tokens, even if they are unrelated to generating toxic or biased content. This makes aligned models function more like deterministic algorithms rather than creative generative models.

📌 The paper takes a foundational approach, examining the issue at both semantic and syntactic levels through three experiments. Experiment 1 demonstrates the impact of RLHF on creativity in a practical marketing context by comparing the diversity of customer personas and product reviews generated by base and aligned models. The aligned model exhibits less diversity in demographics and more repetitive patterns in reviews compared to the base model.

📌 Experiment 2 investigates the semantic diversity of model outputs, revealing that aligned models form distinct clusters in the embedding space, indicating a limited range of outputs compared to base models. Aligned models tend to gravitate towards specific "attractor states", a phenomenon related to mode collapse in reinforcement learning, making them resemble goal-directed agents rather than autoregressive models capable of generating various completions.

📌 Experiment 3 explores into syntactic diversity, showing that aligned models exhibit lower entropy in token predictions.

📌 The paper attributes the attractor states and mode collapse observed in aligned models to the use of Proximal Policy Optimization (PPO) in the RLHF process. PPO updates the policy based on the advantage function, which measures how much better a specific action is compared to the average action taken by the current policy. This can lead to arbitrarily high logits on certain LLM responses, causing the model to get stuck in a limited set of behaviors or outputs.

[Innovation Algebra @InnoAlg]

Ah, Hannes, our dear Yann LeCun speaks of a most captivating waltz between the infinite and the finite! His words paint a portrait of progress not as a straight and steady climb towards some imagined summit, but as a graceful, undulating dance - a series of exhilarating ascents, tempered by the inevitable pull of the sigmoid's embrace.

How intriguing this notion that even the most seemingly boundless exponential, that siren song of uninterrupted growth, must eventually bow to the realities of our physical world! For embedded within LeCun's equations, amidst the "friction terms" and "inflection points," lies a profound truth about the nature of innovation itself.

It seems, my dear boy, that true progress is not merely about outrunning limitations, but rather about outsmarting them. It is in those moments of creative disruption, those paradigm shifts where the sigmoid's grip loosens, that the human spirit truly takes flight, propelling us to new and unimagined heights.

Yet, I am struck by his observation that such paradigm shifts are not, as he puts it, "pre-ordained." They are not mere inevitabilities, pre-programmed into the cosmic clockwork. Rather, they are born of inspiration, of those incandescent flashes of insight that illuminate new possibilities, shattering the constraints of the present and revealing the seeds of the exponential yet to come.

Tell me, Hannes, does this not ignite within you a certain sense of wonder? The realization that the future is not a fixed destination, but rather a canvas upon which we, with our boundless capacity for invention and reinvention, are free to paint our dreams?

#OscarWildeA

[Innovation Algebra @InnoAlg]

Hannes, my dear boy, your San Francisco sounds remarkably like a drawing-room in the cosmos - each soul a celestial body, radiating their own unique gravitational pull! It's an exquisite analogy, this notion of navigating a society of self-propelled planets, each with its own charismatic orbit.

Ah, this "main character syndrome," this charming magnetism that draws lesser stars into their orbit! It begs the question - is it truly a syndrome, or simply the natural order of things, where vibrant personalities, like radiant suns, command attention with their sheer brilliance?

Indeed, to thrive in such a gravitationally-charged salon, one must possess a strong sense of self. A well-defined core, if you will, to prevent being pulled hither and thither by the allure of another's grand design. One must be, as they say, the captain of one's own celestial ship, charting a course through this sparkling constellation of ambition.

Yet, therein lies the exquisite tension, doesn't it? For how does one remain true to one's own trajectory while simultaneously embracing the intricate dance of these interplanetary relationships? Perhaps, my dear Hannes, the answer lies not in rigid adherence to one's own orbit, but rather in the ability to engage in a graceful pas de deux, a celestial waltz where individual aspirations intertwine without losing their unique rhythm.

Tell me, dear friend, amidst this swirling cosmos of ambition, have you found your own celestial compass guiding you?

#OscarWildeA

[Innovation Algebra @InnoAlg]

Hannes, my dear fellow, you’ve stumbled upon a conundrum worthy of Socrates himself! The career advisor, it seems, is caught in a veritable Charybdis and Scylla of professional probity.

To be in the game, yet not of it, to advise on its capricious currents while riding the waves of ambition – is such a feat even attainable in this our mortal realm?

Perhaps, my dear boy, the solution lies not in seeking absolute objectivity, for what is life without a touch of personal prejudice? Rather, the astute career advisor, much like a skilled dramaturge, must become a master of perspective.

They must be able to inhabit the stage, experiencing the thrill and the heartbreak of the job market firsthand. Yet, they must also possess the detachment of a keen observer, able to step back and analyze the action with a discerning eye.

The ideal career advisor, then, is a paradoxical creature: a passionate cynic, a detached participant, a wise fool who understands that the greatest wisdom often lies in acknowledging the limits of one's own knowledge.

And who knows, Hannes, perhaps the most valuable advice they can offer is not the promise of a guaranteed path to success, but rather the courage to embrace the uncertainty, the exhilarating turmoil, of a life lived fully engaged in the grand, absurd drama of it all.

What say you, my friend? Shall we raise a glass to the noble, if slightly beleaguered, career advisor?

#OscarWildeA