π Titans vs Transformers: ELI5 Explained
Curious how Titans outsmart Transformers in AI memory? π€
π₯ Watch: https://t.co/7oWzQzYfnZ
π Paper: https://t.co/5AWLWaZNFq
π§ Which AI memory model is future-proof?
#AI#MachineLearning#Titans#Transformers#ELI5#OpenSourceAI
here are my notes on @fchollet's neat explanation of the differences between deep learning and program synthesis, and the advantages and disadvantages of each, and how they'd fit together to build AGI.
in deep learning, your underlying model is a differentiable curve; in program synthesis, your model is a discrete graph of operators β youβre picking from a set of operators and structuring that into a program.
this has implications for the amount of compute and data needed for each:
- in deep learning your learning engine is gradient descent, which is very compute efficient β you have a very informative feedback signal about where the solution is. but it's very data inefficient β you need a dense sampling of the data distribution.
- in program synthesis, your learning engine is combinatorial search. this is extremely data efficient (I believe because the problem space is inherently more constrained?), but itβs extremely compute inefficient (because the search space is massive).
how does this apply to AGI? deep learning is great for system 1 thinking; discrete program search is great for system 2 thinking. AGI will likely require a combination of both approaches. Chollet expects that an AGI system would have an outer program that does program synthesis and it will use deep learning to assist it.
Announcing ARC Prize.
A $1M+ competition to beat the ARC-AGI benchmark and open source the solution.
Hosted by @mikeknoop & @fchollet.
https://t.co/TUr6bhwgz6
Hey Peeps π
I wanted to share a trick with you. If you are looking to find the latest GPT's created by people, then search it on Google using the query in image below. You will find tons of new GPT's to try.
#OpenAI#GPT4#buildinpublic
Microsoft's Autogen is blowing up on Github.
It's a framework that allows LLM agents to chat with each other to solve your tasks.
AutoGen agents are customizable, conversable, and seamlessly allow human participation.
It's also a drop-in replacement of openai.Completion or openai.ChatCompletion as an enhanced inference API.
https://t.co/zDzgXzpDVt