Composo

The LLM performance Lab.

No more testing by vibes. Build high-performance AI apps you can trust.

[Composo @composo_ai]

First proposed in a 2009 paper from Palatucci et al, zero-shot inference - the idea that models could perform on tasks not seen during training - evolved and was popularized in LLMs through generative pretraining with GPT-1 and decoder-only models.

[Composo @composo_ai]

The emergence of LLM prompting was surely a milestone🎯in the democratization of AI. (1/3)

#programming #chatgpt #ai [arXiv 2304.13712 Yang et al.]

[Composo @composo_ai]

LLMs could have easily been powerful, but hard to operate; indeed many of the early models were designed to perform a specific task that was unconfigurable at inference time. Examples are sentiment-analysis, part-of-speech tagging, sentence classifications etc.

[Composo @composo_ai]

kani is one such new framework in this space which provides fine-grained control over the underlying tokens dispatched to the LLM - one to watch…

[Composo @composo_ai]

🧵it feels like today's LLM applications only scratch the surface of what's possible with today's LLMs

we're starting to see empirical results that suggest the more tokens spent working through a problem the more likely the output is to be intelligent
#ChatGPT #programming

[Composo @composo_ai]

the ML community is constantly finding new general abstractions to increase token spend and improve performance: chain-of-thought, self-critique, long-term memory, etc. new frameworks will be key to making these techniques easier to stack without large engineering overhead

[Composo @composo_ai]

instruction finetuning + RLHF + chat = 🔥🔥🔥

#AI #chatgpt #llm #promptengineering [Zhao 2023 2303.18223]

[Composo @composo_ai]

LLMs are the core technology, but it will take years to create the ecosystem application developers need to build the killer app what do you think are the first fundamental technologies to emerge? model triage and tool usage perhaps? 🤔 (3/3)

[Composo @composo_ai]

💾😱 using the chatgpt research preview today is like buying a server and hosting a html webpage in 1992 🧵 (1/3)

#AI #chatgpt #llm #promptengineering [Qin 2023 2304.08354]

[Composo @composo_ai]

📱apps, IM and social media only became possible because of the boom in the mid to late 90s which brought javascript, server-side programming, cookies, and more recently web frameworks, design tools and mobile networks (2/3)

[Composo @composo_ai]

6/6 At @composo_ai, we believe that LLM model evaluation should be an accessible and empowering process. Join us in shaping the future of #ai! 🔥🚀

[Composo @composo_ai]

1/6: 🌊Calling all #AI aficionados! Who's ready for a deep-dive into language model evaluation? We're hitting a high note with 4 game-changing elements! 🚀 #PerformanceTesting

[Composo @composo_ai]

5/6 Last but not least, the varied test cases: Our fourth stepping stone. 🗺️ Imagine real-life scenarios. We have everything, right from user queries to adversarial testing covered for a robust assessment. 🍀