Saikat Mitra

Building for AI engineers - @empiricalrun

[Arjun Attam @arjunattam]

✨ just shipped: Appwright

we're building AI agents for software QA (more on that soon!) – and we saw how painfully e2e testing for mobile apps can be

to solve this, we built a new test framework that combines the best of @AppiumDevs and @playwrightweb — and it's called Appwright

https://t.co/5bb9uyxvp6

[Saikat Mitra @Phsychi]

@dharmendrakumar @narendramodi @IncomeTaxIndia @FinMinIndia even though if you are lucky to fill the form without any interruptions .. you will get stuck on the final submission screen.

[Saikat Mitra @Phsychi]

@Dinosn Given the number of spam calls I get, I thought @airtelindia's customer data is already public 😏

[Arjun Attam @arjunattam]

the "hype" is coming in the way of a foundational understanding of what LLMs are, and what a developer needs to know about them

to fix this, I've put together a pragmatic intro guide—something I wish I had a year ago

[Arjun Attam @arjunattam]

no serious developer would build an app that only runs on AWS, but chances are that your LLM app only runs with OpenAI models

the new smaller Llama 3 model is insanely good: model accuracy is as good as GPT 3.5, while being cheaper and faster. but chances are that your prompts are tightly coupled with your LLM, making it harder to switch

@empiricalrun enables you to track and compare model performance, and iterate towards the best answer for your use-case—so that you can ride the tailwinds of LLM improvements

[Aditya Lahiri @cross_entrippy]

🧪rapid prompt-engineering + intuitive evals 🩺
[prompt-learner 🤝empirical]

🟢tailor-made prompting, specific to task & model, aids smaller models significantly more than larger models

🟢2x ⏫ on sql execution with haiku

🟢sql syntax correctness 🔼99%🤯from 56% on llama3

[Arjun Attam @arjunattam]

every LLM launch uses benchmarks like HumanEval to showcase performance

these benchmarks are everywhere—but they have little utility for app developers. they are mere numbers on a blog post. what's missing is a "vibe check"

what if we could play with these benchmarks, tweak them for our use-cases, and do this quickly with fast iteration cycles?

[Arjun Attam @arjunattam]

🦉 new on @empiricalrun

test out the new Claude models on scenarios that matter for your application in our side-by-side comparison UI

demo link in 🧵

[Arjun Attam @arjunattam]

obvious reasoning wins on the new mistral-large

available for side-by-side comparisons on @empiricalrun 🦉

https://t.co/uAq4ZPIW1Q

[Arjun Attam @arjunattam]

🦉 new on @empiricalrun

we've built the best playground for developers to try the new Gemma models

compare models and prompts side-by-side, and across multiple scenarios at the same time

link in thread ⬇️

[Arjun Attam @arjunattam]

🦉 new on @empiricalrun

prompt engineering can be frustrating, especially when you need to support multiple scenarios

we shipped a better playground that lets you "prompt in bulk". demo ⤵️

[Arjun Attam @arjunattam]

introducing @empiricalrun 🦉

new LLMs are launching daily, and showcasing their capabilities with theoretical benchmarks

BUT app devs need more: they want to play with and observe model behavior on real-world scenarios

we're solving that with empirical. launch post ⬇️

[Amit Raj @oyeBob]

India in last 3 decades at Asian Games

From that 1 Gold to 25 Gold,
From those 23 Medals to 100 Medals.

India is becoming that sports powerhouse, we dreamed of back then.

[Pushpam Saxena @saxena_pushpam]

Speed vs. Customizability -> Speed and Customizability

Checkout 100ms Prebuilt Components Here: https://t.co/zuwqSfkC9s

[Saikat Mitra @Phsychi]

@oyeBob

[Kashish Sharma @kashisharma_]

.@100mslive making it infinitely easier for developers to build live-streaming within their apps and websites!

[100ms @100mslive]

We just launched our Interactive Live Stream SDK on Product Hunt 🐱

Devs can now build Instagram Live-like experiences right inside their apps

100% customisable. Scale to millions. Go live in hours!

Show us some love on PH
https://t.co/2UgE7sNOUi

[100ms @100mslive]

🚀 Super excited to launch something we've been working on for a bit with @vercel!!!

🗣️ The all-new virtual events starter kit with live video!

We're bringing the power of custom live video to all developers with a self-hosted starter kit:

Demo- https://t.co/GOWbwb8y7d

(1/4)

[fru @Oinkoo]

The doctor asks me to download the http://t.co/4gyCrgTUUg app and book an appointment any time! Sweet.