We're officially SOC2 Type 2 compliant at Libretto! 🎉
But forget the usual corporate speak—here's an honest look at the weird, messy reality of SOC2 compliance at a startup. Check out what we learned the hard way: #StartupLife#SOC2#RealTalk
https://t.co/XeSIrqOzSp
7/7 I wrote up the whole detective story, including:
* How we caught it (with receipts!)
* Why this is terrifying for LLM-dependent products
* What we can do about it
Read more: https://t.co/vaBOlkCBui
6/ It's like your reliable coworker suddenly deciding they're only going to do 50% of their job, but only for certain tasks, and not telling anyone about it 🫠
1/ Excited to announce a new blog series: "Building an LLM-based App with Libretto"! In Part 1, we dive into turning simple LLM demos into robust, production-ready applications. Here's a quick rundown 🧵👇
https://t.co/caGsubjcvG
7/ What's next? Stay tuned for upcoming posts where we'll cover:
* Integrating Libretto directly into your app's codebase for real-time monitoring.
* Deep dives into customizing and calibrating Evals for nuanced assessments.
9/ We're eager to see how prompt engineers leverage this tool to elevate their work. If you're interested in more precise, scalable LLM evaluations, join our Libretto Beta and experience the difference firsthand!
https://t.co/n3JRd1YUxt
3/ Even a year and a half into the GPT revolution, most folks we talk to are testing their prompts with "vibes checks"—running a few examples and manually reviewing the outputs. Effective, yes, but slow and hard to scale.
8/ The result? A more efficient process that not only speeds up evaluation but also enhances the relevance & precision of feedback, allowing you to focus on refining the most promising prompt variations. Go from manually checking 5 or 6 prompt inputs to spot-checking 100 inputs.