Jack Fratto

- Agentic Legal Workspace

- Frontier Model Benchmarking

[Jack Fratto @JackFratto]

Day 2 #buildinpublic

- Added 3D Pareto frontier for the following dimensions:
1. Performance
2. Cost
3. Speed
- Added a filter for speed.
- Added support for viewing on mobile.

What should I work on next?

https://t.co/fWobOYsQiB

[Jack Fratto @JackFratto]

@gmax_digital_ @X Sick! Looking forward to it

[Jack Fratto @JackFratto]

H! @X, trying to connect with everyone interested in:

• AI/ML
• Gen AI
• Data Science
• Full-Stack Development
• Building in Public
• OSS
• Founding
• AI Agents & Automation
• SaaS & Startups

Let's build something awesome!!
#connect

[Jack Fratto @JackFratto]

@mytwillot Looks pretty good! I think users would prefer to see the product as the first thing when they hit the landing page. I've also heard cinematic shots of people build trust quickly

[Jack Fratto @JackFratto]

I'm going to start doing weekly reviews of peoples products and feature the top 3 every Saturday.

Let's get started! Can't wait to see what you're all working on.

Drop your product below 👇

[Jack Fratto @JackFratto]

@mediumbuckley @daniellevine Appreciate that!

[Jack Fratto @JackFratto]

@Sil0Sec @X Sent ya a follow!

[Jack Fratto @JackFratto]

@mediumbuckley @daniellevine Appreciate that advice! I try my hardest but getting connected with lawyers who are open to new technology is very difficult. Especially if it's relevant to AI.

[Jack Fratto @JackFratto]

@fchralph @Certainium @daniellevine The citation issue is pretty easy imo... The largest roadblock I'm running into is rate-limits with CourtListener and I'm not ready to commit $600 for infra to download the bulk data and serve embeddings.

[Jack Fratto @JackFratto]

@fchralph Custom evaluators are going to be the final phase! It will likely be closed source and the app displaying the benchmarks will be the core. Custom evals are also quite a lot of work so I'm hoping to find some contributors before I get to that point 😂

[Jack Fratto @JackFratto]

Day 1 #buildinpublic

Open Source LLM benchmarking leaderboard.

gh: https://t.co/geCggJweRY

Future plans
-------------
1. Add benchmarks for domains affected by world models
2. Capture agentic performance
3. Create custom evaluator
4. Add section for relevant research

[Jack Fratto @JackFratto]

This is the peak of the AI bubble

[tae kim @firstadopter]

"No HBM for you. I need HBM" - Jensen

Fact check. True. $NVDA

[Jack Fratto @JackFratto]

@BuildWithOm whatever middle management currently does

[Jack Fratto @JackFratto]

@Proximatwt lets get it!

[Jack Fratto @JackFratto]

@MatthewBerman peep https://t.co/geCggJweRY

We're building analysis/benchmarks for the best AI model to use in any domain!

[Jack Fratto @JackFratto]

@yuhasbeentaken Most CS classes are not for Software Engineering! They mostly gear you up for research rather than industry.

[Jack Fratto @JackFratto]

@roudyhermez @X iOS! definitely not my wheelhouse but can't wait to see what you build!

[Jack Fratto @JackFratto]

@daniellevine Appreciate that advice!

[Jack Fratto @JackFratto]

@RebornTechGlob @X Sounds great! Excited for the future

[Jack Fratto @JackFratto]

Can't wait for this model to get distilled at 0.1x the cost 😎

[🚨 AI News | TestingCatalog @testingcatalog]

ANTHROPIC 🔥: A new "claude-oceanus-v1-p" has been made available to Red Teams.

This appearance may signal an upcoming release of newer Mythos models, referenced earlier by Antropic.

Soon? 👀

[Jack Fratto @JackFratto]

@Rj_Lopez_ @X Sent ya a follow!

[Jack Fratto @JackFratto]

@persona_nett @X Sounds dope! Are you creating embedding for each user and grouping based on that?

[Jack Fratto @JackFratto]

@1Umairshaikh sell