@traceliteai that's a good addition. i was thinking about provenance in terms of maintaining the connection between the original trace and the derived eval, but validating whether the derived case actually preserves the original failure conditions is another important step.
Weโre excited to launch Atlan Trident ๐ฑ
The shared context workspace for humans and agents.
One place for every skill, agent, metric and ontology.
Shared across your team, in the tools they already use.
Better with every run.
@pass_at_k yes, this is an important pointer. having a separate tune/test dataset split helps ensure we're measuring actual improvements rather than just optimizing for the evaluation set. refreshing the holdout with new production failures is equally important.