Koen van der Veen

Building PySyft at @OpenMinedOrg
previously @Memriio

[Koen van der Veen @koenvanderveen]

@DynamicWebPaige 🫡

[Andrew Trask @iamtrask]

If all AI companies and AI eval organizations used double-blind evals, test set contamination in training data would disappear overnight, and confidence in AI eval veracity would rise across all subject areas.

[OpenMined @openminedorg]

After nearly a decade of research and development including contributions from over 400 contributors from around the world, we are pleased to announce that PySyft has been used by @GoogleDeepMind, @AVERIorg, Singapore AISI, and @MLCommons to facilitate the world’s first double-blind evaluations of a proprietary, frontier class AI model.

[Andrew Trask @iamtrask]

@GoogleDeepMind @AVERIorg @MLCommons I'm exceptionally proud to have been a part of this pilot. And after 9 years of R&D, massive congratulations to the @OpenMinedOrg team on reaching this milestone with #PySyft.

[Koen van der Veen @koenvanderveen]

Today, we released our work on double blind eval of frontier models using PySyft. To the best of my knowledge this is the first of its kind. Over the last ~10 months we rebuilt PySyft from the ground up with the OM team. Still a lot to do but very happy with these results!

[Google DeepMind @GoogleDeepMind]

In an industry first, we’re piloting double-blind evaluations for frontier AI.

By creating a secure environment where neither test prompts nor model weights are revealed, we can ensure external safety and performance evaluations of our models remain private, robust, and trustworthy. → https://t.co/ocwQ2iWFDz

[Koen van der Veen @koenvanderveen]

@antirez I would want this to work. However, in practice it feels like this is still super challenging and there are so many small things that go wrong making it impossible to work at this level of abstraction. Do you feel like you are succeeding at it?

[Koen van der Veen @koenvanderveen]

@iamtrask Similarly, it might be useful to sample multiple answers from multiple populations to see: what would person X say?

[Koen van der Veen @koenvanderveen]

@iamtrask In terms of use cases the type of questions where the answer depends on your appetite for risk might be useful. Because most LLMs are so risk averse. for example:
1) At what time do I need to be at the aiport
2) Should I eat this food that has been outside of the fridge for x hrs

[FAR.AI @farairesearch]

How do you verify what an AI system is actually doing, from training through deployment?

On May 17, https://t.co/CpCOFQPSHr and @ai_risks co-hosted the Assurance & Verification of AI Development (AViD) Workshop, bringing researchers and engineers together to build the technical foundations for AI assurance.

The day covered domestic auditing, international treaty verification, and internal oversight, with sources of trust spanning TEEs, novel hardware, and cryptographic methods.

Lightning talks from Shahin Tajik (WPI), @koenvanderveen (@OpenMinedOrg), @bing_jyue (UIUC), @RoyRinberg (Harvard), Adam Chlipala (MIT), Amean Asad (Confidential AI), Quintus Kilbourn (Flashbots), and @AriJuels (Cornell Tech).

As AI capabilities advance, the infrastructure for verifying these systems has to advance with them. Independent verification gives society what it needs to respond and adapt.

Recordings coming soon.

[Koen van der Veen @koenvanderveen]

@levelsio @sw33tLie Is it actually more prone to prompt injection than just claude + user data? How?

[Koen van der Veen @koenvanderveen]

@charliermarsh Just use claude!

[Koen van der Veen @koenvanderveen]

@levelsio I am using the same setup but when claude is thinking I cannot scroll and the screen is flickering. Do you have the same or found a solution for that?

[Koen van der Veen @koenvanderveen]

@badlogicgames ah that makes sense, thanks!

[Koen van der Veen @koenvanderveen]

@adriaandotcom @mikewiendels I tried a few times to make it write an issue that made sense, but it was so far off that it was in the end much easier to implement it from scratch (with cursor though). It also took a long time thinking about the response and researching the code

[Koen van der Veen @koenvanderveen]

@simonw perhaps mcp servers and clients could be modified in a way that warns the user when these are combined. Would require labeling mcp endpoints, but that may be fine. Especially now that clients are not implementing security measures like CaMeL yet

[OpenMined @openminedorg]

OpenMined recently completed a test pilot using GPU enclaves in the evaluation of an AI model across two organizations. Details here → https://t.co/QSsp2jG0cm

[OpenMined @openminedorg]

We are excited to announce or work with @AiEleuther to show how third-party evaluators can query a non-public AI training dataset.

Read about it in our latest post: https://t.co/jYj6fYSBB7

[CSET @CSETGeorgetown]

What: CSET Webinar 📺
When: Thursday, 9/19 at 12PM ET 📅

How can AI developers innovate while respecting data privacy?

@iamtrask & Irina Bejan of @openminedorg will join CSET’s @hlntnr to discuss their work on privacy-enhancing tech.

Register now: https://t.co/id8VN7yb48