cTuning foundation

Empowering everyone to participate in collaborative research (AI/ML/Sys), reproducible experiments and open science to solve the world's most complex challenges

[Google AI @GoogleAI]

Today on the blog, we’re excited to announce the release of @MLCommons Croissant, a metadata format to make ML datasets more easily discoverable and usable across a wide array of tools and platforms. Learn more and try it today →https://t.co/fdjBDannpx #ml #datasets

[HPCA @HpcaArchConf]

Distinguished Artifact Runner Up:
Title: "An Optimizing Framework on MLIR for Efficient FPGA-based Accelerator Generation"
Authors: Weichuang Zhang, Jieru Zhao, Guan Shen, Quan Chen, Chen Chen, Minyi Guo

[Grigori Fursin @grigori_fursin]

We have released a new CM-MLPerf automation to benchmark commodity hardware for AI performance, power and cost efficiency - it helped to automate ~90% of MLPerf inference v4.0 submissions while achieving several top performance and power results: https://t.co/8BqZs03U9P

[Grigori Fursin @grigori_fursin]

What an exciting week for our open-source #CollectiveMind project to support collaborative and reproducible benchmarking of AI models and systems (#MLPerf training v3.1, @TheOfficialACM Youtube channel; @Supercomputing ; @MicroArchConf; #LLM):
https://t.co/FHoiJkyHg3

[Grigori Fursin @grigori_fursin]

Check out our artifact evaluation report for the 56th IEEE/ACM International Symposium on Microarchitecture with the pilot @MLCommons project @https://www.linkedin.com/pulse/micro-2023-artifact-evaluation-report-56th-ieeeacm-symposium-fursin-bsgwe @MicroArchConf

[Grigori Fursin @grigori_fursin]

Check out my latest article: A new milestone to make MLPerf benchmarks accessible to everyone: https://t.co/LIzNqyUH8x via @LinkedIn

[Grigori Fursin @grigori_fursin]

Urgent: @MicroArchConf'23 is looking for motivated artifact evaluators to #reproduce results for accepted papers. If you already have relevant AE experience, please use this form for self-nomination: https://t.co/8uMj6fxwrB . Deadline for self-nomination is August 14. Thank you!

[HiPEAC @hipeac]

📢 Calling all students!
👩‍🎓HiPEAC Reproducibility Student Challenge now open for registrations
⏳Deadline: 15 September
🔬 Get hands-on experience of the latest research
📜📜Contribute to reproducibility
🤝Network with top researchers at #HiPEAC24
👉 https://t.co/lByaZdvnj7

[Grigori Fursin @grigori_fursin]

Are you interested to know how fast the open-source GPT-J 6B #LLM model from @AiEleuther runs on your @nvidia GPU? Join the new MLPerf@home challenge to crowd-benchmark GPT-J and submit your results to MLPerf inference v3.1 round (deadline: August 3rd):
https://t.co/y3ieMm06lu

[MLCommons @MLCommons]

Today we announced MedPerf an open benchmarking platform for medical AI from the MLCommons Medical WG with major contributions from @DanaFarber @Google @huggingface @intel @MSFTResearch @nutanix @PennMedicine and many others. https://t.co/vABY1HDrt2

[Grigori Fursin @grigori_fursin]

Very excited to announce the 1st Collective Knowledge Cup - a set of open challenges prepared by @MLCommons organizations to let the community benchmark and optimize AI and ML Systems (latency, throughput, energy, accuracy, cost)
https://t.co/8to8d5EyGG via @LinkedIn

[Grigori Fursin @grigori_fursin]

@TheSeaMouse Oh, that's so true! After being hugely frustrated with all that, we developed a simple automation language to address some of these issues: https://t.co/muOBA3BiHn - we just tested it to reproduce MLPerf inference benchmarks and it seems to work but we will need more feedback ;)

[MLCommons @MLCommons]

New @MLPerf Tiny results improved 1000-fold in some areas. Congrats to all our Tiny v1.1 submitters: @BoschGlobal @c_tuning @fpgaConvNet, Kai Jiang, @krai4ai @nuvoton @plumerai @skymizer @ST_World @Syntiantcorp on their fantastic results. See the results https://t.co/Po0KOMpWxD

[Grigori Fursin @grigori_fursin]

Very excited to give a keynote along with @thoefler and @jfreirenet at @acmrep - the 1st ACM conference on #reproducibility and #replicability: https://t.co/QvJz9ZAN8h . You can find abstracts at https://t.co/q8poEyyb4o . Looking forward to discussions!

[Andrej Karpathy @karpathy]

Very nice & inspiring, "no-gradient architecture" for high-level skills/learning. LLM here is the "prefrontal cortex" orchestrating the lower-level mineflayer API via code generation++.

Meta-comment is that I remember how hopeless it felt to work on agents in environments like Minecraft around ~2016, feeling stuck on how RL at the time would ever randomly explore their way into performing long-horizon tasks from super sparse rewards. This block has now to a very large extent been lifted - the correct thing was to forget all that, first train LLMs that learn (1) world knowledge, (2) reasoning and (3) tool-use (esp writing code) all from internet text, then point them back at the problem in this kind of a way. TLDR If I had read about this "no-gradient" approach to agents in 2016 my mind would certainly be blown.

Also haha @ source code in the voyager/prompts/*.txt directory :D

[Grigori Fursin @grigori_fursin]

Thank you, Flavio (@Flav1oV)! It was great to collaborate with you on the original CK framework to facilitate #reproducible research and technology transfer! I am looking forward to collaborate with you on the 2nd generation of our @MLCommons CM automation language in the future!

[ACM REP '24 @acmrep]

#registration for #acmrep23 is open. Check out the exciting program of the first ACM conference dedicated to #reproducibility. The #conference will take place at the beautiful Cowell Hay Barn at @UCSC. But space is limited, so register early. https://t.co/uV5SDANbaX

[ACM REP '24 @acmrep]

Really happy to add @jfreirenet to our keynote speakers at ACM REP '23, @thoefler and @grigori_fursin . Check out our exciting program and register soon. Space for in-person attendance is limited. https://t.co/uV5SDANbaX. #acmrep23 #acmrep

[Carlos Maltzahn, linkedin: carlosmaltzahn @CarlosMaltzahn]

We are delighted to have @grigori_fursin confirmed as keynote speaker at #acmrep23. The keynote's title is "Toward a common language to facilitate reproducible research and technology transfer: challenges and solutions".

[Grigori Fursin @grigori_fursin]

The English version of the proposed #EuropeanUnion #ArtificialIntelligence law: https://t.co/px0TuUuRk9