@[email protected]

HARPO (AKA Carlos A. Catania). A Nerdo with decent social skills? #DataScience #Algorithms. English, Spanish and Français. From Mendoza Republic. South America

[Sebastian Raschka @rasbt]

Since Mixture of Expert (MoE) LLMs are all the rage as of this weekend, thanks to the Mixtral-8x-7B release, here's a quick explainer. The figure below shows the architecture behind the Switch Transformer (https://t.co/6jowgQx0DV), a great intro to MoEs.

The model depicted in this figure uses 1 expert per token with 4 experts in total. Mixtral-8x-7B, on the other hand, consists of 8 experts and uses 2 experts per token.

Why MoEs? Combined, the 8 experts in a 7B model like Mixtral are still ~56B parameters. (Actually, it's less than 56B, because the MoE approach is only applied to the MoE layers, not the self-attention weight matrices. So, it's likely closer to 40-50B parameters.)

However, since the router reroutes the tokens such that only 7B parameters (instead of all 56B) are used at a time for the forward pass, the training (and especially inference) will be much faster compared to the traditional non-MoE approach.

If you read my AI and Open Source in 2023 article (https://t.co/C8SGseRHNZ) approx. 2 months ago, I mentioned that "It will be interesting to see if MoE approaches can lift open-source models to new heights in 2024". It looks like Mixtral started this trend early, and I am sure that this is just the beginning :).

[Seba García @eldracote]

Today ⁦@verovaleros⁩ gave the talk “Four Key Problems in OSINT for Cyber Threat Intelligence” in ⁦@enisa_eu⁩ CTI conf. #cti

[Seba García @eldracote]

"Researchers Pre-trained LLM Agents Acting as Human Penetration Testers" Our work w/@mrigaki @ondrej_lukas @harpolabs and me, as part of the Aidojo w/@muni_cz was covered in @The_Cyber_News https://t.co/1kSu8qgjfV @StratosphereIPS #RL #agents #automaticpentest

[Seba García @eldracote]

Our work with @MurisSladic @verovaleros @harpolabs in LLM honeypots was highlighted in @The_Cyber_News "shelLM – A New AI-Based Honeypot to Engage Attackers as a Real System" @StratosphereIPS https://t.co/0PbysS49Sx via @The_Cyber_News

[Maria Rigaki @mrigaki]

🚨New Paper Alert! In our latest research we used LLMs as reinforcement learning red team agents: "Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments" https://t.co/LmMHoUlHYh with @ondrej_lukas @harpolabs @eldraco #AI #LLM #ML #infosec

[Max Kuhn @topepos]

@predict_addict I'm talking on this at #PositConf2023 in Chicago (https://t.co/vAkF2WBi7y) in about 2 weeks!

[__veronica__ @verovaleros]

It’s time to admit that we were all overly optimistic during COVID, thinking that organizations will finally understand the “remote work” culture. Not only is back to office, but also back to paper and physical meets. We thought we could make a change and we lost. #fail

[Maria Rigaki @mrigaki]

We are happy to announce that our paper with @eldracote was accepted in ESORICS 2023: "The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning" https://t.co/MhtHQ4x5cf @StratosphereIPS @ctu_cs #ai #adversarial #MachineLearning #malware #security

[Seba García @eldracote]

Thanks for the kits and the workshop! Help us get into the community and have fun!

[Leo Oliva @loliva]

Desde el nuevo Media Lab de la @UNCUYO acabamos de lanzar un podcast dedicado a la inteligencia artificial. Se llama "Inteligencia natural" y acá están los primeros 3 episodios. https://t.co/0HAtJ6CcHV

[Magdalena Day @magdalenaday]

Los Harpo están conquistando el mundo @harpolabs 😂😂#LiliFiallo muy interesante arte y IA #podcasts

[Seba García @eldracote]

Join us in Las Vegas for our @BlackHatEvents
hands-on Advanced Malware Traffic Analysis training with @verovaleros! Learn how to detect botnets, ransomware, lateral movement, and more! Learn to detect attacks w/machine learning tools! #BlackHat #BHUSA https://t.co/LmYEnqG9Dz

[Cancu Rodriguez 🇦🇷 @CancuCS]

Automated Code Reviews by ChatGPT through GH Actions? Sure! Learn how to do it at https://t.co/VdDh6eRJ0o
#chatgpt #githubactions #codereview #pullrequest #llm

[Seba García @eldracote]

Our paper w/@MasarahClouston "On the dynamics behind profit-driven cybercrime from contextual factors to perceived group structures, and the workforce at the periphery" was finally published! I'm so proud and happy for this collaboration! #cybercrime https://t.co/oWrgygd6Ky

[Hugging Face @huggingface]

Let's gooo! 🔥

We're happy to introduce two new official Space Templates in collaboration with @posit_pbc: Build and share your R and Python Shiny apps directly on the Hub! 🥳

[@[email protected] @harpolabs]

Mornings are made for reading, nights are made for coding.🤓
... Except when you have kids 🤷🏻‍♀️

[clem 🤗 @ClementDelangue]

I believe we need open-source alternatives to ChatGPT for more transparency, inclusivity, accountability and distribution of power.

Excited to introduce HuggingChat, an open-source early prototype interface, powered by OpenAssistant, a model that was released a few weeks ago.

[Nicolás Wolovick ⭐⭐⭐ @nwolovick]

¡El #SAHTI2023 de las #52JAIIO @jaiio_oficial se prepara!
@bmassare desde CABA, Karina Bianculli desde MdP y yo desde Cba, armando un Simposio de Historias, Tecnología e Informática que va a estar muy bueno.

¡Hasta el 1 de mayo tenés tiempo de presentar el resúmen!
Te esperamos.

[@[email protected] @harpolabs]

Another collaboration with visual artist Lili Fiallo. Using #dreambooth we fine tuned #stablediffusion to incorporate Lili's toy art style. incredibly simple using diffusers and gradio libraries.

https://t.co/7TK1n6tTDj
https://t.co/1TXvs5iQop

[Stratosphere Laboratory AIC FEE CTU @StratosphereIPS]

We have planned outages in our datasets server today (20/04/23) between 9:30 and 12:00 CET. We will let you know when things are back online. ⚙️