Check out NotebookLM! Create a notebook, upload one or more sources (e.g. PDFs of research papers, your favorite PhD thesis, a newspaper article, etc) then click on 'Generate' to create a podcast of two voices talking about the content you've uploaded.
https://t.co/FSCBvsr8tw
Muchas gracias @RiuHoteles y especialmente al hotel de Ocho Rios Jamaica y a su director por su amabilidad y por cuidar de los pequeños detalles. ¡¡¡Seguro que repetiremos!!!
@LadillaRusa ¿Pasado? El otro día me sirvieron el limón, sobre una taza de agua caliente y con un soplete. ¿Puedo decir el restaurante? Arena en S'Arenal des Castell.
We’re excited to announce that 'Help me write' in @googledocs and @gmail is available in Spanish and Portuguese. Sign up to try Gemini for #GoogleWorkspace today. → https://t.co/G8C01ccqqJ
Google Cloud and Oracle announced an exciting new strategic partnership that aims to provide our customers with a variety of best-in-class solutions to migrate, modernize, and manage their Oracle-based applications in the cloud.
Read the announcement: https://t.co/UhCSidNDHO
Google Cloud’s new partnership with Oracle provides a path for the world’s largest enterprises to use next generation cloud and AI by combining Oracle Cloud database and Google Cloud services. https://t.co/30FK2zZnXJ
This is really a 'WOW' paper. 🤯
Claims that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales and by utilizing an optimized kernel during inference, their model’s memory consumption can be reduced by more than 10× compared to unoptimized models. 🤯
'Scalable MatMul-free Language Modeling'
Concludes that it is possible to create the first scalable MatMul-free LLM that achieves performance on par with state-of-the-art Transformers at billion-parameter scales.
📌 The proposed MatMul-free LLM replaces MatMul operations in dense layers with ternary accumulations using weights constrained to {-1, 0, +1}. This reduces computational cost and memory utilization while preserving network expressiveness.
📌 To remove MatMul from self-attention, the Gated Recurrent Unit (GRU) is optimized to rely solely on element-wise products, creating the MatMul-free Linear GRU (MLGRU) token mixer. The MLGRU simplifies the GRU by removing hidden-state related weights, enabling parallel computation, and replacing remaining weights with ternary matrices.
📌 For MatMul-free channel mixing, the Gated Linear Unit (GLU) is adapted to use BitLinear layers with ternary weights, eliminating expensive MatMuls while maintaining effectiveness in mixing information across channels.
📌 The paper introduces a hardware-efficient fused BitLinear layer that optimizes RMSNorm and BitLinear operations. By fusing these operations and utilizing shared memory, training speed improves by 25.6% and memory consumption reduces by 61% over an unoptimized baseline.
📌 Experimental results show that the MatMul-free LLM achieves competitive performance compared to Transformer++ baselines on downstream tasks, with the performance gap narrowing as model size increases. The scaling law projections suggest MatMul-free LLM can outperform Transformer++ in efficiency and potentially in loss when scaled up.
📌 A custom FPGA accelerator is built to exploit the lightweight operations of the MatMul-free LLM. The accelerator processes billion-parameter scale models at 13W beyond human-readable throughput, demonstrating the potential for brain-like efficiency in future lightweight LLMs.
Day 1 of the International Conference on Education and #Innovation in #Museums (#ICEIM) in Riyadh! Excited to share our insights on creativity-tech tools for enhancing visitor experiences and learning. Incorporating global expertise from institutions and colleagues! 🌍✨
Tengo 2 entradas en pista para AC/DC de un amigo que no puede ir por problemas médicos. ¿Algún interesado en la sala?
Y si me ayudáis a difundirlo, os agradecería.
For Reinforcement Learning to be successful on top of LLMs, it is critical to have a very powerful and accurate reward model. Reward in most language tasks isn't as clearly defined as, say, in chess, where the winning condition is a simple computation.
Generative reward models are the most powerful class as they are themselves general. It's great to see how the best LLMs are starting to top this important benchmark!
https://t.co/ecUIIMttpw
Everything is ready for our largest ever Google event in Spain, Google Cloud Summit Madrid. Looking forward to hosting thousands of customers and partners Today #AI#GoogleCloud
Excited to share my new post on how Gemini 1.5 Pro can now analyze entire binaries, offering a new & scalable approach to threat detection.
Working on producing these reports at scale - stay tuned!
https://t.co/9EEZ6zNQXL
#malwareanalysis#reverseengineering#AI