Zirui Song

MBZUAI PhD.

[Zirui Song @stevenS28456474]

@BitcoinPulseX wow

[Zirui Song @stevenS28456474]

@NCEMAUAE @grok 翻译中文

[AIDB @ai_database]

LLM内部で怒りや悲しみ、喜びなど感情を表現する神経回路のようなものが存在していることを発見したとのことです。
MBZUAIと北京大学の研究者らによる報告。

こうした、いわば「感情回路」として実装されているニューロンは、LLMが感情を文章で表現する時に働くこと��判明。

そして重要なニューロンはわずか数個程度であり、それらを操作するだけで感情表現を自在にコントロールできることが分かっています。

内部で感情に相当する何らかの��造を形成しているのであれば、モデルは単に訓練データの表面的なパターンを真似ているだけではないのかもしれません。

なお、安全性を重視したモデルでは喜びや驚きといったポジティブな感情は簡単に誘導できる一方、怒りや嫌悪などのネガティブな感情の誘導は意図的にブロックされていました。
AIの安全性設計が感情回路レベルで機能していることを示唆しています。

[Zhongzhi Li @ZhongzhiLi4]

Our team from the Microsoft Research Asia, UCLA, Chinese Academy of Sciences, Tsinghua University, and released a paper, “TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compression”proposing an innovative training method that effectively compresses the reasoning.

[Zhongzhi Li @ZhongzhiLi4]

Hi everyone!
The field of LLM-based reasoning has seen tremendous progress and rapid development over the past few months.
We’ve updated our survey for exciting advances, now covering over 500 papers!
《From system 1 to system 2 a survey of reasoning large language models》

[Xiao Liang @MasterVito0601]

🙋‍♂️ Can RL training address model weaknesses without external distillation?

🚀 Please check our latest work on RL for LLM reasoning!

💯 TL;DR: We propose augmenting RL training with synthetic problems targeting model’s reasoning weaknesses.

📊Qwen2.5-32B: 42.9 → SwS-32B: 68.4

[naveen manwani @NaveenManwani17]

🚨Paper Alert 🚨

➡️Paper Title: Motion Anything: Any to Motion Generation

🌟Few pointers from the paper

🎯Conditional motion generation has been extensively studied in computer vision, yet two critical challenges remain.

🎯First, while masked autoregressive methods have recently outperformed diffusion-based approaches, existing masking models lack a mechanism to prioritize dynamic frames and body parts based on given conditions.

🎯Second, existing methods for different conditioning modalities often fail to integrate multiple modalities effectively, limiting control and coherence in generated motion.

🎯To address these challenges, authors of this paper proposed “Motion Anything”, a multimodal motion generation framework that introduces an Attention-based Mask Modeling approach, enabling fine-grained spatial and temporal control over key frames and actions.

🎯Their model adaptively encodes multimodal conditions, including text and music, improving controllability.

🎯Additionally, they introduced Text-Music-Dance (TMD), a new motion dataset consisting of 2,153 pairs of text, music, and dance, making it twice the size of AIST++, thereby filling a critical gap in the community.

🎯Extensive experiments demonstrate that Motion Anything surpasses state-of-the-art methods across multiple benchmarks, achieving a 15% improvement in FID on HumanML3D and showing consistent performance gains on AIST++ and TMD.

🏢Organization: The Australian National University [@ANUmedia ], The University of Sydney, @TencentGlobal , @mcgillu , https://t.co/Yi2aODZ6eQ, University of Technology Sydney, @mbzuai , @ZJU_China , @Google Research

🧙Paper Authors: Zeyu Zhang, Yiran Wang, Wei Mao, Danning Li, Rui Zhao, Biao Wu, @stevenS28456474 , @supremeZhuang , Ian Reid, Richard Hartley

📝 Read the Full Paper here: https://t.co/09U3yREKN4

🗂️ Project Page: https://t.co/HdezgF4MN2

🎥 Be sure to watch the attached Demo Video- Sound on 🔊🔊

Find this Valuable 💎 ?

♻️QT and teach your network something new

Follow me 👣, @NaveenManwani17 , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

[Zhenhao Chen @DavisChan1998]

We propose BenchLMM to assess the robustness of LMMs against three different styles: artistic image style, imaging sensor style, and application style, where each style has five sub-styles. (1/7)
Paper: https://t.co/5FKRagcRd6
Project website: https://t.co/QFYXHmYmzR

@_akhaliq

[青龍聖者 @bdsqlsz]

I tested anydoor from alibaba,it works great in replace object.
I run it in local cost 12GB VRAM.
Publish local demo soon after i pruned its ckpt models and upload to huggingface.
https://t.co/59gsxqf5gG

[AK @_akhaliq]

OneLLM: One Framework to Align All Modalities with Language

paper page: https://t.co/Fzwgg3TMZn

Multimodal large language models (MLLMs) have gained significant attention due to their strong multimodal understanding capability. However, existing works rely heavily on modality-specific encoders, which usually differ in architecture and are limited to common modalities. In this paper, we present OneLLM, an MLLM that aligns eight modalities to language using a unified framework. We achieve this through a unified multimodal encoder and a progressive multimodal alignment pipeline. In detail, we first train an image projection module to connect a vision encoder with LLM. Then, we build a universal projection module (UPM) by mixing multiple image projection modules and dynamic routing. Finally, we progressively align more modalities to LLM with the UPM. To fully leverage the potential of OneLLM in following instructions, we also curated a comprehensive multimodal instruction dataset, including 2M items from image, audio, video, point cloud, depth/normal map, IMU and fMRI brain activity. OneLLM is evaluated on 25 diverse benchmarks, encompassing tasks such as multimodal captioning, question answering and reasoning, where it delivers excellent performance.

[AK @_akhaliq]

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians

paper page: https://t.co/2QAdqpBaHS

Creating high-fidelity 3D head avatars has always been a research hotspot, but there remains a great challenge under lightweight sparse view setups. In this paper, we propose Gaussian Head Avatar represented by controllable 3D Gaussians for high-fidelity head avatar modeling. We optimize the neutral 3D Gaussians and a fully learned MLP-based deformation field to capture complex expressions. The two parts benefit each other, thereby our method can model fine-grained dynamic details while ensuring expression accuracy. Furthermore, we devise a well-designed geometry-guided initialization strategy based on implicit SDF and Deep Marching Tetrahedra for the stability and convergence of the training procedure. Experiments show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.

[AK @_akhaliq]

One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion

paper page: https://t.co/r7ylXY0Tlv

Recent advancements in open-world 3D object generation have been remarkable, with image-to-3D methods offering superior fine-grained control over their text-to-3D counterparts. However, most existing models fall short in simultaneously providing rapid generation speeds and high fidelity to input images - two features essential for practical applications. In this paper, we present One-2-3-45++, an innovative method that transforms a single image into a detailed 3D textured mesh in approximately one minute. Our approach aims to fully harness the extensive knowledge embedded in 2D diffusion models and priors from valuable yet limited 3D data. This is achieved by initially finetuning a 2D diffusion model for consistent multi-view image generation, followed by elevating these images to 3D with the aid of multi-view conditioned 3D native diffusion models. Extensive experimental evaluations demonstrate that our method can produce high-quality, diverse 3D assets that closely mirror the original input image.

[AK @_akhaliq]

VR-NeRF: High-Fidelity Virtualized Walkable Spaces

paper page: https://t.co/gYGk2gb9vn

present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields. To this end, we designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density. We extend instant neural graphics primitives with a novel perceptual color space for learning accurate HDR appearance, and an efficient mip-mapping mechanism for level-of-detail rendering with anti-aliasing, while carefully optimizing the trade-off between quality and speed. Our multi-GPU renderer enables high-fidelity volume rendering of our neural radiance field model at the full VR resolution of dual 2Ktimes2K at 36 Hz on our custom demo machine. We demonstrate the quality of our results on our challenging high-fidelity datasets, and compare our method and datasets to existing baselines. We release our dataset on our project website.

[Zirui Song @stevenS28456474]

Sydney

[Computer Vision News @CVCND]

#CVPR2023: Papers with code >

https://t.co/g0Cmd1NP2Q

[AK @_akhaliq]

Fine-Tuning Language Models with Just Forward Passes

propose a memory-efficient zerothorder optimizer (MeZO), adapting the classical ZO-SGD method to operate in-place, thereby fine-tuning LMs with the same memory footprint as inference. For example, with a single A100 80GB GPU, MeZO can train a 30-billion parameter model, whereas fine-tuning with backpropagation can train only a 2.7B LM with the same budget. We conduct comprehensive experiments across model types (masked and autoregressive LMs), model scales (up to 66B), and downstream tasks (classification, multiple-choice, and generation). Our results demonstrate that (1) MeZO significantly outperforms in-context learning and linear probing; (2) MeZO achieves comparable performance to fine-tuning with backpropagation across multiple tasks, with up to 12x memory reduction; (3) MeZO is compatible with both full-parameter and parameter-efficient tuning techniques such as LoRA and prefix tuning; (4) MeZO can effectively optimize non-differentiable objectives (e.g., maximizing accuracy or F1). We support our empirical findings with theoretical insights, highlighting how adequate pre-training and task prompts enable MeZO to fine-tune huge models, despite classical ZO analyses suggesting otherwise.

paper page: https://t.co/jIooPQmCd8

[Aran Komatsuzaki @arankomatsuzaki]

The False Promise of Imitating Proprietary LLMs

Open-sourced LLMs are adept at mimicking ChatGPT’s style but not its factuality. There exists a substantial capabilities gap, which requires better base LM.

https://t.co/GYORtrN14H

[Kris Kashtanova @icreatelife]

A free app to turn your drawings into AI generations

https://t.co/Cq6KFhAYqY

[AK @_akhaliq]

LIMA: Less Is More for Alignment

LIMA, a 65B parameter LLaMa language model fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses, without any reinforcement learning or human preference modeling. LIMA demonstrates remarkably strong performance, learning to follow specific response formats from only a handful of examples in the training data, including complex queries that range from planning trip itineraries to speculating about alternate history. Moreover, the model tends to generalize well to unseen tasks that did not appear in the training data. In a controlled human study, responses from LIMA are either equivalent or strictly preferred to GPT-4 in 43% of cases; this statistic is as high as 58% when compared to Bard and 65% versus DaVinci003, which was trained with human feedback

paper page: https://t.co/MjiZ6mZn0x