Alibaba presents MIMO
Controllable Character Video Synthesis with Spatial Decomposed Modeling
Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. As a fundamental problem in the computer vision and graphics community, 3D works typically require multi-view captures for per-case training, which severely limits their applicability of modeling arbitrary characters in a short time. Recent 2D methods break this limitation via pre-trained diffusion models, but they struggle for pose generality and scene interaction. To this end, we propose MIMO, a novel framework which can not only synthesize character videos with controllable attributes (i.e., character, motion and scene) provided by simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. The core idea is to encode the 2D video to compact spatial codes, considering the inherent 3D nature of video occurrence. Concretely, we lift the 2D frame pixels into 3D using monocular depth estimators, and decompose the video clip to three spatial components (i.e., main human, underlying scene, and floating occlusion) in hierarchical layers based on the 3D depth. These components are further encoded to canonical identity code, structured motion code and full scene code, which are utilized as control signals of synthesis process. The design of spatial decomposed modeling enables flexible user control, complex motion expression, as well as 3D-aware synthesis for scene interactions. Experimental results demonstrate effectiveness and robustness of the proposed method.
Robotaxis en China: los viajes en los robotaxis de Baidu están valorados con una puntuación de 4,9 sobre 5 puntos y que la mayor parte de las quejas hacen referencia a un avanzar demasiado lento de los vehículos cuando el semáforo se pone en verde. https://t.co/k81aIPinp4
Pablo Jarillo-Herrero: "Una de las razones por las que España no atrae talento en ciencia es por los salarios mediocres y porque culturalmente no es tan meritocrática"
https://t.co/CovssxnKka
Prohibieron a los robotaxis de General Motors circular porque tenían demasiados accidentes. Los de Google van por el mismo camino https://t.co/S5LqceA6aS
Si los resultados parecen peores es porque la web es peor en sí misma. ¿Y por qué es peor?, "Creo que es porque hay un montón de incentivos económicos para la desinformación, para los clics, para las compras".
https://t.co/Tw61OHFkik
Zelenski unificó las emisiones de las cadenas de TV en una llamada telemaratón. "Es pura propaganda. Dice que no muere nadie". "Si no nos envía más armamento, moriremos todos. Vayan a Leópolis, hay cinco mujeres por cada hombre. Ya no quedan hombres". https://t.co/EMPxti88jW