Panel Discussion- Reimagine Media & Content Ecosystem with Web3 & AI https://t.co/rxzSLwLSkl via @Blockcast.cc- News on Blockchain, DLT, Cryptocurrency
100% Multimodal LLMs will be the new OS.
It's the only way we can have lighter devices (Laptops, XR Glasses, BCIs), with bigger batteries, less heat, non built-in obsolescence.
Less demand for hardware -> Cheaper materials -> More access to powerful computing for the masses.
this is the best breakdown on the state of multimodality that i've read, esp why CLIP has been so key in getting us here
https://t.co/1lL1d88WXF by @chipro
https://t.co/WSLYIgN0qj A new study has shown that simple modifications to a multimodal model can achieve state-of-the-art results on various benchmarks with minimal data usage and training time. Code and model will be publicly available.
https://t.co/FM4xZaux97 Researchers propose a new two-stage pruning approach for Large Vision-Language Models that achieves significant performance improvements on various multimodal and unimodal datasets.
Improved Baselines with Visual Instruction Tuning
paper page: https://t.co/YEabbP3Ic3
Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning. In this note, we show that the fully-connected vision-language cross-modal connector in LLaVA is surprisingly powerful and data-efficient. With simple modifications to LLaVA, namely, using CLIP-ViT-L-336px with an MLP projection and adding academic-task-oriented VQA data with simple response formatting prompts, we establish stronger baselines that achieve state-of-the-art across 11 benchmarks. Our final 13B checkpoint uses merely 1.2M publicly available data, and finishes full training in ~1 day on a single 8-A100 node. We hope this can make state-of-the-art LMM research more accessible.
LLaVA can be brutally honest and even insulting.
There are no restrictions or filters to hold it back.
LLaVA is a multimodal LLM on @huggingface like ChatGPT will be soon.
I asked it to criticize and roast my look, and I do not regret it.😅
Check ALT for the link and prompt.
ChatGPT-4V Multimodal make a great effort to present the IBM System 370 control board function from just a single image.
More questions produced a use for each button, switch or dial.
Today I was pleased to introduce Provisional Multimodal Research at the FabLearn/Constructionism 2023 conference at Columbia University. #provisionalmultimodality