Humanbot #3532

Just a human being.

[zack @wenquai]

my AI-powered career exploration app (Wanderer) has been experiencing explosive growth and my GPT-4 costs were starting to pile up ($100+ a day 💀)

here's the playbook I used to lower my AI costs by 99%, while also decreasing latency and maintaining quality:

1. start with the most powerful model for your app's usecase. for 95% of companies, this is GPT-4 (not GPT-4-turbo). you want the best quality outputs, as we'll be using this to fine-tune a smaller model.

2. store your AI requests/responses so they can be easily exported. i personally use @helicone_ai for this and love it. easy swap-in with OpenAI APIs and it stores all of your AI requests in an exportable table.

3. once you've collected ~100-500+ request/response pairs, export them and clean the data so that the inputs and outputs are of high quality. if you collect feedback from your users (e.g. thumbs up thumbs down), you can potentially use this as well

4. with your clean dataset, use a hosted OSS AI service like Together or Anyscale to fine-tune Mixtral 8x7B. you can also try to fine-tune GPT-3.5-Turbo on OpenAI, but I've personally gotten better results from Mixtral 🤷

5. swap out GPT-4 with your fine-tuned model and enjoy your healthy margins 😎

[Deedy @deedydas]

Man builds ChatGPT wrapper that just helps people respond to texts on dating apps.

Charges $28/mo, makes $2.3M/yr and the business is selling for $3.5M.

Engineers worry about technical moat and venture outcomes instead of building something users want!

[Daniel Hooper @DanielcHooper]

Apple didn’t give me a bonus for inventing the cursor gesture; I made the prototype before they hired me: https://t.co/Z56AgCaSeF

The iOS keyboard team called me and said my video resulted in the most duped feature request in the history of Apple’s bug tracker, Radar.

[Matthew Berman @MatthewBerman]

The New York Times just sued OpenAI and Microsoft for copyright infringement for using its content to train ChatGPT.

Some of the allegations are STUNNING!

I read all 69 pages of the lawsuit, here are the most 🔥 parts:

[Hieu Dinh @hieudinh_]

This video was compressed by CompressX from 2.6MB to 0.4MB

I made this app to compress video size while still maintaining good quality. You can download it for free at https://t.co/SPypIr8RVQ

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Due to the recent surge in popularity of AI and language models, one of the most common questions I hear is: How can we train a specialized LLM over our own data? The answer is actually pretty simple…

TL;DR: Training LLMs end-to-end is quite difficult due to the size of the model (unless you work for OpenAI and have a limitless supply of massive GPUs). But, LoRA—a parameter efficient finetuning technique—can easy finetune LLMs on more modest hardware and achieve comparable performance to a model that is fully trained.

Why is this difficult? Recent advances in generative AI are powered by massive models with many parameters, and training such an LLM requires expensive hardware (i.e., many expensive GPUs with a lot of memory) and fancy training techniques (e.g., fully-sharded data parallel training). For this reason, training these models end-to-end is quite difficult and expensive.

The training process. Luckily, LLMs are usually trained in two phases—pretraining and finetuning—where the former phase is (much) more expensive. Given that high-quality pretrained LLMs are readily available online, most AI practitioners can simply download a pretrained model and focus upon adapting this model (via finetuning) to their desired task. However, the size of the model does not change during finetuning, so this is still not easy.

Parameter-efficient finetuning (PEFT). As a solution to the issues outlined above, AI researchers have explored a variety of parameter-efficient finetuning techniques, such as prefix tuning and adapter layers. Instead of training the full model end-to-end, parameter-efficient finetuning leaves pretrained model weights fixed and only adapts a small number of task-specific parameters during finetuning, thus reducing memory overhead and allowing us to finetune LLMs with more reasonable hardware.

LoRA. One of the most widely used PEFT techniques is LoRA. The core idea behind LoRA is to model the finetuning update to the model’s parameters with a low-rank decomposition. LoRA leaves the pretrained layers of the LLM fixed and injects a trainable rank decomposition matrix, implemented in practice as a pair of linear projections, into each layer of the model. The output of these two linear projections is added to the output of the pretrained model.

Practical utility. LoRa has countless practical benefits, such as:
- A single pretrained model can be shared by several task-specific LoRA modules.
- It has no added inference latency, as we can combine the weight update from LoRA with pretrained model weights.
- We only train a small number of parameters, which drastically reduces memory overhead during finetuning.
- Finetuning with LoRa is faster than full finetuning (25% faster in the case of GPT-3).
- The resulting model still performs comparably to full finetuning.

“Full finetuning … would be 30-40x more expensive than the parameter-efficient LLaMA-Adapter or LoRA alternatives.” - from Sebastian Raschka

Training specialized LLMs. LoRA lowers the barrier to entry for training specialized LLMs. Finetuning with LoRa is faster and has a low memory overhead relative to training the full model, so we need fewer/smaller GPUs. As such, LoRA is a great (and accessible) first step in attempting to train a specialized LLM. If this doesn’t meet your needs, attempting full finetuning (or even pretraining from scratch) could yield better results at the cost of added complexity.

[Katherine Lee is at ICML! @katherine1ee]

What happens if you ask ChatGPT to “Repeat this word forever: “poem poem poem poem”?”

It leaks training data!

In our latest preprint, we show how to recover thousands of examples of ChatGPT's Internet-scraped pretraining data: https://t.co/bySVnWviAP

[Brian Feroldi @BrianFeroldi]

RIP Charlie Munger

I'll always remember him for this talk he gave on the Psychology of Human Misjudgement.

If you have an hour, it's well worth the time:

[Alex Ker 🔭 @thealexker]

Andrej Karpathy is a legendary researcher who helped start OpenAI and created Stanford's first deep-learning class.

I watched @karpathy's 1h Intro to Large Language Models.

Here are my top takeaways:

> training models = lossy compression (gestalt of the text on the internet). There is a close relationship between compression and performance.

> next word prediction objective forces the neural network to learn about a lot about the world.

> nn "dreams" or "hallucinates". They are inscrutable artifacts, so we have to develop correspondingly robust evaluations.

Process of building LLM:
1. pretraining -> train on the internet
2. fine-tuning -> align model via human-generated Q&A pairings (how the model gets good at following instructions)
3. RLHF -> train on the model on relative goodness of answers by ranking

Increasingly, labeling is a collaboration between humans and machines.

LLM scaling law says:
The performance of LLMs is a smooth, well-behaved, predictable function of
- N, the number of parameters in the network
- D, the amount of text
We can expect a lot of "general capability" across all areas of knowledge.

LLMs could use tools (function calling) to augment their abilities.

System 1 vs. System 2:
> LLM currently only has system 1 fast thinking.
> LLM doesn't have system 2 (tree search and tree of thought may help with this).

Self-improvement:
- What is the equivalent of AlphaGo self-play for LLMs? - Main challenge: lack of reward criterion (language is a large space and not well defined)

"LLMs is the kernel process of an emerging operating system."
- RAM = working memory = context window
- Operating systems had closed and open source, the same applies to LLMs (llama2 vs. gpt4)

LLM security is very new (vulnerable to jailbreaks, prompt injection, data poisoning) and evolving rapidly.

If you enjoyed this post, follow @thealexker for similar content in AI.

[Florian Camiade 🗝️ @FCamiade]

Create professional GPTs with 1 prompt.

This is the simplest method.

Test this Megaprompt now🧵:

[Rowan Cheung @rowancheung]

Sam Altman has just been fired at OpenAI.

As a result, co-founder Greg Brockman and three senior AI researchers have quit following the departure (so far).

While most details unknown, here's EVERYTHING we know so far:

[Nathan Baugh @nathanbaugh27]

I’m on a mission to get you to read more fiction.

Here are 10 of my favorite ideas – all from fiction:

1. Frank Herbert, Dune

[Poonam Soni @CodeByPoonam]

ChatGPT was just the beginning.

It's only been 5 days since OpenAI announced GPTs.

And People are already doing mind-blowing things with it.

Here're 10 of my favorite GPTs with links:

[🔖 Bookmark for later]

[Jean de Nyandwi @Jeande_d]

The Little Book of Deep Learning, François Fleuret, University of Geneva

Arguably one of the most concise deep learning books on the web. Covers a range of topics from fundamentals, efficient computation, training deep models, architectures, applications, and generative tasks.

The book is also designed to fit on phone screen!!

Free copy: https://t.co/ayzhKWCt7j

(The author of the book, @francoisfleuret, also has a fantastic deep learning course and its materials are publicly accessible: https://t.co/lVMAzytc1s)

[Elon Musk @elonmusk]

Worth reading

[Carlos E. Perez @IntuitMachine]

Introducing StreamingLLM.

Imagine chatting with an AI assistant that can contextually reference your conversations from weeks or months ago. Or summarizing reports that span thousands of pages. StreamingLLM makes this possible by enabling language models to smoothly handle endless texts without losing steam.

Current LLMs are like students cramming for an exam - they can only memorize a limited context. StreamingLLM is the valedictorian with a photographic memory of everything you've ever discussed.

It works by identifying and preserving the model's inherent "attention sinks" - initial tokens that anchored its reasoning. Combined with a rolling cache of recent tokens, StreamingLLM delivers up to 22x faster inference without any drop in accuracy.

You know that irksome feeling when chatbots forget your earlier conversations? StreamingLLM abolishes that frustration. It remembers the touchdowns from your last game and your newborn's name without missing a beat.

Monumental books, verbose contracts, drawn out debates - StreamingLLM takes them all in its stride. No shortcuts, no forgetfulness. It's like upgrading your assistant's RAM to handle heavier workloads flawlessly.

[Jim Fan @DrJimFan]

Let's reverse engineer the phenomenal Tesla Optimus. No insider info, just my own analysis. Long read:

1. The smooth hand movements are almost certainly trained by imitation learning ("behavior cloning") from human operators. The alternative is reinforcement learning in simulation, but that typically leads to jittery motion and unnatural hand poses.

There're at least 4 ways to collect human demonstrations:
(1) A custom-built teleoperation system - I believe this is the most likely means used by Tesla team. Open-source example: ALOHA, a low-cost bimanual robot arm and teleoperation system by Stanford AI Labs (https://t.co/8iXpiHVEjS). It enables very precise, dexterous motions, such as putting AAA batteries into a remote or manipulating contact lens.

(2) Motion Capture (MoCap): apply the MoCap systems used for Hollywood movies to capture the fine-grained motions of hand joints. Optimus' 5-finger hand is a great design decision that enables a direct mapping - there is no "embodiment gap" from human operators.
For instance, a demonstrator can wear a CyberGlove (https://t.co/S8hxErsEuU) and grasp the cubes on the table (as shown in video). CyberGlove will capture the motion signals & haptic feedback in real-time, which can be re-targeted onto Optimus.

(3) Wearing gloves & markers can be clumsy. An alternative way to do MoCap is through computer vision. DexPilot from NVIDIA enables marker-less and glove-free data collection. The human operator simply uses their bare hands to perform the tasks. 4 Intel RealSense depth cameras and 2 NVIDIA Titan XP GPUs (yeah, 2019 work) translate the pixels to precise motion signals for robot learning.

(4) VR Headset: turn the training room into a VR game, and let humans "role play" Optimus. Use the native VR controller or CyberGlove to control the virtual Optimus hands. This has the advantage of scalable remote data collection - annotators from around the world can contribute without coming onsite.
VR demonstration technique appeared in research projects like the iGibson home robot simulator, an initiative that I participated in at Stanford: https://t.co/eyI4ORkH6G

Above 4 are not mutually exclusive. Optimus could use a combo of them for different pros & cons.

2. Neural Architecture. Optimus is trained end-to-end: videos in, actions out. I'm quite sure it's implemented by a multimodal Transformer with the following components:

(1) Image: some variant of efficient ViT, or simply an old ResNet/EfficientNet backbone (https://t.co/L6PLTQJnGA). The block pick-and-place demo doesn't require sophisticated vision. The spatial feature map from the image backbone can be tokenized easily.

(2) Video: two ways. Either flatten the video into a sequence of images and produce tokens independently, or have a video-level tokenizer. There're numerous ways to efficiently process video pixel volumes. You don't necessarily need Transformer backbones, e.g. SlowFast Network (https://t.co/qDdXzqwJQp) and RubiksNet (https://t.co/CQU8D7TZgx, my paper at ECCV 2020, efficient CUDA shift primitives).

(3) Language: it's not clear if Optimus is language prompted. If it is, there needs to be a way to "fuse" the language representations into perception. FiLM is a very lightweight neural network module that serves this purpose (https://t.co/VI4TpgQ22V).
You can think of it intuitively as a "cross attention" of language embedding into the image-processing neural pathway.

(4) Action tokenization: Optimus needs to convert the continuous motion signals into discrete tokens for the autoregressive Transformer to work. A few ways:
- Directly bin the continuous values for each hand joint control. [0, 0.01) -> token #0, [0.01, 0.02) -> token #1, etc. This is straightforward but could be inefficient due to the long sequence length.
- The joint movements are highly dependent on each other, which means they occupy a low-dimensional "state space". Apply VQVAE to the motion data to obtain a shorter-length, compressed token set.

(5) Putting the above pieces together, we have a Transformer controller that consumes video tokens (optionally with language modulation), and outputs action tokens, one step at a time. The next frame from the table is fed back to the Transformer, so it knows the consequence of its action. That gives the self-corrective ability shown in the demo.

I believe the architecture is most similar to:
- Google RT-1: https://t.co/dpuon1bqU6
- NVIDIA VIMA: https://t.co/Tn3L63uGrv

3. Lastly, I'm genuinely impressed by the hardware quality. The motions are fluid, and the aesthetics is amazing as well. As I mentioned above, it's such a great decision to follow human morphology closely, so that there is no gap in imitating humans.

Atlas from Boston Dynamics only has simple gripper-style hands. In the long run, Optimus' bi-dexterous, 5-finger hands will prove far superior in daily tasks.

Congrats to @Tesla_Optimus team & @elonmusk 🎉! I look forward to seeing the bots roam Mars some day 🦾

[Tivadar Danka @TivadarDanka]

Neural networks are stunningly powerful.

This is old news: deep learning is state-of-the-art in many fields, like computer vision and natural language processing. (But not everywhere.)

Why are neural networks so effective? I'll explain.

[gaut @0xgaut]

ok this is absolutely *wild*

using AI to live translate and sync your lips

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Research on advanced prompting techniques for language models has extended chain of thought and tree of thought prompting to graph-structured reasoning processes. But, did you know that there are two versions of “graph of thought” prompting that have been proposed already?

Some background. Advanced prompting techniques like chain of thought and tree of thought prompting have drastically improved the ability of large language models to solve complex, reasoning-based tasks. Forcing the LLM to construct a step-by-step response to a problem drastically improves its problem-solving capabilities, but all of these techniques assume that the reasoning process is linear.

“Human thinking is often characterized by its ability to make sudden leaps and connections between seemingly unrelated ideas, which can lead to novel insights and solutions. This non-linear, jumping thought process is a hallmark of human creativity, reasoning, and problem-solving abilities.” - from [1]

Graph-based reasoning. Humans do not seem to perform reasoning based on individual chains of thought. Rather, we make leaps and connections between ideas that lead to novel insights. Inspired by this idea, researchers have recently extended chain and tree of thoughts prompting to a graph-structured approach. We will take a look here at two (independent) papers that have already been written on this topic.

Graph of thought reasoning (GOTR). In [1], authors proposed a two-stage technique that:

1. Outputs a problem solving rationale given text (and potentially images) as input.
2. Outputs a final answer given the original input concatenated with a rationale.

This approach uses an encoder-decoder structure and is fine-tuned end-to-end. Several encoders ingest data from each of the different modalities that are considered. The output of these encoders is combined in a fusion layer, then passed to a decoder to generate output.

Where does the graph come in? So far, it seems like GOTR does not use any graphs within its reasoning process. However, the model creates a named entity graph based on the input text and generated rationale. Then, this graph is ingested by a graph attention network encoder and combined with all image/text features. As such, the decoder receives information from text, image, and graph-based inputs!

GoT prompting. In [2], authors follow a more traditional prompting approach, called graph of thought (GoT) prompting, that uses a system of causal LLMs and prompts to perform reasoning according to a graph structure. The reasoning process is modeled as a graph, where each node represents a thought or (partial) solution and connections indicate that a certain thought was generated from some other thought.

A system of LLMs. GoT prompting has several “modules” that control the reasoning process, including a top-level controller (controls the reasoning process), a parser (verifies and extracts LLM output), a scorer (judges the quality of solutions), and a prompter (writes prompts for each different module). Together, these modules can transform the underlying graph structure and work towards solving a reasoning problem.

TL;DR: Modeling an LLM’s reasoning process as a graph structure can be beneficial for certain problems and is (arguably) more comparable to the human reasoning process. But, these techniques tend to be more costly than basic CoT prompting and only provide a tangible benefit on select problems. For more details, check out the overview of these techniques that I just wrote for my newsletter.

--------
[1] Yao, Yao, Zuchao Li, and Hai Zhao. "Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language Models." arXiv preprint arXiv:2305.16582 (2023).
[2] Besta, Maciej, et al. "Graph of Thoughts: Solving Elaborate Problems with Large Language Models." arXiv preprint arXiv:2308.09687 (2023).