Taeyoung Lim

Social Enterpreneur;
Cloud Computing, IoT, Connected Car Business Professional;
Hyperscale generative AI Business Development;

[Taeyoung Lim @Haha_Dad]

[스타트업 28주차 — 책 두 권, 그리고 슈퍼 개인 https://t.co/OTfJVFKVgT
최근 두 권의 책에서 각각 다른 방향의 답을 찾았다.
첫 번째 책은 퍼블리 박소령 대표의 《실패를 통과하는 일》이었다.
두 번째 책은 윤석빈 저자의 《AI 오케스트레이터와 슈퍼 1인 기업》이었다.

[Taeyoung Lim @Haha_Dad]

검색-문서제작-PPT-캔바-노션을 한번에 자동화? felo ai 사용기 https://t.co/CTi6LZS9gW

[Taeyoung Lim @Haha_Dad]

[PTM] cursor를 이용한 개발자의 파이썬 정복기(라고 쓰고 고생기라고 읽자) https://t.co/CDft3ZgkfP

[Taeyoung Lim @Haha_Dad]

🐢거북이처럼 n8n 노드 구축 끙끙대며 한땀한땀 시작하기 (feat. AI agent with RAG 처음으로 구현) https://t.co/2OcbCsRkxt

[Taeyoung Lim @Haha_Dad]

TRIZ 발명 원리 및 특허 분석 GPTs https://t.co/RQEoxzpDyX

[Taeyoung Lim @Haha_Dad]

ChatGPT + Notion 사용자를 위한 구원자, Typora로 전자책 쉽게 완성하기 https://t.co/ByPYWpbfFC
아주 유용한 팁

[Taeyoung Lim @Haha_Dad]

Why people are falling in love with AI chatbots https://t.co/nYkE899JYX - @Verge

[Taeyoung Lim @Haha_Dad]

Directly from Greg!

[Greg Brockman @gdb]

Sam and I are shocked and saddened by what the board did today.

Let us first say thank you to all the incredible people who we have worked with at OpenAI, our customers, our investors, and all of those who have been reaching out.

We too are still trying to figure out exactly what happened. Here is what we know:

- Last night, Sam got a text from Ilya asking to talk at noon Friday. Sam joined a Google Meet and the whole board, except Greg, was there. Ilya told Sam he was being fired and that the news was going out very soon.

- At 12:19pm, Greg got a text from Ilya asking for a quick call. At 12:23pm, Ilya sent a Google Meet link. Greg was told that he was being removed from the board (but was vital to the company and would retain his role) and that Sam had been fired. Around the same time, OpenAI published a blog post.

- As far as we know, the management team was made aware of this shortly after, other than Mira who found out the night prior.

The outpouring of support has been really nice; thank you, but please don’t spend any time being concerned. We will be fine. Greater things coming soon.

[Taeyoung Lim @Haha_Dad]

128K Contexts& Fresher knowledge, Retrieval & Code Interpreter, Multimodel GPT4 w/ Vision& Speech

[OpenAI @OpenAI]

We're rolling out new features and improvements that developers have been asking for:

1. Our new model GPT-4 Turbo supports 128K context and has fresher knowledge than GPT-4. Its input and output tokens are respectively 3× and 2× less expensive than GPT-4. It’s available now to all developers in preview.

2. Assistants API and new tools (Retrieval, Code Interpreter) will help developers build world-class AI assistants within their own apps.

3. The platform is becoming multimodal. GPT-4 Turbo with Vision, DALL·E 3, and text-to-speech are all now available to developers.

Oh… and we’re doubling GPT-4 rate limits. https://t.co/BMnsBAHorI

[OpenAI @OpenAI]

We're rolling out new features and improvements that developers have been asking for:

1. Our new model GPT-4 Turbo supports 128K context and has fresher knowledge than GPT-4. Its input and output tokens are respectively 3× and 2× less expensive than GPT-4. It’s available now to all developers in preview.

2. Assistants API and new tools (Retrieval, Code Interpreter) will help developers build world-class AI assistants within their own apps.

3. The platform is becoming multimodal. GPT-4 Turbo with Vision, DALL·E 3, and text-to-speech are all now available to developers.

Oh… and we’re doubling GPT-4 rate limits. https://t.co/BMnsBAHorI

[Taeyoung Lim @Haha_Dad]

Karikó, Higgs, and patience we need.

[Mushtaq Bilal, PhD @MushtaqBilalPhD]

Nature rejected her paper for not being original,

University of Pennsylvania (her employer) demoted her,

and yesterday Katalin Karikó won the Noble Prize in physiology.

In mid-2000s, Karikó and her Drew Weismann submitted their paper on mRNA (messenger Ribonucleic Acid) to Nature.

Nature desk rejected their paper for being "an incremental contribution" only. The paper was later published in another journal, Immunity.

Earlier in her career at the University of Pennsylvania, Karikó was demoted because her applications for grants kept getting rejected.
But Karikó persevered and kept on going.

In 2013, she joined BioNTech, a German company founded by two scientists, Uğur Şahin and Özlem Türeci. In 2018, BioNTech partnered with Pfizer to develop mRNA vaccines against the influenza virus.

When the COVID-19 hit the world, Karikó's research helped Pfizer to produce the first vaccine against the disease.

I don't know how the Nature editors who desk rejected Karikó's paper and the Penn administration who demoted her feel about Karikó Nobel Prize.

Takeaway: Many academics and scientists worry about getting published in "prestigious" journals. Instead of worrying about prestige, we should try to put our work out as quickly as possible like Karikó did.

Once you put your work out without caring about prestige, two good things happen:

1. Your work will lead to newer opportunities.
2. You will start getting feedback from the scholarly community, which you can use to iterate and improve.

Here's another interesting Nobel Prize story.

Peter Higgs, a British physicist, joined the University of Edinburgh in 1956. By 1964, Higgs has published his groundbreaking work about subatomic particles.

After 1964, Higgs published less than 10 papers.
When his department would ask him how many papers, he published in a given year, he would reply "None."

It happened so often that he stared feeling like an "embarrassment to the department."

The University of Edinburgh, however, never fired Higgs because in 1980 he had been nominated for the Nobel Prize.

Higgs retired in 1996 and stayed on as an emeritus professor at Edinburg.

In 2012, experiments conducted at the CERN laboratory confirmed Higgs work and the existence of Higgs Particle.

And in 2013, Higgs was awarded the Noble Prize in physics and the University of Edinburgh got rewarded for being patient.

[Taeyoung Lim @Haha_Dad]

안드로이드 코딩 보조 AI, Studio Bot

[GeekNews @GeekNewsHada]

구글, 안드로이드 AI 코딩 보조 프로그램 Studio Bot을 170개국에 공개 https://t.co/OfRsENxAqm
- 올해 Google I/O 에서 발표한 Android용 AI 코딩 보조 프로그램
- 자연어를 이해하여 코드 생성, 질문 답변, 개발 관련 리소스를 찾아줌
- 이제 전 세계 170개국 이상의 Android Studio...

[Taeyoung Lim @Haha_Dad]

!!

[Bishal Nandi @LearnWithBishal]

Google just released 10 FREE courses to master Generative AI. ( 4 New Courses🔥)

1. Introduction to Generative AI

This is an introductory-level microlearning course aimed at explaining Generative AI, how it is used, and how it differs.

👉 https://t.co/7pxVVOtKTb

Thread 🧵

[Taeyoung Lim @Haha_Dad]

LLM as Optimizers by Google DeepMind

[elvis @omarsar0]

LLMs as Optimizers

This is a really neat idea. This new paper from Google DeepMind proposes an approach where the optimization problem is described in natural language.

An LLM is then instructed to iteratively generate new solutions based on the defined problem and previously found solutions.

It was first tested on linear regression and the traveling salesman problem. Leveraging LLMs with simple prompting match or surpass hand-designed heuristic algorithms. This shows good potential for using LLMs as optimizers.

The idea is then applied to prompt optimization that aims to maximize task accuracy on different tasks like math word problem-solving.

The first piece of the proposed meta-prompt takes in previously generated prompts along with corresponding training accuracies. The second piece includes the optimization problem description with samples obtained from a training set representing the task.

At each optimization step, the goal is to generate new prompts that increase test accuracy based on the trajectory of previously generated prompts.

The optimized prompts outperform human-designed prompts on GSM8K and Big-Bench Hard, sometimes by over 50%!

For math word problem solving, one of the most effective instructions found begins with "Take a deep breath and work on this problem step-by-step".

https://t.co/GsF8fzjevX

[Taeyoung Lim @Haha_Dad]

#1 Open LLM so far!!!

[Dr. Ebtesam Almazrouei @Dr_Almazrouei]

Falcon 180B

[Taeyoung Lim @Haha_Dad]

New Prompting skills

[elvis @omarsar0]

Graph of Thoughts

Presents a prompting approach that models text generated by LLMs as an arbitrary graph.

It enables combining arbitrary "thoughts" and enhancing them using feedback loops.

The core idea is to enhance the LLM capabilities through "network reasoning" and without any model updates. This could be seen as a generalization of the now popular Chain-of-Thought and Tree-of-Thought.

The interesting part is that this approach can enable rapid prototyping of even more novel prompting ideas. I've been thinking about something like this for some time.

It makes sense to build prompts that operate more generally and that could work with all kinds of models regardless of formatting or style. I think the key with prompting models is how and when information is combined and graphs are excellent at solving these type of problems.

I will be writing an extended summary of this and adding it to our prompting guide as well: https://t.co/o4KzoHfCSu

The authors suggest that this work brings "LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks."

Paper: https://t.co/EAKsDPkFDV

[Taeyoung Lim @Haha_Dad]

AI21 Labs Announces Series C Funding Round at $1.4 Billion Valuation https://t.co/DPT9kOqSDz

[Taeyoung Lim @Haha_Dad]

!!!

[WizardLM @WizardLM_AI]

🔥🔥🔥
Introduce the newest WizardMath models (70B/13B/7B) !

WizardMath 70B achieves:
1. Surpasses ChatGPT-3.5, Claude Instant-1, PaLM-2 and Chinchilla on GSM8k with 81.6 Pass@1

2. Surpasses Text-davinci-002, GAL, PaLM, GPT-3 on MATH with 22.7 Pass@1

3. Surpasses all other open-source LLMs on both GSM8k and MATH by a substantial margin

4. Paper is coming next week, with brand-new Reinforced Evol-Instruct method for math LLMs

🏇
HF Weights: https://t.co/pAyBYhPGbL
Github: https://t.co/OwvWSO0q4Z
Discord: https://t.co/jjYGPeA6AS

[Taeyoung Lim @Haha_Dad]

Fine-tuning of Falcon-7B Large Language Model using QLoRA on Mental Health Conversational Dataset by @iamarunbrahma https://t.co/pZE3J3VQVz

[Taeyoung Lim @Haha_Dad]

OSS LLM모델에 대한 객관적인 평가. Llama도 T5를 넘어서질 못하는구나

[Yi Tay @YiTayML]

Hot take 🔥: Lots of buzz these days about new foundation open-source models but what if I told you there have been no real advance since 2019's T5 models 😀

Take a look at this table from this new InstructEval paper: https://t.co/Pe3ajGtavs. Some thoughts/observations:

1. Flan-T5 beats everything, including Alpaca (LLama-based), Flan-Alpaca, Mosiac-Chat/MPT, Dolly.
2. If you arrange this table in terms of "compute-match", encoder-decoder should have been in a different (lower) weight class. Basically, Flan-T5 3B is like a 1B+ decoder and Flan-UL2 is more like a 8B+ model. With this perspective, the gap is so dramatically huge that it's not even funny.
3. Flan-UL2 basically wrecks Alpaca-Lora 30B despite being so much smaller and effectively 4x less compute.
4. This is not entirely about Flan series models - it's more about the base models! The point is that the base T5 models are already ridiculously strong. 1 trillion tokens, just blatantly repeating C4 to heart's content. There's also mT5 and uMT5 which are strongly multilingual and ridiculously good. The base models are not long context, but Flan mitigates this.
5. The weakness is that T5/UL2 models are not diverse and are only C4 trained that means they probably don't do well at code/math whatever (code eval 0 score below lol). But its scary how strong a C4-only baseline that we had since 2019 is performing.
6. If you look at fastchat-t5 on the chatbot arena by @lmsysorg the 3B fastchat model does as well as MPT-7B et al despite being only 3B (and if you paid attention till now you know that's a 1B+ equivalent decoder-only model). That's really insane if you think of a 1B+ model on a leaderboard of all these new "OSS LLM advances".
7. It's likely at compute match, T5 >> Llama. The only problem is that we don't have T5 models at 30B and 65B.

You're welcome.