Sébastien Toth

[Sébastien Toth @TothSebastien]

So #GPT-3.5-turbo has 20B params ?

Didn't know that, extracted from this paper: https://t.co/5sNYYsp7oK from Microsoft Research.

[Sébastien Toth @TothSebastien]

2023 is an amazing time for entrepreneurship with so many new use-cases unlocked by A.I.

Funny that it's also this specific year many VC decided to buckle up and get tight on investments

I feel like BAs are taking back a huge role on seeding start ups and that's pretty good

[Sébastien Toth @TothSebastien]

@arvidkahl Congratz man, watching you keeping it up is extremely inspiring.

[Sébastien Toth @TothSebastien]

Med-PaLM-2 outsandingly beating GPT3.5 shows a very interesting yet under-developed domain for research & business opportunities:

Expert LLMs

[Sébastien Toth @TothSebastien]

@huggingface now that we know that #Bloom wasn't trained #chinchilla optimal, please start the training again !

Considering what we know now, Bloom is a hell of a LM, @Thom_Wolf.

#opensource #chatgpt

[Sébastien Toth @TothSebastien]

Google's "We don't have any MOTA" leak made quite a lot of noise.

People keep complaining about OpenAI, Anthropic & Google switch to closer & closer source.

It's not about them getting open, it's about giving strength to open-source players #huggingface

[Sébastien Toth @TothSebastien]

Remember when you needed 6 months of research & data labeling to learn one simple task such as text classification or some kind of context-based generation ?

That was 6 months ago.

Now it takes a few minutes of prompt-tuning and you're good to go.

[Sébastien Toth @TothSebastien]

I may be too old school but I think these kind of content have no place in an "academic research paper"

Looks like that "papers" have become yet another form of content no different than twitter threads or medium posts.

h-index is losing value, fast.

[Hervé "pyannote" Bredin @hbredin]

#pyannote models are now #2 and #3 most downloaded audio models on @huggingface hub.

Yet, I am struggling finding applicants for the two open positions to make it even better (2 years research engineer, 3 years PhD scholarshop)

Can anyone help?

https://t.co/2ghHcpy7IU

[Sébastien Toth @TothSebastien]

@garrytan Hey @garrytan , https://t.co/BHMS1120XB can actually do that pretty easily !

[Sébastien Toth @TothSebastien]

In the end all these GAFAs are just comparing the size of their LLMs

Talk about maturity

[Sébastien Toth @TothSebastien]

I think people don't get how much ChatGPT & its successors will significantly raise the worldwide average human intelligence over time.

StockFish & its predecessor was a huge improvement for chess performance, same here.

[Sébastien Toth @TothSebastien]

prediction:
LLM-generated content will rapidly become worthless and low-value compaired to human-generated content (except for synthesis).

We use language to dialog with other humans, it's losing value if there is no empathy towards the writer or the reader.

Happened with chess

[Sébastien Toth @TothSebastien]

Isn't it already considered rude to send a ChatGPT-generated piece of content ?

[Sébastien Toth @TothSebastien]

@NafetsWirth @fba Master teach me to be so productive yet so calm !!

I'm admirative !

[Sébastien Toth @TothSebastien]

Interestingly, #ChatGPT is behind fine-tuned #SOTA in most tasks.

Of course, those #benchmarks are not perfectly designed, and metrics are flawed.

But what's interesting is that ChatGPT has been trained to... look like it's helping us.

And we love that.

[Sébastien Toth @TothSebastien]

@arvidkahl not about the number of connections but about the strength of each connection

[Sébastien Toth @TothSebastien]

The worst thing about the 9-5 job is the tyranny of low expectations.

For many it's becoming unbearable.

Having hard challenges & being responsible to overcome them is part of what makes entrepreneurship so rewarding

Let me fail, as long as I fought properly

[Sébastien Toth @TothSebastien]

It would be great for the scientific community to have the total compute budget spent (in term of AWS-equivalent pricing GPU hours) for each papers & model.

[Sébastien Toth @TothSebastien]

@Alexmathers84 Congratz on the growth @Alexmathers84 btw !