Cristian Simionescu

PhD Student at the Faculty of Computer Science Iași

[Cristian Simionescu @CrisSimi2525]

@j_mcgraph I have been trying to upgrade from Plus to Pro subscription for two weeks now with no success.
So this is why

[Cristian Simionescu @CrisSimi2525]

@yegordb Do you mean productivity as in output per hour?
That might work in the sense that "practice makes perfect", but not sure how big that multipler can get when moving past the standard 40h/week.

[Cristian Simionescu @CrisSimi2525]

@jamesdouma @MikeStirner Wouldn’t more sensors (helping crutch) become less and less beneficial as the software matures more and more? So either they add more sensors at the start and start cutting down over time, or this is the most sensors they will ever have.

[Cristian Simionescu @CrisSimi2525]

Finished this year’s edition of Pavia University Winter School. A very well timed school to have thoughtful multi-disciplinary discussions about the current and future use of AI!

[Cristian Simionescu @CrisSimi2525]

@_albertgu @tri_dao I can't tell why delta comprises both a parameter + s delta, when B and C are just taken from the projection.
Is there a reason why the parameter is needed and can't simply be learned by the projection?
And is there a reason why B and C wouldn't benefit from having parameters?

[Cristian Simionescu @CrisSimi2525]

@zacharylipton Yes, if you consider the bucket tool in Paint or Instagram filters

[Thomas Wolf @Thom_Wolf]

Over the past weeks the H4 team has been busy pushing the Zephyr 7B model to new heights 🗻

The new version is now topping all 7b models on chat evals and even 10x larger models 🤯🔥

Here are the intuitions on it

1/ Start with the strongest pretrained model you can find: Mistral 7B is amazing, by far the strongest 7B pretrained model. Props to the Mistral AI team 😍

2/ Scale human-preference annotations: several studies have show how for many tasks GPT4 is on-par with the average human annotators while making scalable annotations as easy as an API call: the H4 team started from the largest and most diverse public GPT4 preference annotation dataset: UltraFeedback 🤖🦾

3/ Drop Reinforcement Learning in favor of DPO (Direct Preference Optimization): while using LLM in RL is definitely much easier compared to the struggles of getting deep-RL to work from scratch, DPO totally remove RL from the preference annotation training and directly optimize the preference model in a much more stable training procedure in the H4 team's experiments

4/ Don't be scared of overfitting on preference dataset: This is maybe the most counterintuitive results of the work. While the train/test loss of DPO training shows signs of overfitting on the feedback dataset after just one epoch, training further still show significant improvements on downstream tasks even up to 3 epochs without signs of performances regression. Would be interesting to dive even further in this surprising behavior

5/ Share everything openly 😁 the recipes, code, model and dataset will be available at https://t.co/qkKhS8eqvt
In the meantime the paper is a great starting point: https://t.co/qzZ0tb8Aq1

[Cristian Simionescu @CrisSimi2525]

@fchollet How sure are we that intelligence isn’t still optimization just on a larger signal? For example, conversational AI might not be “intelligent” simply because it optimizes on a very slim (and distorted) projection that is digital text.

[Cristian Simionescu @CrisSimi2525]

@zacharylipton Now I am curious if anyone tried NAS using a large collection of multiple datasets from different domains and different modalities

[Cristian Simionescu @CrisSimi2525]

@zacharylipton Probably current methods overfit the data that is used for searching and/or the random seeds

[Cristian Simionescu @CrisSimi2525]

Great presentations and discussions today at #KES2023

[Yann LeCun @ylecun]

This is huge: Llama-v2 is open source, with a license that authorizes commercial use!

This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers

Pretrained and fine-tuned models are available with 7B, 13B and 70B parameters.

Llama-2 website: https://t.co/PKrrXgHdem
Llama-2 paper: https://t.co/aINNrXNhMb

A number of personalities from industry and academia have endorsed our open source approach: https://t.co/N7HwgW9Suh

[Cristian Simionescu @CrisSimi2525]

@sama This should be included in the humblebrag hall of fame

[Cristian Simionescu @CrisSimi2525]

@alex_avoigt Technologically regressing, a few decades, from more nuclear and less coal to using more coal and no nuclear is pretty wild. No sanctions needed, all self-imposed. Incredible

[Cristian Simionescu @CrisSimi2525]

@ykilcher Was about to close the video during the lil trolling at the start, almost got me there, nice one 😂

[Cristian Simionescu @CrisSimi2525]

Attending #AAAI23 @RealAAAI this week. Any *must see* posters/presentations recommendations? Especially interested in topics related to self-supervision, interested to have interesting discussions, reach out if interested to talk 🙂

[Cristian Simionescu @CrisSimi2525]

@patanjal @RayDalio Not every business has the same level of debt. If everyone raises prices in proportion to how much debt they have, the ones who have low debt will stay very competitive and capture the market, if the high debt ones raise prices a lot, they might lose all their customers

[Cristian Simionescu @CrisSimi2525]

@karpathy Even if the AI is trained on human data, which is limited by human understanding and intelligence, the fact that a single entity could learn ALL specializations, not just one niche like humans, should be superhuman. The ultimate interdisciplinary researcher, no more blindspots

[Cristian Simionescu @CrisSimi2525]

@AndreiCaramitru With the Leu being defacto pegged to the Euro (with a ~4.95 Eur-Leu rate limit), wouldn't going straight to leu bonds with higher yield be better?

[Cristian Simionescu @CrisSimi2525]

@Jason @Google If the email said "because a super useful assistant AI would reduce ad revenue from Search and watching content on Youtube", it might have been more credible