Luigi Laura

Big Data, Big Graphs, Algorithms, and some competitive programming. Books, comics, novels, and aphorisms.

[Future Internet_MDPI @FutureInternet6]

🔔 New Published Papers of #MDPIfutureinternet

Title: Transfer Learning Strategies for Comic Character Recognition in Low-Data Regimes: A Comparative Study

https://t.co/TsM4DNxKp5

#transferlearning #ConvolutionalNeuralNetworks #CNNs #featureextraction #dataaugmentation

[Luigi Laura @luigilaura]

Come ha detto una volta Andy Warhol: “Nel futuro ogni LLM sarà il migliore al mondo per quindici minuti”

[Luigi Laura @luigilaura]

@rberetta83 Meravigliosa!

[TITAN - AI Coaching to Fight Disinformation @TITAN_Thinking]

🔹How can we stop #Disinformation ?
🔹How can we create trustful news?

Find the answers during the #EURegionsWeek ws. 👇

Separating fact from fiction: Creating a smarter, more truthful and trusted Europe

https://t.co/cWE8SpDgDA

#Crossbordercooperation @EUinmyRegion @EU_CoR

[TITAN - AI Coaching to Fight Disinformation @TITAN_Thinking]

We've published 🧑‍🤝‍🧑user requirements 🧑‍🤝‍🧑from our work with citizens and experts on AI data and ethics. Check them out here 👇
https://t.co/JX5z94GtJL

[gaut @0xgaut]

Someone built a frontend to Reddit that looks like Outlook so you can browse it all day at work without worrying about who is looking over your shoulder.

I love the internet.

[Bindu Reddy @bindureddy]

Types of Neural Networks - Evolution Of Deep Learning Architectures.

Oppenheimer, the movie, has all of us thinking about the 40s and WW2. Believe it or not, the first neural networks (NN) were invented around the same time, in 1943!

Warren McCulloch and Walter Pitts the founding fathers of NNs, were intrigued by how biological neurons worked and proposed a mathematical model for a NN

It was not until 1958, that Frank Rosenblatt invented the "Perceptron" which was basically a computer program designed to learn from its mistakes. It ran on a very big machine and essentially did binary classification. While there was a lot of excitement around these baby NNs they required a lot of compute and data, which meant that they needed some serious funding.

In 1969, a paper titled "Perceptrons" by Minsky and Papert, killed almost all innovation in NNs. The paper proved that the single perceptron, couldn't solve simple problems including the XOR problem, and was severely limiting and all funding stopped. At the same time, algorithms like Support Vector Machines (SVMs) start taking off and NNs took a back seat.

Multi-layer perceptrons (MLPs) were viewed as a way to address the issues that single-layer perceptrons had, but training these MLPs proved to be very difficult. Not until 1986, did we see the resurgence NNs. Rumelhart, Hinton, and Williams introduced the backpropagation algorithm, and suddenly training multi-layer NNs became tractable. Computers were becoming more powerful and more data become available. NNs were back in business.

In the late 80s, Yann LeCunn introduced CNNs, The convolutional layers of a CNN can model the spatial hierarchy of images and NNs started to become useful in image-processing applications. Still, SVMs were the cool kids and NNs were being used for niche tasks like handwriting recognition.

Only in the 2000s, did we see a true renaissance of NNs. Geoff Hinton introduced Deep Belief Networks and the term deep learning (DL) begin to take off.

In 2012, Deep Learning had a seminal breakthrough with a CNN called AlexNet that outperformed all other algorithms in image classification. Since then we have seen an explosion in NN architectures.

Recurrent Neural Networks (RNNs) and Long Short-Term Memory Networks were useful in understanding patterns in sequential data. In 2015, ResNets helped solve the vanishing gradient problem (another pesky issue with DL training), and DL research was exploding.

In 2014, generative NNs had a big moment - Generative Adversarial Networks (GANs) were invented by Ian Goodfellow et. al. GANs were really good at generating realistic images. The first deep fake was born :)

Finally, in 2017, Vaswani et al introduced Transformers. Transformers, with their self-attention mechanism allowed the model to weigh the importance of each word in relation to others and better understand language.

BERT in 2018, was a specific implementation on Transformers and can look and understand text in both directions. BERT is pre-trained on massive amounts of data (e.g. Wikipedia) and can be adapted to specific tasks with fine-tuning. BERT can be adapted to multiple tasks like Q/A and text classification

Just a few months earlier, also in 2018, OpenAI introduced the GPT models. These were unidirectional but also were trained on massive amounts of data. Unlike BERT, GPTs are fine-tuned for generation or next-word prediction. Since 2018, we have seen better and more sophisticated versions of the GPT series...with GPT-4 released in 2023, being capable of human-level cognition, generation, and basic reasoning!!

So what started almost 80 years ago is now finally beginning to take over and transform the world completely!! ���

[Turing Post @TheTuringPost]

The LLaMa model sparked an open-source revolution, giving birth to 11 incredible projects:

🦙 Alpaca
🦙 Vicuna
🦙 Koala
🦙 Dalai
🦙 GPT4ALL
🦙 ChatLLama
🦙 ColossalChat
🦙 FreedomGPT
🦙 Lit-LLaMA
🦙 KoAlpaca
🦙 ParroT

Retweet this and we'll send you the links to all models!

[Luigi Laura @luigilaura]

@Setapp Thanks, just did it!

[Jim Fan @DrJimFan]

I believe next-gen LLMs will heavily borrow insights from a decade of game AI research.

▸ Noam Brown, creator of Libratus poker AI, is joining OpenAI.
▸ Demis Hassabis says that DeepMind Gemini will tap techniques from AlphaGo.

These moves make a lot of sense. Methods like self-play (training) and tree search (inference) helped machines beat human champions in Go, Poker, Dota, and StarCraft. They improve a model's reasoning capabilities in a highly scalable fashion.

We are already seeing such ideas being added to the LLM arsenal. Voyager is an inference time algorithm that enables an agent to continuously write code and bootstrap its skills in Minecraft. Tree of Thought combines search with LLM in-context to boost reasoning.

Many more will follow.

[Massimo Mecella @mecellone]

Thanks to #summersoc2023 for inviting us to give a tutorial on I4.0 and services
A great experience and a wonderful community
With @__fleotta @iaiauno Luciana Silo
Supported (also) by MICS spoke 8

[Marco Angelini @Lynos79]

Yesterday we presented our State-of-the-Art report on Visual Analytics for eXplainable Deep Learning. It was great to discuss it in presence and receive feedback at #EuroVis2023!!

Paper: https://t.co/hh7ylcl2cw
Live explorer: https://t.co/iq74OEsOWC

[Luigi Laura @luigilaura]

@A_Sassano Io ho avuto il privilegio di seguirne due 😁

[Luigi Laura @luigilaura]

TITAN meeting in Rome!

[Luigi Laura @luigilaura]

@heyeaslo @readwise save thread

[Andy Matuschak @andy_matuschak]

It's probably parochial, but I guess I've developed a deep faith in "the internet delivers." Like: you write a lot of good words, and then you ask for things, and the mob that is the internet grants your wishes? I know this is naive, but it seems like a pretty predictive model!

[Andy Matuschak @andy_matuschak]

Just spent a few days with friends, often in conversation about various kinds of stuckness. Amazing how often "what if… you wrote seriously about your project(s) on the internet?" seemed like the best path forward.

[Kanjun 🐙 @kanjun]

Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why?

We're launching @genintelligent & open sourcing our research environment, Avalon, so as a community we can answer these questions: https://t.co/dNs0TpQeh3

[Emanuele Rossi @emaros96]

Italian meetup at NeurIps 2022 🇮🇹