Cheng-Te Li

Professor on #datascience and #machinelearning at National Cheng Kung University at Taiwan, applications to #graphdata, #socialmedia, #recsys.

[Canyu Chen @CanyuChen3]

🔥New #ICLR2024 Paper🔥: "Can LLM-Generated Misinformation Be Detected?"

Also very honored to receive the🏆Didactic Paper Award in workshop ICBINB@NeurIPS 2023 @ICBINBWorkshop

Project website (paper, code, dataset): https://t.co/OlViWkz8g8

1/🧵

[Hyung Won Chung @hwchung27]

I gave a talk at Seoul National University.

I titled the talk “Large Language Models (in 2023)”. This was an ambitious attempt to summarize our exploding field.

Video: https://t.co/vumzAtUvBl
Slides: https://t.co/IidLe4JfrC

Trying to summarize the field forced me to think about what really matters in the field. While scaling undeniably stands out, its far-reaching implications are more nuanced. I share my thoughts on scaling from three angles:

1) Change in perspective is necessary because some abilities only emerge at a certain scale. Even if some abilities don’t work with the current generation LLMs, we should not claim that it doesn’t work. Rather, we should think it doesn’t work yet. Once larger models are available many conclusions change.

This also means that some conclusions from the past are invalidated and we need to constantly unlearn intuitions built on top of such ideas.

2) From first-principles, scaling up the Transformer amounts to efficiently doing matrix multiplications with many, many machines. I see many researchers in the field of LLM who are not familiar with how scaling is actually done. This section is targeted for technical audiences who want to understand what it means to train large models.

3) I talk about what we should think about for further scaling (think 10000x GPT-4 scale). To me scaling isn’t just doing the same thing with more machines. It entails finding the inductive bias that is the bottleneck in further scaling.

I believe that the maximum likelihood objective function is the bottleneck in achieving the scale of 10000x GPT-4 level. Learning the objective function with an expressive neural net is the next paradigm that is a lot more scalable. With the compute cost going down exponentially, scalable methods eventually win. Don’t compete with that.

In all of these sections, I strive to describe everything from first-principles. In an extremely fast moving field like LLM, no one can keep up. I believe that understanding the core ideas by deriving from first-principles is the only scalable approach.

[Cheng-Te Li @chengtel]

We also prepare the #awesome #GNN4TDL page on GitHub!

https://t.co/rRbjfRGsot

[Cheng-Te Li @chengtel]

🚀 Our survey paper “Graph Neural Networks for Tabular Data Learning” #GNN4TDL is now out! Dive into how and why can the recent advancements in #GNNs enhance tabular data predictions. 📊💡

https://t.co/EjUI8sKA4a

#GraphNeuralNetworks
#MachineLearning
#DataScience
#TabularData

[elvis @omarsar0]

LLMs in Medicine

A comprehensive survey (analyzing 300+ papers) on LLMs in medicine.

Includes an overview of the principles, applications, and challenges faced by LLMs in medicine.

https://t.co/OqLxc32GjE

[Camilo Ruiz @_camiloruiz]

Can deep learning work on small data with far more features than samples? We present PLATO: a method that achieves the state-of-the-art on such datasets by using prior domain information! https://t.co/GdsqIHRL8a 🧵

Published in #NeurIPS2023 with @ren_hongyu @kexinhuang5 @jure

[Jure Leskovec @jure]

Here's a great #gaming use case for #GraphLearning https://t.co/Mq8dGZEwvC #GraphNeuralNetworks

[Jure Leskovec @jure]

There's a lot of goodness in #GraphNeuralNetworks including better model quality over traditional #MachineLearning, you can learn more here: https://t.co/DwJFxA2ECV

[Jure Leskovec @jure]

Relational Deep Learning is brings the power of Graph Representation Learning to a Relational Database.

Slides from my keynote at @LogConference yesterday:
https://t.co/qaRXaMqojl

[Jure Leskovec @jure]

Relational Deep Learning is bringing the power of representation learning to relational databases and data warehouses. This breakthrough technology offers lots of opportunity for impact and raises exciting new research questions.
Check out the paper here: https://t.co/9zXQqVlWKV and benchmark datasets and code here: https://t.co/BzwWxv9lNb
#RelationalDeepLearning #RepresentationLearning #AI #Research

[Kirk Borne @KirkDBorne]

Understanding #DeepLearning (download 541-page PDF eBook): https://t.co/Z3ZjaEnk2m
——————
#BigData #DataScience #AI #ML #MachineLearning #NeuralNetworks #ReinforcementLearning #NLProc #ComputerVision #Algorithms #DataScientists #mathematics

[Bindu Reddy @bindureddy]

Common Probability Distributions That Occur In Nature And Why Mathematics Is The Language Of The Universe

Fundamentally the patterns of the universe repeat themselves making machine learning and AI possible.

Basically, when training ML models, including NNs we are attempting to model the patterns in the data. A good model can predict unseen data points that are part of the original data distribution.

Here are the common probability distributions and where they occur naturally

Symmetry and normal distributions - Nature loves balance. You see it in butterfly wings, faces, and even in physical laws. Natural phenomena that exhibit symmetry often fit well with a normal distribution.

The Central Limit Theorem tells us that the sum of many independent, identically distributed variables approaches a normal distribution. These distributions are characterized by a mean and a standard deviation and everything from human heights, IQ scores, and even the velocities of molecules in a gas follow a normal distribution

Log-normal distribution - If the logarithm of the variable follows a normal distribution, then the variable itself has a log-normal distribution. The distribution of financial assets, or the sizes of particles generated by grinding, blood pressure, and metabolic rates in various organisms often follow a log-normal distribution.

The Log-Normal Distribution often arises from multiplicative growth processes, where you're repeatedly multiplying by some random factor.

Uniform Distribution. This is self-explanatory. All outcomes are equally likely. In the continuous case, the probability density function is flat between the minimum and maximum values. Rolling a fair die, or picking a card from a well-shuffled deck follows uniform distributions

Poisson Distribution It describes the probability of a given number of events occurring in a fixed interval of time or space. The events must occur with a known constant mean rate and independently of each other. It is the go-to for predicting rare events.

Poisson distribution models the number of times a radioactive atom decays in a given time period, or the number of cars passing through a certain point in a given time frame.

If a particular intersection sees an average of 10 cars passing through it per minute, the Poisson Distribution can estimate the probability of 15 cars passing through in the next minute. Similarly in sports like hockey or soccer where goals are rare, you can use distribution to model the number of goals scored by each team.

Bernoulli Distribution, the binary star of the probability universe! Scientifically speaking, it's a discrete probability distribution that models a single experiment with exactly two outcomes, often termed "success" and "failure," and is characterized by a single parameter p, the probability of success.

The flip of a coin, genetic inheritance (you either have a trait or you don't), and the polarization of a single photon (either vertical or horizontal) model this distribution.

Binomial Distribution It describes the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success.

For example, The number of heads when flipping a coin multiple times, or the number of defective items in a batch of manufactured goods can be modeled with this distribution.

Exponential Distribution models the time you have to wait for the next event in scenarios where events happen independently and at a constant average rate. It's often used to describe phenomena like the time between bus arrivals or the intervals between radioactive decay events.

In essence, it's the go-to distribution for understanding "waiting times" in various natural and man-made systems.

This is a summary of the most common distributions. There are a bunch of other distributions that are related or variations on the above distributions and the image below captures some of them.

These naturally occurring distributions speak to why mathematics is the language of the universe and how you can use it to model and predict its behavior

[Lior Alexander @LiorOnAI]

Just found out about scikit-llm.

Sklearn Meets Large Language Models. It allows you to integrate language models like ChatGPT into scikit-learn for text analysis tasks.

https://t.co/gOsLli9Dx4

[Srijan Kumar - Lighthouz AI (YC S24) 🤖 @srijankedia]

Dynamic Graph Neural Networks #GNNs are popular for modeling dynamic data (financial transactions, communication networks, etc). But are dynamic GNNs stable to smart adversarial attacks?

@ksartik answers this #KDD2023!
Paper https://t.co/SnwCrXuPrn
Code https://t.co/TE9jYcvIGg

[Srijan Kumar - Lighthouz AI (YC S24) 🤖 @srijankedia]

We all know information spreads across communities and platforms. But how? What are the typical pathways of spread? Can we predict spread? @AhrenJin answers these questions in #KDD2023 presentation!

Paper: https://t.co/Ss4sdpy2oH
Website: https://t.co/aJzXo4pTNG

[Sahar Abdelnabi 🕊 @sahar_abdelnabi]

I, unfortunately, was not able to present our @USENIXSecurity paper in person due to visa issues. If you are around and interested, please check out my recorded talk later today. Of course, I would be very happy to chat here, via email, or via the conference Slack! #usesec23

[Clint Kruger @clintkruger]

Write Essays Like a Pro with ChatGPT

Use this ChatGPT Prompt to Structure Your Next Academic Essay

Prompt:

"Develop a comprehensive structure for a 1000-word paper on the subject of 'YOUR TOPIC'. The article should examine how 'YOUR SUBJECT' will transform a specific process or phenomenon.

This structure should be organized coherently and logically, beginning with an introductory section that introduces the subject matter and a thesis statement. The main body of the essay should be partitioned into several distinct segments, each dedicated to a different facet of the subject. The concluding part should encapsulate the primary points and rephrase the thesis in the context of the evidence presented.

The structure should adhere to the following format:

1. Introduction
→ Presentation of the subject
→ Thesis statement

2. Main Body
→ Section 1: Facet of the subject
→ Section 2: Facet of the subject
→ Section 3: Facet of the subject

3. Conclusion
→ Recapitulation of key arguments
→ Reiteration of the thesis

Each segment should include a concise summary of the specific points or arguments to be presented. The text should maintain a formal and academic tone, with points being clear, precise, and succinct."

Specific Example:

Structure for a 1000-word paper on 'Yamanaka Factors and Their Effect on Aging'.

ChatGPT Output:

1. Introduction

→ Presentation of the subject: Provide an overview of Yamanaka factors and their relevance in biology.

→ Thesis statement: State how the study of Yamanaka factors might revolutionize our understanding of ageing and its process.

2. Main Body

→ Section 1: Define and explain Yamanaka factors. Discuss their discovery, their function in inducing pluripotency, and their relevance in biological research.

→ Section 2: Discuss the ageing process. Briefly outline the scientific understanding of ageing and the various factors contributing to it.

→ Section 3: Link Yamanaka factors and ageing. Analyze the research indicating the effect of Yamanaka factors on the ageing process. Discuss the potential implications of these findings for ageing and anti-ageing research.

3. Conclusion

→ Recapitulation of key arguments: Summarize the explanation of Yamanaka factors, the discussion on ageing, and how Yamanaka factors might influence the ageing process.

→ Reiteration of the thesis: Reaffirm the potential transformative effect that the study of Yamanaka factors may have on our understanding and handling of the ageing process.

Remember, the objective is not merely to define Yamanaka factors, but to critically assess their potential impact on the ageing process.

👇If you found this helpful, share with others and follow me for more prompt tutorials.

[Simone Scardapane @s_scardapane]

*Designing and explaining GNNs*

An updated slide deck and code introducing graph neural networks and their explainability, presented at #ebiss2023 in @fib_upc!

See below for a quick overview. 😎 /n

Link to slides & notebook: https://t.co/pQyKYMbasA

[Luke Gessler @LukeGessler]

this paper's nuts. for sentence classification on out-of-domain datasets, all neural (Transformer or not) approaches lose to good old kNN on representations generated by.... gzip https://t.co/6eZiXlJxOX

[Shirui Pan @ShiruiPan]

[GNNs + Time Series] Time series data is omnipresent, with graph neural networks (GNNs) emerging as a powerful tool for its analysis. We have recently encapsulated the progressive strides of GNNs in time series analysis (GNN4TS).
ArXiv: https://t.co/U2mjwkcIih