2026 - ARC DECRA fellow
2025 - Lecturer at Monash University. Data Science and AI Group
2020-2024 - Research scientist in machine learning at CSIRO's Data61.
Squared families can be viewed as simplified versions of previous models, such as squared probabilistic circuits (@loreloc_ ), or squared neural families. We fix the "hidden feature" for extra tractable theory, while still allowing for rich representations.
Squared families are in some precise sense "rare" families which always have tractable (often closed-form):
1. Normalising constant
2. Fisher info
3. Statistical divergence
all linked through a tractable convex Bregman generator and integral. Applied to MLE for params/density.
Squared Neural Families: A New Class of Tractable Density Models [spotlight], with @RussellTsuchida, @ChengSoonOng, a simple neural network density model with closed form normalising constants, generalising exponential family.
https://t.co/jBvkZDEJGi
https://t.co/i0ctvAwOWF 4/4
We find latent variables under an exponential family model using SGD. We call inner products of these latents Deep Equilibrium Kernels (DEKers). DEKers are built from and generalise NNGP components. DEKers may admit fixed points and are kernel versions of deep equilibrium models.
Pleased to share our freshly accepted paper at TMLR!
If you are interested in neural network kernels (NNGP, NTK), implicit neural networks, SGD, or exponential families, you might enjoy this paper.
https://t.co/ecBwxItZpb
Nice to be recognised as a Top Reviewer (top 10%) by @aistats_conf!
Also looking forward to attending in April, where I will be presenting https://t.co/3so8rOs01v, which views exponential family PCA in light of deep equilibrium models. Joint work with @ChengSoonOng.
Happy to announce that work emerging from a successful collaboration between myself, @sukyeeyong, Ali Armin, Lars Peterrson and @ChengSoonOng has been accepted into @iclr_conf!
Excited to present our work Gaussian Process Bandits with Aggregated Feedback (accepted by #AAAI2022) at #NeurIPS2021#WiML2021 poster session #1 (https://t.co/o0cFa5LlyS) at 3:45-4:45pm AEST .
Pre-print https://t.co/ruE2GTIcYy
Joint work with @RussellTsuchida@ChengSoonOng
@aaronsnoswell I see you found the answer yourself :) Cool stuff! The question was tagged as recreational so I'm curious; do you have an application in mind or is just for fun?
Back in February, @Tea_Pearce, Chris van der Heide, Fred Roosta, @marcus_marcusg and I did some work on the kernels of infinitely wide deep neural networks with GELU and ELU activations. We also studied the fixed points of these kernels. Check it out here:
https://t.co/fJoJP43aYn