Sebastian Popescu

ML scientist @FindnOrder, taming GP to respect physics. PhD Machine Learning & Medical Image Analysis @imperialcollege

[Sebastian Popescu @SebastianGP13]

Chuffed to finally share my PhD thesis. If you're interested in learning more about creating proper distribution shift-aware DeepGP for reliable anomaly detection or how to create inverse-free Sparse GPs through a probabilistic framework, do check it out: https://t.co/21P2bU7mXj

[Sebastian Popescu @SebastianGP13]

@langer_han @miniapeur Due to the mean function given by PCA in each hidden layer, this should be less of a problem for DGPs compared to DNNs. It is a strong assumption but I think it's needed if you want OOD capabilities. I propose making the layers Lipschitz so that it's respected.

[Sebastian Popescu @SebastianGP13]

@langer_han @miniapeur I explored a bit the OOD problem with DGPs here: https://t.co/32tSAwrOCD

What I found was that using Optimal Transport inspired kernels to be crucial to maintaining a sense of what an outlier is across layers.

However, the OOD capabilities diminish above 5-6 layers.

[Sebastian Popescu @SebastianGP13]

@langer_han @miniapeur Have you seen any recent work dealing with the "depth-pathology"? The latest paper that I remeber dealing with this was "Compositional uncertainty in deep gaussian processes". Has been some years that I didn't touch base on this though.

[XY Han @XYHan_]

What if you pass out-of-distribution data into NNs showing neural collapse? How's that useful?

Turns out, the out-of-distribution data become orthogonal to in-distribution data & you can then use that to detect those OOD points.

[Sebastian Popescu @SebastianGP13]

Glad to have contributed to this paper exploring how to use the geometry imposed by Neural Collapse on over-parametrized neural networks and how we can exploit it for out-of-distribution detection. Check it out at #ICLR2024

[Gianni Franchi @GianniFranchi10]

https://t.co/7Z9brwJWvK employs Neural Collapse and dimensionality reduction for OOD detection.

[Sebastian Popescu @SebastianGP13]

@miniapeur there are several papers on using Wasserstein distance based kernels for GPs. The latest that comes to mind is:
https://t.co/kbdlgBc81I

[Mathieu @miniapeur]

For users of Gaussian processes. What do you use? GPflow, GPyTorch and GPJax? What are the advantages and disadvantages of each?

[Sebastian Popescu @SebastianGP13]

@emmsskyyy @DanielCrisan90 @DragosCGruia I couldn't even find this news.

[Sebastian Popescu @SebastianGP13]

@e_soreq interesting that the double descent is regarded as a core ML idea...quite hard to reproduce in practice AFAIK

[Machine Learning for Biomedical Imaging @MELBAJournal]

New publication alert🔖⚠️:"Distributional Gaussian Processes Layers for Out-of-Distribution Detection." By S G Popescu, D J Sharp, J H Cole, @kostaskamnitsas, @glockerben. Free to read: https://t.co/Ek0enUh6m9

[Vincent Dutordoir @vdutor]

Introducing *Neural Diffusion Processes (NDPs)*: a novel approach, based upon diffusion models, that learns to sample from a distribution over functions.

Work with @alandanielsaul, @ZoubinGhahrama1, @frgsimpson

[1/11]

[Sebastian Popescu @SebastianGP13]

Wondering how to perform matrix inversion-free inference in Student-T Processes? Come to our poster tonight at https://t.co/dIPdQAmhOp #AABI2022 Full paper: https://t.co/59SMSUFH2x

[Sebastian Popescu @SebastianGP13]

@dccastr0 I see your sister has even more talent in producing figures:)

[Sebastian Popescu @SebastianGP13]

An extended version of this paper will soon be published, including some interesting theoretical function-space properties of the newly introduced module, additional architectures and results on generic OOD tasks plus other medical imaging modalities.

[Sebastian Popescu @SebastianGP13]

Our work "Distributional Gaussian Process Layers for Outlier Detection in Image Segmentation" on using Lipschitz filters + Distributional GP layers for uncertainty quantification in medical image segmentation has been accepted as an oral at @ipmi2021
https://t.co/PNIjVj342e

[Sebastian Popescu @SebastianGP13]

work done with @Neurosharp @JamesCole_Neuro @KostasKamnitsas @GlockerBen

[Sebastian Popescu @SebastianGP13]

We trained our network to segment white matter, gray matter and cerebrospinal fluid on healthy T2 scans (normative data).
At testing time, we used BRATS (T2 scans with tumors) to assess the capability of our model to detect tumors as outliers.

[Sebastian Popescu @SebastianGP13]

The former encapsulates uncertainities still present within the data manifold (after taking into consideration aleatoric unc.), while the latter measures the departure from normative data (out-of-distribution detection)

[Sebastian Popescu @SebastianGP13]

Our Segmentation network is capable to separate two types of uncertainties, the within-data uncertainty (associated to parametric part of DistGP)
and distributional uncertainty (non-parametric part of DistGP).

[Sebastian Popescu @SebastianGP13]

Through this formulation, the "heavy-lifting" convolutional part is performed by the Lipschitz constrained affine layer, while the DistGP is tasked
to devise non-linearities and propagate uncertainty.

[Sebastian Popescu @SebastianGP13]

"Measure preserving DistGP layer" consists of convolving a filter on the first two moments of the previous layer.
Subsequently, we introduce an "activation function" as defined by a Distributional GP (building on work introduced in https://t.co/GQSuZRylAr).