Flora Salim

Prof @UNSWComputing | CI @AdmsCentre | Deputy Director @unsw_ai | @MonashUni👩🏻‍🎓 | #AI #ML #MFM #AI4Science #GeoAI #UbiComp #LLM #timeseries #spatiotemporal

[Patrick Langer @PatrickLangerAI]

We’re releasing TimeNet today, joining forces with researchers across the US, Europe, Asia and Australia to build the future of Temporal AI that understands our world as it changes.

An open data standard and scalable infrastructure for next-generation Temporal Foundation Models.

[Flora Salim @flosalim]

Can satellite VLMs and spatial datasets improve how we map human mobility? 🛰️🏙️
Two of our 9 papers from our lab accepted in #SIGSPATIAL2026 are tackling urban mobility from orbit and from the ground.

🛰️ PAPER 1: Do Satellites See Commuters? A Critical Benchmark of Vision Foundation Models
The Problem: Travel census data is sparse and expensive. Can satellite foundation models replace it globally?
🛠️ Contributions:
• Benchmarked vision encoders (RemoteCLIP, DINOv3, SatCLIP, AlphaEarth) via a WeDAN graph diffusion pipeline across 1,925 US counties, 325 UK districts, and 14 cities.
🎯 Takeaways:
1️⃣ Language vs. Geography: Language-supervised models (RemoteCLIP) dominate locally, recovering 98.3% of census performance. Geographically grounded models (AlphaEarth) win at cross-continental transfers by 33%.
2️⃣ Scale isnt everything: Despite massive datasets, self-supervised models like DINOv3 collapsed to near-zero accuracy globally.
3️⃣ Cross-Continental Wall: Seamlessly transferring mobility logic across radically different global cities remains an unsolved problem.

Proud of my undergrad student Ashiq Shukoor Iqbal. His first paper, jointly authored with Wilson Wongso is a solid output from his 1 year Honours thesis project with us.

Preprint: https://t.co/IHJ2L1Vr09
Code: https://t.co/M3FOGJyCfz

📍PAPER 2: Massive-STEPS: Massive Semantic Trajectories for Understanding POI Check-ins -- Dataset and Benchmarks 
Human mobility AI is suffering from a massive data bottleneck. Most models are still trained on outdated check-in datasets from 2012-2013, over-indexed on a few Western tech-hubs, and isolated from modern open-source mapping data.

🛠️ Technical Contributions:
• Massive-STEPS: A newly released, ultra-large-scale semantic trajectory benchmark dataset.
• Expanded temporal and geographic horizons, capturing 24 months of data across two distinct eras (2012-2013 and 2017-2018) spanning 15 global cities—with an emphasis on under-represented markets like Jakarta, Kuwait City, and Petaling Jaya.
• Formally aligned check-in data with Foursquares Open Source Places metadata to benchmark traditional models alongside zero-shot Large Language Models (LLMs).

🎯 Key Takeaways:
• Cultural Nuance Matters: Mobility algorithms trained solely on US patterns fail in Southeast Asian or Middle Eastern contexts; diverse benchmarks are a necessity, not an option.
• Benchmarking LLMs: The cross-era dataset proves that modern LLMs can reason through shifting urban structures, opening new avenues for zero-shot location recommendation. 

Preprint: https://t.co/6vRPQwpvqx
Code: https://t.co/XpaZtetYr0
🤗 dataset: https://t.co/NRqmeMGycG
(total download to date: >26K)

Awesome effort by my PhD student Wilson Wongso.

More #SIGSPATIAL2026 posts to come...

#SpatialAI #ComputerVision #RemoteSensing #DataScience #UrbanMobility

[Flora Salim @flosalim]

If you're attending #UbiComp in Shanghai this week, find my students Baiyu Chen @Br33z3lled, Tahia Tazin, and Hada Melino Muhammad who are presenting their papers.

1. 🚀 Camera-Free Egocentric HAR: Distilling Video Intelligence into Low-Power Wearable Sensors. (see the 1st fig)
with COMODO (Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition (IMWUT journal), Baiyu Chen et al.

Always-on video in egocentric AI drains batteries, generates high heat, and raises privacy flags. 
COMODO is a self-supervised cross-modal distillation framework that transfers the semantic structure of pretrained video representations to IMU representations through similarity-distribution alignment, enabling efficient and generalisable human activity recognition with IMU-only inference.
COMODO prevents loss of temporal structure during video-to-IMU translation.
By mapping motion data into the same semantic space as video, COMODO enables headworn wearables to track activities without cameras.

Why this matters:
⚡ 115x Compute Drop: Slashes operations from 196B to just 1.7B FLOPs and latency to 75.7ms.
🔋 Battery: Camera sensors can safely duty-cycle while the ultra-low-power IMU runs continuously.
👁️ Privacy: Recognises complex activities without recording user spaces.

paper: https://t.co/z5D5B80G1W
code: https://t.co/1jZvieNoYI
I presented this work also in the invite-only @Meta Wearables summit in July.

2. How stressed are ICU nurses during a shift? 🏥 (ISWC 2026) by Tahia Tazin et al. (see the second figure).
In our study, we tracked 20 Surgical ICU nurses using non-invasive wearable sensors—measuring brainwaves (EEG), sweat gland activity (EDA), and pulse (BVP) during routine workflows. By using multimodal physiological sensing, the system can predict cognitive load in real time, paving the way for smarter, safer clinical environments.
paper: https://t.co/TRdo84jjjD

3. How few self-reported labels does a mobile-sensing model need to adapt to a users mental health? 📱 by Hada Melino Muhammad et al. (see the 3rd figure)
Continuous mental health surveys cause massive user fatigue, leading to high drop-out rates in digital interventions.
The Solution: Models can achieve robust screening accuracy by collecting only a few highly strategic baseline labels rather than continuous daily prompts.
💡 Key Takeaways: The 5-Label Plateau: Accuracy gains drop off heavily after the fifth label.
Base Rate vs. Case-Finding: Early labels mostly teach the model a user’s baseline symptom rate, leaving macro-F1 and ROC-AUC flat.
Zero-Adaptation Baseline: High discrimination is often fully available without personalised tuning.
Lets cut survey burden! 🧠
paper: https://t.co/Vx34cTNp52

#ubicomp2026 #sensing #wearables #smartglasses #TSFM #digitalhealth

[Flora Salim @flosalim]

So proud of this work led by Breeze. Check this out. #neurips2026

[Baiyu (Breeze) Chen @Br33z3lled]

Excited to share our work AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild, accepted at #NeurIPS2026! 🎉

AnyMo enables IMU understanding across different sensor setups and datasets for recognition, retrieval, and captioning.
https://t.co/T3xYNP5lez
🧵⬇️

[Flora Salim @flosalim]

Ok we end up with 9 NeurIPS papers. I'll do a post for each of the 9 NeurIPS papers that got through -- I promise! (One done already - GlucoFM). Congrats to my students, postdocs, and collaborators.

The list of papers.
1. FOGO: Forgetting-aware Orthogonalization Optimiser - Toan Nguyen, YANG LIU et al.
2. GlucoFM - Zechen Li, Ahmed Abdelhadi Metwally, and the Google team.
3. MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs - Ruiyi Yang Imran Razzak
4. Robust Residual Correction via Selective Deployment for Time Series Forecasting - Jianxiang Xie, Yuncheng Hua, Hao Xue
5. AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild - Baiyu Chen et al.
6. Discovering Programmatic Policies from Reinforcement Learning-Based Traffic Signal Controllers Lindong Xie
7. Out-of-Distribution Detection in Continual Learning - Nimeshika Udayangani, Sarah M. Erfani, Christopher Leckie (collaboration)

And two evaluation track papers
8. When root cause analysis fails: a retrieval-reranking decomposition - Hada Melino Muhammad, Luan (Miles) Pham
9. Beyond high and low: evaluating graded cognitive diversity in LLM Persona simulation - Sion Weatherhead

NeurIPS Sydney, here we come. :)

#neurips2026 #neurips #ai #foundationmodels #timeseries #llm #tsfm #continuallearning #user #simulation #worldmodel

[Flora Salim @flosalim]

Congrats to all my students, postdocs and collaborators
1. Gluco FM -  my student Zechen Li & Google team.
2. Discovering Programmatic Policies from Reinforcement Learning-Based Traffic Signal Controllers - Lindong Xie et al
3. MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs - Ruiyi Yang et al
4. Robust Residual Correction via Selective Deployment for Time Series Forecasting - Jianxiang Xie, et al.
5. FOGO: Forgetting-aware Orthogonalization Optimizer - @KhanhToanNguye6 et al
6. AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild - Breeze Chen at al
7. Out-of-Distribution Detection in Continual Learning - Nimeshika et al

[Flora Salim @flosalim]

7 #neurips papers (main track) accepted!! Happy days.... Still waiting for D&B track result...

[Flora Salim @flosalim]

One dataset, one generator. As datasets multiply, that approach becomes hard to scale.
Why should every new dataset or scenario mean another separate model to train and maintain?

The problem: synthetic time-series generation is fragmented. Want traffic data? Train a model. Want energy load? Train another. Want to switch between morning-peak, evening-peak, or high-volatility scenarios? That choice should be an explicit input to the generator.

UPLOTS takes a different approach. We developed one shared diffusion generator, guided by a pretrained transformer and scenario prompts. Choose a supported dataset and scenario through its prompt, and the same jointly trained model generates samples from the corresponding learned distribution. No separate model for every scenario. No retraining just to switch between supported prompts.

1. Breaking the One-Model-Per-Dataset Pattern
UPLOTS brings multiple dataset–scenario pools into a single training framework. At each denoising step, a pretrained transformer combines the selected prompt with the current noisy sequence, producing representations that guide the Diffusion-TS generator. This makes scenario selection part of the generation process itself, while reducing the need to train and maintain separate generators.

2. A Unified Representation Layer
Instead of engineering distinct neural architectures for different data behaviors, UPLOTS maps time-series inputs from different datasets into a shared embedding space.

Shared Embeddings: From traffic flows to energy-related measurements, time-series inputs are projected into a common representation space. A frozen pretrained language-model backbone provides a shared component across these datasets.

Prompt-to-Pattern Architecture: Textual scenario identifiers are embedded alongside the current noisy sequence. These prompts let one model distinguish among learned scenarios, including peak-period, calendar, load-level, and volatility subsets. A simple change of prompt selects a different supported scenario.

3. Multi-Dataset Learning — and a Path Toward Broader Generalisation
The shared framework brings different datasets into one model, making joint training and scenario selection central to the design.

Train Together, Switch by Prompt: UPLOTS jointly learns from multiple predefined dataset–scenario pools. Curriculum-aware loss modulation and rolling-loss dynamic sampling balance their contributions during training. Once trained, the model switches among these learned scenarios without separate fine-tuning for each one.

We evaluate UPLOTS on four real-world datasets, with an extended study of 14 prompt configurations on ETTh and Energy. Downstream forecasting experiments also show the value of its synthetic data, particularly when real training data are scarce.

This #CIKM2026 paper will be presented by my student Du Yin.
Preprint: https://t.co/k112c4K2na

#TimeSeries #GenerativeAI #FoundationModels #MachineLearning #Diffusion #TSFM

[Flora Salim @flosalim]

🧠 How can a video-language model continually adapt to new tasks and domains without storing a separate set of adaptation tokens for each task?

This is especially challenging for multimodal LLMs, where each new task may require different visual reasoning and adaptation, while task-specific prompts and parameters can quickly accumulate over time.

I’m very pleased to share another work accepted to the EMNLP 2026 Main Conference! 🎉

DynaTokens: Controlling Token Dynamics for Continual Video-Language Understanding

💡 DynaTokens explores a simple idea: instead of storing a separate set of prompts for every task, can the model learn to generate the right adaptation tokens when they are needed?

We build DynaTokens on a frozen LLaMA-2-7B backbone with CLIP ViT-L/14, using LLaMA-Adapter for efficient video-language adaptation. So while the model has a 7B-parameter LLM backbone, we do not fine-tune those 7B parameters. DynaTokens trains only a small token generator and lightweight adapter-side components, around 9.6M trainable parameters, less than 0.15% of the 7B backbone.

⚙️ Dynamic Token Generation: A token hypernetwork generates the task-specific prompts used by LLaMA-Adapter on demand, rather than learning and storing a new prompt block for every task.

🔄 Look-Ahead Regularisation: Anticipates how new-task updates affect past tasks, with a theoretical connection to sharpness-aware optimisation that helps reduce forgetting.

🪶 Very Lightweight: Each new task requires only ~26 KB of storage, around 200× smaller than storing its full generated prompt tensor.

📈 Strong Continual Learning: Higher accuracy and lower forgetting on continual NExT-QA and DramaQA, with strong performance across a 13-task cross-domain sequence.

🌐 Beyond VideoQA: DynaTokens also shows better zero-shot generalisation to unseen video domains and promising results in a challenging ImageQA → VideoQA continual learning setting.

An interesting aspect is the efficiency trade-off. We retain the capacity of a large frozen LLaMA-2-7B model, while continual adaptation happens through a very small fraction of trainable parameters and only kilobytes of new storage per task.

This work was led by my postdoc @KhanhToanNguye6.

📄 Paper: https://t.co/S03cwfpkDb

Demo: https://t.co/NpTAMPPAle

💻 Code: https://t.co/keXWU6tzgA

@KhanhToanNguye6 will present this work at EMNLP 2026 in Budapest! 🇭🇺

#EMNLP2026 #ContinualLearning #MultimodalAI #MachineLearning #VideoUnderstanding #LLM #VLM

[Flora Salim @flosalim]

Introducing two of our works on robust and efficient agentic retrieval to be presented at EMNLP 2026 by my student Ruiyi Yang, who led three out of our six accepted papers.

SPLIT-RAG (Main)
When queries sent to an agentic RAG system are too complex and require multi-hop reasoning, retrieving over a large knowledge graph can introduce substantial search overhead, irrelevant evidence, and conflicts across retrieved facts. SPLIT-RAG is a multi-agent framework with:
- Question-Driven Graph Partitioning: It uses reasoning patterns observed in training questions to organize a large knowledge graph into compact, reusable partitions aligned with recurring information needs.
- Selective Multi-Agent Retrieval: Each query activates only a small set of partition-specialized agents, which retrieve evidence from their assigned graph regions in parallel.
- Compatibility-Based Aggregation: Retrieved triples are scored and filtered through a compatibility graph before being passed to the head agent, reducing conflicts across evidence from different agents.

SPLIT-RAG achieves the strongest results among all the evaluated KG+LLM methods, and the lowest end-to-end latency under given budget (29.4-33.7s per query).
Preprint: https://t.co/BH1t4YNVq0

RELOOP (Findings)
Multi-hop QA becomes particularly challenging when the required evidence is distributed across text, tables, and knowledge graphs, as different sources have different structures and often require separate retrieval pipelines. RELOOP introduces a structure-aware iterative retrieval framework built on Hierarchical Sequence (HSEQ), providing a unified interface for retrieving and reasoning:
- Hierarchical Sequence (HSEQ): converts text, tables, and KGs into a common sequence of typed segments while preserving their original structure and provenance.
- Structure-Aware Iterative Retrieval: A lightweight iterator selects and expands evidence through modality-specific structural links, such as paragraph–sentence relations, table rows/columns, and KG neighbourhoods.
- Sufficiency-Controlled Stopping: Instead of retrieving for a fixed number of rounds, RELOOP estimates when enough evidence has been collected and stops retrieval accordingly.
RELOOP achieves the highest accuracy on all evaluated tasks. RELOOP also substantially reduces iterative retrieval cost: under latency-sensitive inference, it uses roughly 3–4 retrieval steps, compared with 11–17 steps for Think-on-Graph, and > 2x faster than other baselines.
Preprint: https://t.co/ImJ76PRJQG

This project is supported by @TIIuae.
We also thank the support of @sharon__ai for the compute support.

#EMNLP2026 #agentic #RAG #LLM #robustness #efficient #retrieval

[Flora Salim @flosalim]

What if your AI agents could actively fight back against invasive web tracking?

Our new paper (EMNLP 2026) introduces GuardianAgent, a privacy framework that evaluates outgoing web traffic against a site's destination policies in real time. Instead of just blocking requests, it automatically sanitises sensitive data or deploys verified adversarial escalation when risk levels spike.

Text anonymisation is not simply about removing as much sensitive information as possible. Stronger anonymisation can improve privacy but also unnecessarily destroy useful content.

GuardianAgent introduces a policy-conditioned, risk-adaptive anonymisation method that determines how strongly content should be anonymized based on disclosure risk and escalates rewriting only when residual leakage is actually supported by the original text.

Core Concepts of GuardianAgent:

- Policy-Conditioned Risk Scoring: AMRSF combines policy-violation likelihood with factors such as data sensitivity, recipient, purpose, action context, and policy transparency to produce an explicit risk score.

- Risk-Adaptive Anonymization: The risk score determines the initial strength of anonymization through a five-level hierarchy, ranging from partial masking to semantic rewriting.

- Verified Adversarial Escalation: An adversarial guesser can trigger stronger anonymization only when its inferred sensitive information is verified against the original text, preventing unsupported guesses from causing unnecessary rewriting.

Results:
Across TAB, SynthPAI, and PII-Masking-300k, GuardianAgent achieves 0.945, 0.944, and 0.901 privacy, respectively, while maintaining strong utility. It is the only evaluated method to exceed 0.90 privacy across all three domains and lies above or beyond the baseline privacy–utility frontier on each benchmark.

Preprint: https://t.co/6Epf3UBZN3
Code: https://t.co/JY8C1j3hmw

This work is led by my student Ruiyi Yang, funded by UNSW-UTS Trustworthy Digital Society, with the team: Salil Kanhere, Rahat Masood, Gayathri Harshila

The genesis of this work was ideated a while ago, described in this Conversation article I wrote with Salil Kanhere.
https://t.co/dsGX4KXDiC

We're now in the middle of user testing of a GuardianAgent browser extension with our collaborators.

#AI #Privacy #CyberSecurity #DataProtection #MachineLearning

[Flora Salim @flosalim]

What if a glucose foundation model could capture both long-term metabolic state and short-term responses to meals?
This is a collaboration led by my student Zechen Li who's currently interning at Google Research under @aametwally1.
Continuous glucose monitoring (CGM) provides a dense view of daily metabolic physiological time series, but general-purpose time-series foundation models are not explicitly designed for its multiscale dynamics, irregular observations, and sensor artifacts.

GlucoFM is a lightweight, CGM-specific foundation model built around a state–event dual-stream architecture.
• Chronological Grid Alignment: Maps each 24-hour CGM segment onto a standardized five-minute grid while preserving absolute time-of-day information.  
• Missingness-Aware Representation: Retains an explicit observation mask to distinguish measured glucose values from unobserved positions.  
• Dual-Stream Encoding: Separately models slower glycemic trends through a state stream and short-term deviations through an event stream before fusing them into a unified representation.  
• JEPA-Style Pretraining: Combines masked latent prediction over fused daily representations with a Temporal Dynamics objective that models hour-to-hour transitions in both streams, avoiding direct reconstruction of potentially noisy raw glucose values.  
• Lightweight Frozen Encoder: Uses a compact 0.72M-parameter encoder that remains frozen during downstream evaluation.

GlucoFM builds on the direction introduced by CGM-JEPA (by my student Hada Melino Muhammad, Zechen Li also in close collaboration with Ahmed Abdelhadi Metwally). While CGM-JEPA focuses on masked temporal representation learning—and X-CGM-JEPA extends it with Glucodensity-derived cross-view information—GlucoFM focuses on the internal multiscale structure of the glucose sequence through state–event decomposition and stream-level dynamics modeling.

Further, pretrained on 109,066 hours of unlabeled CGM data from 477 subjects, GlucoFM demonstrated strong performance in two complementary settings:
• Linear probing: Across four cohorts and seven metabolic phenotype tasks, GlucoFM achieved the strongest average subject-disjoint performance among the evaluated models.
• Postprandial glycemic response forecasting: When combined with recent CGM, meal nutrition, and participant context, the same frozen encoder achieved the lowest two-hour PPGR errors across both Dexcom and Libre.

To sum up, these results highlight the potential of multiscale, sensing-aware FMs to enable reusable CGM representations for metabolic phenotyping and glucose-response prediction, bringing scalable and personalized metabolic risk assessment closer to practice.
GlucoFM: https://t.co/D4cBAxl0Be
CGM-JEPA: https://t.co/broljpC1V2

#CGM #glucose #T1D #T2D #sensor #timeseries #FM #TSFM #JEPA

[Google Research @GoogleResearch]

Introducing GlucoFM, a lightweight, self-supervised continuous glucose monitoring foundation model that separates metabolic baselines from transient spikes, producing transferable representations and setting new performance standards across diverse metabolic prediction tasks, such as diabetes risk assessment, insulin resistance, and post-prandial glycemic response.

Learn more →https://t.co/PA7wpFz2xo

[Flora Salim @flosalim]

🚀 Fine-Tuning LLMs/VLMs with <1.5% of the parameters without Catastrophic Forgetting: Introducing CP-MoE 🧠
When scaling foundation models sequentially, traditional efficient methods (like LoRA) still suffer from catastrophic forgetting. Parameter overwriting and rigid routing load-balancing consistently erase historical task knowledge.

CP-MoE (Consistency-Preserving Mixture-of-Experts) solves this. It acts as a post-training framework that governs how LoRA-based experts are routed and updated while keeping the core 7B foundation model entirely frozen.

🔍 The Core Strategy: Assess Then Update
Instead of jumping directly into fine-tuning, CP-MoE introduces a temporary probe mechanism:
* The Transient Expert: A disposable expert adapts to a new task during a brief warm-up phase. Its trajectory guides the learning process before it is discarded, adding zero long-term overhead.
* Consistency-Preserving Routing Bias: CKA-based compatibility score between the transient expert and each stable expert is injected as a bias on the router logits, steering the new task toward the most aligned stable experts
* Transient Expert-Guided Regularisation: Parameter importance is calculated dynamically to shield crucial historical knowledge with selective regularisation.

📊 Benchmark Breakthroughs
Evaluated on frozen 7B backbones (Llama-2-7B for language, LLaVA-1.5-7B for vision-language), training only the LoRA experts and router. Specifically, by training only 1.48% of total parameters on SuperNI and 0.47% on VQA v2, CP-MoE outperforms traditional CL-MoE and LoRA baselines:

* SuperNI (8 sequential language tasks):  Achieved 50.84% average performance with 1.32% forgetting, matching recent LoRA-based continual learning baselines (InfLoRA, O-LoRA, GainLoRA) in-sequence and 35.80% zero-shot transfer on unseen tasks, beating the strongest baseline (GainLoRA at 33.80%).
* VQA v2 (10 sequential visual reasoning tasks): Reached 62.30% average accuracy with near-zero forgetting (just 0.35%), outperforming the prior state-of-the-art CL-MoE.

CP-MoE proves that we do not need to choose between knowledge transfer and parameter isolation. Dynamic, consistency-guided routing provides a promising path toward scalable and sustainable continual AI training.

🔗 The full paper accepted at CoLLAs 2026: https://t.co/ELmuQtIMov
⚒️ Github : https://t.co/xTwrm95DpV

#MachineLearning #LLM #GenerativeAI #MOE #LORA #PEFT #ContinualLearning #CoLLAs2026 #CoLLAs

p.s. Very proud of my Master thesis student Yang Liu who did this research as part of his 2-term thesis coursework project, leading to this output in just 8 months, and my postdoc @KhanhToanNguye6 the cosupervisor. This is Yang's first paper as a first-author, and we trained him from scratch to do research, yet with his emergent capabilities, we shall see many more brilliant outputs from him :)