Did you know that you can think about confirmatory factor analysis as a multilevel model? SEM and MLM can seem like disparate methods, but they aren't! @maireadkshaw and I developed this to demonstrate the connection: https://t.co/4XVabhyanf
So what's in this module? 1/
First paper of our new line of work combining Generative AI and Network Psychometrics: "Decoding Emotion Dynamics in Videos using Dynamic Exploratory Graph Analysis and Zero-Shot Image Classification: A Simulation and Tutorial using the transforEmotion R package".
In this paper, @atomasevic, Alexander Christensen, and I present several innovations:
1) Using Generative AI to compute emotion scores of videos,
2) Then modeling the emotion scores using Dynamic Exploratory Graph Analysis,
3) We validate the use of DynEGA as a way to model emotion dynamics via a simulation study.
In the simulation, we developed a new data-generation mechanism called the "Damped Linear Oscillator with a Measurement Model".
We implemented everything in a new SUPER COOL R package called "transforEmotion" (available on Github and on CRAN).
The paper also presents a method using OpenAI's CLIP transformer neural network for facial expression recognition in political speeches. Trained on a diverse dataset of 400 million image-text pairs, CLIP can execute various tasks, including emotion recognition, through natural language instructions, akin to the zero-shot capabilities of GPT models. To demonstrate this, the paper uses sampled frames from a public address by Boris Johnson. Using the transforEmotion R package, every 15th frame from the video was extracted, totaling 350 frames. These frames were then analyzed using CLIP's zero-shot image classification with labels for emotions like anger, fear, and happiness, focusing on Johnson's face. The analysis revealed predominant emotions of anger and fear, with their intensities changing across frames. This application produced six time series of Facial Expression Recognition (FER) scores, showcasing CLIP's effectiveness in capturing the emotional nuances in political speeches. Our proposed pipeline is summarized in the figure below:
Visual abstract. Top row: four frames of Boris Johnson's video; Second row: four bar plots showing the results of emotion detection procedure for each frame from the top row; Third row: time series of 6 emotions from 350 extracted frames of the video; Bottom row: network plot of dynamic EGA model of 6 time series with negative emotion dimension (red) and positive emotion dimension (blue).
The pre-print of our paper can be found here: psyarxiv [dot] com/hf3g7
This research was supported by the Fulbright Visiting Scholars Program (2023) and the Jefferson Trust (2024).
Added the cutest example of how a linear model is a pretty good approximation of a logistic curve *for a small range of values*
When nova was a puppy, the vet said to walk her ~5 minutes at a time per month of age.
That worked great for the first 8 months of her life...
Forgive me, for I am about to Bayes. Lesson: Don't trust intuition, for even simple prior+likelihood scenarios defy it. Four examples below, each producing radically different posteriors. Can you guess what each does? Revealed in next tweet >>
has anyone ever lost a full week's worth of work from @Box?? I'm trying to find some documents I worked on but all the the version history is saying the last time they were modified was months before I last worked on them
very frustrating
Congratulations to @SheaDuarte for being awarded an NRSA fellowship to study multisensory memory using virtual reality and fMRI! Looking forward to all the cool studies to come 👩🎓@UCD_CMB@UCDavisPsych
I'm delighted to announce that I have accepted the offer of admission to the Ph.D. program in Psychology at @ucdavis. This Fall, I will join @rastlab as a graduate student!
I'm delighted to announce that I have accepted the offer of admission to the Ph.D. program in Psychology at @ucdavis. This Fall, I will join @rastlab as a graduate student!
The cat is officially out of the bag! On March 1, I will be starting in a new role at @ucdavisCAES@UCDavisPlants as an Assistant Professor in the Department of Plant Sciences and Director-Elect of the Strawberry Breeding Program.
On the one hand, I was rejected from a dream job this week. On the other, one of my student evals said "I like Josue, he’s a chill dude and immensely knowledgeable about the material."
You win some and you lose some I guess
Working on a revealjs presentation with #QuartoPub. Document renders, but the resulting HTML file is blank when I try to preview or open it.
Anyone have any thoughts on what might be happening? #RStats