Julian Ashwin

Assistant Professor in Macroeconomics @umsbe. Previously post-doc @LBS, PhD @UniofOxford.

[Vijayendra Rao @bijurao]

An interesting paper, but neglects a crucial point – for more complex/nuanced annotation tasks, particularly in non-Western contexts, LLMs can display a substantial bias leading to misleading results (see our paper with @JulianAshwin and @AdityaKChhabra https://t.co/p8FPDzEGkT.🧵

•The PNAS paper shows that in a range of straightforward applications like sentiment analysis, GPT does well in annotating documents across a range of languages. This may be valuable for low resource languages, but contexts in LLMs might be applied involve more nuanced or complex concepts. 
•Indeed the final example that the paper presents, an analysis of moral foundations, involves more complex concepts. In this case the LLM’s prediction accuracy appears to fall substantially below that of a model trained on the specific sample at hand.
•But more concerning that this poor performance is the possibility of bias. Even in cases where the LLM is relatively accurate, it will make mistakes and if these mistakes are not random then it can lead to misleading conclusions. We show that this is a real concern in our paper. https://t.co/p8FPDzEGkT
•It is easy to check for such a bias – just test if LLM prediction errors are correlated with relevant features (e.g. gender, ethnicity). Uncritical applications of LLMs may lead to “results” that are based entirely on the biases LLMs inherit from their training data.…

[NBER @nberpubs]

Neural networks can be used to study learnability in non-linear economies with application to monetary policy, from Julian Ashwin, Paul Beaudry, and @NuCampEconomics https://t.co/S6CfTTLuqq

[Julian Ashwin @JulianAshwin]

Both full papers and research-in-progress are welcome!

[Julian Ashwin @JulianAshwin]

@GiuliaPiccillo and I will be chairing a track on Expectations and Narratives in Macroeconomics at the 4th Annual MORSE conference. The deadline has been extended until July 31st, so you still have a couple of days to send us your work! https://t.co/4i9vdqviGs

[Julian Ashwin @JulianAshwin]

By bringing together these two literatures, this track aims to further our understanding of how narratives evolve and how they influence economic agents’ expectations. This includes, but is not limited to, applications to narratives around resilience and the climate crisis.

[Julian Ashwin @JulianAshwin]

The effect also spills over into firms related by the structure of the production network, but does not affect the aggregate level of volatility. This media coverage effect can be thought of as reallocating volatility across the market. (7/7)

[Julian Ashwin @JulianAshwin]

Delighted to see my paper “Financial news media and volatility: Is there more to newspapers than news?” in the Journal of Financial Markets. It's the final form of the first paper I worked on in my PhD, so it’s great to see it find such a good home. (1/7)
https://t.co/qP8FKUtlEP

[Julian Ashwin @JulianAshwin]

The effect is economically significant: across all firms and trading days, the average daily value of the extra price fluctuations is $173 million and the value of the extra shares traded is $3.94 billion (2015 USD). (6/7)

[Michalis Moatsos @mmoatsos]

It is with great pleasure that we @MorseMaastricht and @MaastrichtU will host
@BrankoMilan presenting: Visions of Inequality. Branko will also hold a masterclass with SBE students and PhD candidates. Details here: https://t.co/eeITiMaH7I

[Michalis Moatsos @mmoatsos]

Wonderful to host @guido_alfani at @SBE @MaastrichtU for a research seminar (12:00- 13:00 A0.23 at TS53) and a public lecture tomorrow!

[Andrew J Scott @ProfAndrewScott]

How valuable is it to age well? Myself, @JulianAshwin, @DavidaSinclair, Martin Ellison, updated our @NatureAging paper to calculate gains from slowing ageing https://t.co/jaAm7N7YS8. Health matters, ageing is the biggest health challenge, gains are multi-trillion $

#ageingwell

[Julian Ashwin @JulianAshwin]

@Prashant_Garg_ That's a good question and we haven't systematically looked into it, but followiung this paper: https://t.co/0VRZJrausF we set temperature to 0.2. We also tried setting termperature to 1 and it didn't really change the bias (although accuracy was lower).

[Julian Ashwin @JulianAshwin]

New paper with @bijurao and @AdityaKChhabra in which we find that using Large Language Models (LLMs) such as ChatGPT and Llama in social science research might lead to serious biases!
https://t.co/h9IbQjk4pG
(1/6)

[Julian Ashwin @JulianAshwin]

So while LLMs may well have many valuable applications for social science, we need to be wary of biases that relying on them can introduce into our analysis. Thoughts and feedback more than welcome! (6/6)

[Julian Ashwin @JulianAshwin]

We argue that as high quality annotations are necessary to assess whether an LLM is introducing bias, it may be preferable to train a "small" language model on these annotations rather than relying on a pre-trained LLM. (5/6)