I had our @ExpectedParrot AI Interviewer play 20 questions w/ @Prolific workers and in half the cases, I randomized the AI interviewer to actually know the object (which the subject chose ahead of time) but instructed the AI not to make use of that information. It couldn't help itself...
@Auto_Conference Our platform is designed to make it straightforward for you to test this for your own specific case, and our Research Agent can create and run those comparison studies for you.
@MaleManlpulator@TheStalwart Lots of uses, including pretesting surveys, exploring hypotheses, and studying how LLMs respond to different questions and experimental conditions. I put your question directly to our Research Agent: https://t.co/SiupXU7Li0
I was curious what the election might mean for higher education and MIT in particular & would like to get the opinions of noted higher ed critics/reformers. So I sent them a survey which they all promptly answered...no just kidding.
I built profiles of a collection of these folks, with details populated from their published views, interviews & so on. I then had these AI profiles react to the news 1/
!!Public good alert!! 🚨🚨
Introducing Lory-4B and Lory-27B v1 🦜🦜, the first open-source decision models post-trained specifically to make simulated survey respondents follow economic logic. Off-the-shelf decision models can follow personas well but barely react to money at stake.
A fine-tune of Kev-4B and Kev-27B. Lory-27B performs on par with GPT-6.1-terra (no reasoning) at 1/5–1/10 of the price. Lory-4B is small enough to run on a 16 GB laptop at ~free cost.
They work best with EDSL (@ExpectedParrot's survey language). They're trained on its prompt format and ship with an adapter that runs any EDSL survey. And they respond to what's actually at stake: change the price, the odds or the payoff and the answer moves, not just the persona. One forward pass gives a full probability for every option.
This is just v1 of these models. I plan to continue post-training to make them the best decision models for simulated personas at the cheapest possible price.
https://t.co/QGr9P2ksbJ
https://t.co/qFIBXWqNs6
Some people have reached out and asked “is this good for practicing for academic interviews?” - yes! Academics are an open book w/ an extensive web presence.
AI-moderated interviews are often presented as an alternative to 'conventional' surveys but that doesn't have to be the case: our tools let you mix them together, piping data & in-out, all in one flow for the respondent 1/
Important research question - what is the market value of candy collected by a kid per hour on Halloween? Participate in this important research question w/ link in next tweet 1/
Eliciting preferences over email is tedious. A short survey that puts few demands on the respondent might be better. But creating such surveys is itself tedious, w/ lots of GUI hunting & pecking, taking screenshots, making file uploads, copying & pasting URLs & so on.
In thread below, I'll show you the easy way to create a purpose-built survey, w/ an application to planning a date night /1
A new very creative method for measuring ambiguity with LLMs co-led by @dsallydt and @LukeThorburn_ (EMNLP Oral 2026).
When I worked with @mhtessler@summerfieldlab et al on the Habermas Machine (a project in which we use LLMs to find statements that maximize agreement within groups of people), we often found that LLMs maximize consensus by simply making statements more ambiguous.
Eg a statement like "peaceful protests should be allowed" will likely lead to high agreement but that's because a lot of disagreement hides in different interpretations of what's "peaceful".
One way to measure ambiguity is to ask lots of people how they would interpret a statement and see if those people disagree. But this is obviously impossible to do at scale. In this work we show that we can use LLMs to help approximate this. Read Sally's thread or the paper for more details.
I think work like this is super important, for example to stress tests AI constitutions and model specs to see which rules are ambigious and in which situations.
If you are in Budapest, please stop by Sally's presentation.
One of my favorite papers in the LLMs & Social Sciences literature is Ashokkumar et al. which shows that LLMs (GPT-4) asked to simulate how people respond in survey experiments is predictive of actual trt effects & the result is not likely to be memorization; their data & code are public and I used it to do a follow-up w/ more recent models 1/
Introducing the free Government & Legal Index for ChatGPT & Claude.
Give your AI agents 30M+ US government pages, from federal statutes and court opinions to city codes and permits.
Research tax rules, permits & case law with the Firecrawl plugin.
https://t.co/WT268coAo9
Aside from just the point estimates, I like seeing how it reasons through the evidence and makes adjustments. It's more interesting than just seeing a prediction market price.
It you want to try it for some question, use this playbook URL: https://t.co/NugrviXfFL
My report: https://t.co/K9oO0l2oa5. /end