GPT-4, which everybody keeps pouring like ketchup on all social science related tasks even in 2026, performs worse than most Tiny models (<3B parameters) that anyone can run on their laptop without paying an extra dime. It only has slight advantage in longer texts (1000+ words).
I am done testing *208* LLMs of various sizes (~50m to 1T+ parameters) and types (open, closed, MoE, Dense) for 18 different social science related tasks.
Stay tuned for analyses!
First takeaway is, smaller models, especially tiny models (<3B parameters), beat their counterparts all the time.
So, choosing a larger, more expensive, API based model is not always the best strategy.
For example, Qwen 3.5 4B beats GPT-4 in all but one task (longer texts).
@jessesmithsoc@kerbygoff Kuhn says revolutions happen when *the old paradigm loses authority* and *a competing one is available*. The old paradigm, as you pointed out, is has the authority, and the alternative is considered illegitimate by majority. That's the “structure” part of that book 😊
Cohen (1972), “Folk Devils and Moral Panics”
The state of academic sociology:
- activist scholarship gets desk rejected at top journals.
- impossible to find a committed Marxist at top departments.
- everyone wants to get rid of Theory clases.
The rest is pure moral panic!
The sociology community of @HdxAcademy is well-represented in this investigative article about the troubled state of academic sociology.
https://t.co/Y5cjR3xjmh
@kerbygoff@jessesmithsoc How about all ideas are worth engaging with without repudiating any of them especially because own own ideas are flawed, and trusting whatever is happening now is just another phase in “normal science”, which will eventually get sorted out, without causing moral panic?
@jessesmithsoc@kerbygoff Also, citation is a function of overproduction. The field-level prestige is often doesn't correlate with size of the topic. For example, findings a job while working on Christian Nationalism is almost impossible unless it is combined with something else (e.g., method).
@jessesmithsoc@kerbygoff I think, for example, Demography is a joke, but that's my flawed, individual opinion. Demographers do wonderful work, and I can only disagree with them, and not try to control the filed by telling them what to work on and what not to work on. Who made me the arbiter of Sociology?
@kerbygoff@jessesmithsoc Totally agree, but we also have do it on individual basis. The trend, however, is sounding moral panic.
I don't think mainstream Sociology printed in say top 10 journals is not methodologically or ideationally rigorous. What happens at the margins is, namely, marginal.
@jessesmithsoc@kerbygoff I guess we have epistemic differences. I too think there is a deluge of pubs with little substance. However, I don't treat it as a threat or crisis. Instead, this is how science “works”. These works are automatically pushed to margins; neither cited nor respected by mainstream.
@jessesmithsoc@kerbygoff It is instead the admission of the limits of our knowledge and experience, which makes any claims of having acess to absolute or even robust truth untenable. We cannot then do about claiming how someone else interprets world is illegitimate. We can only offer falwed perspective.
@jessesmithsoc@kerbygoff Valid points; I've imited space for rebuttal.
Suggesting because what we do rests on so many disputable assumptions and interpretations, even when some are valid, we cannot be confident enough of our own objectivity, let alone police others, is not “giving up on objectivity.”
@kerbygoff@jessesmithsoc While agreeing with both of you, I reiterate that while objectivity of empirical facts can be established, the objectivity of ideas is far more difficult. If we start accusing others of conducting empirical analysis with preconceived bias, we may end up in infinite mud-slinging.
@jessesmithsoc One of those papers is with Iceland and we often disagree, politely 😊
To reiterate, I agree sociology is normative and go far to say, Enlightenment was normative. Hence, I'm skeptical of “objectivity” claims. Striving for it is noble, but saying claiming objectivity is ironic.
@jessesmithsoc To reiterate, viewpoint diversity is great. We should have more, not less. For everyone, not like Leftist hegemons pushing out Rightist, and Rightists punishing out Leftists. That's kulturkampf.
@jessesmithsoc You are right, but I don't have to write full length papers to point the irony of fighting normativity with normativity, Critical Theory with critical theory, Left DEI with Right DEI. That said, I've two empirical papers under review on these topics. Happy to share once published
@jessesmithsoc The “flippant dismissal” would be accusing one side of being normative, ideological, biased, but other as rigorous, scientific, objective.
Most realistic stance would be, neither of the sides is objective, which I'm at least willing to accommodate as that's how science “works”.
@jessesmithsoc Nope, my objection is epistemological and phenomenological. Not playing the “science” rhetoric game.
Simply, we may strive for objectivity, we are not objective. All knowledge (esp. ideas) is biased because human reason and experience is limited. Our ideas influence our work.