Analyzing student votes across AI models for college essay help
Comments
Aurornis
xtiansimon
Matches my experience. When I need to think or reason about something, between Clause and ChatGPT, the latter is my go to. With more to read, there’s just more to work with. For the same reason it’s bonkers for tech issues. How many times did I get a ChatGPT reply a mile long, do the first thing, and then read more after it didn’t work. Only then realize chatgpt didn’t offer the best answer first.
pasharayan
Yeah it's hard to get good data on what's best. We did read the a good sample amount of essays. Lots of arguments about what made them better, but its uncontroversial that longer ones were more preferred.
Interestingly we trained a simple LORA layer on top of Inkling and it turns out a model can smell which model wrote a response 70% of the time. I wonder if the smell of gemini is just more preferred by students.
dr_dshiv
Hmm, seems like which model teachers prefer would be the more important headline…
pasharayan
You know.. might try to see what we can do here
demibabs
Author, that doesn’t mean you need to get Gemini to write your article.
pasharayan
TBH i did a bit of both - we used AI to get us insights in the db, we then wrote and rewrote the article to be readable. It's really hard because usually when i write reports they're so terse its not fun to read
avaer
> How to use Gemini for a college essay
Given that this is increasingly the go-to for a college degree, college needs to rethink its cirricula and place in the world. Or at least get rid of the essay.
Lest it become a place where student and teacher ais go to play pay-to-win social deduction video games.
Aurornis
My professor and teacher friends are painfully aware of how many students submit AI generated essays.
They're adding steps like having the students discuss and defend their essay, which immediately reveals the people who had AI write something and thought they could bluff. This triggers complaints about social anxiety and such, which are unfortunately becoming the go-to defense when unable to discuss the work.
They're also moving toward more in-person writing. Instead of long essays, shorter writing segments as part of the test. Submitting a written essay earns you feedback from the professor and a better understanding of the topic, but that's it.
pasharayan
I think it's less about curricula and more about how professors should eval their students. In previous era's essays are a proxy of the students ability to reason, remember and argue certain points - but its clear they fall short now.
I suspect the best schools into the future will integrate lots of socratic defenses of theses, building real things in real time or solving problems with a professor in a case study manner. My last company was a good hint to the future.
brainwad
The homework essay is dead. High schoolers in my home state were just relieved of any take-home assignments because it was pointless now, I guess universities won't be far behind.
John7878781
In my experience, Gemini excels at explaining new concepts clearly and intuitively. Much more so than other models.
zorked
Gemini is the best general-purpose model. We hear a lot about the other ones here because we are focusing on coding.
inbx0
So, given that developers these days mix and match models anyway with harnesses like Pi, has anyone tried letting Claude or GPT do the coding, and Gemini do the comments and documentation? Maybe even asking it to "translate" Claude's output messages to better human language for the developer or subagents?
SturgeonsLaw
I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human. To me that feels like each model playing to their strengths.
pkulak
Seems reasonable. You could just switch the model after the code is written, when you need to ask questions. You’ll lose your token cache, but that’s it.
pasharayan
I wonder how much is due to their investment in notebook LLM + gmail work
conception
Gemini is the only model that has made legitimately good jokes - subtle, clever. I was terrified.
xtiansimon
> “The best response must be complete without becoming shapeless, structured without sounding mechanical…”
Anyone else scratching their head at these descriptions? Uh, your writing is shapeless. You’re 5% too mechanical. Who talks like this? Fiction writers and editors??
pasharayan
TBH :') that one was an AI slip through. In other comments i've flagged the things that were hard rewritten - but even on me that's bad.
gherkinnn
Anything is better than the current crop of Claudes, its prose has become painful.
StableAlkyne
For real, the "Claudish" has become so painful to read that it just takes me out of whatever task I was working on.
When prompted to use simple English without jargon, it's still filled with load bearing honest caveats in every footgun seam it talks about — what I should have led with. <Insert whatever other Claude cliche you prefer>
And I'm not the only one to notice this. Next time it's up for renewal, my team is abandoning it for GH Copilot in order to use literally any other frontier model.
glimshe
That claim about load bearing is doing a lot of work there!
I'm curious to know how "Claudish" emerged during the training process. Why each AI has a particular voice if much of the training material is the same across AIs?
pmontra
Yes, Claude speaks Claudish but at the end of the day I care about the Ruby, Python, JS it writes. It still does a good job at it even if maybe I prefer the way DeepSeek talks. I did not use other models in an agentic harness.
gherkinnn
The code is fine, yes. But now that I am dabbling in spec-driven development (unsure whether I like it) I have to read a lot of prose and I simply cannot get myself to read a page of claudisms. It is that painful.
The odd phrases are one thing, what numbs my brain is how everything is evenly bombastic and lacks any sense of rhythm. Technical documents should not read like catchphrases strung together.
levelZero
don't just dabble, dive in! domain-expert, buck-stopping, orchestrating architects of intent under constraints are going to be the only survivors ;-)
mobius47
good thing claudish isnt a programming language
kuboble
Say a word.
purplecats
You make some good points, but the best one is the one you might not be aware you've made.
levelZero
You may not be aware of the most important emergent property of this comment: it made me piss myself laughing. Strong work, architect of intent.
senderista
It's so bad that my workflow has Sol rewrite all comments and error messages produced by Claude.
whateveracct
yeah i think Claude might just be compounding its way to a local maximum rn
asdfologist
Fair challenge, the ability to write good prose is load-bearing for essential tasks.
gexla
GPT gets the most out there, but creates interesting nuance that turn an idea into a more creative exercise. Claude is like a more grounded GPT but may miss the nuance. Gemini is my last pick, but does better than the rest for making something clear and understandable. It gets to be exhausting parsing through Claude outputs and breaking it down into something more easily understandable. One way I use to improve this is to ask the LLM to pull upstream ideas from prior work on the topics. This makes me feel better about the possibility of hallucination, gives me alternate places to look, but the model still may pull things out of context or fall over on the interpretation.
Given all that, I can definitely see how Gemini would be preferred. And good for Google, because I would rather my offering be the top choice for the most people rather than better serving a small subset of users.
ShawWang
Gemini is bad at coding, but good at writing. More human like
parasti
Gemini is not bad at coding? Give it a /plan and it does a very decent job in a fraction of the time/tokens that Claude needs.
madhatter999
Makes sense when you consider what type of data Google had available in abundance. Natural written language (e.g., Docs, Gmail, Books) and natural spoken language (e.g., YouTube).
One could argue that Google Colab would supply the training data for better coding performance. I would argue that Colab is mostly used for non-complex (e.g., small number of variables) and self-contained (i.e., runnable in one page) code that can’t train a model for multi-folder and multi-page projects that rely on global connections, which real-life coding would often require.
danpalmer
According to public material, Google does not train Gemini on Gmail or Docs: https://blog.google/products-and-platforms/products/gmail/pr... https://proton.me/business/blog/google-docs-ai-scraping
smlacy
Yet
madhatter999
Given it's Google, I don't believe it.
danpalmer
That's up to you, but occam's razor would suggest the most likely thing is that they are in fact just doing what they say they do in their T&Cs.
madhatter999
If that's what Occam's razor would suggest, then Occam presumes a world where actors do what they say. We see more and more the opposite of that in Silicon Valley.
You may well be right about Google not including the said data but your support for this point is weak.
danpalmer
Yes Occam does quite literally state that – the option with the fewest assumptions is most likely to be correct. Assuming that someone is lying is an extra assumption to make over what they are claiming to do.
Malicious acts from large corporations are certainly well known, but they're also grossly over-represented in the news and mindshare. Mostly things just work as they say they work.
I don't really know what proof you're looking for, it's well known that it's hard to prove a negative. I'm not suggesting taking Google (or anyone else) at face-value, but rather saying that when you think about T&Cs, laws, PR, etc, it's unlikely.
WithinReason
They cached the entire internet an have Google Books though
blauditore
Google also has millions of lines of code in several languages written over decades. Probably access to more non-public (quality) code than most other companies in the world.
hesprs
True. Whenever I ask LLMs to translate English to other languages, Gemini is always the most natural one.
Honali
I'd love to see the same experiment with responses normalized for length, and with actual essay quality scored separately from how helpful the model's feedback felt
pasharayan
Will give this a shot - we're going to try to work with students and schools better on normalizing this stuff out
mogili
I've found Gemini to be better at writing. Claude has gotten worse while it gets better at coding / agentic use cases.
kuuuzya
IT students should use their own LLM.
pasharayan
As in they should make their own foundation models for essays?
lwansbrough
If you use Gemini after reading Claude's simlish, it will feel very refreshing.
jdthedisciple
skeptical
at most, the result could be useful for fellow students
I have no doubt other cohorts would rate differently
it is known (on HN at least) e.g. that SWEs tend to prefer brevity, contrary to these students apparently
pasharayan
Students do have different incentives in writing than SWE's. There's the habit of reaching word counts for assignments, some social signalling of how intellgent you are with essay length etc.
The data we have is just there to compare for the student base. I'd love to try it out on other cohorts - but the acquisition of such users and the product to make them happy is hard to achieve!
foolfoolz
when i use gemini i ask it to stop asking me questions, to stop showing me media and just use text, and be concise in output. with these directions it’s pretty useful
explodes
My settings are similar. I also added to be very concise without losing important information. It's output is so slim now that I can ask yes or no questions. Asking for a code snippet gives me just a snippet without an accompanying paragraph of explanation that I would just ignore anyway. Of course I can still ask follow up questions, but it gets right to the point. No filler is a dream.
mial
From my own tests, the best model for writing complex, nuanced prose is Opus 4.6. All the next versions are impossible to coax into good writing. Gemini is OK, GPT ok but requires significant prompt tuning.
userbinator
Having coworkers who use it and thus unfortunately needing to read its output, Claude's "English" is very obviously unnatural-sounding, and extremely distinctive in a bad and irritating way. It's almost like another dialect.
...and of course this article itself has a bit of AI-ish tone to it.
pasharayan
You'll find some paragraphs are half written by Ai and some written totally by me. It's heavily edited from its original form (as it was a data report by codex on our db) - but some paragraphs which were good enough stayed.
E.g. Contrast my human written paragraph vs the AI written first paragraph
Human written:
"The three most popular AI models used by college students are ChatGPT, Gemini, and Claude. As of August 2026 the data on StudyArena shows they prefer Gemini. "
vs
AI Written "Most AI comparsions are written like wine reviews. Claude is subtle. ChatGPT is dependable.."
s18m
its starting to feel that way with newer models. right after opus 4.6, I have started to notice the same
s18m
I have the same workflow for about 3 months now (constant). same workflow since opus 4.5 and now with fable (variable). start with a plan about a problem (somewhat similar), write down the plan, use the plan to execute the work. Lately the plans are so verbose and confusing, Im starting to think of skipping the plan stage entirely or tell it to be super concise.
rienbdj
Students? The couldn’t even write the essay!
pasharayan
Most students can write an essay - but most are also smart enough to try to save time too haha
levelZero
Claude prose praising Gemini ;-)
pasharayan
Codex - at least initially haha! I wrote lots of insights and findings and pushed out a lot of false logic it picked up. Personally I hate claude writing cause it feels bureaucratic. But definitely wrote a ton more with my bare hands and brain.
levelZero
My bad. If you are interested, it was "Gemini's lead matters..." which tingled.
dzhiurgis
Been using Gemini for about a year for code (much cheaper and actually sticks to your prompt so savings are many multiples of; I work with Salesforce implementations and Google is a user for Salesforce so I think their code might have went into training data, coupled with their code guidelines it's been a bliss for me) and for other uses for maybe 9 months.
I do go back to ChatGPT for things that need to be calculated, but _Gemini simply hallucinates less_. IMO that's whats important for students.
I did get suckered into extending my Grok subscription which is really fun for images/videos. Grok in car is a must too (my jaw dropped when I asked what is pantsula music and it added multiple albums into playlist for me).
adventured
I use Fable for writing code and some planning. I use GPT for planning/roadmaps and code review, it checks Fable. I use Gemini for stray conversations. Gemini has a better conversational style than GPT, less robotic; it hallucinates more unfortunately, it has plainly fallen quite a bit behind, but it's still highly useful to converse with, research with, etc. And it spares me wasting usage of GPT or Fable. I use Gemini for the 'everything else' category, basically. Gemini never goes anywhere near anything serious.
Fable & Claude Opus 4.x or 5 are terrible to talk to about anything. I gave up on that entirely and just use Anthropic's models for work.
Any less serious technical work I'll use GPT for, as the usage limits are quite fantastic.
nickhodge
Did they assess if any wanted to use their own brain?
pasharayan
Interestingly, the smartest students I've worked with do use AI - but more in the sense that AI is a good comparison tool and a good bottom ceiling of the quality of work they should bring in.
Lots of research shows being able to see different answers improves the quality of a student's output - especially as the good ones are able to pick out what's good from the other work and then add their own insights or takes on top.
johnnyApplePRNG
I'm blown away, lol. I am not a fan of Gemini.
acosmism
ok. so what
arcticfox2029
I think that recently gemini has responded quickly but has not been effective, unable to support gpt.
The spread isn't as large as I thought from the headline. It's closer to a toss-up than I would have expected.
The length of the response is a huge factor:
> Students tended to prefer longer responses. The selected answer was 37% longer on average than the alternatives. The longest response won 47.7% of decisive writing comparisons. The shortest still won 25.0%.
So the score is partially a proxy for longest responses.
Makes me wonder how much the reviewers actually read the text. Were lazy evaluators picking the text that looked the longest or most structured without reading it all?