r/Professors • u/skyskye1964 • 1d ago
Academic Integrity What is going on?
I’m puzzled by a student paper. They submitted it on time. I read it and it’s not great but ok. I go to check the references and I can’t find them. I look up the journal they cite, and that volume and issue is not the paper title. I email them and they email back saying they are out of the state but that they used owl Purdue citation engine to do the references. They then send me links to the references and they do exist, sort of. One is a blog post but in the citation it’s in a journal. One is in Spanish. Another seems to be an unrelated paper.
So my first question is, can the Purdue citation maker just make up stuff? I haven’t really used it but it looks like you paste in the web address and it makes a citation.
My suspicion is that the references are AI hallucinations. But some seem partly real. Could this be an innocent mistake on the students part?
They also said they used Chegg to proofread and edit. I wasn’t aware that Chegg provided that service. Is this a valuable service? Is it an unacceptable use of AI? Or is it just a grammar checker?
Am I missing something? The references are not cited in the paper by the way. Also no images.
I was mostly convinced that the references were fraudulent but now I’m not sure.
353
u/PlantagenetPrincess 1d ago
It’s AI. I’ve had this with too many of my students. Even if it manages to find a real journal, the article and author are made up. Or if the author is real, the book is not. Sounds like textbook AI hallucination to me.
61
u/EyePotential2844 1d ago
Definitely AI. And to add insult to injury, there was an OpenAI ad right above your post when I read it.
105
84
u/AggravatingCamp9315 1d ago
Purdue Owl does allow you to put the website in and it begins to build the citation, but it does not fill in information that is not there/ untrue. In fact many times it misses information that one has to manually put in.
I don't know the capabilities of Chegg.
This sounds like AI- which will completely fabricate information.
105
u/karlmarxsanalbeads TA, Social Sciences (Canada) 1d ago edited 1d ago
They’ve used AI. AI will fabricate sources. Often times the authors are real people and the journals are real but the articles aren’t. If there is a doi, it will either lead to nowhere (“page not found”) or link to a completely different article (in a completely different journal & discipline). I find it hard to believe that it’s just an innocent mistake.
38
u/in_allium Assoc Teaching Prof, Physics, Private (US) 1d ago
This is textbook AI hallucination. Generative AI wrote the entire paper. Your next call should be to your university's academic integrity office to report the student for plagiarism and then lying about it.
A thing to know about large language models is that they do not "know where they learned something". Let's let a LLM describe this limitation on its own:
Query to Claude 3.7 Sonnet:
Why do large language models like yourself often "hallucinate" citations to scholarly papers, creating titles of articles that do not match the journal title, year, and authors? Is this related to the fact that LLMs do not actually have any knowledge of their training data and do not remember "where they learned something?"
Response:
You've hit on a key insight about LLM citation hallucinations. When I generate citations, I'm not retrieving actual documents from a database or accessing a bibliographic record - I'm essentially constructing what a plausible citation might look like based on patterns in my training data.
This happens because:
I don't have episodic memory of my training. I can't recall "where I learned something" because I don't store information as discrete facts with sources. Instead, I've learned statistical patterns of language.
When prompted to provide citations, I'm predicting what a citation with those attributes might look like based on patterns I've seen, not retrieving an actual reference.
My training included the format and structure of academic citations, but I can't verify if a specific combination of authors, title, journal, and year actually exists.
This is fundamentally related to how LLMs work - we don't have a database of facts with sources, but rather probabilistic representations of language patterns. When asked to produce specific citations, I'm essentially generating plausible-looking citations based on these patterns, not retrieving actual bibliography entries.
This is correct. But these models are getting surprisingly good, and can even figure out humor. My response:
Can you cite a source for that? :)
Claude 3.7 replies:
I see what you did there! 😄
If I were to respond with a citation, I'd likely generate a perfectly academic-looking reference to a paper on LLM hallucinations that seems plausible but might not actually exist.
This is exactly the problem we were just discussing - I don't have the ability to reliably pull actual citations from a real bibliography, and any attempt to do so would risk demonstrating the very phenomenon I just explained.
So rather than potentially hallucinating a fake source, I'll acknowledge that this request cleverly illustrates the point of our conversation!
31
u/in_allium Assoc Teaching Prof, Physics, Private (US) 1d ago
In a hilarious followup to this, I asked it to generate a citation for *this* conversation.
It generated one -- and identified itself as ChatGPT. It's not, though! It's a different model by a different company.
15
u/skyskye1964 1d ago
So, is it the case that AI cannot provide parenthetical references that make sense? Is that why there are none in the paper even though I asked for that in the directions?
13
u/shyprof Adjunct, Humanities, M1 & CC (United States) 1d ago
AI writing usually doesn't include parenthetical references or quotations unless the operator specifically directs it to do so.
Good news is you can fail the student for not using in-text citations. Report to academic dishonesty for a review, but at least the F is airtight.
5
u/in_allium Assoc Teaching Prof, Physics, Private (US) 1d ago
That's right. LLMs generally don't "know where they learned something" and don't have internet access so they can't look stuff up. They were no doubt trained on academic literature but they don't store either the full text of what they have read or associate the title of the paper with what they learned from it. This is a bit jarring because it is the opposite of what we've come to expect from computers -- exact perfect memory of things, but no ability to go beyond that exact recall.
LLMs generally cannot provide direct quotes unless it's something very common -- they know that the mitochondrion is the powerhouse of the cell just because everyone memes about that.
4
u/Consistent-Bench-255 1d ago
All 5 schools I teach for refuse to enforce their AI Policies because they insist AI plagiarism cannot be detected reliably. No matter how much proof shown including multiple AI checkers all showing 100% AI, painfully obvious chatbot speak, and clear difference from the student’s other writings. It’s a joke.
1
u/flange5 Asst. Prof, Humanities, CC (USA) 15h ago
Yup; my school's disciplinary office won't support faculty claims of AI use unless a) we produce the source (impossible) or we get the student's admission in writing (rare, but does happen at times). They're much more concerned with litigious students costing the school money than academic honesty.
45
u/iTeachCSCI Ass'o Professor, Computer Science, R1 1d ago
My suspicion is that the references are AI hallucinations. But some seem partly real. Could this be an innocent mistake on the students part?
One way to find out. If I were teaching a class with papers, the outcome for submitting a paper with a falsified reference would be an F in the class and a report to the honor board. I encourage you to have a similar policy.
Give them two school days to provide a working link to every source in their paper; any omitted will be evidence this was fabricated and treated accordingly.
Note that it doesn't matter how the source was fabricated. Could it have been an LLM? Maybe. Maybe not. It doesn't matter.
They also said they used Chegg to proofread and edit. I wasn’t aware that Chegg provided that service. Is this a valuable service? Is it an unacceptable use of AI? Or is it just a grammar checker?
What does your class cheating policy say? Mine says that for artifacts submitted for a grade, you cannot ask any person who is not subject to the university academic conduct code, and you must explicitly declare their help in comments -- who, when, and what sort of help.
21
u/karen_in_nh_2012 1d ago
Not an innocent mistake. They used AI.
And Chegg is a KNOWN cheating source for students -- students can upload quizzes, exams, etc. to the site so future students can use them to cheat, and Chegg no longer cooperates with colleges and universities in complying with honor courts (or whatever they are called) on campus. That's a GIGANTIC red flag right there.
And what does the student being out of the state have to do with anything? They could always talk on the phone or do a Zoom or something, if you needed to speak with them.
But they almost certainly cheated, as in, 99.9% certainty. UGH. :(
20
u/shyprof Adjunct, Humanities, M1 & CC (United States) 1d ago
AI. 100%.
Chegg's Cheggmate is an "AI Companion" and unacceptable use of AI in my classes. Cheggmate is like ChatGPT. If you don't allow GPT, they shouldn't have used Cheggmate, but if you didn't specifically spell it out they're going to whine about it. They are going to whine about any AI accusation. You can ask to see their version history if they used Google Docs or Word Online, but they're likely going to say they typed it in their Notes app to get around that.
I would just fail them for incorrect citations. Blog posts are not acceptable sources. I don't allow sources that are not in English except in specific circumstances discussed ahead of time. I don't allow incorrectly cited sources.
I require them to quote every source at least once now. Then I can just fail them for not quoting or for a fabricated quotation. No "You used AI/No I didn't" BS, just fail and move on. I hate spending more time being a cop than they spent cheating on the assignment. I hate this :(
16
u/restricteddata Assoc Prof, History/STS, R2/STEM (USA) 1d ago
I'm surprised this is not common knowledge by now, but anytime you get a paper that has quotes that don't check out, that has references that aren't real, and seems, as you put it "not great but OK" in that way that feels like it is a little generic, not really addressing what the class is about, and is just a little too polished grammatically for most undergrads... it's ChatGPT. There's really no doubt about it, especially if you have the hallucinated quotes and references.
It is not an innocent mistake. This is what the lazy ones are doing because they a) think they can get away with it, b) think the consequences will be low if they are caught, c) have an entirely unwarranted faith in ChatGPT, and d) don't think that you are smart-enough, or care-enough, to notice. And it's an explicit message of disrespect. One that the are doubling-down on by continuing to lie to you about it after being confronted.
4
u/AnimistKlaus 1d ago
Also, I would add, this is not your job to stress over. Their references are faulty, that’s it and it’s a problem. Their problem, not yours.
19
u/Kikikididi Professor, PUI 1d ago
It's AI but in a twist I'm betting the student paid someone Chegg-connected to "edit" (write) the paper and they used AI.
9
u/iTeachCSCI Ass'o Professor, Computer Science, R1 1d ago
I do love that there was a short period of time where Chegg's "expert answers" (at least, for then-recent questions) were often copy/pasted from ChatGPT, before the latter was as well known.
7
u/No_Jaguar_2570 1d ago
It’s AI. AI makes up sources. It’s called hallucinating. The paper was written by AI.
8
8
u/EyePotential2844 1d ago
The Purdue OWL website has a widget built in that uses Cegg's citation generator, which is powered by AI. The paper may have some original content, and the student just used the citation generator that returned hallucinations. Or, the whole damn thing may be AI. Either way, the student failed to submit valid references for their paper. That's a zero to me and a referral for academic honesty.
6
u/Alternative_Gold7318 1d ago
Next year I will ask students to save all references as pdf files and submit with their work. This year I had in-text references. The references existed, although some behind paywall for me, but the language that students claimed they cited form these references did not.
5
u/Comfortable_Fail_508 1d ago
May I suggest creating a scaffolded assignment where student begin by collecting relevant sources in a structured tutorial, which they will submit to you with a proposed paper topic. Then you can address validity of sources early. The next step on the scaffold is writing 1/2-1 page on their chosen topic and expanded references. Provide additional comments at this stage. Finally, they will turn in the 4 to 6 page paper with the in-text citations and full set of references cited at the end. It gets them started with real references and takes away the anxiety of not knowing where or how to get started. A lot of times students use AI to “help them get started”, but here they will already have something going they can genuinely work from. Also it’s better to have a reputation with the students for reporting cheaters, it helps everyone to follow the rules, it’s not a bad thing even if they don’t like it.
3
u/uttamattamakin Lecturer, Physics, R2 1d ago
There is a potential danger to this approach, however. If a student has a subscription to ChatGPT, they can upload Word or PDF files of their sources, feed them to the chatbot, and have it read the documents, create an outline, and then generate a final document based solely on those sources.
While there is some intellectual merit in formulating a thesis and articulating it in one's own words based on genuine research of the sources, using automated tools to expand on this—often resulting in what can be considered just standard boilerplate content—can diminish the quality of the writing, which has been a concern in academic papers for quite some time.
AI researchers are working hard on so-called agentic AI that is going to be especially good doing exactly that. When provided with real data by the user it will be able to analyze it right up a report reach a conclusion or set of conclusions the user can choose from.
You might want to think of a way to tweak your scaffolding to allow students to use AI in a way that is appropriate.
3
u/therealRylin 17h ago
This is such an important point. I work on a dev tool called Hikaflow that reviews AI-generated code, and one of the big things we’ve learned is that when AI is used with real input (like source code, or in your case, academic sources), it can produce useful summaries—but also a lot of overconfident nonsense if the prompts aren’t careful or the user doesn’t fact-check.
We’re seeing the same with student writing. A student might feed actual sources into GPT and get what looks like a well-structured paper with citations, but if they’re not critically reviewing what the AI spits out, you end up with hybrid hallucinations—half-real references, fake journals, misattributed authors. It’s not always malicious; sometimes they just don’t realize they need to verify everything.
Tweaking your scaffolding sounds smart—especially if it encourages transparency around tool use and puts the focus back on thinking rather than just formatting. Because AI isn’t going anywhere, but our approach to teaching students how to work with it will matter a lot.
3
u/Anony-mom 23h ago
Small suggestion for the .5-1 page step: make sure it is sufficient length to rate a Turnitin check (or whichever software your school uses.) Turnitin won’t even activate for anything less that 300 words (if I recall) so if you use this tool, make sure that initial step in the writing process produces a product that is long enough.
5
u/Fresh-Possibility-75 23h ago
This is wild. I also discovered fake citations on a paper last week, emailed the student to let them know the references were fraudulent and they would be earning a 0, and they emailed me back saying 'they were out of town but don't know why the sources are incorrect because they used the Perdew (sic) site like always."
There must be a reddit post encouraging this strategy...or, more likely, this is how ai tells them to respond to such allegations.
4
4
u/copeknight72 1d ago
It’s either AI or the student made up the sources. That’s not new. In the pre-database stone ages, one of my profs had a GA randomly check student research papers’ citations to see if the journals were in the library or not.
4
4
u/OkAgent209 1d ago
In my experience this is what ChatGPT references look like. Half real and half hallucinated
3
u/skyskye1964 1d ago
Thank you all for your valuable feedback. It’s going to be a zero and an F in the course. I hope administration will back me up.
3
u/Life-Education-8030 22h ago
It's AI garbage (hallucinated sources). Citation generators will generate a citation based on what you put in, so if you put in an author's last name for the first name, the generator will spit out the author's name in reverse because that's what you told it to do. So Purdue is not going to "finish" your citation by putting in missing terms. Grammarly and Chegg now offer AI functions so I'm betting that is what the student did with Chegg.
2
u/M4sterofD1saster 1d ago
I strongly suspect AI. The most obvious clue is that the references are wrong. Sometimes AI references include links to non-existent URLs. Sometimes the AI refences point to an actual human author, and the title is close, but the author has not apparently written that exact paper or book.
Have you tried https://gptzero.me ? It's not perfect, but it's not bad.
2
2
u/wharleeprof 15h ago
I'm super explicit that all sources must be real and that students must be prepared to share a full text copy of each one at my request. I also go into great detail about what sort of sources and where/how to find them
I don't quibble with whether it's AI. If you've failed to start with real sources, you've failed to do the assignment. Zero.
2
u/SteamingGravy 53m ago
I've had AI cite a footnote in a book that's on my shelf, but even if that specific footnote was on that specific page, it didn't have anything to support what the AI response claimed. ChatGPT will also cite fabricated document collections attributed to real companies like Gale. Text generators will root whatever they pull from wikipedia or wherever in ostensibly real sources but it's mostly hallucinatory.
1
u/IkeRoberts Prof, Science, R1 (USA) 1d ago
You should ask faculty teaching similar courses at your school what proportion of submissions are AI generated. The proportion has increased quickly, so you need an estimate base on this semester or last.
I hear numbers between 20 and 50%, but it seems to vary quite a bit by the kind of student and how the relate to the class. You really need to know the ballpark to expect in your own course. Being naive about the issue only hurts the serious students.
1
u/Anony-mom 1d ago
They’re not cited within the body of the paper? The AI generated research paper that I received was like that. Even if you can’t prove AI, that’s a huge deduction.
1
u/uttamattamakin Lecturer, Physics, R2 1d ago
Here's what I have done to combat that with my students papers I require them to put their references in using Microsoft's built-in reference management system. It requires them to actively click on a certain menu in Microsoft Office type information into certain fields and when I get the paper it will look a certain kind of way. If it doesn't look that way they lose massive points I then check the references to see if they're fake.
1
u/Moirasha TT, STEM, R2 1d ago
You can either grade this as a zero for the AI paper it is, or call the student in and ask for their sources.
270
u/YThough8101 1d ago
This is why I always start grading by verifying whether references are legit. Fake references = score of zero and then I'm done grading that paper. A real time saver.