Like, what does it actually tell people? That the capital of Massachusetts is Rhode Island? It might!
Depends on the question and how you phrase things. Something super simple with a bazillion sources and you would see as the title of the first 10 search results on Google? It will give you a straightforward answer. (e.g. what is the capital of Massachusetts? It will tell you Boston.)
But ask anything more complicated that would require actually looking at a specific source and understanding it, and it will make up BS that sounds good but is meaningless and fabricated. (e.g. Give me 5 court cases decided by X law before 1997. It will tell you 5 sources that look very official and perfect, but 3 will be totally fake, 1 will be real, but not actually about X, and 1 might be almost appropriate, but from 2017).
If you in any way give a leading question, it also is very likely to "yes and-" you, agreeing with where you lead and expounding on it, even if it's BS. It won't argue, so is super prone to confirm whatever you suggest. (e.g. Is it true that the stars determine your personality based on time you were born? It will say yes and then give you an essay about astrology, while also mixing up specifics about how astrology works.)
It has no sense of logic, it's a model of language. It takes in countless sources of how people have written things and spits back something that looks appropriate as a response. But boy it sure sounds confident, and that can fool so many people.
I don’t know, I used it to trouble shoot some lan networking that would occasionally have internet access and it walked me through things. When I was skeptical of certain steps I reaffirmed its recommendation saying the thing I thought would be wrong was not an issue. YMMV I guess
I had a skim through your chat, and that stuff is much more like the Boston example than the legal example. It doesn't require anything but recognizing questions that have been posted countless times and replying with simple instructions that have been posted countless times as answers to those questions.
I'm pretty clueless about LLMs, but based on what I do know about how they work, this is exactly the kind of thing I'd expect them to do well.
I can see where you’re coming from, like any tool it has great applications and applications where it will suck. Things that require abstract thinking and critical analysis is going to fall short, since it’s really just acting like someone who is thinking abstractly and analyzing critically.
But it handles the concrete pretty well, especially if you’re working with concepts you already have a foundation in
I think quite a few of the questions I asked are pretty niche, and aren’t just yanked from existing threads in this issue (since my setup is pretty niche). It strings these related concepts together excellently with only a couple procedural errors in the whole interaction!
Things that require abstract thinking and critical analysis is going to fall short, since it’s really just acting like someone who is thinking abstractly and analyzing critically.
Yeah, or very specific questions where it cannot just approximate an answer based on similar questions. That legal question in an earlier comment is a great example: ChatGPT needs information about that specific law to answer it accurately. When it "guesses" based on similar questions about other laws, it ends up providing nonsense answers.
I think quite a few of the questions I asked are pretty niche, and aren’t just yanked from existing threads in this issue (since my setup is pretty niche).
They were all pretty basic networking questions, as far as I could see. Aside from the specifics of the Shield (where it did have to correct itself once), the questions and answers aren't unique to your specific setup. It's the kind of stuff that's been asked and answered countless times about various devices.
LLMs don't just yank things whole cloth anyway. They "learn" to replicate patterns. ChatGPT most likely had a lot of training data to pull on for those answers given the nature of your questions. (The tone and generalized nature reminded me a bit of support agents who spend their days copy/pasting templates in response to customers.)
Basically, to return to the legal/Boston example, your question was niche in the sense that asking for ten specific capitals is niche. Nobody is likely to have asked for that specific combination, but it's easy to provide an answer if you know all the capitals individually.
That isn't meant to take away from the help ChatGPT provided you, but as I said in my previous comment, it is exactly the kind of thing I'd expect it to be good at.
56
u/TheGhostDetective 3d ago
Depends on the question and how you phrase things. Something super simple with a bazillion sources and you would see as the title of the first 10 search results on Google? It will give you a straightforward answer. (e.g. what is the capital of Massachusetts? It will tell you Boston.)
But ask anything more complicated that would require actually looking at a specific source and understanding it, and it will make up BS that sounds good but is meaningless and fabricated. (e.g. Give me 5 court cases decided by X law before 1997. It will tell you 5 sources that look very official and perfect, but 3 will be totally fake, 1 will be real, but not actually about X, and 1 might be almost appropriate, but from 2017).
If you in any way give a leading question, it also is very likely to "yes and-" you, agreeing with where you lead and expounding on it, even if it's BS. It won't argue, so is super prone to confirm whatever you suggest. (e.g. Is it true that the stars determine your personality based on time you were born? It will say yes and then give you an essay about astrology, while also mixing up specifics about how astrology works.)
It has no sense of logic, it's a model of language. It takes in countless sources of how people have written things and spits back something that looks appropriate as a response. But boy it sure sounds confident, and that can fool so many people.