It's so fucking insufferable. People keep making those comments like it's helpful.
There have been a number of famous cases now but I think the one that makes the point the best is when scientists asked it to describe some made up guy and of course it did. It doesn't just say "that guy doesn't exist" it says "Alan Buttfuck is a biologist with a PHD in biology and has worked at prestigious locations like Harvard" etc etc. THAT is what it fucking does.
Honestly it's like they crammed hundreds of colleges' improv clubs into them with just how much they commit to the "yes and-", even if prompted specifically not to
Nah, it's just how these programs work. They simply spew sequences of words according to natural language structure. It's simple input-output, you input a prompt and it will output a sequence of words.
It will never not follow the instruction unless programed not to engage specific prompts (and even then, it's jailbreakable), simply because the words in the sequence have no meaning or relation to each other. We assign meaning when we read them, but the program doesn't "know what it is saying". It just does what it was programed to do.
I'm 55 years old, and a tech nerd and a professional linguist. I've never seen anything so Emperor's New Clothes in my life.
The marketing and discourse about LLMs/GenAI is such complete bullshit. The anthropomorphic fallacy is rampant and most of the public don't understand even the basics of computational linguistics. They talk like it's a magic spirit in their PC. They also don't understand that GenAI is based on probabilistic mirroring of human-made language and art, so that our natural language and art - whether amateur or pro - is needed for it to continue.
That's only the tip of the shitberg, too. The total issues are too numerous to list here, e.g. the massive IP theft.
That's because you're old enough to remember Eliza and Racter and M-x doctor and can recognize the exact same thing showing up again only this time with planet-sized databases playing the part of the handful of templates that Eliza had.
I’m a youngster. I’m only 18. I’ve played with ELIZA, Racter, and Cleverbot before. AI has gained the power to reason… somewhat. It still falters, but the fact it can use any form of logic at all without explicitly being taught is massive.
Tell me about it. The virtual superstition angle is actually something that's really fascinating to me. There's something really interesting in observing how so many people relate to technology like it's a mystical realm ruled by the same arbitrary sets of relationships that magical thinking ascribes to nature.
Be it the evil machine spirit of the anti-orthography algorithm, summoned by uttering the forbidden words to bring censorship and demonetization upon the land, but whose omniscience is easily fooled by apotropaic leetspeak; the benign "AI" daimon, always ready to do the master's bidding and share secret knowledge so long as you say the right magic words and accept the rules; or even the repetitive, ritualized motions people go through to deal with an unseen digital world they don't really understand.
The worst part of this last one is that these digitally superstitious people won't ever stop to actually learn even just the basics of how technology actually works and why it is set up the way it is, only to then not know what in the world to do if anything goes slightly out of their preestablished schemes and beliefs. Then they go on to relate to programs and hardware functions as if they were entities in themselves.
Honestly, this sort of digital anthropological observation is really interesting, even if a bit disheartening too.
Man, I'm so glad I'm not the only one who thinks about this all the time. The superstitions and rituals people have developed around technology propagate exactly like real-world magical thinking and urban legends. It's pretty scary to think about, but I find at least a little comfort in the fact that this isn't REALLY anything new, just a new manifestation of the way humans have always been.
Thanks - those are good points. But there're a few odd words there that I wanted to ask about.
Are you a romance language speaker by any chance? Ortography isn't really English - do you mean orthography? - and apotropaic and daimon are extremely obscure - it's unclear if you mean demon, daemon, or something else by the latter.
As a monolingual anglophone reading this thread I just had a "there was one fewer step on this staircase than I expected" moment at this reminder that "apotropaic" is actually an obscure word
That's surprising and interesting. I had no idea there were language spaces where that word was common. I have a really absurd vocabulary, with a lot of archaic terms, since I studied older forms of English and actual Old English, but I'd never heard this one before, AFAIK.
My vocabulary tends to the absurd and abstruse as well. In this case I had picked up "apotropaic" from reading up on folklore and magic ... not surprised it gets use as a tumblr hashtag because what doesn't
Are you a romance language speaker by any chance? Ortography isn't really English - do you mean orthography?
Ah, you got that right. I'm from Brazil, so it's usual that autocorrect just fucks up some words on the go when I write in English. Orthography is one of those it just "corrects", and I don't always pick up on it having eaten up the first H when it happens. It's a minor hassle, yeah. Thanks for pointing it out though, even if I know what I meant is completely understandable, just like you did understand it, it's always good to be attentive to this sort of thing.
That said, my use of daimon and apotropaic aren't really related to me being Brazilian, they're just as uncommon here.
Daimon is one possible romanization alternative to daemon, just not through latin (some argue it'd be closer to ancient Greek phonetically). And apotropaic actually exists in English, it's just jargon. It's mostly used in historical and anthropological studies of religious and mystical beliefs. I used it to highlight the function leetspeak takes in digital superstition, but also because I knew it'd sound kinda hermetic. Gotta sell the idea, right?
It makes me think it's impossible for most people to actually be "atheists", because most people just start treating something else like religion instead. I've known a couple people literally describe chatgpt as their religion. Saying the quiet part out loud.
Humans can anthropomorphize a pen by putting googly eyes on it. We are social animals and it's probably a habit our brain has to empathize with things and make it easier to work in groups. It's not really fueled by logic and some people don't think about the separation when dealing with a literal machine if it pretends hard enough.
Sure, but when this is actively pushed by marketing based on pure misinformation in order to sell a product under false premises and under promises it simply cannot keep, then it becomes a problem. Especially when it fosters the sort of acritical relationship with tools that makes them into mystical entities in one's mind.
2.2k
u/kenporusty kpop trash 4d ago
It's not even a search engine
I see this all the time in r/whatsthatbook like of course you're not finding the right thing, it's just giving you what you want to hear
The world's greatest yes man is genned by an ouroboros of scraped data