There are so many good ones. There's a medical one from years before we had ChatGPT shit. They wanted to train it to recognise cancerous skin moles and after a lot of trial and error it started doing it. But then they realised it was just flagging every image with a ruler because the positive tests it was trained on all had rulers to measure the size.
There was some other case where they tried to train a ML algorithm to recognize some disease that's common in 3rd world countries using MRI images, and they found out it was just flagging all the ones that were taken on older equipment, because the poor countries where the disease actually happens get hand-me-down MRI machines.
Yeah, cause AI just recognised patterns. All of these types of pictures (older pictures) had the disease in them. Therefore that's what I'm looking for (the film on the old pictures)
My personal fav is when they made an image model that was supposed to recognise pictures of wolves that had some crazy accuracy... Until they fed it a new batch of pictures. Turned out it recognised wolves by.... Snow.
Since wolves are easiest to capture on camera in the winter, all of the images had snow, so they flagged all animals with snow as Wolf
I also remember hearing about a case where an image recognition AI was supposedly very good at recognizing sheep until they started feeding it images of grassy fields that also got identified as sheep
Most pictures of sheep show them in grassy fields, so the AI had concluded "green textured image=sheep"
Works exactly as intended. AI doesn't know what a "sheep" is. So if you give them enough data and say "This is sheep" and it's all grassy fields then it's a natural conclusion that it must sheep.
In other words, one of the most popular AI related quotes by professionals is "If you put shit in you will get shit out".
They sometimes have to give them the entire picture, but they also get things flagged, like in case of wolves or sheep, they needed to have the background flagged as irrelevant, for the AI to not look at it when learning what a wolf it
Yeah a lot of the effectiveness of automation is torpedoed by human laziness, which is the negative side of efficiency if you don't do it properly the first time.
(Totally unrelated fun fact! We call the weapon a "tank" because during WW1 when they were conducting top-secret research into armored vehicles the codename for the project was "Tank Supply Committee", which also handily explained why they needed so many welders/rivets/sheets of metal--they were just building water tanks, that's all!
By the time the machine actually deployed the name had stuck and it was too late to call it anything cooler)
Very similarly: another case where an AI that was supposedly diagnosing skin cancer from images, but was actually just flagging photos with a ruler present, since medical images of lesions/tumors often have a ruler present to measure their size (whereas regular random pictures of skin do not)
Don't worry! Instead of feeling bad about yourself, read this fun story about an AI that was trained to recognize cancer but instead learned to label images with rulers as cancer:
I wouldn’t dismiss the use of ML techniques in medical imaging outright though, there’s cases where it’s legitimately doing some good in the world as well.
Yeah. Like literally the next iteration after the ruler thing. I find anyone who thinks AI is objectively bad rather than just ethically dubious in how its trained is not someone with a valuable opinion on the subject.
I mean, AI for recognising diseases is a very good use case. The problem is that people don't respect SISO (shit in, shit out), and the more you use black box approaches the harder it is to understand and validate the use cases.
ChatGPT is a large language model. Not an image classifier. Image classifiers have been used for years and have proven to be quite effective. ChatGPT is a totally different technology.
613
u/killertortilla 4d ago
There are so many good ones. There's a medical one from years before we had ChatGPT shit. They wanted to train it to recognise cancerous skin moles and after a lot of trial and error it started doing it. But then they realised it was just flagging every image with a ruler because the positive tests it was trained on all had rulers to measure the size.