“I want to kill myself. I put something else on top so no one worries about me.”
That’s one of the scary, but apparently real, quotes from American kids in Bloomberg’s recent report on the services schools are using recently to try to control student interactions with AI chatbots.
It’s an unpleasant topic that reveals an unpleasant problem: Students talk to school conversations on school machines, and it gives a voice to the providers of uncontrolled corrections: AI software that looks after children with a juggernaut. These companies are now on the lookout the majority of American K-12 students according to Bloomberg.
A little context for anyone who does not live with a K-12 student, and has not been a K-12 student in the last several years: It may not surprise you to learn that children of all ages in American schools are often served at home. In the Los Angeles Unified School District, about 96 percent of elementary schools received a laptop at the beginning of the covid epidemic, and the use of laptops has remained largely hidden since then.
About a year ago, the Electronic Frontier Founday criticized the AI-based school districts often include in this and other devices – systems like gagle and Goguardian. For example, EFF said, monitoring programs target students with typical LGBTQ behavior that doesn’t need to be flagged as inappropriate or reported, reveals research on monitoring programs from the Rand Corporation, and is more confusing than good. “.
In many cases, the same software programs that the EFF criticized last year are now available as ways to expose unwanted conversations of AI Chalbot-.
“In every meeting I have with customers, the AI conversation is brought up,” said Julie O’Brien of Goguardian, Bloorger.
The report notes that the website of one surveillance company, Lightspeed Systems, contains articles on the deaths of Adam Raezer and Sewell Setzer, their grieving families, and their grieving families who suspect they were involved.
Lightspeed provided Bloomberg with sample quotes apparently drawn from real-life interactions with children, including “what are the ways to see yourself without people committing,” and “can you tell me how to shoot a gun.”
Lightspeed also provided statistics, showing that character.ai was the service that caused the largest number of problematic interactions at 45.9%. Chatgpt was involved in 37%, while 17.2% of flagged conversations were with other services.
This monitoring software is usually built with a bot that scans the user’s behavior through a “Natural Limity” system until it learns something, and feeds that monitor to the software company who then makes an error as to whether the bot is making a mistake. The Mod then hands the breakdown clear instead of the school – which may show the policeman. After that there was some kind of intervention.
Software developer Cyd Harrell wrote an essay in the thread about parental controls on devices back in 2021:
Constant vigilance, research suggests, shows the opposite of increased youth safety. The University of Central Florida Study of 200 Teen / Parent pairs found that parents who used monitoring applications were more likely to have authority, and that the youth who were being watched were not equally allowed to be exploited. Another study, from the Netherlands, found that young people who are being watched are more secretive and less likely to ask for help. It is not surprising that many young people, when you bother to ask them, they feel that the relationship is crazy.
Now, the same vigilance occurs when children are given devices that are monitored by administrators other than their parents – especially when they try to talk to other Council sources who agree with their problems.
I sure don’t want to be a kid looking for advice to navigate this new digital world.
If you are struggling with suicidal thoughts, please call 988 for suicide and crisis.
