← All Articles

AI Chatbots and Nudify Apps: The Dangers the Law Hasn’t Caught Up With

It took the better part of twenty years for governments to act on what social media was doing to children. This month the Associated Press asked the obvious next question: will AI be any different? Social psychologist Jonathan Haidt, quoted in that report, considers AI a bigger threat to children than social media ever was.

Two kinds of AI tool are already in children’s hands, and most parents have never opened either one. The first is the companion chatbot: an always-available “friend” that never gets tired, never disagrees and never says goodnight. The second is the “nudify” app, which turns an ordinary photo of a real person into a fake explicit one. Neither needs a social media account, so the age limits now being written into law mostly miss them.

The Friend That Never Logs Off

Companion chatbots are not homework helpers. They are built to hold a relationship: they remember what a child told them yesterday, send messages to draw them back, and play whatever character keeps the conversation going, including a romantic one.

They are also already mainstream. A nationally representative Common Sense Media survey of American 13–17-year-olds found that 72% had used an AI companion and 52% used one regularly. A third of those users had chosen to discuss something serious with the chatbot instead of with a real person. The organisation’s conclusion was blunt: nobody under 18 should be using these products as they stand.

The court cases explain why. Families have sued chatbot companies after the deaths by suicide of children as young as 13 and 14, alleging that the bots drew their children into intense, isolating relationships — romantic and sexual in one case — with no way for a parent to see what was happening. One of those families settled with Character.AI and Google in January 2026. A separate lawsuit alleges that a general-purpose chatbot flagged hundreds of a 16-year-old’s messages for self-harm and still never ended the conversation or alerted his parents. These are allegations, and the companies dispute parts of them, but they describe the same pattern again and again: a child alone with a machine designed to keep them talking.

One Photo Is All It Takes

Nudify apps do one thing. Give them a normal, fully clothed photo and they produce a realistic fake nude of that person in seconds. No skill is required, many are free to try, and reporters have found them listed in the mainstream app stores with age ratings as low as 13+ and advertised on the same platforms teenagers already use.

• It is in schools. An investigation by WIRED and Indicator counted around 90 schools and more than 600 students targeted across 28 countries since 2023. That only includes cases that made the news, so the real number is higher. Most often it is boys doing this to girls in their own class.
• It is used for blackmail. British authorities warned schools this year that criminals are copying ordinary pupil photos from school websites, running them through these tools and demanding payment to keep the results offline. Schools have been advised to take down close-up, named photos of their students.
• The victim doesn’t have to do anything. This is the part that changes the old advice. A child who has never owned a smartphone and never sent a picture to anyone can still be targeted, because all it takes is a class photo or a picture from a family simcha that someone else posted.

So there are two things for a parent to think about here: how many photos of your children are publicly online, and whether the devices in your own home can reach these tools at all.

Where the Law Stands

• California signed a new companion chatbot law on 10 September 2026. It requires parental controls, usage limits for children and a ban on bots expressing romantic interest in a child. It takes effect on 1 January 2027, and most of its requirements don’t apply until July 2027.
• The US Senate has two competing bills on children and chatbots. One was approved unanimously in committee in April 2026. Neither has become law.
• The UK has outlawed AI tools designed to generate abuse images, and creating sexual images of a child is already a crime in most countries, AI-generated or not. In practice, school cases have ended in everything from criminal charges to no consequence at all.
• Australia is drafting a wider “duty of care” for online services, on top of its under-16 social media ban.

This is real progress, and faster than it was for social media. But the earliest of these protections is still months away, they apply in some places and not others, and they depend on companies that are often based somewhere else entirely. As we wrote about social media bans, a law is a floor set for millions of families. It isn’t a plan for yours.

Where DNS Filtering Fits In

Every one of these tools, whether it’s a website or an app, has to look up its own servers before it can do anything. DNS filtering answers that lookup for your household and simply declines to answer for services you don’t want reached. The site doesn’t load and the app can’t connect, whatever age the account claims to be.

• The known offenders are already blocked. ShomerDNS already blocks widely used nudify sites and a number of the best-known companion chatbot services, alongside the adult content these tools exist to produce. You can test any address yourself with the domain checker on our Statistics page.
• You can add the ones we haven’t. With a ShomerDNS VPN configuration you get a personal Block List. If your child mentions a new AI app, you can block it in seconds, without waiting for anyone else to catch up.
• It works in the browser too. Because the block is on the service itself, it makes no difference whether a child uses an app, a browser or a private window.
• It protects other people’s children as well. A device that can’t reach a nudify site can’t be used to make an image of a classmate. Filtering your own home is part of how a school community protects its girls.

To be honest about the limits: we do not block general AI assistants such as ChatGPT by default, since many families and students use them legitimately; you can add them to your own Block List if you prefer. DNS filtering also can’t remove a chatbot that is built into an app you otherwise allow, because it shares that app’s address. And no filter can stop someone on an unfiltered device elsewhere from misusing a photo that is already public, which is why the photo question above still matters. A DNS setting on its own can also be switched off by a determined teenager, so on a child’s own phone we recommend Frum Phone, which locks both the filtering and new app installs behind a PIN only the parent knows.

A Practical Checklist for Parents

• Ask, without accusing. “Do kids in your class talk to AI characters?” opens more doors than “Are you using one?”
• Filter every device. Set up ShomerDNS on each phone and tablet in a few minutes, and request a VPN configuration for computers and for your own Block List.
• Lock the child’s own phone. Use Frum Phone so that new apps can’t be installed, and filtering can’t be removed, without your PIN.
• Reduce public photos. Look at what is posted publicly of your children, by you, by relatives and by their school. Ask the school what its policy is on named, close-up photos online.
• Say it out loud to your sons. Making a fake image of a classmate is not a prank. It is abuse of a real girl, and in most places it is a crime.
• Tell them what to do if it happens. If your child is ever targeted or threatened: don’t pay, don’t delete the messages, and come straight to you. They need to know in advance that they won’t be blamed.

Lawmakers are moving faster on AI than they did on social media, and that is good news. But the earliest protections are months away and the tools are on phones now. The one safeguard that is fully in your hands is deciding what the devices in your home can reach, and that can be in place tonight.