If you’ve ever shopped on Amazon you probably know that feeling when they “learn” you’re interested in something. Like when you buy a vacuum and so the site sends you vacuum recommendations, emails about vacuums, and vacuum related products for days or weeks. The recommendations seem dumb.
Well, dumb recommendations can be dangerous. YouTube is letting a supremely imperfect algorithm drive attention to things that are not appropriate for anyone. And even worse, they are doing this in the official YouTube Kids app which is supposed to be a safe place for parents to let kids go.
My main take aways are: never trust an algorithm without regularly vetting the real outputs of it and don’t offer services you aren’t willing to support, police, and make better.