Hello Barbie – AI Making Children’s Dreams Come True
Since there have been toys, every kid has fantasized about having conversations with their beloved doll or stuffed animal. The partnership between Barbie's parent company and an artificial intelligence company has made that dream come true.
Hello Barbie is the fulfillment of any child’s dreams: since there have been toys, every kid has fantasized about having conversations with their beloved doll or stuffed animal. In the 20th century, toy makers got a step closer to turning dreams into reality with the commercialization of dolls that spoke several phrases and teddy bears that moved their ears while telling stories.
In the last ten years, advances in artificial intelligence and big data have provided adults with devices that engage on conversations, such as Apple’s Siri and Google’s Assistant. But how are kids benefitting from these developments?
The advances in artificial intelligence and speech recognition have spilled over to the toy industry. According to Mattel, the toy manufacturer and parent company of Barbie: “The number one request we hear from girls is that they want to have a two-way conversation with Barbie doll.” By partnering with ToyTalk, an artificial intelligence company based in San Francisco, they’ve now made that possible.
Hello Barbie is the first AI-powered doll in the market that uses speech recognition and progressive learning to engage with children. Priced at a premium to other dolls (over $80), the doll is programmed with more than 8,000 lines of dialogues and 20 interactive games. Its speech recognition technology is equipped with a microphone, a speaker and two LEDs, and is activated by a push-and-hold button located on the Barbie’s belt. The LEDs light up when the doll’s speech recognition is active.
After purchasing a Hello Barbie, parents connect the doll to a WiFi network and create an account, which they can later use to access the child-doll conversations. Kids can start a conversation with Hello Barbie by pressing the button on her belt. To hear Barbie’s response, the button must be released. The recorded dialogue is sent to a control center via a cloud system, where ToyTalk’s speech recognition technology uses natural language processing to decide possible responses and select the best one.
The conversation and data are sent back and forth between the Barbie and the ToyTalk control center, with all the storage of the recordings and data-processing occurring in the cloud system. The full process (from the child speaking to Hello Barbie replying) takes approximately only a second. The stored recordings are used by the ToyTalk team to improve its speech recognition technology. They are also used to remember things about that child in particular, so that Barbie doesn’t ask the same question twice and is prepared to ask follow-up questions. For instance, Hello Barbie can answer questions build on data from previous conversations, such as “what should I be when I grow up?.” According to several sources, in one demonstration the doll suggested being a dancer or a politician, or a “dancing politician” after learning that the child liked acting.
Despite being in the forefront of the AI-powered toy revolution, Hello Barbie received strong criticism from sceptical parents. First, Hello Barbie calls into question the value of privacy. Should children’s privacy be taken away without their consent or ability to understand what monitoring is? Second, many customers were doubtful about the use of the recorded data. Would Mattel and ToyTalk sell their children’s data to marketers? Mattel tried to dismiss these concerns: “ToyTalk and Mattel will only use the conversations recorded through Hello Barbie to operate and improve our products, to develop better speech recognition for children, and to improve the natural language processing of children’s speech,” a ToyTalk spokesperson said. Third, as with any AI-powered device, Mattel has to decide how the doll would respond to sensitive questions about gender, religion or self-esteem. Lastly, US security researchers warned that when connected to Wi-Fi the doll was vulnerable to hacking. According to several sources, hackers could easily access Hello Barbie’s stored information and take control of its microphone.
Despite these concerns, it is clear the Hello Barbie has opened up a new horizon for toy manufacturers around the world: whether and how to integrate AI capabilities in their product portfolios. Its speech recognition technology and natural language processing ability use the data collected from children around the world to become “smarter”: learning about children’s interests, remembering previous conversations and providing each child with a customized and unique experience. As the creator and market leader of a whole new toy category, Mattel will benefit from its first mover advantage and ability to price these dolls at a premium. It has been reported that the toy manufacturer has been exploring a $300 hologram version of the doll. This premium, I believe, is justified: will children prefer a teddy bear that wiggles its ears or a Barbie who they can speak with for hours? I think that the answer is obvious.
Sources:
“Hello Barbie Messaging / Q&A” – Mattel Website http://hellobarbiefaq.mattel.com/wp-content/uploads/2015/12/hellobarbie-faq-v3.pdf
“Teardown: The electronics of Hello Barbie” – Microcontroller Tips (Apr-2017) https://www.microcontrollertips.com/teardown-electronics-hello-barbie/
“Barbie wants to get to know your child” – The New York Times Magazine (Sept-2015) https://www.nytimes.com/2015/09/20/magazine/barbie-wants-to-get-to-know-your-child.html
“Hackers can hijack Wi-Fi Hello Barbie to spy on your children” – The Guardian (Nov-2015) https://www.theguardian.com/technology/2015/nov/26/hackers-can-hijack-wi-fi-hello-barbie-to-spy-on-your-children
“Privacy fears over ‘smart’ Barbie that can listen to your kids” – The Guardian (Mar-2015) https://www.theguardian.com/technology/2015/mar/13/smart-barbie-that-can-listen-to-your-kids-privacy-fears-mattel
“Barbie: Making products smarter with Artificial Intelligence” – Bernard Marr & Co https://www.bernardmarr.com/default.asp?contentID=730
This is fascinating, Loti! Speech recognition in toys! You have elaborated on concerns around data privacy and security. I was wondering if the company is using collective data to perform some kind of analytics on it to improve its software or individual conversation data that goes to the control center is used to modify each conversation individually? Would it be valuable ( albeit ethically disputable) to analyse all the data they get to see trends in children conversation so that they can train barbies better for the next set of buyers?
And ha! Dancing politician sounds like a good career option!
Super interesting – this reminds me a bit of my own childhood when ‘Furbies’ were becoming popular. I remember how fun it was to talk with them at first, but quickly how creepy they became. My parents actually had to put it in the storage room in our basement and we could hear it babbling all night! Straight out of a horror movie.
I think you elaborated on the privacy concerns really well, especially with the vulnerability to hacking. Even more concerning I think are the issues you raised around sensitive questions about gender, religion or self-esteem. These Barbies could have a significant influence on how children think – do parents want Mattel deciding that narrative? Something to think about for sure!
Love this post! Such an interesting new frontier. I thought you did a great job of bringing up all the potential privacy and regulatory concerns around this product. With regards to selling data, I believe this is against COPPA which makes it illegal to use data to sell targeted ads to kids.
I think as a kid I would love this, but I wonder if some parents might think that this makes ‘play’ easier for kids and robs some of the creative fantasy-play from the Barbie experience.
I had read this sometime back and the idea sounds so amazing. I just think that the adoption of these barbies would be very slow. The low income part of the population can mostly not afford such as an expensive barbie with voice recognition. The part of the population that can afford it is educated and sensitive to what they tell their children. I see the challenge of using further AI in the dialogue as very big since parents are very worried about what their kids are getting taught. One of the solutions that comes to my mind is some kind of consortium regulation and rules that companies have to follow when they design kids toys with AI capabilities.
Great post Loti, super interesting! As others mentioned, great coverage on the privacy and regulatory concerns. I’m wondering if this could also open the door to a new segment of physical toys capable to educate children while they are playing, and how they would compete with educational gaming apps. Maybe would not be the case of such an established brand like Barbie, but I think there will be a new space that could combine joy and learning in a new way.
Great Post Loti. As a father of two girls, I must confess that I am scared of this kind of technology, obviously the privacy issues (that I believe can be solved) but more on the AI side, and responses to sensitive questions, but I believe we cannot stop progress. One idea to address a parent’s concern would be to offer different personalities of Barbies, more conservative or more progressive, for instance, according to the parent’s preference ; )
I had no idea this existed but it is a very cool application of AI. I share your concerns around data & privacy – it made me think about all of the issues around voice assistants speaking without their prompts or recording things they were not supposed to. Given these toys would be around children, these issues are even more sensitive. Although they would have to be careful how they used the data, one potential utilization would be to use the data to inform trends and help generate toy ideas for the future.
Interesting blog Loti and a really unique AI application cutting across consumer products to evolve a historically static segment. I totally agree that the concern about data privacy and voice recording is a troubling one, which adds a new business responsibility to Mattel. I am curious as to what organizational changes Mattel has made as they change their customer interactions and required loyalty/trust. This is a very unique transition for both the Barbie and Mattel brands as they dive into more content than physical products. As we have observed within the media and entertainment industry content is king, so do you think ToyTalk will come out as the winner in the long run by collecting and analyzing this data to create various products and offerings based on their learnings?
This is interesting! Can definitely see the value proposition of a barbie able to have a 2 way conversation with a kid. I was wondering if there were any concerns or potential risks around children becoming too attached their barbie and preferring to talk with a doll instead of other children? In this spirit, should there be a limit on just how well the barbie should be able to speak?
Very interesting article. Thank you, Loti!
I never played with the Barbie doll so I don’t know if I ever want to talk with her. But I definitely want her to be present in front of Zoom camera and make a comment during the class instead of me.
It raises a question about posing some sort of biases for children. Your Barbie now has a voice that is preset with particular tone, accent, and vocabularies which ultimately indicate particular ethnicity, nationality, and so on. My children, hypothetically, would want to mimic the voice and way of speaking of the Barbie. Is this a good thing or a bad thing? If I purchased an African American type Barbie and the doll spoke in “typical-media-reinforced” Black Vernacular English, is this good for my children? is there any source of biases?
Turning fantasy into real-life sounds great, but sometimes fantasy should stay only in dreams.
Thank you for sharing this article. I find it very interesting. I have a four year old daughter and am sure she will appreciate a talking Barbie. I think with evolution of technology, the children entertainment industry has launched quite interesting toys and games to keep children entertained. Like in many cases, kids end up being glued to a screen and in this case a Barbie doll, limiting interaction with people and the things around them, which in my opinion is better for the development of a child. However, I know that once my daughter hears about this talking doll, am going to have to buy one for her!