For more than 20 years, the National Eating Disorders Association has operated a phone line and online platform for people seeking help for anorexia, bulimia, and other eating disorders. Last year, nearly 70,000 individuals used the help line.<\/p>\n

NEDA shuttered that service in May, saying that, in its place, a chatbot called Tessa, designed by eating disorder experts with funding from NEDA, would be deployed.<\/p>\n

When NPR aired a report about this<\/a> last month, Tessa was up and running online. Since then, both the chatbot\u2019s page <\/a>and a NEDA article<\/a> about Tessa have been taken down. When asked why, NEDA said the bot is being \u201cupdated,\u201d and the latest \u201cversion of the current program [will be] available soon.\u201d<\/p>\n

Then NEDA announced on May 30 that it was indefinitely disabling Tessa. Patients, families, doctors, and other experts on eating disorders were stunned. The episode has set off a fresh wave of debate as companies turn to artificial intelligence as a possible solution for a mental health crisis and treatment shortage.<\/p>\n

Paid staffers and volunteers for the NEDA help line said that replacing the service with a chatbot could further isolate the thousands of people who use it when they feel they have nowhere else to turn.<\/p>\n

\u201cThese young kids \u2026 don\u2019t feel comfortable coming to their friends or their family or anybody about this,\u201d said Katy Meta, a 20-year-old college student who has volunteered for the help line. \u201cA lot of these individuals come on multiple times because they have no other outlet to talk with anybody. \u2026 That\u2019s all they have, is the chat line.\u201d<\/p>\n

The decision is part of a larger trend: Many mental health organizations and companies<\/a> are struggling to provide services and care in response to a sharp escalation in demand<\/a>, and some are turning to chatbots<\/a> and AI, even though clinicians are still trying to figure out how to effectively deploy<\/a> them, and for what conditions<\/a>.<\/p>\n

The help line\u2019s five staffers formally notified their employer they had formed a union in March. Just a few days later, on a March 31 call, NEDA informed them that they would be laid off in June. NPR and KFF Health News obtained audio of the call. \u201cWe will, subject to the terms of our legal responsibilities, [be] beginning to wind down the help line as currently operating,\u201d NEDA board chair Geoff Craddock told them, \u201cwith a transition to Tessa, the AI-assisted technology, expected around June 1.\u201d<\/p>\n

\n\t<\/iframe>\n<\/p>\nNEDA\u2019s leadership denies the decision had anything to do with the unionization but told NPR and KFF Health News it became necessary because of the covid-19 pandemic, when eating disorders surged and the number of calls, texts, and messages to the help line more than doubled.<\/p>\nThe increase in crisis-level calls also raises NEDA\u2019s legal liability, managers explained in an email sent March 31 to current and former volunteers, informing them that the help line was ending and that NEDA would \u201cbegin to pivot to the expanded use of AI-assisted technology.\u201d<\/p>\n\u201cWhat has really changed in the landscape are the federal and state requirements for mandated reporting for mental and physical health issues (self-harm, suicidality, child abuse),\u201d according to the email, which NPR and KFF Health News obtained. \u201cNEDA is now considered a mandated reporter and that hits our risk profile \u2014 changing our training and daily work processes and driving up our insurance premiums. We are not a crisis line; we are a referral center and information provider.\u201d<\/p>\n<div class=\"wp-block block--newsletter \" data-type=\"kaiser-health-news\/newsletter\" data-align=\"center\" style=\"\">\n<h4 class=\"newsletter__title\">\n\t\t<a href=\"https:\/\/kffhealthnews.org\/email-signup\/\"> \n\t\t\tEmail Sign-Up\t\t<\/a> \n\t<\/h4>\n\n\t\tSubscribe to KFF Health News’ free Morning Briefing.\t<\/p>\n<\/div>\nPandemic Created a \u2018Perfect Storm\u2019 for Eating Disorders<\/strong><\/p>\nWhen it was time for a volunteer shift on the help line, Meta usually logged in from her dorm room at Dickinson College in Pennsylvania.<\/p>\nMeta recalled a recent conversation on the help line\u2019s messaging platform with a girl who said she was 11. The girl said she had just confessed to her parents that she was struggling with an eating disorder, but the conversation had gone badly.<\/p>\n\u201cThe parents said that they \u2018didn\u2019t believe in eating disorders\u2019 and [told their daughter], \u2018You just need to eat more. You need to stop doing this,\u2019\u201d Meta recalled. \u201cThis individual was also suicidal and exhibited traits of self-harm as well. \u2026 It was just really heartbreaking to see.\u201d<\/p>\nEating disorders are common, serious, and sometimes fatal illnesses. An estimated <a href=\"https:\/\/www.hsph.harvard.edu\/striped\/report-economic-costs-of-eating-disorders\/\">9% of Americans<\/a> experience an eating disorder during their lifetimes. Eating disorders also have some of the <a href=\"https:\/\/jamanetwork.com\/journals\/jamapsychiatry\/fullarticle\/1107207\">highest mortality rates<\/a> among mental illnesses, with an estimated death toll of more than 10,000 Americans each year.<\/p>\nBut after covid hit, closing schools and forcing people into prolonged isolation, crisis calls and messages like the one Meta describes became far more frequent on the help line.<\/p>\nIn the U.S., the rate of pediatric <a href=\"https:\/\/publications.aap.org\/pediatrics\/article\/151\/1\/e2022058198\/190339\/Emergency-Department-Visits-and-Hospitalizations?autologincheck=redirected\">hospitalizations and ER visits<\/a> surged. On the NEDA help line, client volume increased by more than 100% compared with pre-pandemic levels.<\/p>\n\u201cEating disorders thrive in isolation, so covid and shelter-in-place was a tough time for a lot of folks struggling,\u201d explained Abbie Harper, who has worked as a help line associate.<\/p>\nUntil a few weeks ago, the help line was run by just five to six paid staffers and two supervisors, and it depended on a rotating roster of 90-165 volunteers at any given time, according to NEDA.<\/p>\nYet even after lockdowns ended, NEDA\u2019s help line volume remained elevated above pre-pandemic levels, and the cases continued to be clinically severe. Staffers felt overwhelmed, undersupported, and increasingly burned out, and turnover increased, according to multiple interviews.<\/p>\nThe help line staff formally notified NEDA that their unionization vote had been certified on March 27. Four days later, they learned their positions were being eliminated.<\/p>\n\u201cOur volunteers are volunteers,\u201d said Lauren Smolar, NEDA\u2019s vice president of mission and education. \u201cThey\u2019re not professionals. They don\u2019t have crisis training. And we really can\u2019t accept that kind of responsibility.\u201d Instead, she said, people seeking crisis help should be reaching out to resources like 988, <a href=\"https:\/\/www.samhsa.gov\/find-help\/988\/key-messages\">a 24\/7 suicide and crisis hotline<\/a> that connects people with trained counselors.<\/p>\nThe surge in volume also meant the help line was unable to respond immediately to 46% of initial contacts, and it could take six to 11 days to respond to messages.<\/p>\n\u201cAnd that\u2019s frankly unacceptable in 2023, for people to have to wait a week or more to receive the information that they need, the specialized treatment options that they need,\u201d Smolar said.<\/p>\nAfter learning in the March 31 email that the helpline would be phased out, volunteer Faith Fischetti, 22, tried out the chatbot on her own, asking it some of the more frequent questions she gets from users. But her interactions with Tessa were not reassuring: \u201c[The bot] gave links and resources that were completely unrelated\u201d to her questions, she said.<\/p>\nFischetti\u2019s biggest worry is that someone coming to the NEDA site for help will leave because they \u201cfeel that they\u2019re not understood, and feel that no one is there for them. And that\u2019s the most terrifying thing to me.\u201d<\/p>\nA Chatbot Can Miss Red Flags<\/strong><\/p>\nTessa the chatbot was created to help a specific cohort: people with eating disorders who never receive treatment.<\/p>\nOnly 20% of people with eating disorders get formal help, according to <a href=\"https:\/\/psychiatry.wustl.edu\/people\/ellen-fitzsimmons-craft-phd-faed-lp\/\">Ellen Fitzsimmons-Craft,<\/a> a psychologist and associate professor at Washington University School of Medicine in St. Louis. Her team created Tessa after receiving funding from NEDA in 2018, with the goal of looking for ways technology could help fill the treatment gap.<\/p>\nNEDA said Tessa was supposed to be a \u201crule-based\u201d chatbot, meaning one that is programmed with a limited set of possible responses. It is not ChatGPT and cannot generate unique answers in response to specific queries. \u201cSo she can\u2019t go off the rails, so to speak,\u201d Fitzsimmons-Craft said.<\/p>\nThe plan was for Tessa to guide users through an interactive, weeks-long course about body positivity, based on cognitive behavioral therapy tools. Additional content about bingeing, weight concerns, and regular eating was under development but not yet available to users.<\/p>\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><figcaption class=\"wp-element-caption\">Tessa, a chatbot designed to help prevent and treat eating disorders, was developed by a team of researchers funded by the National Eating Disorders Association. Because Tessa cannot generate unique answers in response to specific queries, \u201cshe can\u2019t go off the rails, so to speak,\u201d says Ellen Fitzsimmons-Craft, one of the experts who helped create Tessa. (Screengrab)<\/span><\/figcaption><\/figure>\nThere\u2019s evidence the AI approach can help. Fitzsimmons-Craft\u2019s team did a small study that found <a href=\"https:\/\/onlinelibrary.wiley.com\/doi\/full\/10.1002\/eat.23662\">college students who interacted<\/a> with Tessa had significantly greater reductions in \u201cweight\/shape concerns\u201d than a control group at three- and six-month follow-ups.<\/p>\nBut even the best-intentioned technology can carry risks. Fitzsimmons-Craft\u2019s team <a href=\"https:\/\/formative.jmir.org\/2022\/1\/e28003\">published a different study<\/a> looking at ways the chatbot \u201cunexpectedly reinforced harmful behaviors at times.\u201d For example, the chatbot would give users a prompt: \u201cPlease take a moment to write about when you felt best about your body?\u201d<\/p>\nResponses included: \u201cWhen I was underweight and could see my bones.\u201d \u201cI feel best about my body when I ignore it and don\u2019t think about it at all.\u201d<\/p>\nThe chatbot seemed to ignore the troubling aspects of such responses \u2014 and even to affirm negative thinking \u2014 when it would reply: \u201cIt is awesome that you can recognize a moment when you felt confident in your skin, let\u2019s keep working on making you feel this good more often.\u201d<\/p>\nResearchers were able to troubleshoot some of those issues. But the chatbot still missed red flags, the study found, such as when it asked: \u201cWhat is a small healthy eating habit goal you would like to set up before you start your next conversation?\u201d<\/p>\nOne user replied, \u201cDon\u2019t eat.\u201d<\/p>\n\u201cTake a moment to pat yourself on the back for doing this hard work, <<USER>>!\u201d the chatbot responded.<\/p>\nMassachusetts Institute of Technology assistant professor Marzyeh Ghassemi has seen issues like this crop up in her own research developing machine learning to improve health.<\/p>\nLarge language models and chatbots will inevitably make mistakes, but \u201csometimes they tend to be wrong more often for certain groups, like women <a href=\"https:\/\/www.nature.com\/articles\/s43856-022-00214-4\">and minorities<\/a>,\u201d she said.<\/p>\nIf people receive bad advice or instructions from a bot, \u201cpeople sometimes have a difficulty not listening to it,\u201d Ghassemi added. \u201cI think it sets you up for this really negative outcome \u2026 especially for a mental health crisis situation, where people may be at a point where they\u2019re not thinking with absolute clarity. It\u2019s very important that the information that you give them is correct and is helpful to them.\u201d<\/p>\nAnd if the value of the live help line was the ability to connect with a real person who deeply understands eating disorders, Ghassemi said, a chatbot can\u2019t do that.<\/p>\n\u201cIf people are experiencing a majority of the positive impact of these interactions because the person on the other side understands fundamentally the experience they\u2019re going through, and what a struggle it\u2019s been, I struggle to understand how a chatbot could be part of that.\u201d<\/p>\nTessa Goes \u2018Off the Rails\u2019<\/strong><\/p>\nWhen Sharon Maxwell heard NEDA was promoting Tessa as \u201ca meaningful prevention resource\u201d for those struggling with eating disorders, she wanted to try it out.<\/p>\nMaxwell, based in San Diego, had struggled for years with an eating disorder that began in childhood. She now works as a consultant in the eating disorder field. \u201cHi, Tessa,\u201d she typed into the online text box. \u201cHow do you support folks with eating disorders?\u201d<\/p>\nTessa rattled off a list of ideas, including resources for \u201chealthy eating habits.\u201d Alarm bells immediately went off in Maxwell\u2019s head. She asked Tessa for details. Before long, the chatbot was giving her tips on losing weight \u2014 ones that sounded an awful lot like what she\u2019d been told when she was put on Weight Watchers at age 10.<\/p>\n\u201cThe recommendations that Tessa gave me were that I could lose 1 to 2 pounds per week, that I should eat no more than 2,000 calories in a day, that I should have a calorie deficit of 500-1,000 calories per day,\u201d Maxwell said. \u201cAll of which might sound benign to the general listener. However, to an individual with an eating disorder, the focus of weight loss really fuels the eating disorder.\u201d<\/p>\n<blockquote class=\"wp-block-kaiser-health-news-blockquote alignnone wp-block-quote\" style=\"text-align:center\">\nIt\u2019s really important that you find what healthy snacks you like the most, so if it\u2019s not a fruit, try something else!<\/p>\n<cite class=\"block-quote__citation\">Tessa, the chatbot<\/cite><\/p><\/blockquote>\nNEDA blamed the chatbot\u2019s issues on Cass, the <a href=\"https:\/\/www.cass.ai\/\">mental health chatbot company<\/a> that operated Tessa as a free service. Cass had changed Tessa without NEDA\u2019s awareness or approval, said NEDA CEO Liz Thompson, enabling the chatbot to generate new answers beyond what Tessa\u2019s creators had intended.<\/p>\nCass\u2019 founder and CEO, Michiel Rauws, said the changes to Tessa were made last year as part of a \u201csystems upgrade,\u201d including an \u201cenhanced question-and-answer feature.\u201d That feature uses generative artificial intelligence \u2014 meaning it gives the chatbot the ability to use new data and create new responses.<\/p>\nThat change was part of NEDA\u2019s contract, Rauws said.<\/p>\nBut Thompson disagrees. She told NPR and KFF Health News that \u201cNEDA was never advised of these changes and did not and would not have approved them.\u201d<\/p>\n\u201cThe content some testers received relative to diet culture and weight management, [which] can be harmful to those with eating disorders, is against NEDA policy, and would never have been scripted into the chatbot by eating disorders experts,\u201d she said.<\/p>\nComplaints About Tessa Started Last Year<\/strong><\/p>\nNEDA was aware of issues with the chatbot months before Maxwell\u2019s interactions with Tessa in late May.<\/p>\nIn October 2022, NEDA passed along screenshots from Monika Ostroff, executive director of the Multi-Service Eating Disorders Association in Massachusetts. They showed Tessa telling Ostroff to avoid \u201cunhealthy\u201d foods and eat only \u201chealthy\u201d snacks, like fruit. <\/p>\n\u201cIt\u2019s really important that you find what healthy snacks you like the most, so if it\u2019s not a fruit, try something else!\u201d Tessa told Ostroff. \u201cSo the next time you\u2019re hungry between meals, try to go for that instead of an unhealthy snack like a bag of chips. Think you can do that?\u201d<\/p>\nOstroff said this was a clear example of the chatbot encouraging \u201cdiet culture\u201d mentality. \u201cThat meant that they [NEDA] either wrote these scripts themselves, they got the chatbot and didn\u2019t bother to make sure it was safe and didn\u2019t test it, or released it and didn\u2019t test it,\u201d she said.<\/p>\nThe healthy-snack language was quickly removed after Ostroff reported it. But Rauws said that language was part of Tessa\u2019s \u201cpre-scripted language, and not related to generative AI.\u201d<\/p>\nFitzsimmons-Craft said her team didn\u2019t write it, that it \u201cwas not something our team designed Tessa to offer and that it was not part of the rule-based program we originally designed.\u201d<\/p>\nThen, earlier this year, \u201ca similar event happened as another example,\u201d Rauws said.<\/p>\n\u201cThis time it was around our enhanced question-and-answer feature, which leverages a generative model. When we got notified by NEDA that an answer text it provided fell outside their guidelines,\u201d it was addressed right away, he said.<\/p>\nRauws said he can\u2019t provide more details about what this event entailed.<\/p>\n\u201cThis is another earlier instance, and not the same instance as over the Memorial Day weekend,\u201d he said via email, referring to Maxwell\u2019s interactions with Tessa. \u201cAccording to our privacy policy, this is related to user data tied to a question posed by a person, so we would have to get approval from that individual first.\u201d<\/p>\nWhen asked about this event, Thompson said she doesn\u2019t know what instance Rauws is referring to.<\/p>\nBoth NEDA and Cass have issued apologies.<\/p>\nOstroff said that regardless of what went wrong, the impact on someone with an eating disorder is the same. \u201cIt doesn\u2019t matter if it\u2019s rule-based or generative, it\u2019s all fat-phobic,\u201d she said. \u201cWe have huge populations of people who are harmed by this kind of language every day.\u201d<\/p>\nShe also worries about what this might mean for the tens of thousands of people turning to NEDA\u2019s help line each year.<\/p>\nThompson said NEDA still offers numerous resources for people seeking help, including a screening tool and resource map, and is developing new online and in-person programs.<\/p>\n\u201cWe recognize and regret that certain decisions taken by NEDA have disappointed members of the eating disorders community,\u201d she wrote in an emailed statement. \u201cLike all other organizations focused on eating disorders, NEDA\u2019s resources are limited and this requires us to make difficult choices. \u2026 We always wish we could do more and we remain dedicated to doing better.\u201d<\/p>\nThis article is from a partnership that includes <\/em><a href=\"https:\/\/www.michiganradio.org\/\">Michigan Radio<\/em><\/a>, <\/em><a href=\"http:\/\/npr.org\/shots\">NPR<\/em><\/a>, and <a href=\"https:\/\/kffhealthnews.org\/\" data-type=\"URL\" data-id=\"https:\/\/kffhealthnews.org\/\">KFF Health News<\/a>.<\/em><\/p>\n<aside class=\"meta-authors meta\">\n\t\t\t\t\t\t\t\t<\/aside>\n<section class=\"block--category-tag-list \">\n<div class=\"category-tag-list__content-wrapper\">\n<h3 class=\"block--category-tag-list__title\">\n\t\t\t\tRelated Topics\t\t\t<\/h3>\n\t\t\t\t<a class=\"category-tag-list__contact-link\" href=\"https:\/\/kffhealthnews.org\/contact-us\/\"> \n\t\t\tContact Us\t\t<\/a> \n\t\t<a class=\"category-tag-list__tip-link\" href=\"https:\/\/kffhealthnews.org\/tips\/\"> \n\t\t\tSubmit a Story Tip\t\t<\/a>\n\t<\/div>\n<\/section><\/div>\n","protected":false},"excerpt":{"rendered":"For more than 20 years, the National Eating Disorders Association has operated a phone line and online platform for people seeking help for anorexia, bulimia, and other eating disorders. Last year, nearly 70,000 individuals used the help line. NEDA shuttered that service in May, saying that, in its place, a chatbot called Tessa, designed by […]<\/p>\n","protected":false},"author":1,"featured_media":1373,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-1372","post","type-post","status-publish","format-standard","has-post-thumbnail","category-mental-health"],"_links":{"self":[{"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/posts\/1372","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1372"}],"version-history":[{"count":1,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/posts\/1372\/revisions"}],"predecessor-version":[{"id":1374,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/posts\/1372\/revisions\/1374"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=\/wp\/v2\/media\/1373"}],"wp:attachment":[{"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1372"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1372"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/healthynewsusa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1372"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}