MDPH used a variety of methods to assess progress toward program goals and objectives, including both qualitative and quantitative data collection.
What MDPH Learned: Mental Health Stigma
One of MDPH's objectives with the ambassador program was to collect information from community members about perceptions of mental health and stigma through surveys and focus groups as described below. MPDH found the following key themes related to stigma and barriers to care:
(1) Negative Attitudes Toward Mental Health: The fear of being judged, labeled, or perceived as weak keeps many community members from discussing mental health. Embarrassment and shame about mental health difficulties also keeps them from seeking help or talking about their mental health. My community does not feel comfortable talking about their mental health with peers or mental health professionals,” one ambassador said. There is a stigma attached to mental health issues and the admission of issues among peers. There is a feeling that mental health issues are a luxury that those in my community cannot admit to.”
(2) Lack of Trust in the Health Care System and Government: Fear of unfair or discriminatory treatment by health care providers discourages people from seeking mental health treatment. Additionally, they are concerned about the security of their health and personal information and how involvement in the health care system interacts with government agencies, specifically the criminal justice system and immigration authorities. I mean, in [the Black] community, we have such a lack of [trust] in the criminal justice system, health care system, all of the systems you can think of, right?” a focus group participant said.
(3) Lack of Culturally Responsive Care: Not having access to a mental health care professional from the same cultural background can be a barrier to seeking care. For the Hispanic/Latinx community in particular, access to Spanish-language services is important for communicating and building trust with providers. As one focus group participant put it, The fact that mental health personnel do not speak the same language as the patients can cause difficulties because a translator is needed, and this complicates the chain of communication.”
(4) Concern for Cost of Care: The cost of mental health services is a deterrent for many community members, especially for those who are uninsured. Many Latinos are not looking for mental health services because they are also concerned about the cost this help may bring,” a focus group participant said.
Objectives Achieved: Outcome Measures Methods and Progress
MDPH's first objective, Objective 1A, is to decrease the percentage of Black or African American, Hispanic/Latinx, and LGBTQ+ populations in metro Denver who have reported eight or more days of poor mental health in the last month by 2025. This objective is a long-term goal that will be measured over the span of a few years. To measure progress, MDPH utilizes data from the statewide Colorado Health Access Survey (CHAS), which is repeated every two years, and a community survey distributed by ambassadors to gather aggregate regional and specific local data about mental health stigma and service utilization.
In addition to analyzing the percentage of respondents who have reported eight or more days of poor mental health, MDPH uses the CHAS to select priority populations to collaborate with the ambassador program. MDPH will use the CHAS on a continuing basis to assess change in stigma and behavioral health access in 2023 and 2025 as well. This includes demographic information and other outcome data such as: (1) reasons why people do not get the mental health care they need, (2) percentage who spoke to a health care provider about mental health in the past year, (3) percentage who spoke to a mental health professional in the past year, and (4) percentage of people who needed but did not receive mental health services. Equity-specific outcome measurements include analyzing disparities in mental health stigma and access among priority race/ethnicities, gender, insurance types, location of residence, housing status, and age groups.
Objectives Achieved: Process Measures Methods and Progress
Focus Groups and Community Survey: Objective 1B is to identify what mental health stigma messages and strategies may resonate with priority populations. To achieve this objective, two ambassadors — one from the Black/African American community and one serving the Hispanic/Latinx community — hosted focus groups to collect feedback on specific mental health messaging. In total, there were 41 focus group participants, 24 from the Black/African American community and 17 from the Hispanic/Latinx community. Focus groups were held both online and in person.
Focus group participants gave feedback on how mental health messaging should be shared and what topics they would like to see promoted in the future. For example, the Hispanic/Latinx focus group highlighted the need for youth-specific mental health messaging in their community. Participants felt that adapting messaging for youth about bullying and substance use was particularly important.
Participants also discussed how mental health messaging should be shared. For example, the Black/African American focus group suggested using a variety of media to reach different age groups in the community, including radio, social media, newspaper, TV commercials, graffiti art, and text messages. The focus groups also provided insight about mental health stigma in their communities; some of the quotes from those focus groups are featured above.
In addition to focus groups, ambassadors surveyed community members to further understand stigma-related barriers among the Black/African American and Hispanic/Latinx populations in the metro region. The community survey, made available in both English and Spanish and in both paper and electronic formats, was distributed by ambassadors in March 2021. In total, 144 people responded: 34% identified as white, 30% identified as Black/African American, and 56% of respondents were Spanish speakers.
MDPH used the survey to further explore CHAS results and asked respondents about a variety of mental health topics, including: (1) outlook on mental health, (2) mental health services, and (3) mental health conversations.
Key findings from each of these areas include:
(1) Outlook on Mental Health: MDPH found that most people (90%) view mental health as part of overall health and talk to their friends and family about it (69%). However, about three out of 10 people don't talk to their friends or family about mental health.
(2) Mental Health Services: Most people view professional services as helpful (85%), but many (54%) do not know what local resources are available or do not have local resources consistent with their beliefs/customs (62%).
(3) Mental Health Conversation: Over half (56%) of people said they are comfortable talking about their mental health when they feel safe and are able to confide in a trusted individual who does not judge them. Other respondents said they feel uncomfortable talking about their mental health because it's not something they talk about much and they are afraid of the other person's response.
MDPH aims to repeat the community survey in the future rounds of programming to assess changes in perceptions of mental health and availability of culturally relevant mental health resources and to inform approaches for the ambassador program moving forward.
These two methods, focus groups and the community survey, allowed MDPH to identify the messages and approaches to mental health conversations that most resonate with community members, fully achieving Objective 1B.
Ambassador Pre/Post Surveys and Monthly Report: Objective 1C is for ambassadors and community members to gain greater knowledge and understanding of stigma and mental health. Ambassadors completed monthly reports outlining the successes and challenges of their activities. Ambassadors also anonymously completed a pre- and post-program survey. The survey asked ambassadors to rate (1) their knowledge level about mental health stigma in their community, (2) comfort having conversations about mental health, and (3) level of partnership that their organization engages in with other public health entities. Results showed that:
(1) Ambassadors felt they were very knowledgeable about mental health stigma after the program and, on average, felt they knew more about mental health resources available in their communities. They also felt community members were more willing to talk about mental health after the program.
(2) On average, ambassadors felt more confident in their ability to decrease stigma in their communities, including by having mental health conversations with community members.
(3) All ambassadors were encouraged to partner with health care or public health agencies. Ambassadors indicated that their partnerships with these entities increased after the program. Additionally, five out of six ambassador organizations reported they were likely to work with public health partners in the future compared with only three out of six before the program.
Results from the pre/post ambassador surveys demonstrate that Objective 1C and Goal 2 (increase the likelihood that ambassador organizations will partner with health care or public health partners) were achieved.
Objective 2A is to promote bidirectional information sharing and best practices in mental health messaging among community partners by hosting monthly Community of Practice (CoP) meetings with community ambassadors and MDPH liaisons. Through the monthly reports, MDPH continually assessed the effectiveness of the CoP meetings and incorporated ambassador feedback and input to make the meetings more useful and informative.
Modifications
One of the most important lessons MDPH learned was to expect variety in ambassador activities and then embrace flexibility when measuring program impact. Ambassador organizations each had different programming capacities, connections to the community, and approaches toward delivering messaging — all of which resulted in a variety of activities. For example, some organizations conducted large-scale media campaigns, while others opted for small, intimate discussion circles. The variety of activities made it difficult at times for liaisons to gauge how ambassadors were doing compared with program requirements and expectations. It is important to anticipate and support a variety of activities and communication strategies from ambassador organizations, without valuing one over another. This is also related to understanding program impact and evaluation. It is difficult to compare strategy outcomes when ambassadors implement diverse interventions. Flexibility, and qualitative data are key, but what constitutes success should be dictated by ambassadors. This is an ongoing and continuous process.