ACTIVATE Evaluation Report | Center for Reducing Health Disparities | UC Davis Health

University of California, Davis, Center for Reducing Health Disparities in partnership with Ampla Health and ACTIVATE Project Partners

ACTIVATE

Accountability, Coordination, and Telehealth in the Valley to Achieve Transformation and Equity

2022-2023 Evaluation Report

The ACTIVATE Community Engagement Team is grateful for all the community partners who shared their personal experiences through their participation in focus groups and key informant interviews.

This collaborative project was funded by a private donation. We are grateful for the donor’s generous support in service of the advancement of telehealth and tele-mental health for historically underserved communities.

ACTIVATE was made possible through the collaboration of various partners in the agricultural community in the Northern Sacramento Valley, Ampla Health, UC Davis Center for Reducing Health Disparities, the Center for Information Technology Research in the Interest of Society and the Banatao Institute (CITRIS), and MITRE.

Lucy Candito – Patient Navigator; Neymi Leon – Lead Community Promotions Worker; Maria Soto – Community Promotions Worker; Adriana Sanchez – Community Promotions Worker; Adriana Castro – Community Promotions Worker; Cynthia Peshek, M.A. – Outreach Program Manager;

Cindy Snelgrove, M.B.A./M.S.N./H.C.M., RN, P.H.N. – Chief Clinical Services Officer; Catharin Manriquez

– Former Lead Community Promotions Worker; Benjamín H. Flores, M.P.H. – Chief Executive Officer; Horacio Paras – Board Chair.

Community Partner: Brian Park

Sergio Aguilar-Gaxiola, M.D., Ph.D. – Director; Katherine Ferry – Project Manager; Karla Ornelas Hernandez – Senior Project Coordinator; Javier Morla Estrada, M.Sc. – Graduate Student Researcher; Andrea C. Nuñez – Chief Administrative Officer. Cassie Chandler, M.P.H.; Jeffrey Duong, Ph.D.; Gustavo Loera, Ed.D. – formerly of CRHD.

David Lindeman, Ph.D. – Executive Director, CITRIS; Gale Berkowitz, Dr.P.H. – Associate Director, CITRIS; Katherine Kim, Ph.D. – Principal, MITRE Labs, Health Innovation Center; Melissa Bruno Valenzuela, M.P.H. – Senior Public Health Specialist, MITRE.

Suggested Citation: Morla Estrada, J.A., Ferry, K., Ornelas Hernandez, K.D., Nuñez, A.C. & Aguilar- Gaxiola, S. (2023). ACTIVATE 2022-2023 Evaluation Report. University of California Davis Center for Reducing Health Disparities.

TABLE OF CONTENTS

1. Executive Summary
2. Purpose
3. Evaluation Questions
4. Methodology
   • ACTIVATE Desired Results and UTAUT
   • Attitudes Survey
   • Knowledge Tests
   • Structured Participant Observation
   • Qualitative Exit Interviews
   • Community-engaged Learning
5. Limitations
6. Findings
   • Attitudes Surveys on Telehealth
   • Knowledge Tests and Sessions Attendance
   • Structured Observations of Participant Engagement and Promotora Facilitation Skills
   • Impacts of ACTIVATE
7. Lessons Learned
8. Best Practices
9. Conclusions
10. References

1. EXECUTIVE SUMMARY

The purpose of this evaluation is to assess the achievement of project objectives, identify impacts on participants and implementing partners, and discern lessons learned and best practices of the Accountability, Coordination, and Telehealth in the Valley to Achieve Transformation and Equity (ACTIVATE) project implemented by the Center for Reducing Health Disparities at UC Davis Health, and implementing partners, Ampla Health and a local grower. This evaluation employs a mixed- methods approach to assess the ACTIVATE project, utilizing quantitative and qualitative techniques, participant observation, and community-engaged learning.

These methods offer diverse data sources to triangulate evidence and measure project impacts on participant Latino farmworkers and implementing partners.

The summarized findings of the evaluation include:

• Health education sessions promoted favorable attitudes toward telehealth care service use among project participants. The results of the attitude survey demonstrate that most participants, on average, expressed agreement with the intention to regularly seek medical care through telehealth technology after completing the health education sessions.
• Health education sessions created awareness about telehealth and mental health care service availability and use among project participants. The results of the knowledge tests show that, on average, participants’ level of knowledge increased 10-22% each session after completing the health education sessions.
• The facilitator’s guide was well received and beneficial for participants and Ampla Health.
• The facilitation training supported the Promotoras’ delivery of health education sessions through developing and practicing public presentation skills.
• ACTIVATE’s financial and promotional incentives were beneficial to motivating participants to attend health education sessions.

During the ACTIVATE project, the following best practices were identified:

On developing the health education curriculum content:

• Solicited input from all project partners, including the grower, regarding curriculum topics promotes buy-in and satisfaction with the project.
• Incorporate new modules or making significant changes to the curriculum requires a team member with a background in curriculum design.
• Review and revising the professionally translated curriculum demanded more effort than anticipated.

On health education content and strategies that worked:

• Roleplay and simulation of telehealth visit between participants and Promotoras.
• Incorporate stories and examples Latino farmworkers can relate to in facilitator’s guide made telehealth and mental health seeking relatable and not scary.

On training for Promotoras:

• Allow more time for Promotoras to rehearse workshop content and review materials in advance.
• Equip Promotoras with strategies for facilitaing discussions when addressing these sensitive topics.
• Enhance training on effectively asking open-ended questions related to mental health.

On community Engagement:

• Involving a grower in a project like this requires personal connections within the agricultural community.
• Allocate sufficient time for the health education workshops to suit the preferences of all implementing partners.

Considerations for Future Revisions or Additional Content to the Curriculum:

• Include a session that delves into the factors contributing to mental health issues, in addition to covering types of services, proves beneficial for Latino farmworkers.
• Consider splitting module 6 into two separate sessions.
• Incorporate visual aids for presenting information, such as infographics or images of a farmworker on a telehealth call.

Additional Best Practices:

• Engage Promotoras throughout the process to co-design the curriculum.
• Provide training to Promotoras in public speaking and workshop facilitation.
• Develop trust with workshop participants before the workshops begin by meeting informally – also known as “convivencia” in Spanish.
• Include telehealth simulation and role- playing activity in the workshops.
• Provide incentives for workshop attendance.
• Build and maintain trust with all project partners.

2. PURPOSE

The purpose of this outcome evaluation is to assess the tangible results, impacts, lessons learned, and best practices of the ACTIVATE project. The evaluation aims to provide a comprehensive appraisal of how the project’s objectives impacted project participant Latino farmworkers and implementing partners, Ampla Health and the local grower.

The key objectives of this outcome evaluation are as follows:

Measurement of Project Success: Quantify the degree to which the ACTIVATE project achieved its intended objectives of promoting Latino farmworkers’ favorable attitudes about tele(mental)health care service use.

Impact Assessment: Examine the effects of project’s key products and inputs on participants, and implementing partners, Ampla Health and the local grower. The products and inputs include the facilitator’s guide, the promotora facilitation training, and the incentives given to participants for attending the health education workshops.

Participant Engagement: Determine the degree to which participants actively participated in ACTIVATE’s health education workshops.

Unintended Consequences: The evaluation identifies any unintended positive or negative consequences resulting from the project’s implementation. By acknowledging these effects, we can adapt strategies to enhance positive outcomes and mitigate potential drawbacks in future endeavors.

Lessons Learned: Identify the knowledge and understanding gained by the experience of implementing the ACTIVATE project. This experience may be based on insights from what worked well and what challenges were encountered, during the project design, implementation, supervision, and evaluation stages.

Best Practices: Identify processes or activities of the project that yield success or meaningful results. These practices are seen as an innovation in comparison to what was done before at Ampla Health or by the local grower.

Accountability and Transparency: The evaluation will enhance accountability by providing a transparent assessment of the project’s achievements and limitations. This transparency fosters trust among partners and supports responsible project management.

Dissemination of Results: The evaluation results will be communicated to partners, including project sponsors, donors, partners, and the wider public. Clear and concise reporting will share the project’s outcomes and impact, facilitating public awareness and shared decision-making.

By conducting this project outcome evaluation, we aim to ensure that the ACTIVATE project has delivered meaningful and replicable results, contributing positively to the health of Latino farmworkers. Finally, results of this evaluation report will inform sections of the ACTIVATE Community Engagement Implementation Guide.

3. EVALUATION QUESTIONS

The evaluation questions and data sources used to answer the questions are presented by the dimension evaluated in Table 1. The dimensions identify what is the aspect the project has examined. The evaluation questions provide what in specific is appraised. The data sources and their collection are discussed in more detail in the Methodology section below.

Evaluation Questions and Data Sources for the ACTIVATE Evaluation

Impacts and Long-term Outcomes

• Evaluation Questions:
• To what extent did attitudes about telehealth change among Latino farmworkers as a result of the ACTIVATE project?
• What significant changes were observed in Latino farmworkers as a result of the ACTIVATE project?
• Data Sources:
• Attitudes Survey
• Exit Interviews

Immediate Outcome

• Evaluation Questions:
• To what extent did knowledge about telehealth and mental health improve among Latino farmworkers as a result of the ACTIVATE project?
• Data Sources:
• Knowledge Tests
• Exit Interviews

Participant Engagement

• Evaluation Questions:
• To what extent were Latino farmworkers actively engaged in ACTIVATE health education sessions?
• How were ACTIVATE health education topics received by Latino farmworkers?
• Data Sources:
• Participant Observation Tool
• End of Project Participant Appraisal
• Exit Interviews

Promotora Facilitator’s Guide and Training

• Evaluation Questions:
• To what extent did Promotoras facilitate ACTIVATE health education sessions effectively?
• How did the Promotora facilitator’s guide support Promotoras’ health education sessions?
• How did the Promotora training support Promotoras to facilitate health education sessions?
• Data Sources:
• Facilitation Observation Tool
• Exit Interviews

Incentives

• Evaluation Questions:
• How did participant incentives (monetary and promotional material) contribute to participant engagement?
• Data Sources:
• Exit Interviews
• Project Records

Lessons Learned

• Evaluation Questions:
• What lessons can be learned from ACTIVATE’s design, planning, implementation, supervision, and evaluation?
• Data Sources:
• Exit Interviews

Best Practices

• Evaluation Questions:
• What best practices can be gleaned from ACTIVATE’s design, planning, implementation, supervision, and evaluation?
• Data Sources:
• Exit Interviews

4. METHODOLOGY

This evaluation utilizes a mixed-methods approach to the assessment of the ACTIVATE project, including quantitative, qualitative, participant observation, and community-engaged learning. In addition, these methods provide complementary data sources to assess the achievement of the project objectives outlined in thenACTIVATE theoretical framework (see 4.1.). The theoretical framework is premised on an adapted PRECEDE-PROCEED model (Glanz, 2015).

The quantitative methods include an attitude survey on telehealth acceptability, and pre-post tests on knowledge gained from health education sessions for Latino farmworkers. The qualitative methods include semi-structured exit interviews of all Ampla Health and CRHD staff and the local grower that participated in the ACTIVATE project. The participant observation includes the use of semi-structured tools to observe and appraise the level of participant engagement during health education sessions and Promotoras’ application of facilitation skills during health education sessions. Finally, the community- engaged learning approach included a series of structured internal workshops between CRHD and Ampla Health, to complete mid- point check-in of the project and review initial findings of exit interviews and identify lessons learned and best practices of the ACTIVATE project.

4.1 ACTIVATE Desired Results and UTAUT

Figure 1 illustrates a simplification of the ACTIVATE’s desired results, which is drawn from the comprehensive theoretical framework (see Annex 1). The figure shows how the intervention’s activities and products produce desired immediate and long-term outcomes in Latino farmworkers. The figure also highlights how these project level objectives contribute to a long-term goal that is outside the scope of this project’s influence, tele(mental)health care utilization among Latino farmworkers. In addition, the figure highlights how these objectives are measured in this evaluation report.

Figure 1. Simplified ACTIVATE Objectives drawn from Comprehensive Theoretical Framework

Inputs, Products & Activities assessed in Project Record Attendance:

Develop Facilitation guide on telehealth and mental health care availability and use.

Develop Facilitation training

Develop Health Education sessions

Immediate Outcomes assessed in Knowledge Tests and Participant Observation:

Build awareness of telehealth and mental health care service availability and use at Ampla Health among Latino farmworkers.

 

Long-term Outcomes Assessed in Attitudes Survey and Exit Interviews:

Promote favorable attitudes toward tele(mental) health care service use at Ampla Health among Latino farmworkers.

 

Goal (Not Assessed Beyond Scope of ACTIVATE):

Tele(mental) health care service utilization at Ampla Health among Latino farmworkers.

 

UTAUT Model

The Unified Theory of Acceptance and Use of Technology (UTAUT) is a widely recognized model that aims to explain and predict individuals’ adoption and usage of information technology in various contexts (Venkatesh, 2003). The technology of interest in ACTIVATE is telehealth. The model identifies several key dimensions that influence users’ decisions to accept and use technology. These dimensions collectively provide insights into the factors that impact users’ long-term behavior to adopt telehealth care services (Harst, 2019).

Dimensions of the UTAUT

The UTAUT model was adapted for the evaluation of the ACTIVATE’s impact on Latino agricultural worker attitudes about telehealth care service use. Below is summary of description of the framework’s dimensions, and how it was amended to meet the needs of the project.

Performance Expectancy

Performance expectancy refers to the user’s perception of how effectively using the technology will help them perform their tasks. It assesses the user’s belief that the technology will lead to improved job performance, efficiency, or effectiveness. For ACTIVATE, the definition was changed to focus on how telehealth helps Latino farmworkers in some aspect of their lives or provides benefit to that aspect of their lives.

 

Effort Expectancy

Effort expectancy pertains to the perceived ease of use of the technology. Users assess how difficult or uncomplicated it is to learn and operate the technology. For ACTIVATE, this dimension was left unchanged.

Social Influence

Involves the impact of others’ opinions and recommendations on a user’s decision to adopt a technology. It encompasses subjective norms, which are the perceived social pressures to adopt or reject the technology based on the expectations of colleagues, supervisors, or friends. For ACTIVATE, this dimension was left unchanged.

Facilitating Conditions

Facilitating conditions refer to the perceived support, resources, and infrastructure available to users for adopting and using the technology. For ACTIVATE, this dimension was left unchanged.

 

Behavioral Intention

Behavioral intention refers to the user’s intention to use the technology. It’s a direct result of the user’s evaluation of the other dimensions. If the user perceives positive performance and effort expectancies, social influence, and facilitating conditions, their intention to adopt and use the technology increases. For ACTIVATE, behavioral intention is measured and deemed the highest-level outcome within its sphere of influence of the ACTIVATE project. Research on behavioral health interventions have demonstrated that behavioral intention is highly correlated with utilization of telehealth (Venkatesh 2003, Harst, 2019). For this reason, measurement of behavior intention is considered a close proxy to telehealth utilization in the ACTIVATE project.

 

Adoption of Technology

Adoption of technology refers to users’ utilization of the technology as intended. It reflects the extent to which users translate their intentions into real-world actions. This evaluation stops short of measuring participant utilization of telehealth services, because it is deemed outside the sphere of influence of the ACTIVATE project. Meaning that the ACTIVATE inputs, products and activities summarized in Figure 1, cannot on their own influence Latino farmworkers to use telehealth services. For the project to influence utilization it would have had to ensure that Ampla Health has a sufficient availability of bilingual medical and mental health care providers, and that telehealth is adopted by providers to deliver health care services. The ACTIVATE project cannot ensure that the latter two conditions are met to meet the demand seeking of telehealth utilization among participants to occur (see Annex 1. Theoretical Framework). Thus, the evaluation measures behavioral intention as its highest-level outcome, which is most closely associated with telehealth utilization.

4.2. Attitudes Survey

To measure the success of ACTIVATE’s aim of creating favorable attitudes toward telehealth care services, we adapted an existing attitudes survey to measure the change in attitudes about telehealth acceptability. The survey included thirteen statements about telehealth health care with a corresponding five-point Likert-scale responses, where 1 is strongly disagree, 2 is disagree, 3 is neither agree nor disagree, 4 is agree and 5 is strongly agree. An additional “I don’t know” response option was provided for each item as well. The survey was administered by Ampla Health Promotoras and CRHD staff who read the instructions of the survey and statements in Spanish for respondents to answer. In total, 14 (out of 15 total workers at the farm) completed the survey before the start of the first health education session and at the end of the last session. The items were developed in English and translated into Spanish. Both Ampla Health Promotoras and CRHD staff participated in a brief orientation session on how to administer the survey tool.

The instrument was adapted from an existing survey on telehealth acceptability used in a patient population in France. The original instrument was developed using the UTAUT, a widely used theoretical framework to evaluate attitudes toward adoption of telemedicine products and interventions. (Hayotte, 2020; Apolinarrio, 2017)

The attitudes survey was pilot-tested before the start of the intervention with a sample of participants that represent demographic characteristics of the expected target population of the intervention. The results of the pilot test results are detailed in Annex 2.

4.3. Knowledge Tests

To measure knowledge gained from attending ACTIVATE health education sessions, we administered knowledge tests to participants.

The tests consisted of five questions with true or false responses and multiple-choice response options. The test was administered in Spanish to participants by Ampla Health Promotoras and CRHD staff before and after each health education session. Test question language was reviewed with the Promotoras to ensure linguistic comprehension of questions.

4.4. Structured Participant Observation

To assess participant engagement in sessions and application of promotora facilitation skills gained in training, two structured observation tools were developed and implemented by CRHD observers. Three CRHD observers completed the participant engagement observation tool at mid-point (during session 3) and end-line (during session 6). In addition, CRHD observers completed one observation per promotora. The results of the combined observations are summarized in the findings section.

The participant engagement structured observation tool required that the observer document whether participants displayed non-verbal and verbal indications that they were partaking in the health education sessions.

In specific the tool required observers to document participants’ display of active listening, verbal response to facilitator prompts, voluntary verbal contributions to discussions, asking questions, body language and distractions during health education sessions.

The promotora facilitation skills structured observation tool used a detailed rubric to determine whether Promotoras did not achieve, were in progress to achieving or achieved the facilitation skills needed to deliver health education session content. The rubric assessed Promotoras’ facilitation skills such as verbal communication, non- verbal communication, active listening, use of strategic questions and optimal use of facilitation guide.

4.5. Qualitative Exit Interviews

We completed qualitative exit interviews with project partners to complement results of surveys, knowledge tests and structured observations, and identify unintended consequences, lessons learned and best practices of the ACTIVATE project. Interviewees included Ampla Health Promotoras, Ampla Health Administrator, CRHD project manager and coordinator, and the local grower. The interviews lasted around 30 minutes and were recorded via zoom and transcribed for analysis.

The questions focused on obtaining background information on the experience of Ampla Health with the health education sessions, the perceived benefits of health education content to participants, perceived benefit of the facilitation training to Promotoras, the perceived challenges to implementing ACTIVATE, and the “most significant change” observed in participants. The interview guides also included questions to elicit feedback on what parts of the facilitator’s guide, facilitation training and intervention worked well, should stay the same or be changed in future implementations.

4.6. Community-engaged Learning

CRHD embraced a community-engaged approach to iteratively devise data collection tools, gather data, and analyze and assess the best practices and lessons derived from the ACTIVATE project. The core activities for implementing this approach involved weekly Zoom meetings with Ampla Health, a mid- point check-in with Ampla Health and the local grower, and a final end-of-project Zoom meeting with Ampla Health. These meetings offered structured opportunities to solicit feedback from the implementation partners, who, in turn, provided insights on the project’s progress and recommended changes. This approach facilitated the establishment of continuous feedback loops between CRHD and the implementing partners, enabling continuous adjustments to activities, products (such as the facilitator’s guide), and evaluation processes.

5. Limitations

The results of this evaluation are limited by challenges in the design of attitudes survey, insufficient pilot testing, and participant guessing in knowledge tests.

First, the results of the attitudes survey were constrained by the availability of existing tools tailored to measuring the project’s objective regarding attitudes toward tele-mental health.

Despite a review of the pertinent literature on tele-mental health care acceptability measurement, no instrument specifically designed for assessing this construct was discovered (Hammer, 2018; Harst, 2019).

The survey tool employed did not encompass the measurement of attitudes toward mental health care services delivered through telehealth; its scope was confined to attitudes related to telehealth acceptability. Moreover, the existing tool had not been subjected to testing within a Spanish-speaking population characterized by reduced literacy proficiency (Hayotte, 2020).

Consequently, adapting the items posed challenges due to the absence of literacy-level adaptations. This limitation further implied that a more extensive and in-depth pilot testing phase was necessary to iteratively evaluate these items among a larger sample of participants. However, resource constraints related to budget and time impeded the feasibility of such comprehensive testing prior to implementation. Thus, the scope of pilot testing was confined to think-aloud cognitive interviews conducted with five Latino farmworkers.

Second, the format of the knowledge tests questions primarily relied on questions with a true/false response format. Although

a response option for “I don’t know” was included in the tests, many participants relied on guessing when responding to post-test questions. For this reason, several

participants correctly guessed the answers to questions with a true/false response format. Thus, it is likely, that the pre-test knowledge scores are overestimated in the results.

Although it is unclear why guessing occurred, it is likely that the following may have contributed to this participant response pattern: perceived participant pressure to demonstrate knowledge or awareness of the topics and forgetting to read all response options, including “I don’t know,” during test administration by Ampla Health and CRHD staff.

6. Findings

6.1. Attitudes Surveys on Telehealth

Perceived Performance Improvement and Benefits

Statements in this dimension gauge participants' perceptions regarding the effectiveness of using telehealth to enhance various aspects of their lives or the lives of other Latino farmworkers. The results presented in Figure 2 demonstrate that three out of the four statements exhibited higher average post-test scores compared to the pre-test scores. This increase in scores means that, on average, participants more frequently expressed agreement or strong agreement with statements concerning the anticipated benefits of telehealth or its potential to enhance various aspects of their lives after attending the health education sessions.

The sole exception to this trend pertains to the responses to the statement “telehealth is helpful for the daily life of farmworkers.” It is probable that this statement, which elicited a high degree of agreement prior to the intervention, reflects participants’ initial

expectations of telehealth care services about which they lacked complete information. This incomplete information likely led participants to overstate the extent to which they agreed with the perceived benefits of telehealth for the daily lives of Latino farmworkers.

Figure 2. Attitude Survey

Results on Participants Perceived Improved Performance and Benefits

Telehealth is helpful for the daily life of agricultural workers.

Pre-test Average Scores: 4.64 / 5

Post-test Average Scores: 4.62 / 5

 

Telehealth is important to avoid missing days of work

Pre-test Average Scores: 3. 64

Post-test Average Scores: 4. 31

 

Telehealth is a replacement for in-person medical care

Pre-test Average Scores: 2.64

Post-test Average Scores: 4.15

Telehealth is helpful for obtaining laboratory or prescription medical services

Pre-test Average scores: 3.08

Post-test Average scores: 4.00

 

Perceived Effort Expectancy

In this dimension, statements measure participants' perceived ease of use of telehealth technology.

The results of the attitudes survey in Figure 3 reveal an increase in the average post-test score compared to the pre-test for all the statements. This finding indicates that, on average, a greater number of participants more frequently agreed or strongly agreed with statements about the ease of using or learning how to use telehealth after attending health education sessions. This contrasts with their initial perceptions at the start of the project, where they expressed disagreement or had neutral perceptions about the ease of using telehealth.

Figure 3. Attitude Survey Results on Participants Perceived Effort Expectancy (Pre and Post Surveys)

 

Participants Perceive Telehealth with Ease After Participating in ACTIVATE

It would be easy to learn how to use telehealth

Pre-test Score: 2.86 / 5

Post-test Score: 4.31 / 5

Telehealth simplifies attending to people’s health needs

Pre-test Score:   3.77 / 5

Post-test Score: 4.38 / 5

 

Telehealth consultations require minimal effort

Pre-test Score: 3.29 / 5

Post-test Score: 4.31 / 5

 

 

Perceived Social Influence

Statements pertaining to social influence encompass the impact of others' opinions and recommendations on participants' decisions to adopt telehealth. The results of the attitude survey in Figure 4 exhibit minimal to no change in participants' perceptions regarding the influence of social factors on their utilization of telehealth services. This outcome could be attributed to challenges associated with understanding the statements included in the survey. During the pilot testing of the original UTAUT-based statements, participants encountered difficulties in grasping the intended meanings of the statements. To address this concern, we introduced the clauses "if my family knew about telehealth" or "if my friends knew about telehealth."

Another potential explanation for the high rate of agreement among participants could be rooted in their initial over-optimistic expectations of telehealth services. These expectations likely emerged from a lack of comprehensive information about telehealth care services, leading participants to express agreement with statements that seemed positive but were not grounded in complete understanding.

Figure 4. Attitude Survey Results on Participants Perceived Social Influence  (Pre and Post Surveys)

 

If my friends knew about telehealth, they would believe that I should use it.

Pre-test Score: 4.07 / 5

Post-test Score: 4.00 / 5

 

If my family knew about telehealth, they would recommend I use it.

Pre-test Score:  4.08 / 5

Post-test Score:  4.15 / 5

 

Facilitating Conditions and Personal Resources

 

Statements in this dimension represent the perceived support, personal and external resources, and infrastructure available to them for adopting and using telehealth care use.

The results of the attitude survey in Figure 5 show an increase in participant average post-test scores relative to pre-test scores.

This finding tells us that on average, a greater number of participants more frequently agreed or strongly agreed with statements about perceived facilitating conditions and personal resources after receiving health education sessions. Even more noteworthy is the pronounced increase in perceived personal resources, such as “knowledge necessary to use telehealth,” which increased from a strong disagreement (1.31) to an agreement (4.38).

Behavioral Intention

Behavioral intention refers to participants’ willingness to utilize telehealth care services. It is a direct outcome of the user’s assessment of the various dimensions. If users perceive positive performance, benefits, and effort expectancies, along with favorable social influence and conducive conditions and resources, their inclination to adopt and employ the technology increases.

The findings from the attitudes survey in Figure 6 shows an increase in the average post-test scores when compared to the pre- test scores. This observation indicates that, on average, a significant majority of participants more consistently expressed agreement with the intention to regularly adopt telehealth for medical visits following the completion of the health education sessions. This stands in contrast to their initial perspectives at the project’s outset, during which they exhibited uncertainty regarding their intention to avail telehealth care services for medical visits.

Figure 5. Attitude Survey Results on Participants Perceived Facilitating Conditions

Telehealth is similar to other technology I use

Pre-test score: 3.21 /5

Post-test score: 4.05 / 5

 

I have the knowledge necessary to use telehealth

Pre-test score 1.31 /5

Post-test score: 4.38 / 5

 

 

I have the technology necessary to use telehealth

Pre-test score: 3.08 /5

Post-test score: 4.46 / 5

 

 

Figure 6. Attitude Survey Results on Participants Perceived Behavioral Intention

Pre-test score:  3.54 /5

Post-test score:: 4.31/ 5

 

6.2 Knowledge Tests and Sessions Attendance

The results obtained from the knowledge tests administered during each health education session reveal that participants achieved an enhanced level of knowledge following each session. As illustrated in Table 2, the percentage change in the average knowledge test scores recorded in all session modules exhibited an increase of at least 10% subsequent to their participation in the health education sessions. It’s worth noting the relatively high average pre-test scores after the initial session, with participants achieving correct responses on 3 out of 5 questions.

This pattern likely emerged because participants correctly guessed the answers to certain knowledge test questions rather than selecting “I don’t know.”

 

Furthermore, it is particularly interesting to observe that the highest average post- test scores were attained in modules characterized by a relatively lower density

of information. In contrast, the lowest post- test score was recorded in module 6, which contained a significant amount of information. Module 6 of the workshops is related to services offered by the FQHC partner, with an emphasis on the FQHC’s offerings related to programs for uninsured and other hardly reached community members, such as sliding fees for services, transportation services and telehealth programs.

Attendance records indicate that participant interest in the training sessions grew over time. Voluntary attendance rose from just 11 participants (out of a total of 15 employees) during the initial health education session to a maximum of 15 participants (out of a total of 15 daytime employees) by the project’s conclusion.

Table 2. Participant Attendance and Knowledge Test Scores

Module 1

Attendance: 12

Pre-test Average 1.83

Post-test Average: 2.92

% Change Average: 22%

 

Module 2

Attendance: 10

Pre-test Average 3.3

Post-test Average: 4.4

% Change Average: 22%

 

Module 3

Attendance: 14

Pre-test Average 3

Post-test Average: 3.54

% Change Average 11%

 

Module 4

Attendance: 13

Pre-test Average: 3.08

Post-test Average 4.00

% Change Average: 18%

 

Module 5

Attendance: 14

Pre-test Average 3.57

Post-test Average 4.36

% Change Average 16%

 

Module 6

Attendance: 15

Pre-test Average 2.53

Post-test Average: 3.27

% Change Average 15%

Participant Engagement

Attendance Rates and Knowledge both Increased.

6.3 Structured Observations of Participant Engagement and Promtora Facilitation Skills

Participant Engagement

Results from the two structured observations completed at mid- point and at the end of the project exhibited the change of level of participant involvement in the health education sessions.

In session three, an observation indicated a moderate level of participant engagement with the session’s content. Although slightly over half of the participants displayed signs of active listening (e.g., maintaining eye contact, nodding), they did not engage on a personal level in the discussion. This lack of involvement was evident as participants were hesitant to voluntarily contribute verbally by sharing their experiences or posing clarifying questions to the Promotoras.

In contrast, during session six, participants displayed a high level of engagement in the health education session. The majority of participants exhibited signs of active listening, demonstrated a willingness to share personal insights and experiences, and actively posed questions to obtain information for addressing their family’s health insurance needs.

Promotora Facilitation

Results from the four structured observations of the five Promotoras, exhibit the varying level of facilitation skills utilized during the health education sessions.

Promotora 1 displayed facilitation skills that were in the progress of meeting the desired level of facilitation skills. She exhibited moderate verbal communication fluency and clarity; however, there were deficits in her non-verbal communication skills. Her reliance on reading from the facilitator’s guide led to occasional presentation errors, but she adeptly employed open-ended questions to encourage participant involvement.

Promotora 2 showcased facilitation skills that were nearly at the desired level to deliver health education sessions. She demonstrated strong verbal and non-verbal communication abilities in Spanish. She effectively utilized open-ended questions to stimulate participant interaction, allowing them to express opinions. Although she refrained from passing judgment while participants shared their ideas, she did tend to read from the guide instead of conveying content in her own words.

Promotora 3 approached the desired level of facilitation skills required for delivering health education sessions. She exhibited strong verbal and non-verbal communication aptitude engaging participants through open-ended questions and active discussion. However, she relied on reading directly from the facilitator’s guide rather than presenting the content in her unique style.

Promotora 4 nearly achieved the desired level of facilitation skills needed for health education sessions. She showcased exceptional verbal and non-verbal communication proficiency, allowing participants to pose questions and providing detailed responses without relying on the guide. Although she predominantly used close-ended questions, she effectively fostered discussion. She tended to read from the guide but maintained the ability to respond to participant inquiries with comprehensive explanations.

Promotora 5 displayed facilitation skills in progress toward achieving the desired level for health education session delivery. While she exhibited moderate verbal communication fluency in Spanish, there were occasional errors in her presentation. Her non-verbal communication skills were strong. She primarily relied on the facilitator’s guide, with occasional presentation errors, and employed some open-ended questions to stimulate participation.

6.4 Impacts of ACTIVATE

The following are results of the exit interviews with Ampla Health Promotoras, Ampla Health Administrators and CRHD project managers. Synthesis of partners’ perceived impacts of ACTIVATE are organized by component of the ACTIVATE project, including its two primary products, the facilitator’s guide and facilitation training to Promotoras, partnership and supervision of the project. Partner quotes are included to illustrate key points of the synthesized results.

6.4.1 Facilitator’s Guide

ACTIVATE partners spoke highly of the quality of facilitator’s guide content and made recommendations for improvements for future iterations of the guide. The facilitator’s guide’s written content was used by Promotoras to deliver the health education sessions over the six weeks of the intervention. Most partners reported that all the content in the facilitator’s guide was well received by project participants. In addition, several partners emphasized that participant interest in the facilitator’s guide content increased over time.

Building interest in the facilitation content was demonstrated by participant reticence during the first module on telehealth and active participation and sharing personal experiences by the last module, on Ampla Health services. When partners were asked what content was most beneficial to participants, both Ampla Health and the grower partners, concurred that the module on mental health provided them provided the most benefits to project participants. Partners mentioned that being aware of differences between mental health care providers and types of mental health problems demystified misperceptions about mental health care, which contributed to them opening up and talking about mental health problems in their community.

What Content of the Facilitator’s Guide was Most Beneficial to the Participants?

“Especially module number three. When we did the mock appointment in front of them for them to see what we were going to be doing to make them feel a little bit more comfortable to do a telehealth visit. And I thought that worked really well.”

—Ampla Health Promotora

ACTIVATE Partners were also asked what content should stay the same or change in the facilitator’s guide. They mentioned that the guide should continue to use questions to facilitate discussion of key themes, role- play to simulate the experience of having a telehealth visit, stories and examples to illustrate the health education messaging. In addition, they mentioned that vocabulary used in the guide should be reviewed and potentially revised to make it even simpler for participants to understand.

This recommendation is a response to Promotora reactions to the translated facilitation guide, which had not been revised an additional time to adapt the Spanish vocabulary to the participant’s literacy level. In addition, Promotoras mentioned the need to use conversational Spanish in the facilitator’s guide. Some partners also mentioned the perceived need to make the stories shorter to improve comprehension by participants. In addition, one partner mentioned the need to include visual aids to help present and communicate facilitator’s guide content during health education sessions.

What about the Facilitator’s Guide Content Should Stay the Same?

“[Modules] one, three and six. Definitely, because I felt that going through the whole [roleplay of] calling to schedule the [telehealth] appointment. That participants were not as open to [telehealth], I felt that [roleplay] made them feel a little more comfortable and understand that it’s not as hard as it may seem

or sound. So, I think that that should stay the same. Telehealth will work and I think it’s good to have that option for them to continue doing that.”

—Ampla Health Promotora

“The actual module, we made it to be relatable. Like we use those stories and made it so that it applies to everybody. We’re not just telling them a story about how my daughter needed urgent medical attention. I like the fact that we talked about ourselves, too in it. It makes [the facilitator’s guide] more personable and easier to relate when you’re presenting.”

—Ampla Health Promotora

“I think the questions, asking [reflection] questions to keep the individuals participating, to keep them focused, to keep their attention. I think the questions [in the facilitator’s guide] are very good.”

—Ampla Health Promotora

What About the Facilitator’s Guide Content Should Change or be Added?

“That story … they said if that it was a different type of farmworker because they travel and they work and collect their money and then send it to their family.So that’s why. And then it kind of didn’t relate to thembecause they are staying in one location and work. Somaybe changing that story up.”

—Ampla Health Promotora

“One of the things that would have been good would be to have the finalized [facilitator’s guide] on hand before the presentations, just to have an opportunity to go over the material before we got in there into the sessions.”

—Ampla Health Promotora

“Some of the wording that I feel some of the farmworkers would be more comfortable with, and maybe easier to understand and receive information. I feel that we did do some small changes [to facilitator’s guide vocabulary] and I think that it was beneficial to, to the quality of the guide.”

—Ampla Health Promotora

6.4.2 ACTIVATE partners reported the positive impact of the facilitation training to support the Promotoras’ delivery of health education sessions. The training curriculum consisted of two in-person trainings of three hours before the start of the first health education session and another training session before the fourth health education session. The training curriculum was related to public speaking and small group facilitation skills. Ampla Health partners reported that Promotoras receive a similar type of training at Ampla Health as part of their job as a Promotora. Most Promotoras reported having experience facilitating similar types of health education sessions at Ampla Health or in their past jobs.

When asked how the facilitation training supported Promotoras’ delivery of the health education sessions, partners reported the trainings were essential for preparing the session content. Several Ampla Health partners emphasized that that content of public speaking techniques and small group facilitation was very helpful delivering the health education sessions. In addition, Ampla Health partners mentioned that the in-person feedback and practice presenting session content was paramount for development of familiarity and confidence for delivering the session content successfully.

What aspects of the Facilitation Trainings were Most Useful in Delivering the Health Education Sessions?

“I think it helped a lot just to introduce us to how to be a Promotora and the language that’s used. I think it did help a lot and it also helped me not feel nervous…because everything’s new to us, whether I’ve done it before, once I think all of this is just a learning experience for us.”

-Ampla Health Promotora

“Just the positive feedback. I think it’s important to have criticism because you can learn new things and that you can use. So, I think I learned a lot of things with you guys as input for speaking in public.”

—Ampla Health Promotora

Partners were also asked what they recommend change in future trainings. They mentioned that more time for in-person training before and during the implementation is needed for preparing Promotoras to deliver the health education sessions. Promotoras mentioned this additional training would help them fulfill their role as health promoters. Others mentioned this training would give them enough time to rehearse and learn the facilitation guide content. A couple of the less experienced Promotoras mentioned the added training would have helped them deliver the health education sessions more confidently. In addition, a couple of Ampla Health partners discussed the importance of providing session content that was fully revised and edited with enough time prior to delivering session content.

What Aspects of the Facilitation Training Would You Recommend Changing to Support Promotoras Delivering Health Education Sessions?

“I think it would have been beneficial to break it up into three sessions just to give everyone more time and an opportunity to go over their stuff a little more.”

—Ampla Health Promotora

“I felt that not all the Promotoras had the opportunity to practice. So maybe a longer training. Maybe for the mental health part, which was a little bit longer [module], maybe do longer training so that way every Promotora have the chance to practice and train and feel good about presenting.”

—Ampla Health Promotora

“A lot of the training was like 80% of us presenting to them what the skills they needed and 20% them applying them as we were training them. I think in that period of time, which was like I want to say four hours, that the trainings lasted, I think that if we had, more time of making this training longer, make it a day long or, maybe a week long, but break it down into the chunks, if we had five topics and one topic each day”

—Ampla Health Promotora

6.4.3 Partnership, Supervision, and Incentives

ACTIVATE partners provided feedback on the intervention overall, including comments on the aspects of the ACTIVATE project

that worked well, including the partnership between CRHD and Ampla Health and the local grower, research, and the incentives. They reported overwhelmingly positive experience with the partnerships.

The local grower praised CRHD and Ampla Health’s professionalism from the design and implementation of the intervention.

The grower also reported appreciation for the clear and frequent communication about the project with CRHD, which helped establish trust between them.

Ampla Health spoke highly of the partnership with CRHD that produced benefits through the project life cycle. Ampla Health partners expressed appreciation for being involved in the needs assessment phase of the project to better understand the needs of the Latino agricultural worker population. They praised CRHD’s management and participatory approach to the development of the facilitator’s guide content. They also expressed satisfaction with the comprehensive approach to evaluation, that Ampla Health did not have the capacity to assume themselves. They also recognized the important role that incentives had in encouraging participant attendance early on in the intervention.

“The partnership with Ampla. I think that was key, the way we approached that. We built trust with the Promotoras because I feel like our team understood that to get to the farmworkers in a community directly, Promotoras are the access points. So, I think we spent a lot of quality time to involve them in every step of the process. In building trust with them.”

“I think incentives helped. They knew that if they came to the module, they would get a little treat. At first it was about the incentive. And then maybe after they [realized], hey, this stuff is beneficial, this stuff is good to know. And then they happy with learning about it and, sharing it with their coworkers.”

—Ampla Health Promotora

6.4.4. Most Significant Change in Participants

Exit interviews also included a question on the most significant change observed in participants after participating in the project. Ampla Health partners expressed that engagement with session content was the most significant change observed in participants. They spoke about participants initial interest in session content may have likely been spurred by financial and promotional material incentives provided by ACTIVATE for attending each session. However, over time participants became more open to engage with Promotoras during sessions and openly talked about telehealth, mental health and health care coverage needs. Partners mentioned that by mid-point of the project, participants spoke comfortably about personal experiences with mental health issues and suicide in their community. This openness was also noticed in their behavior with each other in workshops, where they responded to co-worker comments and openly joked amongst themselves. Ampla Health’s perceived openness and communication was corroborated by the local grower as well.

What aspects of the Project Worked Really Well?

“Just their attitudes. And I feel that the very first session, I saw how they were more laid back, not really answering questions, just listening to us. And then the second one that I did, they were more talkative and the very last one they were more comfortable sharing. So I think they opened up.”

—Ampla Health Promotora

“Their feedback. Their participation. They were pretty open, interactive with receiving the information and giving us their opinion on what they, you know, how they felt about mental health, mental health. And I felt that they were pretty open to what it was received very well and they all participated.

—Ampla Health Promotora

The local grower mentioned that participants’ increased cohesiveness was the most significant change observed after the ACTIVATE project. The grower expressed surprise to see participants communicating more openly with peers. In addition, he pointed out that his workers were more willing to speak to other workers they did not communicate well with before the start of the project. The grower compared the result of the workshop to corporate teambuilding exercises, that intend to develop well-functioning teams at work. The grower spoke highly of the potential benefits of this result, which includes being more open to asking questions and problem solving amongst them.

What was the Most Significant Change you Observed from Participants after the ACTIVATE Project?

“One sounds kind of weird, but it might be that there’s a little more cohesiveness amongst the group. Um, you know, because they all, not all of them have known each other for a long time. And so,

again, this kind of this might just be the act of sitting in any workshop together, right? Like, like, like it’s almost like a team function, right? ... Obviously it’s only going to help, right? I mean, the more comfortable they are around one another, the more likely they’re going to be able to ask, you know, a coworker a question when maybe before they were apprehensive to because they didn’t know him that well or they

weren’t that comfortable around him.”

—Grower

7. LESSONS LEARNED

The lessons learned are knowledge or understanding gained by experience. The experience may be positive, as in a successful strategy or activity, or negative, as in a mishap or failure. They were drawn from exit interviews with Ampla Health and local grower, community-engaged learning meetings with Ampla Health and internal record keeping by CRHD throughout the life of the ACTIVATE project.

The lessons learned include:

On developing the Health Education Curriculum Content:

• Soliciting input from all project partners, including the grower, regarding curriculum topics promoted buy-in and satisfaction with the project.
• Involving the Promotoras in the curriculum design and revision process led to buy- in and the creation of a higher-quality facilitator’s guide. Tailoring the content to suit the needs of local farmworkers is essential.
• Reviewing and revising the professionally translated curriculum demanded more effort than anticipated. Simply reviewing the translation with our team was insufficient. Allocating time to review the entire Spanish translation with the promotora team, who understand the nuances of the local language, was necessary. This last revision of the facilitator’s guide should focus on adapting language to the literacy proficiency of participants and to use conversational Spanish vocabulary as much as possible.

On health education content and strategies that worked:

• Roleplay and simulation of telehealth visit between participants and Promotoras. This demystified the process of completing a telehealth visit and gave Promotoras the confidence to do it in the future.
• Integration of stories and examples in facilitator’s guide made telehealth and mental health seeking relatable and not scary.

Considerations for Future Revisions or Additional Content to the Curriculum:

• Incorporating new modules or making significant changes to the curriculum requires a team member with a background in curriculum design.
• Including a session that delves into the factors contributing to mental health issues, in addition to covering types of services, proves beneficial for Latino farmworkers. During the delivery of mental health modules, several participants requested more information about how mental health problems arise in Latino farmworkers. This new session would address this information to need.
• Consider splitting module 6 into two separate sessions. Perhaps dividing insurance/Medi-Cal into one session and Ampla services into another. Splitting the content into two sessions would allow more time to answer participant questions about eligibility for types of insurance providers or programs. In addition, two sessions would allow Promotoras to provide more stories and examples of how these services may be accessed by Latino farmworkers.
• Incorporating visual aids for presenting information, such as infographics or images of a farmworker on a telehealth call. Providing participants with visual aids enhances engagement and comprehension. A poster with essential requirements for Medi-Cal could be useful to employ visual learning strategies.

On training for Promotoras:

• Allowing more time for Promotoras to practice workshop content and review materials in advance. Project managers must ensure that materials are thoroughly edited with ample time for Promotoras to review and provide feedback on facilitator’s guide content. This time should be used by Promotoras to rehearse presenting the health education session. In addition, this additional time can be used by Promotoras to ask questions about the meaning or intention of facilitator’s guide content, to make any revisions if deemed necessary.
• Equipping Promotoras with strategies for facilitating discussions when addressing these sensitive topics. This type of training may be necessary to address participant sharing personal mental health

experiences, such as a loved one’s suicide, during training.

• Enhancing training on effectively asking open-ended questions related to mental health,

 

What Aspects of the Project were Challenging or Difficult?

“At the beginning, they were kind of quiet. Um, and I think there were some people that were hesitant to participate. Um, so I think those were some of the difficult things.”

—Ampla Health Promotora

 

On Community Engagement:

Involving a grower in a project like this requires personal connections within the agricultural community. Many growers may be unwilling to participate due to concerns about the project’s impact on workers’ time and the underlying intentions. Timing of the health education workshops to suit the preferences of all implementing organizations is crucial. Conducting sessions during the workday, rather than after, ensures consistent participant attendance.

Collaborating with the local grower to ensure workshops do not disrupt farm operations is important. The timing of the sessions should also align with periods when workers are present at the farm. Ideally, this would avoid interfering with harvest while capitalizing on the workers’ presence.

What Aspects of the Project were Challenging or Difficult?

“So, for sure, the difficult part of it was there were guys in between tasks and then they were stopping to go to the meeting and the task wasn’t going to be done on time.”

—Local Grower

What would you Change or do Differently About Facilitation Training?

“I don’t think I would do anything different. The only thing that I think could have come up would have been maybe they might have had competing priorities at the time. So, it was a little bit of a struggle to make sure they all could attend, but that was just based on their individual workload.”

—Local Grower

8. BEST PRACTICES

Best practices are examples of processes or activities that yield success or at least meaningful results. Very often, best practices are seen as an innovation in comparison to what was done before.

During the ACTIVATE project, the following best practices were identified:

Engaged Promotoras throughout the process of designing the curriculum. Promotora input is crucial for both content development and ensuring that the curriculum is culturally and linguistically appropriate. The curriculum must be co- designed by Promotoras who, as members of the local community, have a deep understanding of the language and culture of the target audience.

Provided training to Promotoras in public speaking and workshop facilitation. Facilitating ACTIVATE workshops requires a skillset that a health promotion worker/ Promotora doesn’t necessarily require. Promotoras may have levels of prior training and experience, and their experience may or may not be relevant to delivering content intended to influence the health behaviors of participants.

• Developed trust with workshop participants before the workshops begin by meeting informally. If possible, hold a lunch with the grower and workers. Meet the participants and share a meal together, so you’re familiar to them when the workshops begin.
• Inclusion of telehealth simulation and role-playing activity in the workshops. Practicing a telehealth call, in addition to learning about the benefits and advantages of using telehealth, is crucial for buy-in by workshop participants.
• Provided incentives for workshop attendance. Participants may attend at first only to receive incentives but will develop interest in the content over time.
• Built and maintained trust with all project partners. Demonstrate professionalism. Communicate frequently. Show that you value your partners’ input by listening and making changes in response.

What Aspects of the Project Worked Really Well?

“I think the transparency was, was nice. The communication, you know, whether it was at the shop or via email. Um, I think that that’s, that’s important to me. Right. And I think it would be important to anyone else going forward. Thought you guys did a really good job at that… And then, you know, sticking to the schedule...and being on time.”

—Local Grower

 

“We did take the time to make adjustments during the trainings for each module. And so whatever questions we had or verify if we were receptive to the content, we talked about it and agreed to modifying certain things in the facilitator’s guide.”

—Ampla Health Promotora

 

“I feel like the health education sessions made the participants feel at ease because the module content wasn’t scary, like how we talked about mental health. It was more comforting.”

—Ampla Health Promotora

 

Especially module number three. When we did the mock appointment in front of them for them to see what we were going to be doing to make them feel a little bit more comfortable to do telehealth visit. And I thought that worked really well.”

—Ampla Health Promotora

 

“I remember the very first meeting, I think there was some gift cards given out and that that seemed like it was… talked about a lot. I was hearing on the side, we got these gift cards and they were stoked about it. I think they were surprised, right? Pleasantly surprised that they got it.”

—Local Grower

 

What Aspects of the Project Worked Really Well?

“I believe this project really helped us make it a priority to do [health education] workshops. When I was hired at Ampla Health we had one long term promotora that would visit the farmworker sites. She retired not long after that. The Board of Directors really wanted us to get back to something similar with a greater impact so that resulted in us hiring three more [promotoras]. We were literally just beginning to plan out what we were going to do. We went out to a couple farms and shared information, took our purple bags with written material, but it wasn’t anything hugely structured like what you proposed. We hadn’t done a significant amount of outreach to the farmworker population for some time prior to this program. Then your project came along.

—Ampla Health Administrator

 

“I would say the support that you gave us. I have to be honest at first I thought I was going to be designing the entire training manual and I was concerned. Then when I saw how much you were going to assist, I was relieved! I knew that I would not have been able to do all of what was required. You gave us a tremendous amount of support to get the final product documented and printed. A lot of the time that task tends to end up becoming the responsibility of the partner and that’s too much for us to take on. We’re not skilled in preparing comprehensive manuals and it’s too

time consuming for us so we were pleased to learn that you were taking that portion on. I was very appreciative of the opportunity to provide feedback.

I do believe that our feedback enhanced the final product. Lastly, we would not have been able to conduct this work without your support and guidance.”

—Ampla Health Administrator

9. CONCLUSIONS

The results of this mixed-methods evaluation indicate that the ACTIVATE pilot project successfully achieved its objectives. Triangulation of information from multiple data sources, including, quantitative surveys, knowledge tests, structured observations, and qualitative interviews, support the following conclusions about ACTIVATE:

Health education sessions promotores favorable attitudes toward telehealth service use among project participants. This finding is supported by the results of the attitudes survey, exit interviews, structured participant observation, and knowledge tests. The results of the attitude survey demonstrate that most participants, on average, expressed agreement with the intention to regularly seek medical care through telehealth technology after completing the health education sessions. Qualitative reports from exit interviews confirm this finding; implementing partners lauded participants’ noticeable change in openness and disposition toward telehealth care services. Moreover, evidence from structured participant observation shows that participants exhibited a high degree of interest and engagement in the health topics. In addition, increase in knowledge obtained from health education sessions further exhibits that participants obtained the informational and experiential tools to support the decision to use telehealth in the future.

Health education sessions created awareness about telehealth and mental health care service availability and use amond project participants. The results of the the knowledge tests show that, on average, participants’ level of knowledge increased 10- 22% each session after completing the health education sessions. Results from the exit interviews confirm the finding that participants learned information that was beneficial to their health care decision-making. In addition, this finding is corroborated by documentation of strong participant engagement during health education sessions.

The Facilitator’s Guide was well received and beneficial for participants and Ampla Health. Reports from exit interviews show that the collaborative development of facilitator’s guide was crucial for delivering health education sessions. The co-development approach employed to design, revise translated content, pilot, and improve the guide’s content throughout implementation produced a better-quality education curriculum embraced by implementing partners and participants alike.

The facilitation training supported the Promotoras’ delivery of health education sessions through developing and practicing presentation skills. Implementing partner feedback from exit interviews denotes the positive impact the trainings had on Promotoras. Partners found it helpful to learn and improve presentation skills, practice session content, and overcome the nervousness of presenting to participants.

ACTIVATE’s financial and promotional incentives were beneficial to motivating participants to attend health education sessions. All ACTIVATE partners reported that project incentives were well received by participants and likely motivated attendance to health education sessions, especially at the beginning of the project’s implementation. It is unclear from the results of the exit interviews whether financial (i.e., gift cards) or promotional gifts (i.e., branded water bottles and personal protective equipment) were more important for encouraging attendance.

The Key Lessons Learned from the Project Include:

Developing the Health Education Curriculum Content:

• Seeking input from all project partners, including the grower, enhances project buy- in and satisfaction.
• Incorporating new curriculum modules or significant changes necessitates a team member with curriculum design expertise, which some FQHCs might lack.
• Engaging Promotoras in curriculum design leads to buy-in and yields a higher-quality facilitator’s guide. Adapting content to cater to local farmworkers’ needs is crucial.
• The process of reviewing and revising the professionally translated curriculum demands more effort than initially expected. A comprehensive review involving the promotora team familiar with local language nuances is essential.

Effective Health Education Content and Strategies:

• Roleplay and simulation of telehealth visits between participants and Promotoras

help demystify the telehealth process, boosting participants’ confidence for future interactions.

• Utilizing stories and examples in the facilitator’s guide makes telehealth and mental health concepts relatable and less intimidating.

Considerations for Future Curriculum Revisions:

• Include a session on factors contributing to mental health issues proves beneficial for Latino farmworkers.
• Potentially divide module 6 into two sessions, addressing insurance/Medi-Cal in one and Ampla Health services in the other.
• Incorporate visual aids like infographics or images to enhance engagement and comprehension.

Promotora Training:

• Allow more time for Promotoras to practice and review workshop content.
• Equip Promotoras with strategies to navigate sensitive discussions around personal mental health experiences shared by participants.
• Enhance training on asking effective open- ended questions related to mental health.

Community Engagement:

• Involving a grower necessitates deep personal connections within the agricultural community due to concerns about project impact and intentions.
• Time the implementation of health education workshops to suit all partners’ preferences is crucial for consistent attendance and minimal interference with farm operations.

Furthermore, several best practices emerged from the ACTIVATE project:

• Engage Promotoras in curriculum design ensures cultural and linguistic appropriateness.
• Provide training to Promotoras in public speaking and workshop facilitation.
• Establish trust with participants through informal meetings before workshops begin.
• Include telehealth simulation and role- playing to promote participant buy-in.
• Offer incentives for workshop attendance to generate initial interest.
• Maintain professionalism and effective communication to build and sustain trust with all project partners.

REFERENCES

1. Apolinário-Hagen, Kemper, J., & Stürmer, C. (2017). Public Acceptability of E-Mental Health Treatment Services for Psychological Problems: A Scoping Review. JMIR Mental Health, 4(2), e10–e10. https://doi.org/10.2196/mental.6186
2. Glanz, Rimer, K., & Viswanath, K. (2015). Health behavior : theory, research, and practice (Glanz, B. K. Rimer, & K. (Kasisomayajula) Viswanath, Eds.; Fifth edition.). Jossey-Bass, a Wiley Brand.
3. Hammer, Parent, M. C., & Spiker, D. A. (2018). Mental Help Seeking Attitudes Scale (MHSAS): Development, Reliability, Validity, and Comparison With the ATSPPH-SF and IASMHS-PO. Journal of Counseling Psychology, 65(1), 74–85.
4. Harst, L., Lantzsch, H., & Scheibe, M. (2019). Theories Predicting End-User Acceptance of Telemedicine Use: Systematic Journal of medical Internet research, 21(5), e13117. https://doi.org/10.2196/13117
5. Hayotte, M., Thérouanne, P., Gray, L., Corrion, K., & d’Arripe-Longueville, F. (2020). The French eHealth Acceptability Scale Using the Unified Theory of Acceptance and Use of Technology 2 Model: Instrument Validation Study. Journal of medical Internet research, 22(4), e16520. https://doi.org/10.2196/16520 https://doi.org/10.1037/cou0000248
6. Venkatesh, Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425–478. https://doi. org/10.2307/30036540