Symposium Schedule

The North Carolina Divisions of Public Health and Social Services and regional and local foundations are investing in the widespread scale-up of the Triple P – Positive Parenting Program, an evidence-based system of parenting and family support. Key stakeholders want to know whether community capacity for successful program implementation and optimization is being put in place to increase the likelihood of sustainably achieving targeted population-level outcomes, including reduced child abuse and neglect, out-of-home placements, and child maltreatment injuries. Investigators from the North Carolina Implementation Capacity for Triple P (NCIC-TP) project conducted a two-year evaluation of implementation capacity and outcomes across cross-sector community coalitions in two counties to inform the planning process for impact and sustainability. Following the evaluation, the NCIC-TP team developed an implementation support plan for North Carolina counties scaling Triple P, including an implementation logic model and core practice components for external implementation support that are both grounded in the implementation science literature. The team is currently demonstrating enhanced implementation support in two regions of the state and prototyping quality and outcome measurement processes. The NCIC-TP team has also worked with statewide stakeholders to identify a potential intermediary organization for ongoing Triple P implementation support. Key evaluation and technical assistance findings, models, and lessons from this project will be briefly shared within the lightning format.

Public health and community-based practitioners increasingly are being asked to adopt, adapt, and implement evidence-based interventions (EBIs) to improve population health. Despite access to a growing menu of EBIs, practitioners underuse them, because EBI adoption and implementation require knowledge, skills, and resources that they may not currently have.
Numerous foundations, universities, governmental agencies, and consultants provide trainings to address the gaps in practitioners’ capacity. Many trainings address similar objectives related to assessment, goal setting, and how to find, adapt, implement, and evaluate EBIs. Despite the number of organizations offering trainings, little is known about how to scale-up trainings related to EBIs to efficiently extend their reach to those with limited access. The national Cancer Prevention and Control Research Network, part of the Prevention Research Centers, developed a training curriculum on evidence-based public health practice, “Putting Public Health Evidence in Action”. A training collaborative in North Carolina and Oregon delivered it in both in-person, distance, and blended formats to local public health and community-based practitioners. This session will detail the specific approaches used to extend the reach of this curriculum using distance learning modalities and share quantitative and qualitative evaluation findings from surveys of 252 participants in eight trainings conducted in North Carolina and Oregon. Overall, comments about the training were positive and evaluation findings suggest that face-to-face is an important delivery method for fostering interaction with peers and instructors. Recommendations will be provided for using multi-modality approaches to efficiently extend a training’s reach while also increasing participants’ competency and practice.

The past decade has seen increasing concern about what is known as the “science to service gap.” As public schools, service organizations and other large systems of care have spent considerable sums on promising innovations it has been documented that the effectiveness of interventions in the lab where they were developed is not often replicated when they are applied to actual service settings. A central issue is that programs are tested in ideal settings with the support of the intervention’s developers. If a program is transferred from the university to the service setting without proper professional development and other implementation structures in place the innovation is pre-destined to be less effective - with at best a 5% transfer of new skills and practices to the work environment. The application of evidence-based “competency drivers” during implementation can increase effective adoption of an innovation to 95%. Implementation coaching is the core competency driver. This presentation describes a coaching framework specifically developed for use in innovation implementation. The framework is based upon research from both implementation science and systems-level consultee-centered consultation. Critical components of the framework include working alliance, situated professional development, innovation configuration, systems change, time/structure/accountability, and management of role change. At the organizational level, implementation coaching helps drive the systemic alignment and facilitative administration needed to champion an innovation through supportive policies and application of adequate resources. For the end-users, coaching provides a framework and process that allows them to acquire the knowledge and skills needed to effectively implement an innovation.

There is growing recognition that considering organizational readiness is important for successful and effective implementation of an innovation. Readiness refers to the extent to which an organization is both willing and able to implement a particular innovation. It is increasingly understood that readiness changes over time and should therefore be monitored across the lifespan of an innovation. A readiness monitoring tool (RMT) based on R=MC2 has been developed to evaluate the areas of strength and weakness in an organization’s readiness over time in order to help inform and tailor training, technical assistance, and quality improvement/quality assurance strategies to foster successful and effective implementation. To date, the RMT has been primarily utilized within coalition and healthcare settings in United States. However, there are a diversity of contexts in which innovations are being implemented, and different contexts may have different needs, both in terms of evaluating and supporting readiness. We will share findings from the work regarding the adaptation and simplification of R= MC2 readiness tool, to assess the readiness of the health care facilities in four pilot districts of Himachal Pradesh, India to implement quality improvement interventions. The presentation will highlight the cross-cultural adaptations made to the readiness tool, how the evaluators administered the tool at three administrative levels (state, district and facility), gathered readiness data from four pilot districts and multiple health care facilities and the recommendations to the Government of Himachal Pradesh, India on the change management strategies that must be adopted to sustain quality improvement programs over time.

Background: Use of simulation is becoming a mandatory aspect of medical training. The goal of simulation is to have medical trainees, especially those in procedural fields such as surgery or gastroenterology, gain familiarity with procedural equipment and technical aspects of the procedure prior to starting their clinical training. However, there is very little data, especially in terms of endoscopy training that supports that simulation improves trainee comfort. Aim: The primary aim is to determine if an endoscopic educational intervention, including endoscopic simulation, improves trainee comfort level and expectations of endoscopic training. Methods: Participants will be gastrointestinal (GI) fellowship trainees who are starting their first year of fellowship at UNC. All trainees who meet the inclusion criteria and who are willing to participate will undergo an educational, endoscopy-focused intervention, prior to starting endoscopic training. The educational intervention will use a simulation model with a prosthetic model stomach that will allow trainees to perform an upper endoscopy on the model with simulated activities such as retrieval of foreign objects. In addition, prior to the simulation activity, orientation to the endoscopy suite, devices, and instructions on how to use the endoscopes will be provided. A pre and post-test will be conducted measuring trainee comfort level and expectations regarding endoscopy education and training. The test will consist of questions regarding comfort level and expectations regarding performing endoscopy prior to their scheduled clinical endoscopy rotations. Analysis will be performed to determine correlation between improved comfort and different interventions of the educational orientation, including simulation.

The Consortium for Implementation Science is a joint endeavor of the UNC Gillings School of Global Public Health and RTI International. The mission of the Consortium is to advance implementation science through research, practice, policy, and education. To achieve this mission, we identify funding opportunities and build collaborative teams to conduct implementation research, translate research into practice, and support evidence-based policy. The Consortium also creates didactic and experiential learning opportunities in implementation science for students and researchers. For example, the Consortium supports the Implementation Science Student Group at UNC-Chapel Hill and sponsors lectures focusing on implementation science and practice. During the past year the Consortium for Implementation Science launched Implementation Science News, which includes editorials from members of the Consortium team, and information about implementation-related publications, funding announcements, training opportunities, job postings, and other useful resources. This poster presentation will provide further information about the Consortium and its future directions, suggest opportunities to become involved, and give attendees the opportunity to provide feedback and suggestions about potentially useful activities to build capacity for implementation research at UNC, RTI, and beyond.

Conducting clinically relevant multi-site research can be met with a multitude of road blocks resulting in delays, unexpected costs, and failed trials. Riding the wave of quality improvement, research organizations* are now establishing techniques to enhance trial conduct and quality. One key technique is to leverage existing clinical data, harmonize relevant data elements, and openly sharing vetted algorithms to better identify potential research subjects with health conditions of interest. Additionally, key stakeholders are engaged early on in the research process and innovative methods are used to reduce the need for “in clinic” research activities. UNC Chapel Hill is part of several of these efforts called “clinical data research networks” and is an active participant in several ongoing and developing trials that are using such data along with centralized and standardized processes to reduce the time it takes to find answers to pressing clinical questions. Our work is administered via the NC TraCS Institute and via our “Outreach” efforts. We are responsible for ensuring that others on campus are aware of these efforts and available consultative services for those interested in using clinical data for single site or multi-site improvement work. For the Lightening Symposium we will share the elements of a CDRN and provide examples of specific projects where UNC investigators and other stakeholders are leveraging CDRN infrastructure in their work in pragmatic and multi-site trials. *These include the Trial Innovation Network, PCORnet, the Carolinas Collaborative and Accrual of Patients to Clinical Trials Network.

A high priority of elementary schools’ principals must be their instructional leadership particularly in the area of literacy. The pressures of high stakes accountability policies along with the need for strong early literacy instructional programming demand that elementary school principals coordinate and manage literacy instructional programming. Comprehensive literacy programs typically include a school's systematic response for students who struggle to read or for those struggling to attain increasingly complex literacy skills by way of supplemental instruction, targeted interventions, or remediation. The Targeted Reading Intervention (TRI) is one response that seeks to equip general education teachers with the skills to identify specific needs of struggling readers and to implement targeted instructional reading strategies to support the readers. As critical managers of a school's instructional program, principal leadership alone is insufficient to maximize the benefits of an instructional approach such as TRI. Much of the literature about instructional leadership addresses teachers, principals, and instructional coaches as leaders separately. However, understanding the interdependence and co-construction of knowledge among teachers, principals, and instructional coaches in implementing literacy interventions is germane to the work of implementation sciences. The distributed leadership of TRI literacy coaches, classroom teachers, and school principals could shed light on the ways the interactions of these leaders promote or limit implementation of the TRI. Understanding the interactions of their leadership in the implementation of a literacy intervention could have implications for implementation of interventions, instructional approaches, and reform curricula that are supported by professional development and subsequent coaching.

Background: The Center of Medicare and Medicaid Services has implemented a new health care reform program in 2017, the Next Generation ACO Model (NGACO). This new model includes approximately 47 providers that are participating including University of North Carolina Health Care (UNC HC), an “early adopter” and the only healthcare organization participating in North Carolina. The goal of the NGACO is to set predictable financial targets, enable providers greater coordination of care, and aim to attain the highest quality standards of care. NGACOs also will be evaluated on their ability to deliver better care for individuals, better health for populations, and decrease of costs. Practice Quality Innovation & Population Health Services (PQIPHS) is working with the care teams using Lean Principles to address mechanisms that can improve care coordination in order to decrease Emergency Department admissions. Objectives: To collect data from the Medical Center and Rex Hospital within UNCHC about gaps within care coordination for NGACO patients through process mapping. Gaps, current state, and future states will be identified in order to address care coordination barriers and facilitators, and strategies will be identified to reduce the ED admissions through greater care coordination. Methods: Environmental scan of effective care models will be conducted. Primary research will be conducted with care managers, physicians, social workers, and health coaches. Primary research will include qualitative interviews and work group meetings with stakeholders. Atlas.Ti and Excel will be use to collect contextual quotes and quantitative data. Results: Evaluation of effective processes such as Kaizen events through Lean Principles will be identified. Operational metrics to measure effective care coordination will be assessed. Exemplar ACO models using effective care models will be discussed. Conclusions: Results and steps on how to implement care models using Lean Principles will be disseminated to providers with UNCHCS and healthcare stakeholders. Effective care models determined through PQIPHS will be recommended to help decrease ED admissions.

The Abecedarian Approach is a suite of strong teaching and learning strategies that can be implemented from birth to school entry. It was created at UNC-CH and has been implemented and tested in multiple randomized, controlled trials. Research has been conducted in the USA, Canada, Denmark, Romania, Australia, India, Pakistan, and Zambia. Training and small implementation trials have occurred in Mexico, Jordan, Singapore, Mongolia, and China. Thirty-five years of longitudinal research in the USA shows that the program improves child cognitive functioning and reading and math achievement throughout the school years. As a result, young adults (who received this birth to school entry program) experience better health, are 4 time more likely to graduate from a university, and are less likely to use social welfare services. A 2016 economic analysis of the program by Nobel Laureate economist James Heckman concludes that the Abecedarian Approach yields a 13% annual return on investment. The Abecedarian Approach is simple and can be learned by parents or early childhood education professionals in a 4-day training session. Yet, for its simplicity and its remarkable long-term benefits, the Approach is minimally used in the USA or in the other countries that have begun to implement it. An intriguing question remains: How can a beneficial early education Approach backed by extensive scientific evidence be made more accessible and taken to scale in multiple countries?

Our approach for dissemination of reformed STEM curriculum and pedagogy is an individual-centered, semester-long, co-teaching model in Chemistry, Biology, Physics & Astronomy, and Mathematics in the College of Arts and Sciences. Key to our implementation is professional development via the support of mentor-apprentice relationships within the context of teaching reformed curricula in gateway courses. In our model, a mentor (often a term faculty member) partners with an apprentice (often a tenure-track or tenured faculty member) in an individual course. Because faculty at our institution and globally report one of the most significant barriers to change is finding time, our project provides funds for a course release to the apprentice while they practice teaching practices within an exact course they will teach in the future. A second key element in our strategy was to leverage social networks in the form of faculty learning communities (FLCs) across three participating departments. Student outcomes such as learning gains, retention, and attitude surveys are collected, and key to our strategy is to provide evidence of “transformed” teaching behaviors via observation instruments. Faculty attitudes and beliefs of all faculty in the departments (involved and not involved in the project) are also being tracked over time too. With three years of implementation, we have some initial results to share as discussion points, as well as advice for implementing a similar model in other settings.

Ambulatory education is fundamental but the availability of ambulatory learning sites has declined. The decline in ambulatory sites is due to competing demands, lack of remuneration, feelings of inadequacy as teachers, and production-based compensation. To improve the numbers of learning sites, we emphasized bidirectional value of population health management to practices and students. We expanded our ambulatory curriculum to include population health management. Students participated in experiential learning doing quality improvement projects to improve the health of a population. Students were encouraged to align with institutional goals and to pair with local quality improvement experts. We also held faculty development sessions on quality improvement and population health management. UNC affiliated practices increased participation by 17 sites. To date, 86 students have participated in the new curriculum. Projects aligned with practice goals and most with metrics followed on practice dashboards. Students documented improvement 78% of the time. There were improvements in diabetes metrics, such as the use of aspirin and statins, eye exams, and foot exams. Improvements in colorectal cancer screening included implementation of FIT testing, improvement in return rate of FIT tests, and improvement in documentation of outside colonoscopies. Mammography rates increased, including those in uninsured patients due to improved processes for scheduling. One student revised the UNC health system dashboard with a “number needed to goal” calculator to make monthly dashboards more understandable. Population health management curriculum improved physician recruitment for ambulatory education. Curricular support for those goals can improve clinical metrics.

An analysis of return on investment (ROI) can help policy makers support, optimize, and advocate for the expansion of health programs in the world’s poorest countries. In order to harness the impactful message of the benefits that accompany investments in vaccination programs across the globe, we used a return on investment measure. This method of measurement can be well understood and used by professionals in all fields, especially by policy makers who require evidence to make financial decisions across sectors. We assessed the return on investment associated with achieving projected coverage levels for vaccinations to prevent diseases related to ten antigens in ninety-four low- and middle-income countries during 2011–20, the Decade of Vaccines. We derived these estimates by using costs of vaccines, supply chains, and service delivery and their associated economic benefits. Based on the costs of illnesses averted, we estimated that projected immunizations will yield a net return about 16 times greater than costs over the decade (uncertainty range: 10–25). Using a full-income approach, which quantifies the value that people place on living longer and healthier lives, we found that net returns amounted to 44 times the costs (uncertainty range: 27–67). Across all antigens, net returns were greater than costs. But to realize the substantial positive return on investment from immunization programs, it is essential that governments and donors provide the requisite investments.

Improvement science is predicated upon feedback loops between scholars and educators that allow policy development and implementation to be an iterative and developmental process. Too often, however, these conversations and feedback loops occur in environments that are resource constrained. While a significant part of the problem may be resource insufficiency, another and often-overlooked part of the problem is that the culture around resource allocation is nestled within the processes and assumptions of incremental-based budgeting. This presentation provides an overview of budgetary practices school districts have undertaken to more tightly link resource allocation and improvement processes. Through the lens of resource allocation and budgeting theory from the field of school finance, this presentation will present zero-based budgeting (ZBB) and modified zero-based budgeting (mZBB) as alternatives to the incremental based budgeting paradigm. This presentation will provide an overview of the incremental and zero-based paradigms with conceptual links to district initiatives such as PDCA cycles. It will provide examples of school districts implementing ZBB and mZBB budgeting plans. Finally, this presentation will provide areas of attention and focus important for districts moving into ZBB and mZBB environments.

Running from 2010 through 2015, the National Center for Scaling up Effective Schools was a five-year partnership between researchers and educators in two of the nation’s largest school districts. Together, its partners enacted a four-stage model of improvement. To begin, the improvement efforts were grounded in a research phase aimed at identifying programs, policies and practices that were effective locally. Research showed that higher-performing schools worked to build strong relationships between students and teachers, ground academic experiences in students’ lives and interests, and increase teachers’ awareness of students’ academic, social, and emotional needs. These findings became the foundation of a “design challenge” that guided the second stage of the improvement process – innovation design and development, as partners worked to construct a prototype. The prototype that emerged became known as Personalization for Academic and Social Emotional Learning, and comprised five components: 1) Rapid check-ins (RCIs), 2) Goal setting, 3) Intentional use of information, 4) Educator teams, and 5) Norms of engagement. Through the integration of these components, Center partners hoped to create and scale a ‘culture’ of personalization in their schools. Following this design and development stage, the process moved into its third phase: testing and implementation, wherein new participants from three schools joined the work and teams were tasked with refining the prototype and adapting it within the complex contexts of their individual schools. To accomplish this, teams used rapid-cycle testing – also known as Plan, Do, Study, Act (PDSA) cycles – through which they systematically learned “what works,” iterated upon their innovation, and improved it over time. Finally, the process shifted to the “scale-up” phase. Here, teams continued to implement the innovation while leveraging the PDSA cycles to gradually scale the innovation within the initial schools and out to new school sites.

Medication misuse, underuse, and overuse contributes to poor quality health care and accounts for nearly $300 billion in health care spending annually, suggesting that efforts to improve national health care must address the growing problems around medication use. The Enhancing Performance in Primary Care Medical Practice through Implementation of Comprehensive Medication Management (CMM) study aims to advance the efficient and effective delivery of CMM in primary care in order to optimize medication use for patients, improve health, and control costs. This requires that we operationalize CMM intervention, integrate into practices’ workflow, and ensure a consistent approach to care delivery. A network of 36 primary care practices across the United States was established through a partnership between UNC Eshelman School of Pharmacy, University of Minnesota College of Pharmacy, American Academy of Family Physicians National Research Network, and the National Implementation Research Network. The Active Implementation Frameworks (AIFs) are being used to conceptually guide the study including the use of the Plan-Do-Study-Act (PDSA) cycles. The presentation will describe how the AIFs including PDSA cycles are being used to support the process of purposeful small tests of change and facilitate site teams in identifying challenges, solving problems, improving processes, and building infrastructure within the primary care practice. The PDSA cycles results are being systematically captured to enhance our understanding of how CMM can be delivered effectively and efficiently in real-world practices. Lessons learned from developing the capacity of study sites to engage in active implementation practices including PDSA cycles will also be shared.

The Incredible Years® (IY) Dina Dinosaur Small Group program is an evidence-based program developed for implementation in clinics with young children diagnosed with Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD). Although it holds potential for benefiting a broader range of children at risk for mental health disorders, the program requires adaptation and implementation supports for school delivery. Moreover, evidence of both feasibility and positive impact on educational outcomes is needed for educator buy-in and program adoption. As part of a randomized controlled trial funded by the Department of Education, the Self-Regulation Skills for Success team has adapted session structure and developed implementation supports which may be informative to other school-based translational work. A key component of our approach has been partnering with school counselors who serve as co-leaders for the intervention groups, facilitate communication with families and other school staff, and help align the program with school policies and procedures. Monthly peer coaching has supported this partnership and enhanced counselors’ capacity for delivering effective social-emotional interventions. Other important implementation supports include consultation provided to teachers and parents as well as “recess coaching” (prompts and reinforcement for children on the playground) to support skill generalizability. Through work to date in 9 schools across 4 districts, program adherence and delivery quality have been high as has stakeholder satisfaction. Challenges have included matching the program to student need, addressing differences between program philosophy and school culture, and educating administrators about research design and outcomes.

Health improvement programs are often designed and implemented by experts. However, improvement programs that incorporate community feedback may be easier to implement and sustain over time. Crowdsourcing, or the process of giving a task to a group instead of an individual expert, can help develop health improvement projects. Crowdsourcing incorporates non-expert, community perspectives into global health campaigns and researches, as it often brings together multiple sectors through contest.1 A crowdsourcing contest usually has four different stages: 1) organizing a community steering committee; 2) engaging community to contribute; 3) evaluating contributions; and 4) recognizing finalists. Crowdsourcing contests have been used to develop health campaigns to promote sexual health, 2, 3 reduce cancer risk, 4-6 and encourage smoking cessation. Our team has conducted 16 crowdsourcing contests to promote HIV testing, condom use, and healthy cites campaigns. In our two RCTs focused on MSM, crowdsourcing improved HIV testing (37% among never testers in three weeks) 2 and condom use (successfully reduced condomless sex by 48% within three months) 3 compared to social marketing approaches. In addition, a crowdsourced intervention was also cost saving. 2 Our qualitative data suggest that crowdsourcing is an effective tool for community engagement, empowering local communities to be more involved in the design, creation, and implementation of a health campaign.

Ongoing efforts to strengthen STEM education have emphasized the importance of students’ engagement with meaningful and challenging content, active roles in problem solving and communication, and perseverance to make sense of ideas. Related developments in instructional improvement underscore the importance of students’ interaction with peers in classrooms as a means to broaden access and foster deep engagement with content, structure opportunities for meaningful discourse, an d build competencies in collaboration. Yet small group learning environments often fail to realize their potential as instructors find it difficult to maintain the intellectual demands tasks and integrity of group interaction processes once the task is turned over to students. A conceptual framework and conceptually-based tools for instructors are foundational to successful implementation and scaling of evidence-based practices to improve small group work in k12 as well as post-secondary settings. In PEARL (Peers Engaged as Resources for Learning), an NSF-funded study of small group learning environments, we developed a conceptual framework of “teacher moves” and lesson-planning and implementation tools for teachers as they guide learners through small group work. The framework and tools were developed from naturalistic study of middle and high school classrooms; refined in collaboration with practicing teachers to respond to implementation challenges; and iteratively tested and modified in middle and high school mathematics classrooms using design-based research methodology. A result of this project is the identification of a set of core instructional strategies that can be applied across small group learning tasks to optimize productive student group work. We will describe the framework and tools, and continuing implementation challenges to be addressed in the final phase of testing of the framework and tools.

Employer scheduling practices are an important source of employment instability that can limit earnings, impede worker performance, and create stress and work-life interferences that undermine worker health and well-being. Widespread acknowledgement of the impact of these challenges on thousands of working class households has triggered several policy debates regarding the need for more stable schedules across a variety of industries. However, evidence on the benefits and challenges of changing scheduling practices is still lacking. Using data from a cluster-randomized experiment conducted in partnership with a national apparel retailer (Gap, Inc.), we propose to examine ways in which the stability of workers' schedules can be improved without affecting the financial bottomline of retailers. Stores in San Francisco and Chicago (N=30) were part of a randomized workplace experiment that began October 2015 and ended August 31, 2016.

The child welfare system in North Carolina seeks to promote the well-being of vulnerable children in our state. The needs of children who come in contact with the child welfare system are complex and almost always involve coordination of services with other child-serving systems (e.g., education, behavioral health, housing, and public assistance). Data from one child-serving system is often not enough to provide an accurate depiction of a child’s individual outcomes, or population-level outcomes. Improving the quality of services is related to our ability to understand these cross-sector child systems. The purpose of this presentation is to briefly discuss the technological advances in data collection, storage, and analyses that have resulted in the availability of large amounts of cross-system, administrative data, but also to discuss the “human” barriers to using data to improve quality. “Big data” has its own advantages, disadvantages, and challenges for researchers and policymakers interested in improving outcomes for children.