Adjustment to population health needs

Reliable and timely health data and information are a cornerstone for success in a health system. Data provides the performance insights that countries need to determine whether they are meeting their goals and targets and if not, why. Thus, countries should strengthen their institutional capacity to collect, analyze, use, and share data (i.e. surveillance data, programme monitoring data, patient outcomes and experience data). Doing so will increase their ability to deliver interventions that meet the needs of their population over time. It will also strengthen their capacity to prevent, prepare for, and respond to health emergencies, as demonstrated by the COVID-19 pandemic. ^8131415 

Key activities

All levels 

The below activities apply to all information systems for health (HIS), including those that are important for M&E (for more information about the key types and functions of HIS, see the information and technology module):

• Strengthen data governance policies: Strengthening and/or implementing policies and guidelines for data collection, analysis, use, and dissemination can increase the quality and reliability of data as well as its use for decision-making processes. Such policies may also include a national strategy for eHealth. It is important to recognize the political complexities around data release and use when devising policies. ¹⁷¹⁸
  • See the WHO’s Global Strategy on Digital Health and their National eHealth Strategy Toolkit for additional guidance.​​​
• Institutionalize, coordinate, and harmonize HIS and M&E systems, such as by ¹⁹²⁰
  • Building internal demand for the active collection, analysis, and use of evidence for informed policymaking, programme design, and M&E. For example, countries should present research evidence and findings in an easy-to-understand format and make this information accessible across sectors and levels of the health system. 
    • See pages 6 - 9 of the UW’s Development Policy and Finance team’s report on government M&E systems and pages 12 - 17 of the WHO’s vision for harmonized and interoperable information systems for additional guidance.
  • Clarifying stakeholder roles and responsibilities in country action plans (including the M&E plan) and using evidence from evaluations to assess whether roles are being carried out as planned. 
    • To aid in this step:
      • Establish a central ministry or agency responsible for stewarding and coordinating health information systems and M&E efforts in the country (these may be separate entities/units, depending on the context). ⁸²⁰ For example, Fiji established a Health Information, Research and Analysis Unit within their MoH and Malawi introduced a joint action plan for strengthening HIS and M&E. 
      • Clearly define staff roles and responsibilities across the different platforms, including who is responsible for collecting, monitoring, and reporting health data and information and who is responsible for conducting evaluations. Ensure that roles and responsibilities are consistent across different projects, agencies, and/or governmental levels. ²⁰²¹
      • Appoint a focal point person(s) who coordinates HIS and M&E activities with the programme unit(s) and other stakeholders. To promote harmonization, focal points should coordinate HIS and M&E activities with any central ministries or agencies for HIS and M&E. ²²
    • See pages 9 - 10 of the UW’s Development Policy and Finance team’s report on government M&E systems and chapter 5 of the WHO’s data quality assurance module for additional guidance.
  • Harmonizing government HIS and M&E activities, for example by establishing a mandate for the office in charge of coordination; building staff capacity to conduct M&E activities; and increasing funding for coordination activities within and between M&E and HIS systems.
    • See pages 10 - 11 of the UW’s Development Policy and Finance team’s report on government M&E systems and chapter 5 of the WHO’s data quality assurance module for additional guidance. 
  • Harmonizing government and development partner HIS and M&E systems ¹⁹
    • To aid in this step, use the following set of principles: ²³
      • Make HIS and M&E systems country-owned and led. For example, development partners/ donors should engage country leadership in the design and implementation of their HIS and M&E systems. 
      • Work with partner countries to rely on country-owned M&E systems. For example, use a participatory approach to strengthen country capacities and demand for results-based M&E.
      • Design M&E systems to meet the priorities and accountability needs of partner countries (rather than just donor’s information and accountability needs)
        Ensure that all M&E systems are simplified, harmonized, and appropriately timed in relation to national policy and budget processes. 
    • See pages 24 - 27 of the UW’s Development Policy and Finance team’s report on government M&E systems and chapter 5 of the WHO’s data quality assurance module for additional guidance. 
• Strengthen data collection and monitoring systems, through ¹⁹²⁰
  • Selecting and aligning indicators across HIS and M&E systems, and ensuring that on-the-ground data collection aligns with this official list of indicators. See step 2 of the sub-action below for a recommended set of indicators for PHC. 
    • See page 13 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
  • Increasing the data collection and aggregation capacity. For example, countries should establish documented rules for data collection, including what data to collect, who collects it, and how often. Countries should also establish rules for data aggregation and ensure that staff at all levels of the health system are trained and equipped to deliver on these rules.
    • See pages 13 - 14 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
  • Introducing more sophisticated data capacities (if feasible), including processes for verifying data quality and integrating data collection, aggregation, and verification using information technology tools and supportive supervision  
    • See pages 14 - 15 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
• Improve the analysis and use of data generated by HIS and M&E systems, including by: ¹⁹²⁰
  • Developing a strategic M&E framework that details planned activities and outputs. The framework should allow governments to evaluate how well they have done in implementing their strategy, as well as expected outcomes. 
    • See page 15 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
  • Conducting an implementation assessment, which is a type of performance evaluation. It uses output and outcomes indicators to examine how well the M&E strategy is being implemented in different contexts. Budget A document that outlines forecasted revenue and planned expenditure for an entity (government, subnational unit, or health facility) over a defined period of time. Budgets indicate how funds are meant to be allocated to achieve certain objectives. or audit evaluations are a common type of implementation assessment.
    • See page 16 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
  • Conducting an impact evaluation, which is another type of performance evaluation. It uses outcomes data to evaluate the impact of M&E activities over time. Well-executed impact evaluations tend to use a combination of time series and control groups to determine whether activities are achieving desired outcomes. 
    • See pages 17 - 20 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
  • Conducting a routine, annual, or periodic assessment of data facility- and/or district reported data ⁸
    • See the WHO’s data quality assurance toolkit for additional guidance 
  • Consistently using HIS and M&E data to influence planning and budgetary decisions. 
    • To aid in this step, countries should: 
      • Outline plans for institutionalizing the use of results information during the creation of their M&E plan. In addition, countries should disseminate M&E results to a diverse array of stakeholders, including the public, civil society organizations, and development partners. 
      • Continuously collect data on diseases and events of public health significance (including the burden of diseases data) from both indicator-based sources and community-based or crowd-sourced data. ^24252627
    • See pages 20 - 21 of the UW’s Development Policy and Finance team’s report on government M&E systems for additional guidance.
• Ensuring timely, systematic reporting of health data across levels of care. Closed feedback loops should be in place between facilities, sub-national regions and the centralized levels of the health system to ensure that the data actually informs action. The flow of information from the national level back to the facility is particularly important for response to incidents, as well as for the overall integration of information systems into population health management and decision-making processes. 
  • To assess the reliability and quality of these feedback loops stakeholders should consider factors such as:
    • Does information flow from the national level back to the facility level and back?
    • Are interoperable, interconnected, and electronic communication channels and information systems in place and consistently functioning? 
    • Do staff have identified communication channels to report to? Are there trained staff with the necessary expertise and processes to investigate and respond to reports?
    • If there are gaps, stakeholders should identify and seek to understand why there are gaps, such as whether they are due to inputs (i.e. information systems and workforce), policies, or other factors.

Related elements

• Policies & leadership
• Information & technology 
• Management of services 

Relevant tools & resources

• Data.org, 2022: Epiverse - distributed pandemic tools program 
• PAHO, 2022: ENLACE - Data Portal on Noncommunicable Diseases, Mental Health, and External Causes
• WHO, 2022: Child health and well-being dashboard 
• WHO, 2022: Hub for pandemic and epidemic intelligence 
• WHO, 2022: SCORE data collection tool
• WHO, 2022: Toolkit for routine health information systems data
• GHS Index, 2021: Global Health Security Index
• UN, 2021: SDG Global Database 
• World Bank Group, 2021: Service delivery indicators health survey refresh fact sheet 
• 3-D Commission, 2020: Data, social determinants, and better decision-making for health  
• Dover and Belon, 2019: The health equity measurement framework: a comprehensive model to measure social inequities in health
• Partners in Health, 2018: UHC Monitoring and Planning Tool
• WHO, 2017: Strategic Framework for Effective Communications
• WHO, 2010: Monitoring the Building Blocks of Health Systems - Information Systems
• WHO, 2012: National eHealth Strategy toolkit

The effective analysis, use, and dissemination of data and information support a country’s ability to adjust to population health needs in multiple ways: ^828293031

• Priority setting The process of making decisions about how best to allocate limited resources to improve population health. and planning: Stakeholders can use data to monitor and clarify the epidemiology of public health problems; and guide priority setting, planning, and policy decisions.

• Programme design and implementation: Stakeholders can use data to monitor and improve the effectiveness of programmes over time. For example, they can use data from past evaluations to inform future policy decisions and programme design. 

• Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. , response, and management: Stakeholders can use data to strengthen public health response efforts in normal times and in a crisis, including via risk communication, testing and contact tracing, and case management and transmission control. 

• M&E and quality improvement: Stakeholders can use data to track and measure the impact of a public health intervention, including progress towards specified goals. It can also feed into progress and performance improvement efforts at different levels of the health system, ultimately strengthening the quality of care. 

• Innovation and learning: Stakeholders can use data to facilitate continuous learning, improvement, and innovation at different levels of the health system. For example, the data generated from biomedical, clinical, and health system research activities can be used to identify and address problems in the health sector. 

Key activities & questions

• How reliably and consistently are data and information used to inform priority setting? To understand the degree to which data are used to set service delivery priorities, you might consider looking at the types of data, variation in data availability, and the use of these data: 
  • Are data and information systematically used to set priorities at the national and sub-national levels?
    • Routine, non-routine, and research data are needed to get a fuller picture of a public health issue. Such data may include survey data, claims and cost data, and patient-generated health data, among others. ⁸
  • Are data available, presented, discussed and applied through consistent processes for nearly all priority setting exercises? 
  • If not, are gaps due to problems with information systems, decision-making processes, policies, or other factors? 

1. How reliably and consistently are the results of priority setting (including data from the situation analysis and past evaluations of the NHS) used to inform resource allocation? The results of the priority setting exercise (the most appropriate programs and interventions) should determine the allocation of resources all or nearly all of the time to improve population health. If resources are not being appropriately allocated, or current programs and interventions are not improving population health needs, you might look to the effectiveness of past priority-setting exercises and consider factors such as:

• Are existing and emerging health needs being assessed? 
• Are stakeholders being engaged and held accountable for decisions? 
• Is an explicit process for setting priorities being used? 
• Are the values and context being appropriately considered? 
• Are priorities in line with these needs, values and the local context? Do they also target/align with areas of poor performance?
• Are any decisions made being communicated to the relevant stakeholders, with systems in place for managing feedback and demands as a result of these decisions? 
• If not, are gaps due to problems with governance, political and financial commitment, regulatory and legal structures, or other factors?

Related elements

• Policy & leadership
• Multi-sectoral approach
• Funding & allocation of resources 
• Information & technology 
• Population health management 

Relevant tools & resources

• Emergency “An extraordinary situation in which people are unable to meet their basic survival needs, or there are serious and immediate threats to human life and well-being. Emergency interventions are required to save and preserve human lives and/or the environment. An emergency situation may arise as a result of a disaster, a cumulative process of neglect or environmental degradation, or when a disaster threatens and emergency measures have to be taken to prevent or at least limit the effects of the eventual impact. (UNDP)” preparedness & response 
  • ReBUILD Consortium, 2022: Health systems resilience in fragile and shock-prone settings through the prism of gender, equity and justice: implications for research, policy and practice
  • WHO, 2022: Hub for pandemic and epidemic intelligence 
  • WHO, 2022: Health systems resilience toolkit
  • World Bank, 2021: Frontline: Preparing healthcare systems for shocks from disasters to pandemics 
  • USAID, 2021: Recommendations for strengthening health systems during the COVID-19 pandemic and beyond
  • Prevent Epidemics: Multiple resources 
  • Exemplars in Global Health: Multiple resources 
• Approaches to priority setting 
  • JLN, 2022: A Health Practitioner’s Handbook and Toolbox for Identifying the Poor and Vulnerable
  • Rubinelli et al., 2022: WHO competency framework for health authorities and institutions to manage infodemics: its development and features
  • DCP3, 2018: Disease Control Priorities Review (third edition)
  • WHO, 2016: Priority setting The process of making decisions about how best to allocate limited resources to improve population health. for national health policies, strategies, and plans  
  • WHO, 2016: Strategizing national health in the 21st century: a handbook
  • Center for Global Development, 2012: Report on Priority Setting in Health: Building Institutions for smarter public spending
• Data-driven decision-making
  • AFIDEP, 2022: Evidence-informed policymaking training curriculum 
  • National Collaborating Centre for Methods and Tools, Learning Centre Module, 2022: Implementing Knowledge Translation Strategies in Public Health
  • 3-D Commission, 2020: Data, social determinants, and better decision-making for health  
  • NNIP, 2013: Strengthening local capacity for data-driven decision making 

M&E supports country efforts to adjust to population health needs in several ways: ¹⁷³²

• Enables countries to diagnose gaps and consequently make evidence-informed decisions about their national health sector strategic plan, including new programs, projects, and/or initiatives that should be implemented to achieve plan objectives

• Helps countries identify ways to allocate and manage resources effectively and efficiently 

• Supports countries to assess the impact of their national health sector strategic plan, gaps in implementation, and programs, projects, &/or initiatives that were implemented successfully temporally and regionally 

• Provides countries with data they can use to convince decision-makers and/or donors to invest more in different programs, projects, and/or initiatives. They may also use this data to come up with alternative ways of addressing problems and/or achieving strategic goals. 

Key activities 

The WHO recommends six steps that countries can take to implement and sustain M&E for PHC at the national and/or sub-national levels (specifics on what should be included in an M&E plan are discussed in the M&E deep dive section of this module): ¹⁷³³

• Step 1: Align PHC monitoring within existing national health sector plans, strategies, review processes, and related accountability mechanisms. For example, can promote accountability to strategic targets by building a results-based accountability framework into the M&E plan.
• Step 2: Tailor selection of indicators based on country policies, priorities, maturity of health system and gaps. See Annex 1 of the WHO’s M&E framework for a menu of indicators for country selection for PHC monitoring, by PHC component. 
• Step 3: Set and monitor baseline values and country targets for PHC. Countries may consider conducting a situation analysis to inform baseline values. 
• Step 4: Identify data sources and address major gaps through innovative methods and tools, such as via the use of EHRs and facility, patient-, and community-based surveys.
• Step 5: Strengthen capacities at national and subnational levels in data analysis, communication, and dissemination of results, including by:
  • Implementing mechanisms to promote data access and dissemination such as annual statistical reports, national health observatories or portals, and an open data policy in the government, among others. 
  • Translating and using M&E data to inform policymaking and legislative proposals. Including via policy-relevant data analyses, evidence synthesis, and structured expert review processes.
  • Using regular independent reviews to ensure data accountability and transparency and to drive remedial action as needed. 
• Step 6: Conduct regular process of policy dialogues to guide actions, interventions, and investments for PHC performance improvement and management. This will help to ensure that data and evidence can be effectively applied within the existing regulatory environment (i.e. data release and use policies) and that decision-makers are actively bought into the use of M&E for performance improvement efforts. 

Related elements

• Policy & leadership
• Multi-sectoral approach
• Funding and allocation of resources 
• Information and technology 
• Population health management (community engagement)
• Management of services (systems for improving quality of care)

Relevant tools & case studies

• Cross-cutting learnings: 
  • DEVEX: Software and resources for M&E professionals 
  • Global Fund: Compendium of resources on developing an M&E framework and strengthening in-country M&E systems 
  • Humans of Data: What is M&E? A guide to the basics 
  • Humans of Data: The 7 types of evaluation you need to know
  • MEASURE Evaluation: Self-guided minicourse on M&E fundamentals 
  • MEASURE Evaluation: Compendium of tools and resources for M&E 
  • WHO: Measurement & monitoring framework for PHC
  • WHO, 2019: Primary health care Primary health care (PHC) is “a whole-of-society approach to health that aims to maximise the level and distribution of health and well-being through three components:  (a) primary care and essential public health functions as the core of integrated health services; (b) multisectoral policy and action; and (c) empowered people and communities.” on the road to universal health coverage: 2019 monitoring report
  • World Bank: Good practice case studies for Resilience “The ability of a system, community, or society exposed to hazards to resist, absorb, accommodate to, and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions through risk management.” M&E 
  • World Bank: Volume on how to build M&E systems to support better government 
  • World Bank: Volume on how to build and embed strong M&E systems into decision-making processes in all sectors, including for policy design and policymaking 
  • World Bank: Handbook and case studies on how to build a results-based monitoring system
  • World Bank: 10 key issues for diagnosis of a government’s M&E system
  • USAID: Toolkit for high-quality M&E
• Using the M&E system
  • WHO: Handbook for strategizing national health in the 21st century
  • World Bank: Case study on using M&E to support performance-based planning and budgeting in Indonesia 
  • World Bank: Report on using care cascade analytics to identify challenges and solutions in service delivery and client outcomes 
  • The Global Fund: Strategic framework for data use for action and improvement at the country level
• Improving the quality of the M&E system
  • WHO: Compendium of data collection and analysis tools for M&E and other performance improvement efforts, including the SCORE data collection tool
  • World Bank: Handbook on the ten steps to a results-based monitoring and evaluation system 
  • MEASURE Evaluation: Portfolio for ensuring data quality for M&E
  • Humans of Data: 5 things you’re doing wrong in your M&E process
  • CGD, 2022: A Higher Bar or an Obstacle Course? Peer Review and Organizational Decision-Making in an International Development Bureaucracy
  • Sustaining the M&E system
    • IDEV: Building supply and demand for evaluation in Africa Vol. 1
    • World Bank: Incentives Incentives refer to, “a particular form of payment which is intended to achieve some specific change in behaviour. Incentives come in a variety of forms, and can be either monetary or non-monetary.” for M&E - how to build demand 

Integrating surveillance into a national plan is important for ensuring institutional capacity meets essential health functions and operations - for example through a public health service mandate and an agency to carry out specific public health functions.

Key activities 

National level

• Regularly assess the national surveillance system, and develop and implement national action plans to improve the core functions of surveillance. Regularly evaluate or test national surveillance procedures, plans, or strategies (either through simulations or real health threats) to ensure continuous operational capacity. ³⁴³⁵
• The national plan should include elements relevant to surveillance: 
  • Plans for developing or enhancing event-based and syndromic surveillance. Users can refer to the WHO guidance on implementation in national legislation for additional support in facilitating surveillance and response activities that meet International Health Regulation obligations. Policy and legislation should also promote investment in infrastructure that support the functions of event-based and syndromic surveillance systems, such as electronic information systems and modernized laboratories and clinics. ³⁶³⁷
  • Public health emergency response plan and strategies for health emergency and disaster risk management. Countries should develop procedures, plans, or strategies to activate responses to public health emergencies at a local level and to elevate to higher levels of response, as needed. This response plan should include a strategy for multi-hazard approaches, protocols for the maintenance of essential health services during response and restoration efforts, and a focus on ethics and rights-based approaches. ³⁸³⁹ Find more information in the resilient facilities & services module.
  • Systems for sending and receiving medical countermeasures and health personnel during a public health emergency. Medical countermeasures are products that can be used for population treatment and prevention in a public health emergency, including biological products, drugs, and devices. Find more information in the Medical Countermeasures and Personnel Deployment Indicator of the WHO’s Joint External Evaluation Tool: International Health Regulations (2005). ⁴⁰⁴¹
  • Rapid response to and systematic monitoring of a comprehensive spectrum of health threats and diseases. Countries’ surveillance strategies may differ between the local, regional, and national levels. Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. systems that engage multiple strategies will require coordination among the different health authorities at multiple levels of the health system. ⁴² Find more information on developing a detailed plan of action for surveillance and response systems in the  WHO’s guide to monitoring and evaluating communicable disease surveillance and response systems here and on the different types of surveillance strategies here. ⁴³⁴⁴ In addition, learn more about how to harmonize and coordinate surveillance systems with existing HIS and M&E systems in the action “collect more and better data on population health needs”. 

Related elements

• Resilient facilities & services
• Multi-sectoral approach
• Information & technology

Relevant tools & resources

• Milken Institute, 2022: A global early warning system for pandemics: a blueprint for coordination 
• WHO, 2022: Compendium of resources on surveillance in emergencies 
• World Bank, 2021: Frontline: Preparing healthcare systems for shocks from disasters to pandemics 
• WHO AFRO, 2019: Technical Guidelines for Integrated Disease Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. and Response in the African Region: Third edition
• Groseclose et al., 2017: Public Health Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. Systems: Recent Advances in Their Use and Evaluation
• WHO, 2017: Health emergency interim guidelines: a WHO guideline development framework and toolkit 
• WHO, 2016: Joint external evaluation tool: International Health Regulations (‎2005)‎
• LHSTM, 2009: The use of epidemiological tools in conflict-affected populations: open-access educational resources for policy-makers
• WHO, 2006: Communicable disease surveillance and response systems: a guide to monitoring and evaluating
• DCP2, 2006: Public Health Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. : A Tool for Targeting and Monitoring Interventions (chapter 53)
• WHO AFRO: SORMAS ( Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. Outbreak Response Management and Analysis System)

Stakeholders must have systems in place to develop innovative ways of dealing with public health problems. However, mechanisms that allow countries to develop sustainable, locally-led solutions to problems, including via public health and health policy and systems research, are often underfunded and/or not well-linked to policy-making efforts. ¹⁵⁴⁵ Furthermore, researchers from LMICs are poorly represented and innovation capacity tends to be low in these countries due to resource and governance constraints. Thus, improving a country’s research and innovation capacity will be an important force for economic growth and better health and policy outcomes. ^4546474849 

Key activities 

• Assess a county’s capacity for innovation: Potential indicators a country might use to assess its innovation capacity include: investment in innovation (including human, material, and technical resources for innovations), technological infrastructure for innovation (such as internet usage for health), the sources for soliciting ideas for innovation (including user-driven and open innovation), the culture for innovation, human capital and education (i.e. tertiary education), technology transfer and participation in globalization, and gender equity. 
• Cultivate a culture of innovation and learning: Organizations or institutions should strive to create a culture that readily identifies and implements innovations, taking into account the norms and values of a given context. Seven key dimensions of culture that distinguish highly innovative organizations include risk-taking, resources (in the form of time and money), knowledge, strategic goals, rewards, tools ( that support innovation), and relationships (people are exposed to a diverse range of thinking and people). ⁵⁰
• Promote PHC-oriented research & development: To support the adoption and creation of innovations that support the goals of high-quality PHC for all, it is important to establish norms and values that fulfil the values of PHC such as inclusiveness and effectiveness. ⁵¹ In addition, to sustain these innovations and improve future public health interventions and programs, it is critical to ensure adequate funding and buy-in for public health and health systems and policy research. ⁸⁴⁹
• Implement mechanisms for recognizing, evaluating, and scaling innovations:
  • Recognize: develop/implement tools, technologies, &/or processes that allow innovators to recognize where opportunities for innovation and learning exist. Examples include technology databases; research and utility models from health and non-health industries; publications; local and international universities and research institutes; and evidence from the field, among others. ⁵²
  • Evaluate: use M&E systems to track the progress and performance of innovations and take corrective action, as needed. If not already in place, develop standards and criteria for evaluating the “success” of innovations via a transparent, participatory process. ⁵³ Link/align the results of the evaluation with future priority-setting exercises.
  • Scale: throughout the evaluation process, identify “successful” innovations to scale based on their value and impact. If not in place, identify which factors/criteria qualify an intervention for scale. Factors/criteria may include: the target population or organization's receptiveness to change; access barriers (financial and geographic); ease of use; quality; convenience; and acceptability will influence the strategy for scale. ⁵⁴⁵⁵ Before scaling an innovation, evaluators should ensure it has undergone multiple iterations at the pilot stage to determine its value, performance, and application to the context/level/location in which it will be implemented, including whether the environment the innovation will be implemented in is sufficiently ready for change. 
• Strengthen institutional capacity and stakeholder buy-in: While countries will likely have their own unique methods/mechanisms/processes that lead to different outputs, scaling up innovations require initiatives built around diverse stakeholder participation, financially sustainable institutions, and supportive governance structures that support their integration into the health system. Examples include: ⁵⁵
  • Awareness: provide information to policymakers, the public, and the media that stimulates innovation and learning. 
  • Performance: measure progress and results of changes that have been implemented against public policy objectives and the market factors (i.e. supply and demand)
  • Signalling and Monitoring: call attention to important innovation trends and growth opportunities.
  • Accountability and evaluation: support research and development budgets and innovation policies, in addition to compliance with regulatory structures and public interests. 
  • Consensus building: develop and implement more effective innovation policies and strategies that strengthen a country’s national innovation capacity.

Related elements

• Policy & leadership
• Multi-sectoral approach
• Funding & allocation of resources 
• Information & technology 
• Population health management
• Management of services

Relevant tools & resources

• Fundamentals
  • IHI: Model for improvement
  • IHI: Quality Improvement Essentials Toolkit
  • USAID and Momentum: Toolkit for adaptive learning in projects & programs
  • EPIS: Implementation framework & toolkit
  • CFIR Research Team-Center for Clinical Management Research, 2022: The Consolidated Framework for Implementation Research 
  • Alliance for health policy and systems research, 2021: What is health policy and systems research?
  • Alliance for health policy and systems research, 2021: Learning health systems: pathways to progress
  • WHO, 2022: Compendium of resources on health research
  • IDIA, 2021: Framework, tools, and reports on strengthening and scaling innovation
• Measuring innovation capacity & building a culture of innovation and learning 
  • Policy innovation research unit: Developing a Global Healthcare Innovation Index
  • Harvard Business Review: What is organizational culture? And why should we care?
  • NHS Institute for Innovation and Improvement: Creating the culture for innovation: a practical guide for leaders 
  • WHO: OpenWHO (web-based knowledge-transfer platform)
• Recognizing, evaluating, and scaling innovations
  • LEAD innovation: identification of innovation potential - 6 important triggers for innovation
  • WHO: How do we ensure that innovation in health service delivery and organization is implemented, sustained and spread?
  • Stanford Social Innovation Review: Innovate and Scale: A Tough Balancing Act
  • R4D: Scaling Innovations 
  • IHI: Science of improvement - implementing changes
  • IHI: Science of improvement - spreading changes
  • IHI: A Framework for Spread: From Local Improvements to System-Wide Change
  • NCMMT, 2022: Implementing Knowledge Translation Strategies in Public Health
  • WHO, 2010: Nine steps for developing a scaling-up strategy 
• Strengthening institutional capacity for innovation 
  • WHO: Social innovation in health: case studies and lessons learned from low- and middle-income countries
  • OECD: Innovation policies for inclusive development 
  • IHI: The improvement guide: a practical approach to enhancing organizational performance
  • World Bank, 2017: The innovation paradox: developing-country capabilities and the unrealized promise of technological catch-up

Encouraging broader social participation in decision-making and evaluation processes (including community-based representation) helps to strengthen accountability across sectors and forge collaborative partnerships for more equitable and sustainable initiatives. ⁵⁶

Key questions & activities 

• Understand who, and to what extent, are involved in decision-making processes. It is important to engage a diverse array of stakeholders in priority-setting, M&E, and innovation and learning processes. Implicit in this is the existence of systems for multisectoral and community engagement at national and sub-national levels. If stakeholder engagement does not occur or is not occurring for all priority setting exercises, you might consider looking at recent examples of priority setting and M&E in your country:
  • Are stakeholders’ roles & responsibilities clearly defined and communicated?
  • Are processes for identifying, communicating with, and convening stakeholders in place? What processes are in place to engage and coordinate the interests and actions of stakeholders from diverse contexts (including the private and non-health sectors)? 
  • Are these processes transparent, inclusive, and consistent? 
  • Are engagements occurring at regular, predefined intervals and when necessary, on an ad-hoc basis? 
  • If not, are gaps due to problems with social accountability mechanisms, communication networks, policies, or other factors?
• See the sub-action on engaging and empowering patients, families, and communities (take action section) for a comprehensive list of activities that countries can implement.

Related elements

• Multi-sectoral approach
• Population health management

Priority setting The process of making decisions about how best to allocate limited resources to improve population health. is the process of making decisions about how best to allocate limited resources to improve population health. It involves the use of information collected from surveillance systems and other health information systems to set, implement, and adjust priorities over time. For this reason, a well-functioning health information system is also a critical input to the priority setting process. If data generated from these systems is incomplete or unreliable, it may lead decision-makers to make false conclusions about the extent of health problems and the effectiveness of strategies. ^40575859

Effective priority setting also involves the use of an evidence-informed deliberative process (EDP). EDPs provide a structured process by which diverse stakeholders identify explicit criteria for priority setting, interpret evidence, and deliberate on recommendations and decisions based on their opinions, needs, and interests. When rooted in a strong multisectoral approach (whereby diverse stakeholders are involved and held accountable to citizen interests), EDPs add both scientific and social credibility to the decision-making process. ^{405758 596061}

Priority setting The process of making decisions about how best to allocate limited resources to improve population health. may occur at all levels of the health system. However, the information here is specific to national and sub-national priority setting. More information on priority setting at the community level can be found in the population health management module. 

Priorities are evidence-based

Priority setting The process of making decisions about how best to allocate limited resources to improve population health. relies on the use of diverse sources of data (including health and burden of disease information, service delivery evaluations, and cost-effectiveness assessments) as well as stakeholder input to prioritize the most appropriate programs and interventions and inform resource allocation. ⁵⁷⁵⁸⁵⁹ 

Priority setting The process of making decisions about how best to allocate limited resources to improve population health. exercises should therefore start with an in-depth situation analysis of the health sector to determine existing and emerging needs. ² A situation analysis identifies the strengths and weaknesses of the health system and is a key initial step in the development of different elements of the national health plan, including the strategic directions and identification of priority diseases and interventions. ⁶² The situation analysis must generate a sufficient evidence base to ensure that stakeholders have access to quality and diverse information needed to make informed decisions consistent with the needs and values of the population. ⁵⁸ While the situation analysis can happen at any level of the health system and on varying themes and scopes, such as health financing or workforce, ²⁶² this module focuses on an overall health sector situation analysis for system-wide priority setting. 

According to the World Health Organization, a well-facilitated situation analysis should be participatory and inclusive, analytical, relevant, comprehensive, and evidence-based and is separated into three distinct streams of analysis: ⁶²

• Health data “In public health, “data” usually refers to statistics reported from health care facilities, survey data or data collected through observational studies. Distinctions can be made between routinely reported data and data that are collected at certain times or over a specific period of time as part of a special study or survey. Both routine and non-routine data, as well as data from research systems, are required and contribute to a  fuller picture of any given public health issue.” - Analyses of health data from all levels of the health system - national, regional, and local - including trends and developments over time, provide information on both the health needs of the population and the current performance of the health system in meeting those needs. Analysts involved in examining health data should include technical experts trained to analyze and interpret data and non-technical experts familiar with health sector activities. Examples of data sources include population health surveys, HMIS, CRVS, facility assessments, patient-reported outcomes, and surveillance system reports.
• Activity and budget - An analysis of the health sector budget and the implementation of health sector activities should assess whether the budgets allocated in the health sector and the policies, strategies, or plans adopted into the national health plan reflect the broader national health plan objectives, including whether activities are sufficiently funded and able to be implemented as per the planned activities and budget. Examples of data sources include Ministry of Health and Ministry of Finance routine financial reports, national health accounts, public expenditure data, performance reports, and facility assessments.
• Effectiveness Effectiveness refers to care that is evidenced-based and adheres to established standards and the extent to which a specific intervention, procedure, regimen or service does what it is intended to do for a specified population when deployed in everyday circumstances. of national health plan activities - This analysis should assess the strengths and weaknesses of different elements in the health system, including programs, sub-policies, and strategies, focusing on whether those elements have achieved the expected results and what changes may need to occur in order to reach higher levels of effectiveness. While the first two streams of analysis rely heavily on technical expertise, this analysis is grounded in a participatory dialogue that considers both the opinions of experts as well as those using the health system on a daily basis - the service providers and the population themselves. 

The situation analysis should be conducted by a core team of working groups comprised of relevant experts and stakeholders who have a sufficient understanding of the issue and are representative of all the categories of the population. ⁶² More information on establishing working groups, identifying the expertise required for situation analysis, and sequencing of work can be found in the WHO chapter on situation analysis of the health sector.

Priorities are based on explicit criteria

The national health planning process always includes priorities. These priorities may be explicitly set or ad hoc. Effective priority setting is explicit, meaning it involves a transparent discussion among diverse stakeholders who determine priorities based on a joint examination of the evidence and explicit criteria. Priority setting The process of making decisions about how best to allocate limited resources to improve population health. that is ad hoc does not encourage accountability and is prone to biases or special interests. The WHO recommends the following steps for an explicit priority setting process: ²⁶³

1. Adopt a clear mandate for the priority setting exercise
2. Define the scope of the priority setting exercise and who will play what role
3. Establish a steering body and a process management group
4. Decide on approach, methods, and tools
5. Develop a work plan for priority setting and assure the availability of the necessary resources
6. Develop an effective communication strategy
7. Inform the public about priority setting and engage internal/external stakeholders
8. Organize the data collection, analysis, and consultation/deliberation processes
9. Identify or develop a scoring system
10. Adopt a plan for monitoring and evaluating the priority-setting exercise
11. Collate and analyze the scores
12. Present the provisional results for discussion; adjust if necessary
13. Distribute the priority list to stakeholders
14. Assure the formal validation of recommendations of the priority setting outcome
15. Plan and organize the follow-up of the priority setting, i.e. the decision-making steps
16. Evaluate the priority setting exercise

This process should result in a set of priorities, ranked by what is considered to be the most important based on the established criteria. Criteria are a set of measures that stakeholders use to weigh and determine which health problems, challenges, and solutions should be made a priority. These criteria should be defined before starting the priority setting process and be the basis for final priority setting decisions. In order to make high-quality PHC a priority, stakeholders need to define the principles that drive high-quality PHC (such as equity, efficiency, and sustainability) and set priorities for their health system based on these principles. ⁶⁴

These principles will inform how stakeholders evaluate criteria relative to each other when considering what is politically feasible, affordable, and technically possible. ⁵⁷⁶⁵ The WHO suggests a non-comprehensive list of five criteria that can be used to set priorities in the health sector. These include:

• The burden of disease: The burden of disease is a quantitative, time-based measure that combines years of life lost due to premature mortality and years of life lost due to time lived in states of sub-optimal health (i.e. injury, disease). 
• The effectiveness of the intervention: This criterion evaluates how well the identified health issue can be addressed (clinically or practically) by the given intervention, including if the intervention is applicable and cost-effective for the local context. 
• Cost of the intervention: This criterion considers the cost of an intervention in terms of affordability and efficiency. It is important to consider the absolute and relative costs to the health sector, target community, and individuals. The cost of an intervention must be both economically feasible and sustainable.
• Acceptability of the intervention: This criterion refers to whether the target community or population accepts the chosen health intervention, taking into account the social and cultural norms as well as the willingness of providers or other health authorities to carry out the intervention (i.e. risk aversion, resistance to change, perceived value). This criterion strongly relates to the applicability and feasibility elements of the effectiveness criteria; both require contextual knowledge to evaluate the intervention.
• Fairness: Fairness is a value judgment made collectively by governments and society based on the principles of equality and equity. The fairness criteria are essential to make well-informed judgements about tradeoffs on the importance of a health need and the effectiveness of an intervention. It also influences how much weight to give to the cost of a solution. For example, it might be important to prioritize the health problems of a specific at-risk or marginalized segment of the population, even if the intervention is not particularly cost-effective. An evaluation of fairness can help direct resources to marginalized populations, even when the intervention is not the most cost-effective solution.

Often, stakeholders will have to make trade-offs between different criteria; for example, stakeholders may consider both equity and cost-effectiveness in the evaluation of a given health intervention and find that the intervention that is the most equitable might not be the most cost-effective. The weight given to different criteria is ultimately a political decision shaped by the country's context, including values, principles, and economic and political environment. ⁵⁷

While the above criteria are a strong starting point for priority-setting conversations, new criteria may need to be added or adapted based on contextual factors, such as the epidemiological and demographic profile of the country, the health system structure, and political and financial capital. ⁵⁷⁶⁶ Accordingly, local needs and norms will influence the relative weight attached to these different criteria. The analyses of these criteria will depend on both the quality of the data and information available - including information on the implementation of interventions.

Priorities are accountable to diverse stakeholder interests

Effective priority setting addresses the differing interests and motivations of stakeholders through a clear process focused on the use of evidence, transparency, and participation to identify the most appropriate programs and interventions to address population health needs. ⁵⁷

Priority setting The process of making decisions about how best to allocate limited resources to improve population health. is a shared and multisectoral responsibility that relies on participatory and inclusive stakeholder engagement, including both people who will be affected by decision-making and people who can influence the implementation of the selected priorities during the priority-setting process. ⁵⁷⁶⁷ Stakeholder engagement plays an important role in priority setting because it ensures that priorities reflect population needs and that the interventions and programs selected are acceptable, appropriate, and desired. ¹⁴ Stakeholder engagement should be systematic, meaning the processes for identifying, communicating with, and convening stakeholders are transparent and consistent, with engagements occurring at regular, predefined intervals as well as on an ad-hoc basis, as necessary. Opportunities should be made for citizens to play an active role in shaping the priority-setting agenda, including through citizen consultations and community leader involvement in decision-making processes. 

The World Health Organization identifies three categories of stakeholders that should be involved in priority setting: ⁵⁷⁶⁸

• Government: The role of the government is to plan, initiate, coordinate, and oversee the priority-setting process within and across stakeholders and organizations. The way in which government stakeholders coordinate the priority setting process and who specifically will engage depends on the economic and political environment of the health system. For example, decentralized environments may need to collaborate more with local governments and providers whereas highly aid-dependent contexts may involve more collaboration with development partners. High-level actors may include policy-makers and planners in the Ministry of Health and other ministries as well as administrative and health authorities at decentralized levels.
• Providers: Service delivery providers are important stakeholders because they can offer insights into the feasibility of prioritized service delivery decisions, including balancing patients’ needs and demands with cost-effectiveness. Provider-level actors may include health professionals in both the public and non-public sectors. 
• Clients/citizens: To ensure stakeholders are accountable for their decisions, citizens should be involved in determining which priorities are set as a part of a democratic process. Citizens should be well-informed in advance about the advantages and disadvantages of various options. Citizen-level actors may include citizens themselves, community representatives, and/or groups of patients. Particular attention should be given to ensuring a diverse and representative group of citizen-level actors in this process. 

Stakeholders must be willing to continue participating in the process and accept priority-setting decisions, even if they disagree with the outcomes. ⁵⁹ Effective stakeholder engagement relies on robust institutional frameworks for multisectoral action and social accountability strategies. ⁵⁷ More information on multi-sectoral engagement and social accountability can be found on the relevant tools and resources page for priority setting.

Priorities are aligned with health sector policy, planning, & review processes

While priority setting for PHC is an integral part of improving population health for all, it must be supported by strong governance, political, and financial commitment as well as regulation and implementation capacities to achieve priority setting goals. ⁶⁴

After the priority setting process, relevant stakeholders will need to translate priorities into the strategic and operational plans for the health sector, followed by costing and budgeting, implementation, and finally, monitoring and evaluation (covered in the Monitoring and Evaluation deep dive). ² Multiple tools exist to assist with the planning and implementation of health interventions set as a part of the priority process, including Partners in Health’s UHC Monitoring and Planning tool. ⁶⁹⁷⁰

More information on resources allocation and planning can be found in the WHO’s chapters on estimating cost implications of a NHPSP, budgeting for health, monitoring, evaluation and review of NHPSP, and strategizing for health at sub-national level. 

Monitoring & evaluation (M&E) is a process by which stakeholders collect, measure, and use data to assess and maximize the impact of projects, programs, or social initiatives over time. It seeks to answer the question--is the project, program, &/or initiative going according to plan? If not, why? And what changes are needed to maximize impact? As opposed to surveillance, M&E is a more “passive” process, whereby the data collected on any disease or intervention is monitored on a weekly, monthly, quarterly, or even yearly basis depending on the project, program, or initiative. It involves two interrelated processes: ^432717273

• Monitoring, which is an ongoing process of collecting and analyzing data on specified indicators to track and measure how a change is happening. This data is used to plan, monitor, and improve projects, programs, &/or initiative activities. 
• Evaluation, which is a process that builds on monitoring. It enables stakeholders to check whether a project, program, &/or initiative has met its objectives. It uses a systematic assessment to measure how successfully the change had the intended impact. An evaluation provides a more balanced interpretation of a project, program, or social initiative’s performance. It should produce evidence-based findings, recommendations, and/or lessons that are credible, reliable, and useful. This should inform future decisions regarding the project, program, or social initiative. 

Below, we summarize the key attributes of a well-functioning system. For guidance on how to build one, see the following actions (take action section):

• Use data to set priorities & tailor response efforts
• Use M&E to track, measure, and improve priority actions and response efforts

M&E is coordinated & harmonized

Many countries already have mechanisms/systems for M&E in place, however, they are not necessarily coordinated across departments, programs, and sectors. Weak coordination of M&E processes (frameworks, indicators, and reporting systems) can lead to insufficient and/or poor quality data collection, analysis, use, and dissemination. This can subsequently limit the availability and use of reliable, accurate data for decision-making and performance improvement efforts. ³⁷⁴⁷⁵

To better coordinate M&E processes countries should harmonize them under a single country-led platform. However, the transition to a single platform will be an ambitious goal for many countries. This will often be the case in countries that have many parallel government and development partner/donor M&E systems. It will also be a challenge in contexts that lack a central ministry or agency for M&E. Thus, improving coordination between M&E platforms will often be the first step. ¹⁹²⁰

To learn more about the actions countries can take to improve the coordination of M&E processes, see the following action: use data to set priorities & tailor response efforts (institutionalize, coordinate, and harmonize HIS and M&E systems activity).

M&E is aligned with the NHS

A well-functioning M&E system can weather changes in administrative or governmental officials. This means that the M&E system is firmly institutionalized in core government processes, including budget cycles. A supportive institutional environment is characterized by:

• Country-led mechanisms for M&E, such as a technical subcommittee for M&E within the MoH
• Clearly defined roles and responsibilities for key institutions and stakeholders
• Well-functioning data sources including civil registration and vital statistics systems, facility surveys, and logistics management systems, among others (key data sources are discussed in the information and technology module)
• Capacity strengthening in the areas of data collection, management, analysis, use, and dissemination (also covered in the information and technology module)

In addition, countries should include a detailed M&E plan in the NHS. Because the mid-and final-evaluation results of the NHS will likely inform future iterations of the NHS (i.e. the final evaluation results are used in the initial situation analysis for the next NHS), it is critical that the NHS and its M&E component are developed via a transparent, evidence-based, and participatory process. ³

The M&E plan should outline the objectives of the NHS, what activities will be implemented to achieve these objectives, and what procedures will be used to evaluate whether or not these objectives have been met. It should also specify the resources required to evaluate and implement the plan. While an M&E plan should be developed at the beginning of health plan reform or design, it may need to be iterated on as country needs and priorities evolve. ^17327677

For additional guidance on planning for M&E, visit the following resources:

• United Nations Development Programme’s handbook on planning, monitoring, and evaluation for development results
• USAID’s core module on developing an M&E work plan
• The Evaluation Toolbox step-by-step guide to creating an M&E plan
• MEASURE Evaluation’s best practices in M&E plan development

M&E is built on a logical, evidence-based framework

The M&E platform should be built on a comprehensive, logical framework. An M&E framework describes the indicators that will be used to measure the level of success of the NHS and related initiatives. This framework should address indicator selection, related data sources, and analysis and synthesis practices, including quality assurance, performance review, dissemination, and use. The M&E framework is clarified and operationalized through the country’s M&E plan (discussed above). ^3177778 

To achieve strong PHC systems, the framework should include a well-balanced and well-defined set of core indicators for PHC that can be tracked and measured at the national and sub-national levels. Countries are encouraged to use the WHO’s PHC monitoring framework and menu of indicators to develop an M&E framework and plan that is tailored to their local context. ¹⁷

M&E includes mechanisms for review & action

The M&E platform should also include mechanisms for regular data dissemination and communication. This implies the existence of a regular and transparent system for progress and performance reviews, by which diverse stakeholders assess the quality and independence of data against agreed-upon targets (i.e. via data auditing). This process should be equipped to cover the entire health system, including all major disease programs. Relevant quantitative and qualitative data would be synthesized and made publicly accessible through country health ‘observatories’ or intelligence portals to monitor country health progress and performance. Ideally, this data would also be incorporated into future decision-making processes in the health sector, including resource allocation and financial disbursement. ³⁷⁵

Innovation and learning (I&L) is a characteristic of a health system that enables flexibility and iteration in order to continuously improve services and ultimately drive improved health outcomes. ⁵³⁷⁹ The goal of innovation and learning is to stimulate and make use of new and existing evidence, research, and data and to adapt and incorporate these learnings into changes at scale. ⁵⁶ This requires routine incorporation of new evidence from research or data and routine reviews and discussions of progress and challenges so that lessons from past events are identified and can be used to predict and/or improve response to future threats or changing health needs. ⁷⁹ While countries will develop their own ways of innovating and learning, it is measured by innovation capacity. 

Innovation capacity Innovation capacity refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves and supporting these innovations from ideation to scale. Innovation capacity Innovation capacity refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. comprises two specific functions: knowledge creation and knowledge adaptation. ⁶⁷⁹

• Knowledge creation is the process of generating new knowledge and technologies
• Knowledge adaptation involves creating new value by accessing, anchoring, and spreading existing global and local knowledge 

Developing innovation capacity at the national level relies on a commitment to innovation and learning as a priority investment through integration in national strategies, regulatory structures, dedicated budgets, and multisectoral stakeholder engagement. ⁶⁵⁵⁸⁰ The key components of strong in-country innovation capacity include:

A system-wide culture of innovation and learning 

The culture for innovation describes the specific values, behaviours, and processes that an organization or system takes to promote continual innovation. ⁵⁴⁸¹ The culture for innovation in an organization is a strong factor in determining whether a change (whether small, medium or large) can occur. It is closely linked with organizational readiness, or the organizational commitment to and efficacy in implementing innovations, and spread, which dictates the overall impact of an innovation. ⁵⁰⁸² Users can read more about creating a culture for innovation in healthcare here.

Processes for recognizing, evaluating, and scaling innovations

While the exact processes for innovation and learning will depend on the country's context, it is essential to ensure there is a systematic, structured, and reliable method in place for evaluating evidence and operationalizing these learnings into changes at scale. ⁶ This requires mechanisms to recognize and evaluate innovations as well as the operational capacity to scale identified interventions and programs:

• Recognition: Mechanisms should enable key stakeholders to recognize where opportunities for innovation and learning exist. This includes the identification of information, practices, processes, structures, or other products that could serve as sources of inspiration for new ideas or tools. The relevance of different information or technologies depends on user needs, the policy and market environment, trends and future projections, and the strengths and competencies of an organization. ⁵²
• Evaluation: Evaluation mechanisms (i.e. M&E systems) should enable stakeholders to track and measure the progress and performance of innovations, including their relevance, feasibility, and utility for the local context. If the innovation was not developed locally, stakeholders should also review lessons from other contexts (i.e. barriers and facilitators to implementation). 
• Scale: Scale refers to the process of spreading successful innovations within or among organizations to achieve a greater level of impact. ⁸³ However, scale does not necessarily imply that the innovation should/will be implemented at all levels of the health system. The level of scale chosen will ultimately depend on what is most impactful/relevant to the local context. For example, innovators may decide between increasing surveillance functions of a particular disease using a novel technology nationwide or decide on district-by-district increases depending on what is most appropriate in their country's context. 

Diverse stakeholder engagement and buy-in

The broad involvement of stakeholders supports an overall culture for innovation, and clearly defined roles help ensure that mechanisms for recognizing success and subsequent scaling are not dependent on specific individuals or relationships between individuals, but formalized into a system. Taken together, broad stakeholder engagement paired with social accountability helps to ensure the benefits of innovations are realized and sustained throughout the broader system. ⁷⁹ It is important that these decision-making processes are aligned with national strategic planning and budgetary cycles and take into consideration local norms, needs, and values in order to incorporate innovations in a sustainable and acceptable manner. ⁵

Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. is the “continuous, systematic collection, analysis, and interpretation of health-related data”. ³¹  Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. systems are dynamic and multimodal networks that combine monitoring and response activities to enable a country to identify and respond to emerging and existing threats and continuously assess and respond to communities’ needs over time. ¹¹⁸⁴

Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. is comprehensively addressed in the information and technology module. This module focuses its discussion on the use of surveillance data (and other information generated from health information systems) to support the aforementioned mechanisms/systems (priority-setting, M&E, and innovation and learning). 

• Deliberative process A deliberative dialogue or process is a “process of collective and procedural discussion where an inclusive and representative set of stakeholders consider facts from multiple perspectives, converse with one another to think critically about options, and through reasoned argument refine and enlarge their perspectives, opinions and understandings.” : In the context of priority setting, a deliberative dialogue or process is a “process of collective and procedural discussion where an inclusive and representative set of stakeholders consider facts from multiple perspectives, converse with one another to think critically about options, and through reasoned argument refine and enlarge their perspectives, opinions and understandings.” ⁶¹
• Explicit criteria Criteria are a set of measures that stakeholders use to weigh and determine which health problems, challenges, and solutions should be made a priority. These criteria should be defined before starting the priority setting process and be the basis for final priority setting decisions. : Criteria are a set of measures that stakeholders use to weigh and determine which health problems, challenges, and solutions should be made a priority. These criteria should be defined before starting the priority setting process and be the basis for final priority setting decisions. 
• Innovation capacity Innovation capacity refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. : Innovation capacity Innovation capacity refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity Innovation capacity refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale. Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. comprises two specific functions: knowledge creation and knowledge adaptation.
• Innovation & learning: a characteristic of a health system that enables flexibility and iteration in order to continuously improve services and ultimately drive improved health outcomes.
• Health data “In public health, “data” usually refers to statistics reported from health care facilities, survey data or data collected through observational studies. Distinctions can be made between routinely reported data and data that are collected at certain times or over a specific period of time as part of a special study or survey. Both routine and non-routine data, as well as data from research systems, are required and contribute to a  fuller picture of any given public health issue.” : “In public health, “data” usually refers to statistics reported from health care facilities, survey data or data collected through observational studies. Distinctions can be made between routinely reported data and data that are collected at certain times or over a specific period of time as part of a special study or survey. Both routine and non-routine data, as well as data from research systems, are required and contribute to a  fuller picture of any given public health issue.” ¹⁵
• Health research “Health research has been broadly defined as the generation of new knowledge using the scientific method to identify and deal with health problems. It encompasses a wide variety of branches and methods of research, including: biomedical; clinical; public health; basic; applied; researcher driven; health system driven; quantitative; and qualitative.” : “ Health research “Health research has been broadly defined as the generation of new knowledge using the scientific method to identify and deal with health problems. It encompasses a wide variety of branches and methods of research, including: biomedical; clinical; public health; basic; applied; researcher driven; health system driven; quantitative; and qualitative.” has been broadly defined as the generation of new knowledge using the scientific method to identify and deal with health problems. It encompasses a wide variety of branches and methods of research, including: biomedical; clinical; public health; basic; applied; researcher driven; health system driven; quantitative; and qualitative.” ⁸
• Information systems for health “A health information system (HIS) is broadly defined as a system that integrates data collection, processing, reporting, and use of the information necessary for improving health service effectiveness and efficiency through better management at all levels of health services. It encompasses all health data sources including health facility and community data; electronic health records for patient care; population-based data; human resources information; financial information; supply chain information; and surveillance information, along with the use and communication of this information.” : “A health information system (HIS) is broadly defined as a system that integrates data collection, processing, reporting, and use of the information necessary for improving health service effectiveness and efficiency through better management at all levels of health services. It encompasses all health data sources including health facility and community data; electronic health records for patient care; population-based data; human resources information; financial information; supply chain information; and surveillance information, along with the use and communication of this information.” ⁸⁵
• M&E: processes by which stakeholders collect, measure, and use data to assess and maximize the impact of projects, programs, or social initiatives over time. 
• M&E platform: “M&E platform” is used to describe various mechanisms used to support national and subnational monitoring, evaluation, and review efforts. The primary aim of the platform is to better align and reduce duplication of these efforts among key partners (i.e. governments, donors, and development organizations, among others). To simplify, we will use M&E system and M&E platform interchangeably in this module. 
• Priority setting The process of making decisions about how best to allocate limited resources to improve population health. : the process of making decisions about how best to allocate limited resources to improve population health.
• Surveillance The ongoing and systematic collection, analysis, and interpretation of health-related data essential to the planning, implementation, and evaluation of service delivery and public health. : a core public health function which involves the “continuous, systematic collection, analysis, and interpretation of health-related data”.