2024 Edition

Performance and Quality Improvement Introduction

Purpose

An organization-wide performance and quality improvement system uses data to promote efficient, effective service delivery and achievement of the organization’s mission and strategic goals.

Introduction

COA’s Performance and Quality Improvement (PQI) standards provide the framework for implementation of a sustainable, organization-wide PQI system that increases the organization’s capacity to make data-informed decisions that support achievement of performance targets, program goals, positive client outcomes, and staff and client satisfaction. Building and sustaining a comprehensive, mission-driven PQI system is dependent upon the active engagement of staff from all departments of the organization, persons served, and other stakeholders throughout the improvement cycle.

Note: Please see the PQI Toolkit for additional guidance on these standards.


Note: Please see the PQI Reference List for the research that informed the development of these standards. 


Note: For information about changes made in the 2020 Edition, please see PQI Crosswalk.

2024 Edition

Performance and Quality Improvement (PQI) 1: Infrastructure

The organization’s PQI system has the capacity to:
  1. evaluate services at all regions and sites;
  2. identify organization-wide and program-specific issues; and
  3. implement solutions that improve overall effectiveness.

1
The organization's practices fully meet the standard, as indicated by full implementation of the practices outlined in the PQI 1 Practice standards. The PQI system has sufficient structure, defined procedures, and resources to ensure its long-term sustainability.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 1 Practice standards; e.g.,
  • The plan and procedures are sufficient to implement and sustain a PQI system.
3
Practice requires significant improvement as noted in the ratings for the PQI 1 Practice standards; e.g.,
  • A PQI plan and procedures have been developed but several areas outlined in the PQI Practice standards are not adequately addressed or a few are not addressed at all; or
  • The PQI system, as reflected in the plan and procedures, does not appear to be sustainable.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 1 Practice standards; e.g.,
  • A PQI system has not been developed, or it is wholly inadequate.
 

PQI 1.01

A written PQI plan and procedures cover each program or service area and, if necessary, outline any variances between regions or sites, and:

  1. articulate the organization's approach to quality improvement and methods used;
  2. describe the PQI system's structure, functions, and activities;
  3. define staff roles and assign responsibility for implementing and coordinating the PQI program (PQI 2);
  4. identify what is being measured and why (PQI 3, PQI 4, Service Standards); and
  5.  include procedures for reporting findings and monitoring results (PQI 5).

Examples: The PQI plan describes how the system is structured and functions, includes an overview of the organization's approach to quality improvement, and may include specific models and/or methodologies it may employ (e.g., Six-Sigma, CQI, Plan/Do/Check/Act, and TQM).

PQI Structure: There are many ways to structure how information and data flow through an organization, mechanisms for review, and decision-making. Many organizations integrate PQI responsibilities into their existing decision-making and support structure, e.g., management teams, committees, or task forces. Others establish a separate, independent PQI committee to oversee and guide their PQI system.

Some small organizations may not have the resources to have a separate PQI structure or committee so they are diligent about including PQI as part of the agenda of regular staff meetings (see PQI 1.04). In effect, the entire staff serves as the PQI committee. Please note that it is especially important to thoroughly document PQI discussions in this scenario.

In regards to element (e), procedures for reporting findings and monitoring results can include:
  1. obtaining feedback about findings from stakeholders;
  2. taking action in response to PQI findings and feedback;
  3. monitoring improvement plans and corrective action plans; and
  4. determining if an implemented change is an improvement.

Note: In regards to element (d), please see the Person-Centered Logic Model Core Concept in each assigned Service Standard for additional information on program outputs and client outcomes to be included in the PQI plan.

1
The written PQI plan provides the organization with a framework for operationalizing and implementing a comprehensive PQI system and includes all of the elements of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The PQI plan and procedures do not cover one or two of the organization's programs or one of its service delivery sites, divisions, or departments, but the organization is actively working to integrate these into their plan and procedures; or
  • One of the elements is not fully addressed.
3

Practice needs significant improvement, e.g.,

  • More than two of the organization's programs or service delivery sites, regions, or divisions are not integrated into the organization-wide plan and procedures; or
  • Two of the elements are not fully addressed; or
  • The PQI plan lacks specificity and it is unclear what is being measured for each program or service area; or
  • One element is not addressed at all.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 1.02

The PQI plan:

  1. defines the organization's stakeholders; and
  2. specifies how important internal and external stakeholder groups will be involved in the PQI process.
Related Standards:
Interpretation: Stakeholder involvement is fundamental to a well-designed, useful PQI system. Ideally, a broad range of internal and external stakeholders including staff from all levels of the organization, the organization's governing body, persons served, and other external stakeholders have a role in the organization's PQI system.

Examples: Examples of stakeholders include:
  1. staff;
  2. governing body members;
  3. persons served, including families, as appropriate;
  4. volunteers;
  5. licensing authorities;
  6. consumer advocates;
  7. funders; and
  8. contractors and partners.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Most of the important internal and external stakeholders have been identified; or
  • Procedures for involving stakeholders lack specificity regarding how some stakeholder groups will be meaningfully involved.
3
Practice needs significant improvement; e.g.,
  • Written documentation does not address involving clients or other external stakeholders; or
  • Written documentation provides only minimal guidance about how stakeholders will be involved.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 1.03

The PQI plan describes how:

  1. staff and their supervisors have timely access to the information they need to clarify expectations and implement practice improvements; and
  2. staff at all levels receive relevant information on PQI findings.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Plan/procedures lack specificity regarding the flow of information.
3
Practice needs significant improvement; e.g.,
  • Plan/procedures provide only minimal guidance.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 1.04

Organization leaders, senior managers, program directors, and supervisors:

  1. keep PQI on the agenda of board, management, and staff meetings;
  2. regularly evaluate the need for and uses of data; and
  3. evaluate the PQI system, infrastructure, processes, and procedures.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Leaders, senior managers, etc. are committed to maintaining a sustainable PQI system, but practice related to one of the standard's elements needs improvement.
3
Practice requires significant improvement; e.g.,
  • Leaders, senior managers, etc. do not consistently put in the effort and attention needed to sustain the organization's PQI system, as indicated by limited implementation of two of the standard's elements.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
2024 Edition

Performance and Quality Improvement (PQI) 2: Roles and Responsibilities

The organization has sufficient qualified staff, representing different departments and levels of the organization, to conduct and sustain its PQI system.
Interpretation: COA does not expect all staff to be involved in PQI.

1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 2 Practice Standards. Staff have the knowledge and experience needed to implement and coordinate the PQI system, including the ability to implement evaluation methods, as per the requirements of the standard.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 2 Practice standards; e.g.,
  • Identified staffing and training deficiencies do not significantly compromise the organization's ability to implement its PQI system or sustain it over time; or
  • Job descriptions reflect the required competences, and the organization seeks to hire and/or assign or train people with the requisite skills.
3
Practice requires significant improvement as noted in the ratings for the PQI 2 Practice standards; e.g.,
  • The organization's inability to hire or train staff is presenting a serious challenge to its ability to implement and sustain a PQI system.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 2 Practice standards.
 

PQI 2.01

Staff responsible for implementing and coordinating the organization's PQI system are competent to:

  1. identify indicators of quality practice;
  2. implement internal and external evaluation methods, such as benchmarking, as appropriate to the programs being evaluated;
  3. ensure proper data entry and data integrity;
  4. collect, analyze, and interpret data; and
  5. communicate evidence and findings to staff in a manner that facilitates their active engagement.
Interpretation: PQI may be a shared responsibility as opposed to being under the leadership of a single staff position.

Examples: Organizations that have limited resources or are new to measuring performance can partner with colleges or universities or other organizations to gain access to knowledge and expertise related to setting up and sustaining their PQI system, collecting and analyzing data, etc.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • PQI staff have the competency and/or support needed to implement and coordinate the PQI system, but one of the elements is not fully addressed.
3
Practice needs significant improvement; e.g.,
  • PQI staff are not sufficiently competent and/or supported to implement and coordinate the PQI system, e.g., one element is not addressed at all, or two of the elements are not fully addressed.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 2.02

Staff receive support, as appropriate to their responsibilities, on:

  1. inputting data into the data management system;
  2. using data collection tools and forms;
  3. reading and interpreting reports; and
  4. using data to improve performance.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Most staff receive the support they need.
3
Practice needs significant improvement, e.g.,
  • Staff support is insufficient for at least two of the standard's elements; or
  • Support is not being provided for either element (a) or (b); or
  • The integrity of the data may be compromised due to insufficient staff support.
4
Implementation of the standard is minimal or there is no evidence of implementation at all; e.g.,
  • Staff are not supported and cannot demonstrate competence, and further support is not being provided.
2024 Edition

Performance and Quality Improvement (PQI) 3: Performance and Outcomes Measures

The organization identifies measures and outcomes related to:
  1. the impact of services on clients;
  2. quality of service delivery; and
  3. management and operations performance.

Examples: Organizations providing child welfare services are encouraged to integrate the Federal Child and Family Service Review (CFSR) Outcomes measures and Systemic Factors, particularly those identified in Performance Improvement Plans, into their overall PQI system.

1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 3 standards.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 3 Practice standards.
3
Practice requires significant improvement as noted in the ratings for the PQI 3 Practice standards.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 3 Practice standards.
 

PQI 3.01

The organization identifies key outputs and individual outcomes for each program or service area, and related:

  1. measurement indicators;
  2. performance targets; and
  3. data sources including data collection tools or instruments for each identified output and outcome.
Related Standards:

Interpretation: Organizations are encouraged to use standardized or recognized outcomes evaluation tools when available and appropriate.

 

Interpretation: Program outputs and individual outcomes must be identified in the logic model submitted in the Person-Centered Logic Model Core Concept in each assigned Service Standard.

Examples:

Outputs are what the program delivers. Examples of program outputs include:

  1. number of educational or clinical sessions provided;
  2. total number of clients served over a specified period; and
  3. number of housing placements made.

 Outcomes are the observable and measurable effects of a program's activities or interventions on its service recipients. Examples include:

  1. improved functioning as measured by the Children's Functional Assessment Rating Scale (CFARS);
  2. number/percent of homeless and runaway youth that are reunited with family during the past quarter;
  3. reduction in criminal justice system involvement; and
  4. improved family/community involvement.

 For some programs, outcomes, outputs, indicators, tools, etc. may be established by contractual and/or funding requirements.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization has not developed indicators or performance targets for some of its programs.
3
Practice needs significant improvement; e.g.,
  • At least one of the standard's elements are not being addressed at all; or
  • Outputs and outcomes have not yet been identified for one of its high-risk programs.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 3.02

The organization surveys clients annually to assess program quality.
Related Standards:

Examples: Types of information that the program may collect from clients can include client satisfaction or outcome information.

According to the Urban Institute, client surveys can be an indispensable source of outcome information. They provide a systematic means of gathering data on service outcomes from all or a portion of clients. Client surveys help organizations learn whether services are producing anticipated or desired results and, if not, provide clues for how to improve them.

Issues covered by a client survey should correspond to the key service outcomes an organization wishes to track. Because survey length generally affects response rates, issues not pertinent to improving outcomes should probably be limited. The goal is to develop the shortest possible list of questions consistent with the survey's objective of assessing outcomes.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement.
3
Practice needs significant improvement.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 3.03

The organization identifies measures for management and operational performance to:

  1. measure progress toward achieving its mission and strategic and annual goals;
  2. evaluate operational functions that influence the capacity to deliver services and meet the needs of persons served; and
  3. identify and mitigate risk.
Related Standards:

Examples: Examples of operations and management performance measures can include:
  1. efficiency in the allocation and utilization of its human and financial resources to further the achievement of organizational objectives;
  2. effectiveness of risk prevention measures;
  3. effectiveness at retaining a competent and qualified workforce through staff retention/turnover and satisfaction;
  4. costs versus benefits of fundraising efforts;
  5. achievement of budgetary objectives;
  6. effectiveness of community education and outreach; and
  7. efforts to diversify the governing body, leadership, or workforce.
Organizations may consider if any data is currently being collected related to these elements. Then, the organization may identify an outcome or goal in some of these areas.
 
Network Examples: Network management entities may also measure important network administrative processes, such as:
  1. the average length of time between receiving a clean claim and paying the claim;
  2. the proportion of services that are evidence-based or meet nationally recognized treatment guidelines developed by consensus groups;
  3. the effectiveness of network training;
  4. the satisfaction of stakeholders, such as high volume referral agents (e.g., judges, court workers, employee assistance agents);
  5. penetration rates, or the proportion of the whole population eligible to be served by the network who actually receive services; and
  6. results of retrospective case record reviews, including the percentage of cases in which a placement decision includes an appropriate application of clinical criteria.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization has identified performance measures related to two of the three elements of the standard.
3
Practice needs significant improvement; e.g.,
  • The organization has identified performance measures related to only one of the standard's elements.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 3.04

Findings and recommendations from external review processes are integrated into the organization's PQI system.

Examples: External reviews can include:
  1. licensing and other reviews related to federal, state, and local requirements;
  2. government and other funder audits;
  3. accreditation reviews; and
  4. other reviews, where appropriate.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The process for review of findings and recommendations can be improved, e.g., while findings are reviewed by management, they are not integrated into the PQI improvement cycle when appropriate.
3
Practice needs significant improvement; e.g.,
  • There is evidence that the organization has not adequately addressed the findings or recommendations of at least one key external review; or
  • It does not review or address findings in a timely manner and thus may be putting itself at risk of sanction.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
2024 Edition

Performance and Quality Improvement (PQI) 4: Case Record Review

The organization conducts case record reviews at least quarterly for each of its services to:
  1. minimize the risks associated with poorly maintained case records;
  2. document the quality of the services being delivered; and
  3. identify barriers and opportunities for improving services.
Interpretation: COA is not prescriptive about who can conduct case record reviews. While a peer review model is recommended, it is acceptable for PQI staff, a consultant, or another person or combination of persons to conduct the reviews. Please note that, regarding PQI 4.03, persons with clinical or service delivery experience may be needed to obtain the relevant qualitative data from the case records.

NA The organization is a network management entity assigned the Network Administration (NET) standards that does not provide any direct services to individuals served by the network and is not being reviewed under any Service Standards.

 

NA The organization is only assigned the Early Childhood Education (ECE) and/or Out-of-School Time Services (OST) standards.

 

NA The organization provides only non-clinical group, crisis intervention, and/or information and referral services.

Currently viewing: CASE RECORD REVIEW

Viewing: PQI 4 - Case Record Review
Previous: PQI 3 - Performance and Outcomes Measures Next: PQI 5 - Analyzing and Reporting Information

VIEW THE STANDARDS

Related Standards:

1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 4 Practice standards.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 4 Practice standards.
3
Practice requires significant improvement as noted in the ratings for the PQI 4 Practice standards; e.g.,
  • Case records may pose a risk to the organization and corrective action has not been implemented.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 4 Practice standards.
 

PQI 4.01

The quarterly case record review process:

  1. includes a random sample of both open and recently closed cases;
  2. uses uniform data collection tools to ensure consistency and permit comparison of data across similar programs and services; and
  3. maintains objectivity by ensuring that reviewers do not review cases in which they have been directly involved as a service provider or supervisor.

Interpretation: Sampling: See recommended sampling guidelines. Organizations may choose a different sampling method as long as a rationale is provided.

 

Closed Cases: COA does not define the percentage of closed cases that must be included in the sample. The majority of cases the organization reviews should be open, but the organization must include a sample of closed cases to evaluate documentation related to discharge planning, case closing, aftercare, and the condition of the case record including whether or not records have been expunged as required by PRG 1.06.

Examples: Organizations can get more from the case record review process by stratifying the random sample of open cases to account for length of service. For example, a program that serves clients for up to six months could divide the sample proportionally between cases that have been open less than one month, one to three months, three to six months, and more than six months.

For generating random numbers, the Research Randomizer is an easy to use tool that is made available for free by the Social Psychology Network and includes short tutorials.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • One of the standard's elements is not sufficiently developed, e.g., cases are not selected randomly for more than half of the organization's programs or services but includes both open and closed cases.
3
Practice needs significant improvement; e.g.,
  • At least two of the standard's elements are not sufficiently developed, e.g., sample size is insufficient to enable the organization to draw conclusions from the data; or
  • The organization only reviews open cases; or
  • Little effort is being made to ensure objectivity, e.g., supervisors frequently were the sole reviewers of supervisee cases.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 4.02

Quarterly reviews of case records evaluate the presence, clarity, quality, continuity, and completeness of required documents.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • One or two important documents are not included in the review; or
  • Procedures need strengthening.
3
Practice needs significant improvement; e.g.,
  • A number of important documents are not included in the review; or
  • Reviews are conducted no more than three times per year; or
  • Reviews are not conducted for one of the organization's services; or
  • The review process is poorly designed or haphazardly conducted; or
  • Case records may pose a risk to the organization and corrective action has not been implemented.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 4.03

The organization identifies indicators and measures the quality of services for each of its programs or services in its quarterly case record review process.

Examples: Quality of services is a very broad category and varies according to the program, service population, service mandates, and any number of other factors and can include criteria for evaluating the appropriateness and/or effectiveness of the services provided to persons served.

Examples of common qualitative measures include:
  1. timeliness and comprehensiveness of individualized assessments;
  2. length of service;
  3. need for continued service;
  4. family involvement; and
  5. achievement of service goals, etc.
 
Some organizations take a utilization management approach to case record review and, rather than review case records quarterly, conduct more frequent or ongoing reviews. A utilization management approach looks at the key decisions and process milestones including, for example:
  1. appropriateness of admissions and authorization decisions;
  2. intake and referral processes;
  3. service planning and service delivery milestones;
  4. need for continued service; and
  5. discharge decisions.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Indicators for one program need strengthening; or
  • Data is aggregated and used to monitor service quality for 75% of the organization’s programs/services including all high risk programs.
3
Practice needs significant improvement; e.g.,
  • Data is not consistently collected, or is collected and aggregated but not used to monitor service quality; or
  • Service quality data is collected for less than 75% of the organization's programs; or
  • Service quality data is not being collected for at least one high-risk program.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
2024 Edition

Performance and Quality Improvement (PQI) 5: Analyzing and Reporting Information

The organization systematically collects, aggregates, analyzes, and maintains data.

1
The organization's practices fully meet the standard as indicated by full implementation of the practices outlined in the PQI 5 Practice standards. Comprehensive PQI data management procedures support the organization's ability to systematically collect, aggregate, analyze and maintain data.
2
Practices are basically sound but there is room for improvement as noted in the ratings for the PQI 5 Practice standards.
3
Practice requires significant improvement as noted in the ratings for the PQI 5 Practice standards.
4
Implementation of the standard is minimal or there is no evidence of implementation at all, as noted in the ratings for the PQI 5 Practice standards.
 

PQI 5.01

Procedures for collecting, reviewing, and aggregating data include:

  1. cleaning data to ensure data integrity including accuracy, completeness, timeliness, uniqueness, and outliers;
  2. protecting personal identifiable information (PII) in data reports;
  3. aggregating data quarterly; and
  4. developing reports for analysis and interpretation.

Interpretation: Data should be collected, aggregated, and reviewed at least quarterly at all three levels of performance measurement as addressed in PQI 3.03, PQI 4, and the Person-Centered Logic Model Core Concept in each assigned Service Standard.


Interpretation: The aggregation of data reduces the risk of disclosing PII in most instances; however, risk of disclosure still exists particularly when data is being disaggregated and unique or easily observable characteristics might allow someone to be identified in the data set. As such, data collection and reporting procedures should include mechanisms for avoiding such disclosure such as data suppression, rounding, reporting in ranges rather than exact counts, combining sub-groups into larger groups, etc.

Examples: Cleaning data, also known as data cleansing, means checking for errors and inconsistencies in order to improve the quality of your data prior to aggregating and analyzing it. Common things to check for include:
  1. accuracy - making sure the data was recorded correctly including misspellings, correct numbers, addresses, etc.;
  2. completeness - making sure all the data was recorded and none is missing;
  3. timeliness - ensuring that the data is current and/or relevant to the current time frame;
  4. uniqueness - ensuring that data was recorded only once and not multiple times; and
  5. outliers - look for data that is unexpected (Note: This could mean you have a PQI issue that warrants attention but sometimes a single extreme result, even if it is legitimate, can tip the results so they are not truly representative).

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Procedures for ensuring data integrity and reliability are sufficient for sustaining the PQI system but need some improvement, e.g., formats for reports are not consistently useful for analysis; or
  • In a few instances, data was not aggregated and reviewed quarterly.
3
Practice needs significant improvement; e.g.,
  • Procedures are insufficient to sustain consistent data review or do not address one of the standard's elements; or
  • Only some of the collected data is reviewed and/or aggregated for review; or
  • Data is rarely aggregated into a form that permits analysis.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 5.02

The organization analyzes disaggregated PQI data to:

  1. track and monitor identified measures;
  2. identify patterns and trends; and
  3. compare performance over time.
Related Standards:

Interpretation: Organizations should disaggregate data to identify patterns of disparity or inequity that can be masked by aggregate data reporting. Common characteristics used to disaggregate data include:

  1. race and ethnicity/country of origin;
  2. generation status;
  3. immigrant/refugee status;
  4. age group;
  5. sexual orientation; and
  6. gender/gender identity.

1
The organization's practices reflect full implementation of the standard. The organization analyzes PQI data per the requirements of the standard.
2

Practices are basically sound but there is room for improvement; e.g.,

  • Data from across the organization is analyzed, but data is not analyzed for one of the organization’s programs; or
  • Data from across the organization is analyzed, but only some of the data has been disaggregated to identify patterns of disparate outcomes; or
  • Data analysis does not include one of the elements of the standard.
3

Practice needs significant improvement; e.g.,

  • Data from across the organization is analyzed, but no data has been disaggregated to identify patterns of disparate outcomes; or 
  • Most of the organization's PQI data has not been analyzed; or
  • Data analysis is not performed for most of the organization's programs or services; or
  • Data related to management and operational performance is not analyzed.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 5.03

Reports of PQI findings are:

  1. shared and discussed with board members, staff, and stakeholders; and
  2. distributed in timeframes and formats that facilitate review, analysis, interpretation, and timely corrective action.

Examples: Discussions with board members, staff, and stakeholders about PQI findings can include:
  1. areas of strength and quality practice;
  2. areas for improvement; and
  3. how to prioritize targeted areas, identify interventions, and monitor the effectiveness of interventions over time.
In order to engage in meaningful discussions about the data being collected, organizations should decide how results will be communicated to staff and stakeholders. Organizations can start by determining who needs what data, with what frequency, and how best to share the information.

Methods for sharing findings include:
  1. performance dashboards, report cards, or other types of summary reports;
  2. discussion at board, staff, and departmental meetings;
  3. using monthly reports of key service delivery outputs and outcomes in staff supervision activities;
  4. conducting focus groups and presentations at community meetings;
  5. soliciting feedback via interviews or surveys;
  6. providing quarterly reports to oversight entities, stakeholder advisory groups, and leaders on important data related to key operations and management functions; and
  7. quality review activities that engage community providers.
 Graphic presentation of data is very useful in communicating results of PQI activities. Data visualization techniques can facilitate understanding of complex information and reveal underlying patterns and relationships within the data that may otherwise go unnoticed.

1
The organization's practices reflect full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • Summary reports are created and distributed, but practice could be improved; or
  • Stakeholders have complained about reports that are hard to read or understand; or
  • Summary reports are not always distributed in a useful timeframe.
3
Practice requires significant improvement, e.g.,
  • There are many examples of relevant PQI data not being provided to stakeholders for review; or
  • Data is not formatted into reports; or
  • The format of reports is unclear and confusing; or
  • Confidentiality concerns have been raised or noted.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
 

PQI 5.04

The organization:

  1. reviews PQI findings and stakeholder feedback and takes action, when indicated; and
  2. monitors the effectiveness of actions taken and modifies implemented improvements, as needed.

Examples: Organizations can use PQI findings and feedback to:
  1. improve services;
  2. eliminate or reduce identified problems;
  3. replicate good practice;
  4. recognize and motivate staff; and
  5. improve organizational systems, processes, policies, and procedures.
Examples: Information generated by the PQI system can be used to:
  1. monitor progress toward achieving its mission and strategic and annual goals;
  2. meet funder requirements; and
  3. promote the organization and its services throughout the community.
Examples: Corrective Action Plans or Improvement Plans can be implemented when issues have been identified that will involve ongoing effort and monitoring.

Improvement Plans formally lay out the actions that will be taken to address areas in need of improvement that are identified by staff and stakeholders as crucial to meeting the organization's goals and delivering quality services. Improvement plans should be implemented when it is necessary to monitor and address the issue over time.

Corrective Action Plans are implemented to correct problems or deficiencies, including those related to compliance with regulatory requirements (e.g., Medicaid documentation requirements). The need for a Corrective Action Plan suggests that the issue has moved beyond program improvement to the level of oversight by the organization's leadership.
 
Organizations may also wish to create an annual summary report for oversight entities, stakeholders, and staff that includes:
  1. key PQI activities that are ongoing, have been resolved, or that need further intervention;
  2. issues that require continued monitoring within the PQI system; and
  3. PQI priorities and goals for the coming year.

1
The organization's practices fully meet the standard as indicated by full implementation of the standard.
2
Practices are basically sound but there is room for improvement; e.g.,
  • The organization uses PQI data to improve programs; however, some available findings and recommendations are not being used; or
  • Actions made in response to findings and feedback are being monitored, and modifications are made when needed, but practice could be improved, e.g., the data is not being reviewed in a timely manner.
3
Practice requires significant improvement, e.g.,
  • PQI data is not routinely used; or
  • Except for a few examples, the organization does not generate enough usable data to take meaningful action, or does not routinely use data in either of the ways listed in the standard; or
  • Important modifications are often not made despite evidence that they are needed.
4
Implementation of the standard is minimal or there is no evidence of implementation at all.
Copyright © 2024 Council on Accreditation