Assessments are the fuel that power both step-wise and continuous improvement engines and this is no different when it comes to risk & compliance programs. Assessments help to identify the gap between where you are now compared to where you intend or need to be. The problem is that not all assessments are the same, and many do not assess the things that really matter.
The Assessment Construction
The purpose of an assessment is to answer the question, "Are we there yet?" so that adjustments (i.e. improvements) can be made to successfully get to there.
What there means may manifest itself in many ways, for example:
The distance between where we are compared to our destination.
The speed we are travelling compared with what is needed to arrive at our destination on time.
The amount of fuel remaining compared with what is needed to get to our destination.
For an assessment to be effective we must have a stated goal that defines what we want to achieve along with a way to measure our progress towards that goal. In our previous example there are several goals:
Zero distance from our destination – this is a Measure of Effectiveness (MoE) The trip will not be successful if we don't reach our destination defined as a distance of zero between were we are and where we want to be.
Minimum speed to reach our destination on time – this is a Measure of Performance (MoP). We need to have the capability to travel at or above the minimum speed limit to reach our destination on time. We also need to sustain that level during the trip.
Non-zero fuel level for the duration of the trip – this is a Measure of Conformance (MoC). To be successful we must ensure that we do not run out of fuel. This may require extra fuel (i.e. margin) to address uncertainty along the way.
An effective travel plan will include a gap assessment conducted as often as needed to make certain that sufficient progress is made towards the desired destination, sustainment of the minimum speed, and that the fuel tank never reaches empty.
The Assessment Conflation
Taiichii Ohno (the father of LEAN) is known to have said, "without objectives there is no improvement," or in other words you need to have a gap, but not just any gap. The gaps you need to assess are the ones connected with your goals.
Every management system standard connected with quality, safety & security, environmental and regulatory programs requires the establishment of objectives. These are instrumental goals towards ultimate goals such as zero incidents, zero harm, zero emissions, zero pollution, zero violations, and so on. To make progress against these objectives each company must perform at targeted levels often described in internal procedures and guidelines.
A full evaluation of a management system will therefore include:
Conformance Assessments (i.e. audits) are conducted to assess work-as-prescribed against work-as-done demonstrated by evidentiary artifacts which may include intermediate or final outputs of critical to compliance processes covering: quality, safety & security, environmental,and regulatory objectives. Conformance assessments answer the question, "Is the system following the standard?"
Performance Assessments are conducted to verify actual performance against planned performance of critical to compliance capabilities. Performance assessments answer the question, "Does the system have the capability, competency, and capacity to advance objectives?"
Effectiveness Assessments are conducted to validate progress against stated compliance outcomes (ex. zero violations, zero harm, zero incidents, etc.) Effectiveness assessments measure the results of a compliance system by answering the question, "Is the system having an impact on our compliance outcomes?"
Audits are traditionally used to perform conformance assessments evaluated against a given standard. These standards are often externally defined with some requiring third-party certification. The purpose of the audit is to identify gaps against the standard that will need to remediated to achieve third-party certification or quality control criteria. This assessment focuses on confirming the existence of policies, procedures, and practices along with process outputs.
However, what audits do not do is tell you whether or not your compliance system is performing or effective at advancing your goals and objectives. The latter requires a contextual evaluation against a company's targeted outcomes of their quality, safety & security, environmental, and regulatory systems.
The Assessment Uncertainty
There is one place where most of have experienced the impact (good or bad) of assessments and that is in the context of education and training. As part of making the grade a teacher will conduct tests throughout the year to measure a students progress culminating in a final exam to determine the grade. Performance in the final exam is often the largest contributor to the final grade which can be considered an outcome of a student's effort during the year.
I remember during my university days sitting down for a final math exam. I felt good about this course. I had attended all of the classes, completed all the term exercises, and as a result felt ready for the final.
On the day of the exam, I sat down, took a deep breath and waited for the start signal to begin the three-hour exam. Two hours went by and I had managed to work through all the questions on the exam page. Having completed the questions comfortably, I thought that all my preparations were paying off.
I decided to use the remaining time to go over my answers and check my work. 50 minutes left. Everything was going as planned. I needed some scrap paper to verify some of my calculations and decided to use the back side of the exam paper. 45 minutes left.
As I turned over the paper, my heart sank and then started to race. I saw another set of questions on the back side. How could I have I missed this? My performance on the previous questions would not be enough to pass the course let alone make a reasonable grade to advance the achievement of my engineering degree.
Many organizations find themselves in a similar situation with respect to their risk & compliance programs. They do all the work to pass an audit, the front-side of the exam, but fail to turn the page over to work on performance and effectiveness.
Unfortunately, when it comes to compliance obligations such as safety, it is too late to work on these after an incident or harm has been done. As for my exam, it was almost too late for me. I raced through the rest of the questions and barely achieved a passing grade. I promised to never let that happen again.
The Assessment Conclusion
Gap assessments are necessary to know what to improve provided you are measuring what really matters.
While conformance assessments can tell you that you are missing a procedure and need to create one, they don't tell you how good a procedure is at achieving intended objectives. For this you will need to conduct performance and effectiveness assessments. These require contextualized comparisons between where you are now and the destination you have targeted which will be different for every organization.
Of course, closing the gaps will not be as simple as creating a procedure. Closing gaps will require improving performance and effectiveness which need the application of problem solving, risk analysis, and change management, among other skills and capabilities.
Progress towards risk & compliance objectives can only begin when companies turn the page to answer the other half of the questions. And remember, you still need to show your work!
It's time to do more than just pass the course, it's time to make the grade.