Energize your Internal Audit Program

Planning for the Internal Audit

The key to an effective, thorough and value added internal audit is in the preparation.  If internal auditors are spending one to two hours preparing for an internal audit, it is not enough time.  To properly prepare for an audit, it should take twice to three times that.  If the actual audit time will take an hour, there should be at between two and three hours spent in preparation.  A good rule of thumb to spend about two and half times as much time in preparation as the audit will take.  Often times, auditors plan for a two hour internal audit and spend 1 hour preparing which leads to them running out of questions about 30 minutes into the audit.  I can’t stress this enough if you want to be a successful internal auditor or manage a successful internal audit program then make certain you spend adequate time in preparation for the audit.

This sounds easy, but it is actually very difficult.  The major obstacles to allocating enough time for preparation are time restrictions placed on the internal auditors.  Chances are they have other responsibilities aside from internal auditing that compete for their precious time.  One method to help remove that obstacle is to have as many trained internal auditors as possible to spread the work load.

Effective planning for an internal audit requires following a few simple steps that are listed below.

1. Learn the process (turtle diagram)
2. Identify the interfaces with the standard
3. Document review (compliance to standard)
4. Identify process interfaces
5. Identify potential process failure modes (pFMEA)
6. Value stream map process to breakdown activities
7. Review old audits
8. Develop audit questions
9. Develop audit plan.

1. Learn the processBefore you can audit a process you must become familiar with it.  You need to learn how it is supposed to work, what it supposed to do, what are the inputs, outputs, activities, resources and controls.  The first step would be to create a turtle diagram of the process (This may have already been done by the organization as part of their documentation, or in previous audits).
A turtle diagram looks at the suppliers, inputs, activities, controls, resources, outputs, and customers.   A turtle diagram is laid out such that the process activity is a box in the middle, the inputs come in from the left and outputs exit from the right of the box.  The supplier is listed in the upper left hand corner and the customer is listed in the upper right hand corner.  The controls are above the process activity and the resources are below the process activity.  The feedback loop is an arrow from the output to the input.

Let’s do an example of a turtle diagram for a process.  For this example, the process will be one that applies to about every business in some way and that’s purchasing. 

Inputs:
This is what the process needs for the activity.  It can be in the form of information or a product.  For this example the inputs are: Demand (what is driving the purchase), Quantity, Type, Specifications and Requirements, Due date and Budget (how much can be spent).

Supplier:
This is who is supplying the inputs to the process.  The supplier can supply information or a material product.  For our example the supplier would be whoever is specifying what to purchase, when to purchase and how many to purchase.

Process Activity:
This is the process.  There are a number of associated tasks contributing to the process.  For our example the process activity is purchasing

Outputs:
This is the result of the process.  It can be information, energy or material.  In our example the output of the purchasing process is the desired product or service delivered when needed.  For our example it could be a product like a computer or piece of test equipment.  It could be information such as a failure analysis, training materials, book or manual.  It could also be a service such as mowing the grass, doing the laundry or processing payroll.

Controls:
These are the items that regulate the rate at which inputs are converted to outputs.  Without controls, the process would operate continuously generating the output.  The controls for our example could be the material requirements planning software, the purchase requisition approval process and inventory analysis.

Resources:
These are the items used or consumed in the process activity.  It could be people’s time, machine time or money.  For our example, the resources would be the buyer or purchasing agent, money, the representative for the company supplying the product or service and possibly other support functions who have input for the purchase.    Additional resources are in the form of computers, material planning software, phones, fax, office space, etc. 

Customer:
The customer is the group that takes the output and uses it.  It is most likely used as an input to another process or as a resource. 

Feedback Loop:
This is the mechanism used to monitor the process.  What metric is used to tell the process owner how the process is performing and when action needs to be taken to correct it.  For a purchasing process it could be supplier performance, dollars spent, on-time delivery or receiving inspection information.
2. Identify the Interfaces to the Standard

The interfaces are the points where the process intersects the standard.  In simple terms it is where the requirements of the ISO 9001:2000 standard are applicable to the process being audited.  The easiest way to accomplish this is to use a matrix with the elements of the standard on one axis and the process name on the other. 
 
To better discern the interfaces of the process to the standard you could break the elements down into the sub elements.  For example, 7.2 Customer Related Processes is comprised of 7.2.1 Determination of requirements related to the product, 7.2.2 Review of requirements related to the product and 7.2.3 Customer communication.   The left side of the matrix would become larger, but you would have a more definitive intersection of the process and standard.  This activity provides you with the understanding of what areas of the standard apply to the process.  You will be developing questions to ensure compliance to the standard and this tells you what areas of the standard to focus on.

3. Document Review

The document review section requires reading and understanding the associated documentation for the process you are auditing.  Start with the level 1 document, the quality manual.  The quality manual should provide an overview of the process and should describe how the process fits into the overall quality system.  The quality manual will explain what processes feed the process you are auditing and what processes are supported by it.  It will describe the interaction and interrelationship of processes within the quality system. 

The main output from the review of the quality manual will be an understanding of all the processes that make up the quality system and how they interact.  The quality manual should provide a good description of how the processes work.

Next, review the level 2 documentation or procedures.  Procedures should describe the process in more detail than the quality manual.  There could be many procedures outlining the quality system, or there could be the minimum required by the ISO 9001:2000 standard, six.  The six required procedures are:

  Control of documents
  Control of records
  Internal Audits
  Control of nonconforming product
  Corrective action
  Preventive action

Since the ISO 9001:2000 standard requires less documentation than previous versions of ISO 9000, there may not be as many procedures to evaluate.  In this case the document review portion will be reduced.   During the document review of the manual and procedures your are trying to understand the process and the system and ensure the requirements of the standard are met.    
4. Identify Process Interfaces

Process interfaces are the “hand off” points from one process to another.  This is where the previous process in providing an input to the audited process and the audited process is providing input to another process.  How are process interfaces different from inputs and outputs?  An input is the deliverable the process uses and the process interface describes how and when the deliverable is achieved.  For example, an input into the purchasing process is the requirements of the purchased item.  Looking at the process interface we want to understand how are the requirements delivered to the purchasing process, when are they delivered and by whom?   In essence we are not looking at do the requirements exist, but are they clearly defined and understood by the process using them.  We want to investigate are the requirements delivered on time and are they accurate?

On the output side, we will look at those things the purchasing process provides to other processes.  Clearly one output is the purchased item on time, to specification and in the correct quantity.  Another consideration is how is it moved from purchasing to receiving and inventory.  There are other outputs of the purchasing process used by other processes.  One could be supplier selection for the item purchased.  Engineering or Quality may need to interface with the supplier and if the selection process is delayed, it could affect the design, or ability to qualify the product.

Understanding the process interfaces can lead to some audit questions concerning how smooth the hand off is between processes.
5. Identify Potential Process Failure Modes

Another tool we want to utilize is the pFMEA, which stands for “process failure modes and effects analysis.  You may have some background in FMEA’s and you may not.  Either way is alright because we are not going in depth in the FMEA process.  An pFMEA is a method to identify potential problems with a process before the process is implemented.  It is a preventive measure that aims to resolve problems before they occur.  For our purposes we will be concerned with the process function, the failure mode and the cause of the failure mode.  Below is an example of an pFMEA for the purchasing process:

Process Function      Failure Mode      Potential Cause               
get good product       bad  product       requirements not understood
                                                                  supplier is not capable
                                                                  not inspected enough
       

product on time        product is late    lack of capacity
                                                                 ordered late
                                                                 supplier out of product

low total cost             too costly            excessive rework
                                                                 excessive freight
                                                                 excessive testing

       
pFMEA’s are an exhaustive approach that generates a large quantity of potential audit directions.  By evaluating the prospective problems associated with a process, you can develop audit questions and an audit approach to ensure the potential problems are addressed.  This can lead to some findings that can have positive impact on the quality management system.      

6. Value Stream Map the Process
If you really want to energize the efficiency factor of your internal audits, then conduct a value stream map.  Value stream mapping is a lean manufacturing tool that aids in finding the activities in the process that are non value added.  Similar to the pFMEA example we will approach this tool in an overview so it can be used but we won’t go into great detail and explicit flowcharting that a lean project might require.  Lean initiatives would include takt time, inventory, etc, we will not include those for this use of the tool.  For this purpose you will flowchart the process activities and look for steps that could be eliminated or reduced.

7. Review Old Audits

A key source of information to develop your audit strategy is to review old audits.  Review both internal and external audits if available.  Look for areas of weakness or where findings were noted and see if action has been taken and if it’s still effective.  In reviewing an old audit of purchasing you find that there was a nonconformity written for the buyer not conveying to the supplier all of the requirements of the product.  Based on this you may want to gear some of the audit to see how effective the process is now at conveying the requirements to the supplier.
8. Develop Audit Questions
What we want to do now in the planning process is develop some questions based on the excercises listed above.

1. Turtle Diagram generated questions
How are the requirements for the purchased item documented and communicated?
Who specifies a budget and who monitors it to ensure it is not exceeded?
What training has the purchasing agent received and what is scheduled?
How is inventory monitored to ensure correct purchases at the right time?
What is the measure of the process?  Who monitors it?  What are the planned results and what happens when they are not achieved?

2. Interface with the Standard generated questions
Is there a procedure or work instructions describing the process?
Is the purchasing process covered in the quality manual?
Does the current process reflect what is documented?
How does the purchasing agent know what their responsibilities and authorities are?
Do they know and understand the quality policy and quality objectives?  What does it mean to them?
How are suppliers selected and rated?  Is it effective?
How are purchased items evaluated when received?
What happens when a purchased item is received and does not meet requirements?
Who reviews the data from the purchasing process?  Does the data get delivered to management?
How has the purchasing process been improved?  Has it shown improvement and what is currently being done to improve it.

9. Develop Audit Plan
Up to now you have developed an understanding of the business process you will audit, you have also used various tools to identify some audit questions or paths.  Now we will take this one step further and develop the audit plan.  The audit plan is your playbook for the audit.  If you fail to plan, then you plan to fail.  This statement couldn’t be any more true than in the auditing functions.  You develop the audit plan based on the questions and who you will audit.

Based on our previous work, we will develop our audit plan as follows:

Auditee:  Purchasing agent

1. Explain to me how the purchasing process works?
Verify that it is consistent with whatever is documented.
Document what is said, does it match what you had perceived?  If not make adjustments in your audit plan.  
2. How are the requirements for the purchased item documented and communicated to you?
Pick a critical purchased part and look for evidence of requirements being specified.  Are they clear and do they communicate the quantity, time frame and budget?       
3. How are the requirements communicated to the supplier?
Look for records that the supplier has acknowledged the requirements or was sent them.  You can also later review the incoming inspection or records relating to problems with this part, quality, delivery, quantity or price, this can be a reflection of how well they understand the requirements.  
4. How are the suppliers selected?
Look for evidence they followed their process and verify the effectiveness based on complaints or issues with the product. 
5. How is it verified the suppliers are capable?
Look for evidence that someone evaluated them for ability to meet the requirements.  Can they produce to the specifications?  Was capability studies done?  Do they have the capacity?

You can continue this process to develop a larger audit plan.  You can even develop questions and expected responses for other people such as engineering, quality, manufacturing, material control, etc.  It depends upon the scope of the purchasing process and who is involved. 
 

Improving Management System Internal Audits

The purpose of internal audits is to continually improve the management system.  When I refer to management system, I mean quality, environmental or your business management system.  In either case the internal audit should be a intricate part of the management system you are utilizing.  Most often the internal audit process or function is under utilized and viewed as a necessary evil in order to meet the requirements of a quality standard.

The time constraints on individuals makes it harder to free employees up to plan, execute, document and follow up an internal audit.  The effort to transition internal audits to improvement based, does not make that issue any easier.  In order to conduct a meaningful, effective audit that promotes and facilitates continual improvement, the auditor must spend an adequate time preparing.  The preparation of the audit dictates how effective it will be.

If we want to improve a process, what is the first step?  In all my years as a quality practitioner, I have heard and deeply believed that the first step to improvement is measuring.  If something is not measured, you should not expect to improve it.  The first step to improving the internal audit process is to measure it.  How do you measure it?  The internal audit process is a ratio that relates the relative importance of the findings or results of the audit.  To keep things easy to understand, let’s break the findings down into three categories and they are not major, minor and opportunity for improvement.  The three categories are effectiveness, improvement and compliance.

Describing these three factors with an equation we have the following:

                       V = (E + 2I)/C

Where V = the audit value, E= the number of findings associated with effectiveness, I= the number of findings associated with improvement and C= the number of compliance related findings.  The resulting equation will yield a value or index you can use as a reference measurement, the higher the index the more improvement focused the audit activity is, the lower the number reflects a focus on standard compliance.

I am not suggesting to disregard compliance to a standard, I am contending the focus of the audit should be improvement, effectiveness and compliance.  Too often compliance is the single biggest focus when conducting audits.  There is not much value in finding out that an employee did not complete a form correctly on a given day.  An isolated one time event is useless as a tool to improve a management system.  The point to remember is that whatever the auditors are focused on is what they will find.  If the focus is on compliance issues, that is what is found.  It’s similar to going bass fishing, you may catch a pike, but for the most part you will tend to catch more bass.

The effectiveness factor relates to those findings associated with improving the effectiveness of the management system.  Effectiveness is the gap between where your current performance lies and where you should be or the expected results.   A company has a expects to have 20 customer complaints per month, based on history.  During an audit, the evidence shows  the last few months have had customer complaints of 35, 48 and 55.  Further investigation uncovers the largest contributor to customer complaints is “poor technical support”.  Therefore, findings associated with improving the technical support will improve the effectiveness of the system as will those findings associated with the lack of effort in analyzing the root cause of customer complaints.   These types of findings are much more beneficial to improving the management system than one that documents one customer complaint out of hundreds, had no follow up activity.

The improvement factor is associated with findings that improve the overall management system.  That seems a bit simplistic but it relates to improving the expectation of the management system.  Using the same example above, lets say the customer complaints have been running at about 20 per month or right around the expectation.   Further analysis of the complaints uncovers the major cause to the complaints is “poor technical support”.  An improvement in technical support results in an improvement in the quality system.

A relatively new management system would experience a larger number of compliance issues as opposed to improvement or effectiveness findings.  If your management system is matured several years and you are still experiencing a large number of compliance findings,  you should evaluate the audit process.   Chances are, the auditors are finding compliance related findings because that is what they were trained to do and it is the easiest thing to find.  If you dig hard enough you can find an instance where a human made a mistake.

The audit value indicator is a measure that can be used to monitor the health and direction of the audit activity.  A continual creep in the audit value could indicate the auditors are focused more on improvement and effectiveness findings or it could mean they have neglected compliance related issues, which is not healthy either.  The audit index is a method to measure the internal audit function, because the first step to improvement is measurement.

  

Drive Decisions with Data

It sounds easy and well understood as best practice, but allowing the data to drive decisions is often times difficult.  Personal agendas, beliefs or past practices can infiltrate the conversation and provide alternative thinking.  Good sound data is the best decision making tool.  Whether related to a personnel decision, business process or outsourcing decision, accurate, valid data is the key.What is good data?  Many managers believe it is that which supports their gut feel and opinion, however, good data is objectively collected, effectively analyzed and acted upon.  Objectively collected data is that which is not contaminated by factors other than pure randomness.  If there are external, assignable causes present within the data you could be directed to false conclusions.  For example, let’s assume we have a transactional process where we are monitoring errors in the number of transactions completed.  We started tracking the data in January 2006 and the data is as follows:                                  

              Jan   Feb   Mar   Apr   May   Jun   Jul   Aug   Sep   Oct   Nov   Dec
Errors     20    18     21     22    25     25    31    28     35    38     38     37

In reviewing the above data one can easily see a moderate trend upward.  I realize this is basic and I am not questioning your statistical intelligence.  The intent of using data is not the utilization of a Monte Carlo study, design of experiment, multiple regression or response surface methodology.  I am referring to simple, basic statistical tools that can go a long way in driving correct, congruent decisions.  Referring back to the data above, anybody can tell you have an unfavorable upward trend that requires some immediate attention, right?  Not so fast my friend. Before you claim yourself winner of the “Jump to Conclusions” game, consider this, in May, torn documents were added to the list of errors, prior to that they were not counted.  Torn documents average 5 per month.  In September, a customer complaint added another error code to the transaction process.  Timeliness of the transaction was included as an error code in September.  Prior to that it was not counted as an error.  Timeliness or the transaction errors average about 10 per month. 

With the new error codes included the data looks like this:

             Jan   Feb   Mar   Apr   May   Jun   Jul   Aug   Sep   Oct   Nov   Dec
Errors    35     33    36     37    35     35    41    38     35    38     38     37

That looks a little different.  You may conclude that the increase in transactional errors was caused by the change in the error coding.  There was no change in the transactional errors, there was a change in the measurement method.  There is a difference.  This sounds basic, but often times measurement methods and data integrity are not understood and false indications are recognized.

Once we have ensured the data is objectively collected, now you must analyze it.  Take a look at the data below, which represents rpm’s of a motor:
                                                                     4600      3400
                                                                     3200      4400
                                                                     3400      3600
                                                                     4600      4400
                                                                     4400      4200
                                                                     3400      3400

Looking at the data, not much initially jumps out.  however lets put it into a simple histogram.

                                                                      RPM       Occurences 
                                                                     3200            1
                                                                     3400            4
                                                                     3600            2
                                                                     3800            0
                                                                     4000            0
                                                                     4200            2
                                                                     4400            4
                                                                     4600            1
                  
Hmm, that looks a little different.  Somewhat appears as if there could be two distributions occurring.  A bimodal distribution could be the result of many causes, either way it requires further investigation.  My point is not to insult your statistical intellegence, my objective is to reinforce the power that lies in simple statistical tools.  Basic data analysis can go a long way in discerning fact from fiction when it comes to what the data is telling you. 

There is a place for more advanced statistical tools and they are invaluable, however these basic tools have a distinct advantage when they can be applied.  For one, they are simple to use.  With some initial training, the basic statistical tools can be utilized with good effectiveness.  Number two and most important they are easy to understand.  Top management can understand and grasp average, run charts, histograms, pareato charts and the other basic statistical tools.  Try getting your point across when you’re blabbing on about a two way analysis of variance or an F-test.

When the job you need to complete, entails putting a nail in a 2×4, you don’t need a hydraulic, computerized numerical controlled force application device, a hammer will work fine.