Seven Steps for a Project Quality Review

In some cases, such as a government project, periodic audits may be called for as a part of the overall contract. This “outside party” could be any qualified person outside of the project manager. In some cases, your organization may have an internal project audit specialist. It is possible that the Project Director or the Project Sponsor could also perform this audit. The outside party could be an outside contractor or consultant, but they do not need to be.

The audit itself focuses on whether effective project management processes are being utilized and whether the project appears to be on-track. A project audit asks questions about the processes used to manage the project and build deliverables. The audit can follow this process:

1. Notify the parties (Auditor) – The auditor notifies the project manager of the upcoming audit and schedules a convenient time and place. Other key stakeholders are notified of the audit as well.
2. Prepare for the audit (Auditor) – The auditor may request certain information up-front. The auditor might also ask the project manager to be prepared to discuss certain aspects of the project. This ensures that the actual meeting time is as productive as possible.
3. Perform initial interview (Auditor, Project Manager) – During the initial meeting, the auditor asks the appropriate questions to ensure the project is on-track. If there are any areas that are not on track, the auditor notes them as such.
4. Perform as many other interviews as necessary (optional)(Auditor, Project Team) – On many projects, the audit might culminate in the initial meeting. If the project is large or complex, the auditor might need to perform follow-up analysis. This includes meeting with other team members and clients, and reviewing further documentation.
5. Document the findings (Auditor) – The auditor documents the status and the processes used on this project against best practices. If the organization has standards and policies in place for managing projects, the auditor determines whether any of these are not being followed on the audited project. The auditor also makes recommendations on things that can be done to provide more effective and proactive management of the project.
6. Review draft audit report (Auditor, Project Manager) – The auditor and the project manager meet again to go over the initial findings. This auditor describes any project management deficiencies and recommendations for changes. This review also provides an opportunity for the project manager to provide a rebuttal when necessary. The initial audit findings might be modified based on specific feedback from the project manager.
7. Issue final report (Auditor) – The auditor issues a final report of findings and recommendations. The project manager may also issue a formal response to the audit. In the formal response, the project manager can accept points and discuss plans to implement them. The project manager may also voice his disagreement with certain audit points, and explain his reason why. In these cases, the project sponsor and the project director (manager of the project manager) will need to decide if the project manager should comply with the recommendations or not.

Escalate a Performance Problem with a Formal Plan

One of the hardest jobs of a manager is to take an employee down a path that may ultimately result in termination.  It is hard enough for most managers to provide performance feedback to begin with – even when the employee performance is good.  When the employee performance is not where it needs to be, it is even harder. 

The first thing you need to do when you see a performance problem is sit down with the employee, discuss the performance observations, try to determine a cause and put a short-term action plan in place so that the employee has a chance to turn the situation around. 

Unfortunately, sometimes the initial performance feedback and short-term plan do not have the desired effect.  If this occurs, the manager needs to take additional actions.  In some companies and in some positions, the next step might be a demotion or termination.  This might also be the case at smaller companies where the management team needs to make personnel decisions quickly, and where the company is not under as many obligations from a Human Resources standpoint. 

In larger companies, however, managers normally don’t have the authority to fire employees on their own.  The Human Resources Department normally has processes in place to make sure that people are treated fairly and within allowable legal guidelines.  If a manager tried to resolve a situation on his or her own but was unsuccessful, it is time to bring a formal Human Resources process into play.

Managers sometimes hesitate to take personnel-related actions because of their concern for how the rest of the team will react.  If the employee is a popular one, there is a tendency to believe that the team will react negatively.  In fact, that might be the case if the manager acts arbitrarily.  However, if the manager gives an employee plenty of time to improve his or her performance but the problem does not go away, termination may still be necessary.  In this situation, the manager should be able to explain to the rest of the team how every effort was made in the employee’s favor.  The rest of the team should understand first-hand that the employee’s performance was weak.  Also, they should understand that replacing that employee is in the best interest of the team, the entire organization and perhaps in the employee’s best interest as well.   

A team knows its weak links.  In many situations, the rest of the team ends up working harder to compensate for the person with the weak performance.  In the best cases, the team does so willingly (and perhaps subconsciously), but their actions mean that they cannot be effective.  In the worst case, teammates start to turn against the poor performer, causing resentment, animosity and friction among team members. 

Sometimes a perceived performance problem hits the team member totally by surprise.  However, in most cases, they already understand the situation.  Poor performers should see that they are missing deadlines or that “completed” work requires a lot of rework.  Once they get on a short-term improvement plan, they become keenly aware of whether or not their performance is meeting expectations.  If they still cannot meet expectations, it will become increasingly obvious to them.  This situation will cause them more anxiety, which can drive performance down even lower.  The situation should be resolved as soon as possible for the sake of the employee as well as the organization. 

Remember that putting a formal performance plan in place is not the same as termination.  The performance plan is really a way to save the person from possible termination.  A good performance plan puts everything into black and white, and it should precisely set expectations.  The performance plan should include the length of the plan, the expectations and how the progress will be communicated.                

     

Once the performance plan is signed, it is activated.  The employee should strive to meet the expectations of the performance plan.  The manager must provide ongoing feedback on the employee’s performance and whether the employee is meeting expectations.  This entire process is set up to manage expectations.  If the employee is not meeting expectations, the manager must continue to say so in the ongoing written feedback.  This ensures there are no surprises. 

Manage Quality and Metrics – Techniques

Gather Subjective Metrics with Client Satisfaction Surveys

Gathering metrics is important because it allows you to see how you are performing against the expectations of your clients. If the world were perfect, all of the metrics you collect would be factual, relevant and accurate. However, in many cases it is impractical or cost-prohibitive to try to gather exact and quantitative numbers.

 

One way to supplement quantifiable metrics is with client satisfaction surveys. For instance, instead of trying to measure the exact response time of an application against some service-level standard, you could simply ask your main users how satisfied they were with the application response time. It should make sense that if you are rated a 4.5 out of 5.0 (5 being the highest), you are probably doing a pretty good job in this area. However, if your survey results come back as a 1.8 out of 5.0 (5 being the highest) then it should be obvious that you are missing the mark. You did not need a complicated system, metric to tell you that. A simple customer satisfaction survey questions gets quickly to the same result.

 

In the same way, let’s say you want to gather metrics that indicate the time it takes to resolve client problems. This could involve tracking when the initial request comes in, when you first responded to the client and when the request was resolved. On the other hand, you could simply send out surveys that ask your clients if they were satisfied with the time it took to resolve the problem.

 

Surveys are by their nature qualitative; that is, they reflect the opinion of the person being surveyed. Therefore, you would not necessarily want to base your entire project success criteria on survey metrics. Some results are more easily obtained quantitatively. For instance, there is usually no reason to send out a survey to the finance department to ask them if your spending is within budget. You should have the facts available to you. However, for many other types of metrics, a qualitative survey question can be used as a substitute for the quantitative metric.

 

Collecting Project Metrics and Demographics Can Assist on Future Projects

Gathering and reporting a consistent set of metrics at the end of a project can help your organization see the trends for delivering projects over a period of time. The metrics should show how well project teams are meeting their commitments in terms of quality, cost and duration. As more and more projects report the metrics, a baseline will be established that will allow your organization to see how it is improving, or slipping, over time.

 

If you collect metrics on an organization-wide basis, you should collect some project demographics in addition to the actual metrics. Project demographics are just predefined project characteristics that provide a description on each particular project. If you store the demographics and metrics in a file or database, they can be analyzed to show the overall trends in a more discreet and granular way.

 

If you do not collect demographics, you can still compare actual cost to deliver versus the estimated cost. The advantage of gathering some project demographics is that you can compare similar projects. For instance, you can compare the cost and effort associated with delivering mainframe development projects versus web development projects. Or you could compare how well projects from the Sales organization meet their deadlines and budgets versus projects from the Finance area.

 

The other benefit of gathering project demographics is that you can use the information as input into future project estimates. For instance, let’s say you have marketing campaigns for different products. Let’s also say that you have been collecting project estimations and the actual results for cost and duration. If you had this information, you could estimate the cost and duration of a current product marketing campaign by comparing your project with similar projects that were completed in the past. This may be of help for estimating your project. The demographics you capture from completed projects can be used to search on later for new projects. (A sample worksheet to capture project demographics and metrics is available in the Template Library.)

 

Make Sure Your Metrics Tell a Complete Story

In many instances the project team publishes the results of a metric in a way that does not allow the reader to fully understand whether the results are good or bad. This is because the reader sees a metric, but they do not understand the target and they do not always understand what the metric is trying to achieve. The project manager and project team may know what a given metric is telling them, but other readers of the information may not. One way to help is to always report the metric along with the target. For instance, if you report your current expenditures to date, also include your expected expenditures at this point in the project. If you report that your project has spent $100,000 so far and your total budget is $150,000, the reader still does not have the context to know whether this is a good or bad situation. Sure you are under budget, but the work is not complete either. The better way to report this information is to state that you have spent $100,000 to date and that according to your estimate you should have spent $110,000 at this point in the project. If the trend continued, you estimate the final cost of the project to be $135,000 versus your budget of $150,000. If you report the metrics with this context, your readers can understand what the numbers are saying.

Remember is that it is impossible to categorize everyone within a profession.  You can make some general assumptions about technical people, but this does not mean that the assumptions apply to everyone.  As a manager, you must ultimately have multiple techniques that you can apply to different people in different circumstances.  One technique will not work for all people at all times. 

That being said, let’s make some generalizations about managing technical staff on your project.

• They tend to be introverts.  Generally speaking, the definition of an introvert is one who is primarily more comfortable with an inward focus in life while an extrovert is generally more comfortable with an outward focus.  For example, when introverts receive a lot of new information, they tend to want to think for a while before speaking or drawing conclusions.  Extroverts, on the other hand, are more comfortable expressing ideas to others.  If they jump to the wrong conclusions, they just change their minds.  Basically extroverts are comfortable thinking out loud.  Introverts would rather think through the “rough drafts” in their minds and then talk when they think they have a coherent and logical position. 
• They tend to think more logically than emotionally.  This tendency should be obvious.  Technical staffers typically are not motivated by a lot of “rah-rah” speeches.  In fact, they tend to be cynical of this type of motivation.  They will usually listen politely (perhaps even snickering to themselves), but the effects are short-term.  On the other hand, they can be persuaded and motivated by a logical argument.  If the logical argument can be combined with some motivational techniques, you might have a chance to actually get them excited. 
• They tend to be problem solvers.  This is a great strength of technical staff as well as one of their weaknesses.  Most technical people love nothing better than to be confronted with a problem.  They get excited and they immediately start to apply their problem-solving skills.  The weakness comes in because there is a tendency to jump on a problem without fully understanding it first.  This often can lead to being less than optimal in the use of resources. This is also a reason why technical people don’t like to spend as much time on project planning. They have a disposition to jump in and execute the project instead of spending time to understand the nature of the project first.
• They tend to be technically creative.  This may seem like a contradiction.  Your first thought might be that the sales and marketing staffs are the creative people.  In fact they are – in the sales and marketing areas.  They will also be the first to tell you so – because they are extroverts.  However, the technical discipline requires a fair degree of creativity as well.  This is especially true in the IT world.  In many cases, there is not one best solution to a business problem.  In the development field, for instance, analysts need creativity when they are defining a solution with the business clients.  Designers need to be creative applying technology in the best manner.  Programmers need to be creative as well in trying to apply the best techniques to build the most elegant solution.

Understanding these general characteristics is the place to start if you are a manager of technical staff.  Once you begin to understand how people work and how they are motivated, you can start to think of the best way to manage them.  Now that you have gone through many ideas, here is a summary of some of the general points you should consider as you manage your staff.

• Try to establish an environment where people feel they have what they need to do their jobs.  This includes having appropriate hardware, software, policies, procedures, etc. 
• Technical people like to understand the work processes in the group, and then they like to be creative in working within that structure.  So, set the high-level rules, but don’t micromanage the details. 
• Give people as much information as they need to do their jobs.  Technical staff tends to reflect on this information.  Ask for their ideas and opinions, but give them time and ample opportunities.  Don’t expect them to react immediately.
• Shield the team from office politics and all of the distractions that can abound in a large company.  Tell people what they need to know (see prior point), but don’t get them bogged down in the organization muck. 
• Give people continuous opportunities to learn.  This includes encouraging people to invest the time to learn, but also helping with some opportunities.  There are many creative ways to learn new things.  Once a person has mastered a certain skill or aspect of his job and he starts to become bored, look for ways he can cross-train and learn new areas of the group. 
• Be there when needed and respond to problems and concerns.  Not all problems can be fixed, but many times the simple act of listening and trying is enough.  People will give you credit for trying, even if the ultimate resolution to a problem is not available. 

You might note that many of these management techniques are not unique to technical staff in general or IT staff in particular.  It is true that many of the techniques can be used in other areas as well.  However, they are particularly applicable to the IT staff.

Many people find quality management to be one of the more difficult project management processes to implement. This is because quality is hard to define, and formal quality management requires you to collect metrics to validate the state of quality. The following process will help create a framework for the quality management process.

 

1. Create a Quality Management Plan

Develop a Quality Management Plan to identify the major deliverables, completeness and correctness criteria, quality control activities and quality assurance activities. The Quality Management Plan also describes how you will ensure that the client’s quality requirements are achieved. It is the place to describe the processes and activities that will be put into place to ensure that quality deliverables are produced.

 

2. Determine the customer requirements for quality

Work with your customer to determine their requirements for quality. The high-level characteristics of quality can be uncovered during the project definition process. The detailed quality requirements should be uncovered when you gather business requirements.

 

3. Define a set of metrics to validate quality requirements are met

Identify a set of metrics that will provide insight into the quality of the deliverables. The project manager should already be capturing overall financial and duration metrics. The quality-related metrics need to be more sophisticated. There are two areas where you are trying to manage quality – in your project work processes and in the actual deliverables you are building. You should try to capture metrics that will measure each.

 

4. Execute quality control activities

Quality control refers to activities that validate the quality of your deliverables. It is also referred to as “inspection”. Ensure that the quality control activities for every deliverable are performed while the project is underway.

 

5. Execute quality assurance activities

Quality assurance refers to the processes used to build deliverables. It is also referred to as “prevention”. Having good processes should results in good quality deliverables.

 

6. Monitor and resolve deliverable quality

You need to validate the quality of your deliverables on an ongoing basis. When quality problems are found, implement a process to determine the cause and to make improvements in the process.

Using this process will help you understand, plan for and manage the state of quality on your project.