MED5208 Monitoring Tools and TechniquesCourse objectives The general objective of this course is to enable the learners to understand, appreciate and apply various monitoring tools and techniques in project management At the end of this course the learners should be able to – Understand and appreciate monitoring as a component of project/program management cycle Understand, appreciate and apply different analytical tools in monitoring projects Discuss and recommend the appropriate monitoring tools and techniques for various activities Construct and analyze project diagrams e.g. networks, charts, tables etc Understand and appreciate ethical issues in the monitoring and evaluation Answer why, when, what, who, how and resources of monitoring Course contents 1. Project monitoring fundamentals Definition of monitoring Purpose and importance of monitoring project performance Planning and developing monitoring systems Developing a monitoring team Indicators and generally monitored indicators in projects Process and criteria for monitoring 2. Project monitoring techniques (planning phase) Log frame(logical framework) Work breakdown structures(WBS) Uses of log frames and WBS Preparation/development of log frames and WBS 3. Project network Definition of network as a tool of monitoring projects project activity, event and dummy Organization responsibility chart techniques (ORCT) Understanding ORCT as a technique of monitoring projects Developing ORCT Using ORCT in assigning responsibilities Challenges in assigning and sequencing 8. The PERT Mathematics(calculations) 6. Optimistic Time Estimate (TOPT). The Gantt chart Definition of Gantt chart as a tool of monitoring projects Developing and applying Gantt chart in monitoring project Advantage and disadvantages of Gantt chart in project monitoring process 7. Program evaluation and review technique (PERT) Concept and rationale behind PERT The three estimation times involved in PERT. and Pessimistic Time Estimate (TPESS). Critical path method Definition of CPM the concept and rationale behind CPM Use of forward and backward pass Estimating activities duration and causes of variation in activity duration Project time reduction procedure(crashing) Rationale for reducing project time Calculating slacks/floats 5. Most Likely Time Estimate (TLIKELY). activity duration Types of resources Elapsed time. Estimating resource and costs requirement Matching resources with the activity(resources loading vs. labor time and production time Budgeting and budget preparation Monte Carlo analysis . Activity -on-arrow and activity –on-node approach for CPM analysis Developing and analyzing project network 4. Sampling. Constant Comparison /Grounded Theory. Case study and Participation 10.9. Data analysis tools and techniques in project monitoring process Understand and appreciate quantitative data analysis tool and techniques such as: Frequency distribution(histogram in excel) Descriptive statistics Comparing means – statistical testing Cross tabulations Correlations Linear regressions Text analytic Understand and appreciate qualitative data analysis tools and techniques such as:Typology.These are guideline only and the learners are advised to read widely!!! . Quasi-statistics Event Analysis/Microanalysis. Metaphorical Analysis. Domain Analysis etc NB: . Analytic Induction. developing and applying various data collection tools such as: Observation. Taxonomy. Content Analysis. Questionnaire and Interview Survey. Data collection tools and techniques in project monitoring process Definition. Logical Analysis /matrix analysis. It is a process of routinely gathering information on all aspects of the project. Using lessons from one project experience on to another. Monitoring also involves giving feedback about the progress of the project to the donors. The data acquired through monitoring is used for evaluation. . Identifying problems facing the community or project and finding solutions. To monitor is to check on how project activities are progressing. Ensuring all activities are carried out properly by the right people and in time. ─ systematic and purposeful observation. Monitoring allows results. implementers and beneficiaries of the project. Monitoring is checking progress against plans. processes and experiences to be documented and used as a basis to steer decision-making and learning processes. defines monitoring as the regular observation and recording of activities taking place in a project or program. According to Phil Bartle (2007). and Determining whether the way the project was planned is the most appropriate way of solving the problem at hand. To have internal and external accountability of the resources used and the results obtained. It is observation. monitoring is very important in project planning and implementation because it provides information that will be useful in: Analyzing the situation in the community and its project. To take informed decisions on the future of the initiative. Phil Bartle (2007). Determining whether the inputs in the project are well utilized. To promote empowerment of beneficiaries of the initiative. It is the systematic and routine collection of information from projects and programs for four main purposes: To learn from experiences to improve practices and activities in the future.Chapter one Project monitoring fundamentals What is monitoring Monitoring is a periodically recurring task already beginning in the planning stage of a project or program. Project staff at various levels will do most data collection. As a management tool. field visits. A first step in designing a monitoring plan is to identify who needs what information for what purpose how frequently and in what form. The careful selection of monitoring indictors organizes and focuses the data collection process. stakeholder meetings. The work plans should describe in details the delivery of inputs. Drawing up a monitoring plan will help the organization to think about how it will carry out monitoring over the year or over the lifetime of a project. Developing a monitoring plan The type quaintly and quality of information should be thought of carefully before planning the monitoring system. It will consider: . the activities to be conducted and the expected results. etc – (how to gather information) The monitoring plan should include who will gather the information and how often. To develop an effect monitoring system. Monitoring systems should be linked to manual plans. the following steps might be followed-: Decide what should be monitored. (When to gather information by whom) Progress reports should be reviewed by project staff and major stakeholders. Feedback should be collected by project managers on a regular basis. Needed funds and staff time should be allocated to ensure effective implementation. monitoring should be organized at each level of management. interviews. analysis and reporting. Select methods to track indicators and report on progress (observation. The monitoring plan should indicate the resources needed to carry out project monitoring. The project manager usually prepare annual work plan that translate the project document into real concrete tasks. Staff should agree on what the monitoring report should include.Reporting enables the gathered information to be used in making decisions for improving project performance. They should clearly indicate schedules and the persons responsible for providing the inputs and producing results. The work plan should be used as the basis for monitoring the progress of program/project implementation. routine reporting. These need to be clearly stated in the early stages of planning. Organizations should think about: Why do we want to do it and for whom? What will we monitor and evaluate? How will we do it? Who will do it and do we have the right skills? When will we do it? What resources will we need. . and quality reviews? You will then be able to prioritize the information you need to collect regularly and on an occasional basis by being clear about the key questions. When will different information be collected? Who will carry out and manage the activities? How will information be collected. and their stakeholders. will usually want to find out how the organization or project is meeting its planned aims and objectives. analyzed and presented? How will monitoring and evaluation findings be used in short-term planning and decision making. The importance of planning Planning is a key stage in the monitoring and evaluation cycle. and they may need different types of information gathering. including outside support? What will we do with the information we get? Organizations themselves. such as: How are resources being used? How appropriate is the management structure? How well are we meeting identified needs? How do we fit within a network of services? How well have we met our expected outcomes? What were the unexpected outcomes? What lessons did we learn? Organizations may focus on different questions at different points in time. Project management on a tight budget can be a lonely business . Selecting and gaining commitment from the best team members . especially if you have team-members imposed on you by the project brief. Generally try to establish your team as soon as possible. and not until they have committed themselves to the project upon terms that are clearly understood and acceptable. Developing a monitoring team Another important part of the planning stage is picking your monitoring team with the data collection and analysis skills. . To plan and manage large complex projects with various parallel and dependent activities you will need to put together a 'Critical Path Analysis' and a spreadsheet on MS Excel or equivalent. Critical Path Analysis will show you the order in which tasks must be performed. who want nothing other than to be involved and a few words of thanks. freelance. Don't imagine that teams need to be full of paid and official project team members. and the relative importance of tasks. mentors. The team must be aware of their duties and responsibilities Take great care. whatever the budget. and maximizes what they can contribute. but before trying it you should understand and concentrate on developing the pure project monitoring skills. much of which is useful. Identifying or appointing one or two people even during the terms of reference stage is possible sometimes.whether directly employed. Many funders and commissioners will ask for outcome information as part of their monitoring requirements. consultants or other partners . Various project management software are available. Some of the most valuable team members are informal advisors. which are described in this process. suppliers.get some help from good people you can trust.is crucial to the quality of the project. helpers. contractors.Defining aims and objectives will help you agree on planned outcomes. The best software in the world will not help you if you can't do the basic things. Appointing the team early maximizes their ownership and buy-in to the project. Some tasks can appear small and insignificant when they might actually be hugely influential in enabling much bigger activities to proceed or give best results. But be very wary of appointing people before you are sure how good they are. and the ease with which you are able to manage it. A Gantt chart is a useful way of showing blocks of activities over time and at a given cost and for managing the project and its costs along the way. Any good project manager knows that a team is a collection of people who are committed to common goals and who depend on one another to do their jobs. Team members who contributed to the proposal can remind themselves of the project’s background and purpose. Monitoring teams consist of members who can and must make a valuable and unique contribution to the project. authority. such as acquiring new skills and knowledge. Emphasize how the results may support your organization’s growth and viability. after they’ve reviewed the plan. Encourage people to think about how their participation may help them achieve personal goals. Meet as a group to discuss people’s thoughts and reactions. have them review the approved project plan to reinforce the project’s goals. As soon as you identify your project monitoring team members. their planned roles. clarify the work planned. and identify issues that may affect the project’s success. Help team members develop and buy into a shared sense of the project goals by doing the following: Discuss the reasons for the project. and assignments Processes: The techniques that team members will use to perform their project tasks Relationships: The attitudes and behaviors of team members toward one another As soon as people join the team. and the impact of its results Clarify how the results may benefit your organization’s clients. take steps to define and establish your team’s identity as well as its operating practices. Develop the following elements. confirm the feasibility of time and resource estimates. Explore how the results may impact each team member’s job. and identify any potential problems. and the work to be done. raise concerns about time frames and budgets. increasing their visibility in . meeting new people. Team members commit to your project when they believe their participation can help them achieve worthwhile professional and personal goals. position on the team. find out about their roles and assignments.Merely assigning people to tasks doesn’t create a project monitoring team. They can also identify situations and circumstances that may have changed since the proposal was prepared and then review and reassess project risks and risk-management plans. New team members can understand the project’s background and purpose. its supporters. making sure your team understands and accepts them:Goals: What the team as a whole and members individually hope to accomplish Roles: Each member’s areas of specialty. Examples of types of indicators for monitoring and evaluating CFSC initiatives Input indictor: Funds covering the planned communication activities Qualified staff .A-15).” An indicator provides evidence that a certain condition exists or certain results have or have not been achieved (Brizius & Campbell. the members’ motivation and commitment to project success will be greater. different definitions are being used. p. May 2002). to reflect changes connected to an intervention. According to OECD/DAC. indicators are an integral part of results. target group(s). As such. and enhancing their opportunities for job advancement. projects aren’t only about helping team members achieve personal benefits. an indicator is:“a variable. when team members can realize personal benefits while performing valued services for the organization. an indicator is: “A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement. indicators come into the picture in a development co-operation context at the moment that specific information is required.the organization. and objectives. In this context. Indicators and generally monitored indicators in projects Indicators The English Language Dictionary describes an indicator as “an instrument which gives you information” In line with this description. outcomes. or to help assess the performance of a development actor” (DAC Glossary of Key Terms in Evaluation. Obviously. which purpose it is to measure change in a phenomena or process” The European Commission describes (planning) indicators as: “a description of the project’s objectives in terms of quantity. Indicators enable decision-makers to assess progress towards the achievement of intended outputs. goals. However. According to the definition adopted by USAID. quality. time and place. . or tracking lifetime statistics for an affected and/or control population. fiscal vs.Process indicator: Number of participatory radio program aired Number of people reached through popular theatre activities Output indicator: Percentage of participants by men and women Exposure to needed information/messages Expanded public and private dialogue Outcome/impact indicator: ICT increasingly used for dialogue and debate Percentage of men and women who know about voting procedures (DANIDA. Reporting schedules may not be synchronized (e. Resources: Ideal indicators might require collecting data to calculate an unknown denominator. reporting year). or national data to compare with project area data. or government officials. 2005) Some examples of these considerations are: Availability of data: Some data may be considered ‘privileged’ information by agencies. Human resources and technical skills may be a constraint as well. Programmatic and external requirements: Indicators may be imposed from above by those not trained in monitoring and evaluation techniques. projects. Standardized indicators should be used if available. etc.g. The cost of collecting appropriate data for ideal indicators is prohibitive. Different stakeholders’ priorities may diverge. Data may be available only on aggregated levels or already calculated into indicators that may not be the ideal indicators for your program or activities. such as information on the perspectives of health care providers and women who come for services. select indicators related for example to providers’ knowledge. If the intervention focuses on training providers. 2009) When quantitative indicators of success cannot be identified. (Bott. Examples include:The number and types of services provided The number of people trained The number and type of materials produced and disseminated . 2004). Select a set of indicators that pertain to the identified objectives for the program. it is almost always possible to gather qualitative data. qualitative methods offer a valuable alternative.g. training. (Gage and Dunn. quantitative terms. When it is difficult or not possible to measure “benefits” or “risks” in simple. that do not currently exist and cannot realistically be collected. ability to provide care. airing of TV spot) No more than 8-10 indicators per area of significant program focus Use a mix of data collection strategies and sources Process versus result/impact indicators: It is important to remember the difference between process and results indicators. or others depending on the specific objective. How many indicators are enough? Some guidelines to follow when selecting indicators: At least one or two indicators per result (ideally.In general. from different sources) At least one indicator for every core activity (e. ability to make referrals. Process Indicators are used to monitor the number and types of activities carried out. Information on the perspectives of women and girls (rights-holders) and service providers (governmental duty-bearers or NGOs) is essential. In many cases. qualitative indicators provide more relevant information with respect to the success and effectiveness of the intervention. or that do not accurately represent the desired outcome. attitudes and practices. Guedes and Claramunt. that are too vague. Information from women accessing or attempting to access services is critical for assessing the effectiveness of an intervention. Evaluation efforts and indicators should include the perspectives of both clients and providers. program should stay away from indicators that activities cannot affect. g. outcome and impact levels. The main indicator of impact should be a reduction in the prevalence and incidence of violence. Precisely.) Outcome indicators relate to change that is demonstrated as a result of the program interventions in the medium-to-longer term (e. and how many people have participated (outputs).g. understanding the purpose of the monitoring process and what is to be achieved.) An important issue that needs to be resolved in order to monitor project progress is how to define success. Commonly. After the tools have been developed. but that takes years to achieve and to measure. Monitoring process include identification. especially over time. Monitoring and evaluation frameworks and plans should incorporate both process and results indicators. organizations are able to track how many events they have held. It also determines the methodology to be used. percentage of traditional leaders in community x who completed the training on international human rights standards related to violence against women and girls whose knowledge improved. you can choose to include the schedule reflecting the timing. costing and the responsibilities and so on. Output indicators illustrate the change related directly to the activities undertaken within the program (e. Process and criteria for monitoring The process of monitoring starts getting information on the background of the project to determine the scope of what is to be monitored. Examples include:Results indicators can be developed at the output.The number and percentage of female clients screened Results Indicators are used to evaluate whether or not the activity achieved the intended objectives or results. Finally. the prevalence of violence against women and girls in community x.g. but not how people have changed their attitudes or behaviors as a result (outcomes).) Impact indicators measure the long-term effect of program interventions (e. monitoring is conducted and a report written to be disseminated to the people or organization in need of the report. the number of decisions in the informal justice system of community x related to violence against women that reflect a human rights-based approach. . before developing the tools for monitoring process start by determining the central question that the process is going to address whereby the specific objectives are selected. methodology.purpose. monitoring and reporting. NOTE-The rest of the chapters will be availed progressively . tools. Appropriate use of Tools and techniques in the monitoring process is prerequisite to a project success and that is what we going to dwell on in the next chapters of this course.