Expert review pedagogical project
for compliance
highest qualification category
Problem
A new model for appraisal of executives and teaching staff OU is based on a competency-based approach. In the structure of professional competence of employees, one of the leading competencies is the possession of modern innovative methods in particular, design technology.
Success indicates the teacher's ability to act in the space of modern changes in the education system, the ability to transform pedagogical reality.
In order to competently prepare a pedagogical project for defense, it is necessary to follow certain requirements for drawing up a project plan.
Approximate structure of a professional project
- The problem / theme of the teacher's professional project. The strategic goal of a professional project. The tasks of a professional project that ensure the achievement of its goals. Characteristics of the conditions for ensuring the quality of education:
The quality of resource provision;
Qualification level of the person being certified.
- Quality of design and implementation of the learning process (management):
Validity of the relevance of the problem of professional experience, the ability to identify and formulate the leading idea;
Disclosure of solutions professional problem(reliance on psychological and pedagogical theories, the culture of using information sources; the validity of the selection of subject content; the expediency of pedagogical tools; the rationale for the use of elements of technologies, methods, forms, teaching methods; the presence of examples from the author's practice illustrating ways to solve a professional problem);
Argumentation of innovative transformations in the field of organization of the educational process.
- The qualitative level of the effectiveness of the implementation of a professional project (achievements of students) in accordance with the stated goal and objectives:
Positive dynamics of educational achievements of students, identified by the assessed;
Positive dynamics of educational achievements of students based on the results of external expertise;
Positive dynamics of the formation of social competencies of students.
- Qualitative level of effectiveness of the implementation of a professional project (achievements of the person being certified):
The level of motivation and interest of students in learning;
Generalization and dissemination of experience in the implementation of a professional project;
Active self-education and the pace of advanced training.
Advisory text
Design development should contain the following structural components:
- Rationale for the need for the project. Goals and objectives of the Project. Project participants. Description of the project: strategy and mechanisms for achieving the set goals. Working plan for project implementation. Projected short-term and long-term results of the Project implementation. Evaluation of the efficiency of the Project implementation. Risk assessment. Further development of the project.
1. Rationale for the need for the project
The description of the pedagogical problem includes an analysis of the existing pedagogical situation or the state of the pedagogical system. As a result of the analysis, contradictions are revealed between the current state of the pedagogical system and its optimal state, which allows it to function effectively at the present stage. The teacher must show the discrepancy between the state of the educational system required by modernity and the real situation.
On the basis of such an analysis, the teacher determines what problems he can solve within the framework of his project, what adjustments can be made to the activities of the educational institution, to the content of the educational process, to his own pedagogical activity, etc. The teacher indicates how the identified problems affect certain target groups, i.e. the persons to whom the project activities will be directed. It is necessary to briefly describe the real needs target audience and conditions that are supposed to be changed during the project to solve their problems.
The project should convincingly prove the relevance of the identified problems, justify the importance of solving these problems to improve the efficiency of the pedagogical process. It is important to indicate the importance of solving these problems for an educational institution, to show that their solution is in line with state priorities in the education system.
Thus, the description of the problem involves formulating answers to two questions: why did the project need to be implemented? Why is solving this problem a priority?
A deep and comprehensive analysis of pedagogical problems will allow you to correctly and correctly formulate goals, objectives and select project methods.
2. Goals and objectives of the Project
The formulation of the goals of the project involves the disclosure of the essence of the planned project aimed at solving the identified problems.
When setting goals, a sequence of steps is determined to change the existing pedagogical situation in accordance with the goal. The teacher indicates what educational, upbringing and social tasks will be solved during the implementation of the project. The goal and objectives of the project must be achievable within the stipulated time frame.
3. Project participants
This paragraph indicates the main target audience / project participants, as well as the criteria for their selection. The qualitative and quantitative composition of the target groups with which the teacher will work within the framework of the project is described in detail.
4. Description of the project: strategy and mechanisms for achieving the set goals
Project activities should contribute to the solution of each of the project objectives. When choosing a strategy and approaches to achieve the goal, it is necessary to justify the choice of methods for solving the problem, while it is possible to use both already developed and tested methods for solving problems in various combinations, and modified methods for solving similar problems.
In fact, the choice of methods involves a description of the activities that need to be organized and carried out to solve the problems of the project. Thus, the teacher will determine how the target audience (teachers, students, parents, caregivers) can participate in the project. The teacher thinks over a plan for organizing work on the project, determining the logical chain of his actions.
The teacher determines the executors of the project activity (those who will be involved in its implementation); beneficiaries (those who will be consumers of each particular service). At the same time, the author of the project must assess the availability of the necessary resources (material, informational, personnel, technical, etc.).
If the project plans to hold events - trainings, seminars, classes, etc., it is necessary to describe each of them, indicating the purpose and planned result, how many people it is designed for, the temporary format of the event, the proposed venue, the number of trainers (including number and from which organizations they will be invited), attach a preliminary .
5. The work plan for the implementation of the project is a schedule for the implementation of the planned activities with the obligatory indication of the expected dates and those responsible for their implementation (a possible table form is given below).
6. Projected short-term and long-term results of the Project implementation
Short-term results - those results that are obtained immediately at the end of the project.
Long-term - results that may appear in the future, some time after the completion of the project.
Already at the stage of writing the project, the teacher should highlight the expected short-term and long-term results of his project, by which it will be possible to judge the success of its implementation. Results are planned by relating them to the goal and objectives of the project.
For example, project outcomes include:
Number of new methodologies or technologies introduced within the project;
number of direct recipients of services (for example, training participants);
development of new procedures that facilitate or improve the educational process;
Testing new methods, etc.
7. Evaluation of the effectiveness of the Project implementation
- description of quantitative and qualitative indicators of achievement of results and diagnostic methods.
When planning a project, it is important to identify evidence, data and indicators that would confirm that the task set in the project has been completed. When describing indicators of performance improvement, provide data that allows you to judge the degree of effectiveness, as well as the significance of the implementation of the planned work on the project (for example, expected technological advances; freeing up students' time for creative development personality; reducing conflicts, increasing student achievement, etc.).
- methods for evaluating success/efficiency.
Evaluation methods are ways to collect data on each of the expected results of the project, which will determine the extent to which a particular project result has been achieved. The most common method is to survey beneficiaries/target groups before and after training or participation in a project.
The teacher can offer independently developed diagnostic tools to assess the effectiveness of the project.
8. Risk assessment and measures planned to minimize the impact of such risk factors
The teacher assesses possible risks that may affect the implementation of the project. A qualitative and quantitative analysis of risks and the conditions for their occurrence is carried out. Qualitative risk assessment determines the degree of its importance, quantitative analysis allows you to determine the likelihood of risks in the project. Based on the analysis, procedures and methods are selected to reduce the negative consequences of risk events.
9. Further development of the project
At the end of the project, activities in the chosen direction should be continued, so it is necessary to propose a mechanism for promoting the results of the project.
Specify how the results of the project can be used in the future in an educational institution, other organizations in the region, in other cities. In what ways is it planned to inform the pedagogical community about the results of the project activities and how it will be taken into account Feedback.
The criteria for evaluating a pedagogical project include:
- the importance and necessity of the implementation of the pedagogical project for the educational institution in general, and the educational process in particular; clarity, clarity, achievability of the goals and objectives of the pedagogical project, the logic of the flow of tasks from the problems identified in the project; the correctness of the choice of project activities, compliance with the sequence of actions, the reasonableness of limiting the set of activities; the adequacy of indicators for evaluating the effectiveness of the project; compliance with the logical concept of the project, the presence of a logical chain: problem - goal - tasks - method - result; the significance of the project for other educational institutions, the possibility of implementing the results of the project in the activities of colleagues.
In the annexes to the project, you can submit:
- educational and thematic plans; structure handouts; subject / approximate structure / circulation / volume of publications / printed materials; structure / list of sections created by Internet resources; approximate structure / scope / methodology / tools / research; topic/ sample program/audience of conferences/ ; subject/scope of consultations.
Design Requirements design development
The total volume of project development should be no more than 20 sheets of computer text.
The volume of applications is not limited, but they must correspond to the text (links to them in the text are required).
Materials must be prepared in MS Office Word text editor.
Page format: A-4 (210*297mm).
Font format: size -14; Times New Romans; line spacing - single, red line -1.25.
Margins: left, right, top, bottom - 20 mm.
Literature:
Erofeeva N. Project Management in Education / N. Erofeeva // National Education. - 2002.- № 5. Ilyin G. Projective education and personality formation / G. Ilyin // Higher education in Russia. - 2001. - No. 4. Innovative project for the development of education: Educational book publishing / .- M .: Mir kn., 2001. - 143 p. Karpov E. Project work at school: it is possible / E. Karpov // SHEJ. -1997.- No. 6. Karpov E. Educational - in search of a new pedagogical alternative / E. Karpov // Economics at school. - 2001. - No. 2 Kolesnikov design /, - Siberian. - M.: Academy, 2005. - 285 p. Moiseev problems: essence, purpose, main characteristics / .- M .: “Russian Political Encyclopedia” (ROSSPEN), 2002. Moiseev ideas (promising projects for the development of education) /, . - M .: APK and PKRO, 2002. Monks of the author's (own) methodological system of the teacher / , // School technology. - 2001. - No. 4 Monakhov design: modern tools for didactic research / // School technologies. - 2002.- № 5. Novikov experiment in educational systems. - M.: APK and PKRO, 2002. Novikova approaches to expertise technology / // School technologies. - 2002.- № 1. management of the formation of design and technological competence of the teacher / L. Shmelkova // School. - 2002. - No. 2.
In the course of evaluating the teacher's pedagogical project, the expert uses a 5-point scale, marks with a “+” sign in the corresponding column.
5 - very high degree of severity indicated in the statement
characteristics. It manifests itself in the vast majority of situations, is stable, and fully corresponds to the typical qualities and behavior of the teacher. The answer of the experts is “yes”.
4 - a high degree of expression of the characteristic. It often manifests itself in pedagogical situations. Sometimes there are cases when the qualities or behavior of the teacher do not correspond to the statement. The answer of the experts is “more likely than not”.
3 - the average degree of expression of the characteristics. In some situations, the qualities and behavior of the teacher correspond to the statement, in others they do not. The answer of the experts is “average value”.
2 - weak degree of expression of the characteristic. It rarely manifests itself in pedagogical situations. The behavior and qualities of the teacher only sometimes correspond to the statement. The answer of the experts is “rather no than yes”.
1 - the characteristic is not presented in the activities of the teacher. The qualities and behavior of the teacher do not correspond to the content of the statement. The answer from the experts is “no”.
When conducting an expert assessment, at least two independent experts are involved. In case of significant discrepancy between their assessments, an additional discussion is held in order to reach the necessary consensus.
To determine the qualification level of a certified teacher, it is necessary to calculate the average value for each indicator and competence in general.
The value of the indicator of the level of qualification (PC) of a teacher from 3.9 points (inclusive) to 5 points indicates that his level of qualification corresponds to the highest qualification category.
Expert opinion
Surname, name, patronymic of the certified person
Place of work, position
requirements of the declared _____________________________ qualification category
Problem / topic of the teacher's professional project _____________________________________________________________________________
_____________________________________________________________________________
The strategic goal of the professional project _________________________________________________________________________
The tasks of a professional project that ensure the achievement of its goal _______________________________________________________________________________
Expertise of a professional pedagogical project
Criterion | Indicators | Documents confirming the result |
||
1. Conditions for ensuring the quality of education | The quality of educational programs: |
|||
Compliance of educational programs and resource provision with the requirements of the State Educational Standard | Representation of the administration of the educational institution |
|||
Informed choice and effective use educational and methodical complexes basic courses | ||||
Competence-based growth (both qualitative and quantitative) of the implementation of circles, electives, elective courses, scientific societies of students | ||||
Resource quality: |
||||
Ability to provide a safe and functional educational process (room conditions, implementation of health-saving technologies) |
Certificate of attestation of the cabinet; Certificates, diplomas, diplomas of the corresponding competitions. |
|||
Approbation of didactic tools, control and measuring materials, innovative experimental equipment | ||||
The ability to use the educational opportunities of society in the educational process | ||||
Qualification level of the person being certified: |
||||
Level of knowledge of the content of the taught subject (depth and versatility of theoretical and practical knowledge of the subject) | Representation of the OS administration; Copies of diplomas of education; Copies of certificates, certificates of advanced training; Certificates of participation in relevant seminars, conferences, etc. The conclusion of the psychological service of the educational institution. |
|||
Psychological and pedagogical competence of the person being certified (the ability to work with students of different levels of preparedness, in a specific social situation of development and various resource conditions, the ability to organize educational cooperation) | ||||
High level of analytical and forecasting activities | ||||
Validity of the relevance of the problem of professional experience, the ability to identify and formulate the leading idea | Professional project materials |
|||
2. Quality of design and implementation of the learning process (management) |
||||
Disclosure of ways to solve a professional problem: |
||||
Reliance on psychological and pedagogical theories, culture of using information sources | Professional project materials |
|||
Validity of subject content selection | ||||
Appropriateness of pedagogical tools | ||||
Justification of the use of technologies, methods, forms, techniques | ||||
Argumentation of innovative transformations in the field of organization of the educational process: |
||||
Professional project materials; Reviews, reviews of specialists on the quality of innovative transformations; Presentation of the administration of the educational institution, specifying the methodological and innovative activities of the person being certified |
||||
Use of adequate (relevant, scientific) information sources for the studied material, including Internet resources, multimedia manuals and materials | ||||
The use of active forms of learning | ||||
Adaptation of technology (methodology) to a specific social situation of development and the level of preparedness of students | ||||
Argumentation of the choice of organization of systems of motivation, evaluation, reflection | ||||
Optimization of educational, psychological and physical load to maintain and improve the health of students | ||||
Development of integrated learning programs for children with disabilities | ||||
3. The qualitative level of the effectiveness of the implementation of a professional project in accordance with the stated goal and objectives 3.1. student achievement | Positive dynamics of educational achievements of students, revealed by the assessed: |
|||
Positive dynamics of the level of training (subject knowledge, skills, methods of activity) for the attestation period | Information about the results of the intermediate and final attestation of students in the subject; Certificates, certificates, diplomas, relevant competitions; The results of monitoring studies, including those conducted by the teacher himself; |
|||
Development of general educational competencies | ||||
An increase in the number of students participating in subject Olympiads, conferences, promotions | ||||
Increasing the number of creative domain-oriented product corresponding to the essence of the implemented technology | ||||
Positive dynamics of educational achievements of students based on the results of external expertise: |
||||
High results of the Unified State Examination, testing during the accreditation of educational institutions, international testing, etc.). | Extracts from protocols of external audits Reviews of colleagues about the continuity of subject competence at different levels of the school; |
For your own, describe the project selection process.
1. Determine the main constraints in the project.
2. Generate a variety of project options.
3. Using the method of expert assessments.
4. Conduct a benefit/cost analysis.
An example of using the method of expert assessments.
The following project options were considered.
The main criteria for choosing a project were: cost; duration of the project; the complexity of the project; accommodation; availability of alternative technical solutions; initial permit documentation for the project.
The factors are then listed in descending order of priority. To do this, it is determined which of them will most affect the progress of the project. An assessment of the weight (rank) of each of the listed factors is made. The sum of the ranks of the factors must be equal to one.
The project option is evaluated for each of the evaluation factors (criteria). The maximum score for any of the factors for the project is 100, the minimum is 0. The expert assessment of the impact of each factor is obtained by multiplying the weight of each factor by the assessment of this factor for each option.
Expert evaluation of project options by expert No. 1
| № | Characteristic | Weight | Project option number | Integral assessment | ||||||||
| A | IN | WITH | D | E | A | IN | C | D | E | |||
| Project duration | 0,2 | |||||||||||
| 0,2 | ||||||||||||
| Price | 0,2 | |||||||||||
| Accommodation | 0,2 | |||||||||||
| Project complexity | 0,1 | |||||||||||
| 0,1 | ||||||||||||
| Total: |
Expert evaluation of project options by expert No. 2
| № | Characteristic | Weight | Project option number | Integral assessment | ||||||||
| A | IN | WITH | D | E | A | IN | C | D | E | |||
| Project duration | 0,3 | 7,5 | 4,5 | 16,5 | ||||||||
| Availability of alternative technical solutions | 0,2 | 9,8 | 13,6 | |||||||||
| Price | 0,2 | 7,4 | ||||||||||
| Accommodation | 0,1 | |||||||||||
| Project complexity | 0,1 | |||||||||||
| Initial permit documentation | 0,1 | 4,5 | 4,5 | |||||||||
| Total: | 60,1 |
Expert evaluation of project options by expert No. 3
| № | Characteristic | Weight | Project option number | Integral assessment | ||||||||
| A | IN | WITH | D | E | A | IN | C | D | E | |||
| Project duration | 0,2 | |||||||||||
| Availability of alternative technical solutions | 0,2 | |||||||||||
| Price | 0,2 | 7,4 | ||||||||||
| Accommodation | 0,2 | |||||||||||
| Project complexity | 0,1 | |||||||||||
| Initial permit documentation | 0,1 | 4,5 | 4,5 | |||||||||
| Total: |
Expert evaluation of project options by expert No. 4
| № | Characteristic | Weight | Project option number | Integral assessment | ||||||||
| A | IN | WITH | D | E | A | IN | C | D | E | |||
| Project duration | 0,25 | 12,5 | ||||||||||
| Availability of alternative technical solutions | 0,2 | |||||||||||
| Price | 0,2 | 7,4 | ||||||||||
| Accommodation | 0,2 | |||||||||||
| Project complexity | 0,1 | |||||||||||
| Initial permit documentation | 0,05 | 2,3 | 2,5 | 2,5 | ||||||||
| Total: | 56,5 |
Expert evaluation of project options by expert No. 5
| № | Characteristic | Weight | Project option number | Integral assessment | ||||||||
| A | IN | WITH | D | E | A | IN | C | D | E | |||
| Project duration | 0,25 | 7,5 | 12,5 | |||||||||
| Availability of alternative technical solutions | 0,25 | 7,5 | 2,5 | 7,5 | 7,5 | 7,5 | ||||||
| Price | 0,2 | 7,4 | ||||||||||
| Accommodation | 0,2 | |||||||||||
| Project complexity | 0,05 | 2,5 | 2,5 | 2,5 | ||||||||
| Initial permit documentation | 0,05 | 2,3 | 2,5 | 2,5 | ||||||||
| Total: | 43,5 |
Definition of integral assessment
| Weight | A | IN | WITH | D | E | |
| Expert 1 | 0,2 | |||||
| Expert 2 | 0,2 | |||||
| Expert 3 | 0,2 | |||||
| Expert 4 | 0,2 | |||||
| Expert 5 | 0,2 | |||||
| Total |
The highest integral estimate corresponds to the project E - the construction of a hypermarket in the village. Kulinichi.
"Project Management Methodology"
Laboratory work No. 4 Hierarchy of works. Phases.
Job Hierarchy
Serves to display the current work structure.
The appearance of the work hierarchy window depends on which Hierarchy View is set in the Hierarchy Diagram Setup dialog box, which is opened from the Diagram–>Customize menu or by clicking the button on the toolbar.
If the Hierarchy view is selected, then the phases in the window are arranged in a vertical hierarchy. Phase information (its Name and Code) is displayed to the right of the phase graphic.
If the Tree view is selected, the phases are displayed as rectangles. The rectangle contains the phase information specified in the template dialog box.
The phases are displayed according to the level of the hierarchy: the top level is the project phase, below are the second level phases, etc.
Each hierarchy level has its own background color, which can be changed from the main program window by clicking the button on the toolbar. In the color settings dialog that opens, you can change the colors of the phases of the corresponding level.
On the work hierarchy, you can display phases up to and including any level (for example, only phases of the first and second levels). The display level is set in the settings dialog box.
Double-clicking on a phase with the left mouse button brings up the phase properties dialog box, in which its main characteristics are set.
By clicking the right mouse button on the phase image, you can call up the work hierarchy phase pop-up menu, which allows you to perform the following actions: edit phase properties, create general reports for the selected phase, create and delete phases.
The job hierarchy is edited with the mouse.
If you move the mouse with the left button pressed on the phase, you can swap phases within each level.
When moving a phase to a lower level, hold down the Shift key, and when moving to the same level, hold down the Alt key.
Phases
Phase properties dialog
It is a dialog box with the following pages: Initial and calculated data, Operations, Sub-phases, standard page Comment and standard page OLE documents and folder.
Page Initial and calculated data
Name - the name of the phase.
Code – a unique phase code.
Type – phase type for communication with directories.
Short name– short name of the phase (usually shorter than the main one).
Priority - phase priority.
Calendar - the calendar used to calculate the duration of the phase in days. For change
calendar, click the Select button, select a calendar from the list in the window that opens, and click OK.
The fields below display the calculated values:
Start (KMR) – the earliest start date of the KMR of the operation included in the phase.
End (KMR) – the latest end date of the KMP of the operation included in the phase - Start (KMP) – the earliest start date of the KMP of the operation included in the phase.
Completion (KMP) – the latest date of completion of the KMP of the operation included in the phase - Level – phase level in the hierarchical structure of works.
Summarize volumes – allows you to summarize the volumes of operations included in this phase. The total volume will be displayed on the line with the phase after the calculation of the costs.
Color in the chart changes the color of the image of this phase on the graphical part of the Gantt chart of works (this opens the object color dialog box).
The Managers frame contains a list of managers to whom the phase will be sent when the subprojects are distributed.
To add a manager to the list, click the Add button and select the required manager in the list of users that opens, then click the OK button.
The Delete button is used to remove the selected manager from the list.
The Responsible button is used to select the responsible manager. The responsible manager is placed first in the list, his name is marked with an asterisk, and during the build, updated subprojects are read from his Prepared for Build [Outbox] folder, the path to which is specified in the users table. Other managers are ignored when assembling subprojects and are required only for distribution of subprojects.
Operations Page
Included operations - a list of operations included in the phase.
Create – button for adding a new operation to the phase.
Properties button to call the properties dialog box of the selected operation.
Delete – button for deleting the selected operation from the project.
Exclude – button for transferring the selected operation to the list of excluded work structure objects.
Excluded operations - list of excluded operations.
Include – a button that inserts an excluded operation into the list of included operations.
Properties – a button that opens the properties dialog box of the selected operation.
Delete – button for deleting the selected excluded operation from the project.
Subphase Page
The field contains a list with subphases included in this phase.
Create – the button for adding a new subphase to the phase.
Delete – button for deleting the selected subphase from the project.
Exclude – button to exclude a sub-phase from the work structure (phases cannot be excluded from the responsibility structure, since it must be complete). When subphases are excluded from the current structure, the objects of the lower level of the hierarchy (operations) are excluded, and empty subphases can be removed from the structure. Excluded objects can be subsequently included in the current structure.
A.A. Shirobokova
O.V. Krivoshchekova
S.V. Uralova
social engineering
Part II
Expertise of social projects
Tutorial
publishing house
Irkutsk State Technical University
Reviewer: Head of the Department of the Governor of the Irkutsk Region for Public Relations and National Relations
Social design. Part II. Examination of social projects: textbook. allowance. - Irkutsk: Publishing House of ISTU, 2007. - 90p.
The textbook contains theoretical and practical experience in the examination of social projects. Presented in study guide teaching materials contain theoretical knowledge about the methods of assessment and how it is applied in the process of social design. The presentation of monitoring and evaluation of social projects is given as social technology. The manual also presents the procedures for selecting applications, samples of official documents used to evaluate applications, the procedure for the work of experts.
The tutorial is the second book in the general series: social engineering. The series consists of four books.
It is intended for organizers and participants of competitions, teachers, students, regulatory authorities and all those interested in the subject of grant competitions, public and other control over them.
The book was published with the support of the Provincial Assembly of the Irkutsk Region
ISBN 978-5-8038-0482-6© Shirobokova A.A., Uralova S.V.,
Krivoshchekova O.V., 2007
© Baikal Regional Union of Women "Angara", 2007
© Irkutsk State
Technical University, 2007
| Foreword…………………………………………………………….4 Expertise of social projects as a form of evaluation project activities………………………..7 Types of evaluation……………..………………………………………….10 What is project evaluation…………….. .……………………………..10 Assessment and project life cycle……………………………………….11 Organizational assessment and project assessment…………………. ..16 External and internal evaluation……………………………………………………………………………………………………………………………. ……………………………….….19 How evaluation is carried out…………………........................... .........................19 Indicators…………………………………………………………………………………………………………………………………. ………………….…..25 Managing the evaluation process……………………………….31 Some approaches to project evaluation…………………………………….34 Expertise social projects………………………………….39 Why is the project expertise carried out……………………………..39 Factors of influence on the formation of expert assessment………………..40 Examination of the application: section by section………………………………………………………………43 Criteria for evaluation of applications…………………………………………………….53 Common Mistakes Grant Applications……………………………………………………………………………………………………………………………………………………………………………………………………………………………..59 …………………. …………………………..66 Bibliographic list ………………………………………..67 Appendix…………………………………………… ……………….69 |
Foreword
The content of the manual takes into account the long-term practice of the authors in the preparation and implementation of social projects of consulting and examination of project applications, teaching special courses, conducting trainings on social design issues for members of local communities, in educational institutions, at specialized courses, advanced training courses for state and municipal employees, etc. The manual also takes into account the experience of international and Russian experts in social design, presented in various publications, presented at conferences, forums, etc.
Social design is a new instrument of Russia's social policy, a mechanism social partnership, parts of local communities in the socio-economic development of their territories, highly effective tool solutions to social problems. Hard work, a clear vision of purpose, organization, and a desire to change lives for the better are often not enough to bring transformation ideas to life. Funding is usually the missing link. Therefore, the practice of social design is increasingly being introduced into the real life of territories not only through the system of international and Russian Foundations, corporations, etc., but also through the system of municipal, regional, regional and all-Russian grants, social orders, etc.
Any sponsor wants to know which project best meets the conditions of the competition announced by him, which grant applicant has more effective specific project results. That's why he orders a project appraisal. And then there is a need to find an expert in the field of project evaluation. An assessment of their own project and the grant applicant is needed. It will help to correct possible errors and avoid failures at the project implementation stage. We hope that the recommendations contained in this manual will help our readers to gain knowledge in the field of project evaluation and create conditions for the development and support of effective projects.
The tutorial is the second part of a general series: social engineering. It is intended for those who want to gain experience in peer review of social projects. It presents the theoretical and practical experience of the examination of social projects. The materials presented in it contain theoretical knowledge about the methods of assessment and how it is applied in the process of social design. The presentation of monitoring and evaluation of social projects as a social technology is given. The manual also presents the procedures for selecting applications, samples of official documents used to evaluate applications, the procedure for the work of experts.
The first part of the series, Social Engineering: A Guide to Grant Applications, provides practical experience in preparing project proposals for grantmakers, sponsors, foundations and other organizations that provide funding to public and non-profit organizations, individual citizens, initiative groups of citizens. From it the reader will learn what social design is, what role it plays in modern social policy, how to write projects. The third one is “Active teaching methods. A guide for trainers” is designed to train trainers in social engineering. It contains general materials on active learning methods, organizing and conducting trainings, methodological developments for trainings on social design. The fourth book "Social design: practical experience" contains positive practical experience of social design of public and non-profit organizations, citizens, initiative groups of citizens of the Irkutsk region. It presents projects that have received support from various funding sources.
A recently widespread form of attracting resources (material and personnel) for solving social problems by various foreign and domestic funds, as well as authorities different levels- from federal to local - grant competitions. If there is competition, then it is necessary to evaluate projects and choose those that solve the problem most effectively and least costly. In addition, the grantor is not indifferent to what the specific results of the project are at a certain point in its implementation or at the end of the project. Of course, the latter can be seen from the reports of the project executors, but the opinion of outsiders is also important. Then the sponsor orders the evaluation. And then, for project evaluation, a specialist is ordered who is able to perform the task of an expert in the field of project evaluation.
It is often important for the performer himself to know how his project is progressing, how it corresponds to the declared results. Timely clarification of this will allow timely adjustments to be made to the implementation of the project. That is why it is important that among the project executors there is a specialist who is familiar with the evaluation of project activities.
This book discusses various aspects of project evaluation. A bibliographic list is provided at the end of the book.
Expertise of social projects as a form of evaluation of project activities
Why have grant competitions become so popular in solving social, economic and civil problems?
Grant competitions have shown themselves to be the most effective form of distributing funds to solve these problems: financing projects on a competitive basis stimulates the attraction of additional resources by the organizations participating in the competitions in the form of their own material and professional resources, gratuitous activities of volunteers. As a rule, one third of the funds invested by the donor, the grantee organization invests or attracts on its own.
Competitive financing allows evaluating the effectiveness of invested funds, as it provides for monitoring the process of implementing grant projects and reporting from grantee organizations on the results of project activities.
Grant competitions allow you to attract a wide range of participants and choose from them those who can most effectively and least costly solve a social problem in the community. Consequently, competitions develop a healthy competitive environment in the non-profit sector.
The use of competitive technologies allows to reduce the investor's costs up to 15-20%. Thus, the introduction of mechanisms stimulating competition can produce a tangible economic effect.
Project logic is applied in grant competitions. It allows you to see and solve problems holistically, as well as achieve long-term results and social impact. Design is also a powerful stimulus for the development of the activities of the organization applying for a grant. Even if the application is rejected: writing a project involves, among other things, an analysis of one's own activities, the shortcomings in it and possible ways to improve. Thus, participation in grant competitions encourages the organization to constantly improve, strive for greater efficiency, to achieve long-term, sustainable results.
Grant competitions also contribute to the development of the market social services, forming the latter in the process of implementing project activities. The goal of any grant competition is as follows: there is a problem that needs to be solved, and all project applications submitted for the competition offer different options for solving it. These solutions are implemented and the most effective ones are fixed in the form of services for which other, regular sources of funding are sought. As a result, new services, services, models, technologies for working with target groups appear and are fixed, which can be reproduced and replicated.
Grant competitions have a wide public resonance, as the ongoing social projects are an interesting informational occasion for the media, and the inhabitants of the territory begin to look optimistically at the solution of social problems, which contributes to the formation of a positive public opinion.
Separately, it is worth highlighting grant competitions, which are held with budget money by government authorities of different levels. Of course, this is a positive development. Using all of the above factors for the effectiveness of competitions, the authorities, in this way, also build constructive interaction with civil institutions.
Recently, the practice of holding grant competitions on the basis of a consolidated budget has also been spreading, when the competition fund is made up of various sources: any fund, sponsorship contributions, administration. In such competitions, not only the competition fund increases, but also the resources (financial, professional, methodological, organizational, influence resources, etc.) of different organizations are combined to solve a social problem.
A special place is occupied by grant competitions in the implementation of the mechanisms of social responsibility of business. They allow the business community to effectively invest in the development of local communities.
If there is competition, then it is necessary to evaluate projects and choose those that solve the problem most effectively and least costly. And here there is a need for experts who are able to competently evaluate the solutions proposed in the project. The grantor is also not indifferent to what the specific results of the project are at a certain point in its implementation or at the end of the project. And here again, a specialist-expert in the field of project evaluation is required.
Evaluation is a special process that requires special knowledge. That is why now more and more attention is paid to the training of evaluators in general and in social sphere in particular. At the same time, it is important to note that among the methods and techniques of evaluation, there are those that can be quite easily mastered by practitioners and used for the examination of social projects at certain stages of its development.
Project activity has a certain duration, stages of its development. And at every stage life cycle project, there is a need to evaluate (*):
- Idea--* --Development--** --You-* -fuller-* -nie--*--Impact--*
Therefore, there are different types of assessments. Each stage imposes its own characteristics on the type of assessment that is carried out at this stage. In this tutorial Special attention Evaluation takes place before the start of the project, after it has been developed, and before funding is approved by the sponsor. Before a project is approved, a special evaluation is usually carried out, which is called expertise (**). A conclusion is made, which compares the possible results of the project and possible costs for its implementation. Compliance of the project with the conditions of the sponsor is clarified. Moreover, it is the expertise that determines the possibility of an effective assessment at subsequent stages. design work, since at the stage of examination the basic conditions of project activity are assessed. In order to control the progress of the project and the achievement of the planned effect, to provide conditions for evaluating project activities, when developing a project and designing a project, at least it is necessary:
Evaluate the initial situation, to change which the project will be aimed at;
Determine what changes the project executors would like to receive "at the exit" - that is, after the completion of the project;
Develop a set of measures that, according to the authors, could provide such changes;
Form a system for tracking and evaluating such changes.
How this is achieved is shown in Part I of the Social Engineering Tutorial - Grant Application Writing Guide.
Valuation types
What is project evaluation
Different people may have very different ideas about what an assessment is. Academic Dictionary Russian language gives the following definitions:
Evaluation is an action aimed at forming an opinion, judgment about quality, dignity, value, etc. something;
Evaluate - study and make a balanced judgment.
The meaning of the term "assessment" has expanded over time. Major changes have taken place over the past 30 years. This is due to the fact that the number of socially oriented projects and programs aimed at positive changes in various areas life activities (protection of the environment, education, women's rights, support for people with disabilities, etc.). And in this case, the assessment acquires an extended meaning. That is why a number of evaluation experts have introduced a new aspect to it - the purpose of the evaluation. For example, Carol Weiss defined the purpose of evaluation as measuring the results of a program and comparing them with the objectives that the program should address. She identified evaluation as one of the means to facilitate subsequent program decision-making and to improve the programming process in the future. A systematic analysis of the results of each project allows you to determine the following:
Which programs worked well and should be expanded;
Which programs have failed and should be discontinued;
Which programs require modification for various reasons.
A feature of this approach to evaluation is not only the need for an evaluation goal, but feedback is important - the flow of information to managers, who, on its basis, should make decisions about the future development or closure of the program (project). However, as Robert Stake points out, evaluation can have many different purposes. Evaluation questions reflect the different goals and information needs of the audience. In a formal evaluation, only a few issues can be prioritized.
Historically, the main consumers of evaluation have been those who financed the evaluation. It was they who formulated the evaluation questions. Today, that focus has begun to shift. Society has begun to realize that there are various groups that have a legal right or certain needs to be informed about the results of projects (programs). Peter H. Rossi and Howard E. Freeman point out that evaluators must be able to understand the views of all stakeholders and take them into account when planning and conducting an evaluation.
Estimation and project life cycle
A project is always a hypothesis. With varying degrees of confidence, it is assumed that the actions developed in the project will lead to the expected result. Many factors, both inside and outside the organization, influence the effectiveness of project activities. Special tools are needed to assess these influences. In many countries of the world, tools such as project monitoring and evaluation.
Grade allows you to form an opinion, a judgment about quality, dignity, value, etc. project and its results. The task of the expert is to study and make a balanced judgment about the project or its actual results.
Monitoring- current control over the results of work. It allows you to determine whether all planned activities have been carried out? Is it in time? Has the planned number of project participants been covered? Were the funds spent according to the budget? Etc.
A project is an evolving process that has different life cycles (stages), and each stage has different types of evaluation. The relationship between the stages and types of assessments of project activities is shown in Scheme 2.
Idea--development--approval--execution--completion--aftereffect
evaluation expertise monitoring evaluation evaluation
impact outcome situations
Assessment of the situation is carried out before the development of the project (application for the competition), program. This is the work that the project developers must complete before the preparation of the project, after they have the idea to write the project. As a rule, this stage of the evaluation is omitted. This problem discussed in Part I of Social Engineering: A Guide to Grant Application Writing (Chapter 1.3, The Five Cycles of Project Management).
Draw on the experience of colleagues in other regions;
Realize an old idea
Prepare an application, because a competition has been announced, and the topic falls into the sphere of interest of the organization;
They do not have the skills to assess the situation.
Today, however, this particular type of assessment, often included in separate section applications becomes significant in the competitive selection of projects. That's why expert task- be able to assess how well this issue has been worked out in the proposed project. After all, it is precisely the assessment of the situation that allows:
Determine exactly what the problem is;
Identify the compliance of the activity with the resources (opportunities) presented in the project, mission, goals and objectives;
Determine the investment attractiveness of the project;
See the long-term possible results, project prospects;
Form performance criteria;
Evaluate the effectiveness of the chosen strategy, the realism of the budget;
- evaluate the qualifications of performers;
Efficiently reallocate resources.
There are special methods, assessment tools that allow you to assess the situation. The same tools are used in other stages of project evaluation. This topic is covered in more detail in Chapter 3, Assessment Tools, of this tutorial.
Expertise (preliminary assessment). This evaluation takes place during the approval phase of the project and is not yet evaluated. completed project. Project as a proposal for future work. Expertise determines the quality of the project. Grant applicants typically face scrutiny when applying for a grant. As experts, the grantor attracts specialists with experience in project evaluation and the nominated area (social, civil, economic, etc.). Experts read applications and determine whether they are worth supporting. In principle, evaluation can have many different purposes. However, by and large, in this case, experts need to answer two questions:
Can the proposed project contribute to the solution of a problem that is this moment priority for this or that donor;
Can this project be implemented under the given conditions by the applicant organization.
These questions include a host of characteristics and criteria (is the project planned well enough; does the organization have experience in project implementation and can it fulfill this project; whether project resources are justified, etc.). It is important to note that even if an organization does not plan to submit a project for a grant competition, it is advisable to conduct an examination of it. Colleagues, employees of the organization can act as experts. The third-party view of an experienced manager will help to correct possible mistakes in advance and avoid failures at the project implementation stage.
This tutorial will focus on this type of assessment. However, without understanding overall picture appraisal activities it is difficult for an expert to prepare a high-quality and professional opinion on a grantor's application.
Monitoring is a control tool that allows you to track the progress of the project, the degree of achievement of intermediate results and make operational changes in order to increase the efficiency of the project. Monitoring is carried out continuously, with a certain frequency or at predetermined stages of work. For its implementation, objective indicators are needed, which are called indicators. In chapter 3.2. "Indicators" of this manual, this concept will be discussed in more detail.
Monitoring, as a control tool, is most often used by employees implementing the project. The grantor may also monitor the project by engaging external evaluators. The sponsor may include in the conditions of the competition the requirement for monitoring by the project executors at certain stages of its implementation. In the latter case, the project implementer, based on the monitoring results, prepares interim report for the donor. It is important to understand that monitoring results are just as important for project implementers as they are for monitoring customers. Monitors perform this work in order to identify strong and weak sides project, to help the performers adjust their actions so that the project is successfully implemented.
The use of monitoring at the project stage provides an opportunity to:
Adjust the further course of the project;
Identify potential implementation problems, identify their causes and resolve them in a timely manner;
Determine the level of implementation of the stated tasks and predict the final result;
Accept literate management decisions;
Discover new aspects of the problem;
Get qualified assessment and advice;
Check how rationally resources are used;
Reveal unintended outcomes.
Below are the basic hallmarks of monitoring and evaluation.
Table 1
The end of the table. 1
Evaluation of results carried out at the time of its completion, upon receipt of immediate results. The donor must require it from the grantee in the form final report. To conduct a final assessment, it is necessary to compare the results obtained with the declared project model. Once again, an important place is given to such an assessment tool as an indicator. As a rule, the grantor requires not only a statement of the achieved quantitative and qualitative indicators, but also an assessment of how and why this happened. Thus, on this stage project activity evaluation is the analysis and interpretation of facts in order to formulate recommendations for future programs and applications.
The use of evaluation at the final stage of the project implementation allows:
Understand whether the project objectives have been achieved;
Learn lessons for the future;
Better understand the problem;
Assess project personnel;
Adjust strategy for the future;
Better understand the strengths and weaknesses of the project, goals and objectives;
Continue or repeat the project at the best level;
Raise a question about further funding;
Minimize risks.
Long-term impact assessment (project effect, impact assessment). It is aimed at assessing changes in the problem / situation in connection with the implementation of the project. Such an evaluation usually analyzes all (or almost all) changes caused by the project, and how stable they are, that is, whether the changes persist without further impact from the project. As a rule, the effect of the project is very difficult to plan. Moreover, it is difficult to evaluate it immediately after the end of the project. But the project can be considered really effective if the immediate results have an impact on the situation in society. In fact, the effect of the project is the degree of success of the project in terms of achieving the goal. But if the authors of the project did not have a clear idea of what change they wanted to achieve, it is impossible to say after the project was completed whether it succeeded or not.
Often, in parallel with the actions of the project executors, other actions (projects, programs) are also implemented to solve the same problem. Therefore, even if it is known exactly what the project implementers were trying to achieve a few months after the end of the project, when the situation actually changes, it will be difficult to say with certainty how much the project merited. And then a long-term impact assessment, an impact assessment, can help answer that question.
It is easy to see that the types of assessment described above overlap with each other. In particular, an impact assessment is nothing more than an assessment of a situation in terms of how it has changed since the implementation of the project. Therefore, these two types of assessment - situation assessment and impact assessment must use the same assessment tools, otherwise it will be impossible to compare the results and understand what exactly has changed.
Evaluation of the organization's activities and project evaluation
Depending on what is the object of evaluation, we can talk about the evaluation of the organization's activities and the evaluation of the project. Further we will talk about project evaluation, but it is important to understand that the evaluation of the organization's performance can also use the methods and techniques presented in this book. It is only important to understand why an assessment of the organization's performance is needed.
From the point of view of the organization itself, it is needed because:
· conducting an assessment, the experts will delve into the analysis of the organization's activities; they will pay attention to both successes and failures, that is, they will come closer to seeing the activities of the organization as a whole;
· experts look at the organization as if from the outside and have a chance to avoid the effect of "blurry eyes", distract from everyday "churn" and become more objective in relation to their organization;
employees, volunteers and service recipients (clients, wards) of the organization will receive visible confirmation that the organization is working for results, to achieve its goals and objectives, its mission;
It becomes possible to find the so-called points of growth, moments that have a decisive influence on the quality and efficiency of the organization. In the future, by influencing precisely these points, it is possible to achieve significant positive changes, thereby saving the strength, time and resources of the organization;
· The evaluation will give an occasion to reflect on the quality standards to which the organization should aspire.
From the point of view of the environment in which the organization operates I:
using the results of the assessment, the organization can quite definitely answer the question of how well it works;
· in negotiations with their potential donors, specialists will be able to provide facts and comparative characteristics that will help to understand what the applicant is like as an organization and how effectively this organization works;
· periodic evaluations make the organization a more attractive, more reliable partner;
· Some types of donors (for example, grant-giving organizations) require that such an assessment as an assessment of the organization's performance be included in the application for funding a particular project.
One way or another, the assessment helps to see the real state of affairs in the organization, to determine ways to consolidate success and correct shortcomings.
External and internal evaluation
External evaluation carried out by an invited specialist. At the same time, it is important that he not only does not work in the organization, but is also not connected with it by relations that can be regarded as a conflict of interest. For example, such a specialist should not be:
A direct relative of any of the leaders of the organization;
One of those who receive help from her;
Participant of the same program, grant competition.
An external evaluation conducted by an external expert is also called an independent evaluation.
There are several reasons why sponsors, grant applicants request evaluation during project implementation:
Lack of good information systems;
They have Information Systems answer the question "what is happening", but do not answer the question "why";
Evaluators are trained to look at the big picture and see how the elements fit together - while others may only look at part of the big picture;
The evaluators have time to interact with project beneficiaries and ask questions that project staff may not have.
Internal evaluation, or self-assessment, is carried out by the same people who work in the organization. It is easier and more difficult at the same time, because, on the one hand, who knows the organization better than its employees, and on the other hand, such an assessment will require additional efforts, and sometimes courage, in order to “rise above the situation” and be truly objective.
What can be analyzed during the assessment
Everything related to the project can be subject to evaluation. If we talk about the most common points that are evaluated, and about the questions that are posed, then this, as a rule:
financial efficiency of the project
How acceptable is the ratio of resources involved in the project (as a whole and / or for the implementation of individual stages) and its results;
Is it possible to achieve a reduction in the cost of similar projects in the future, while maintaining the same quality and the same volume of results;
project management efficiency
Whether the selected management scheme is appropriate for the type of project and activities to be carried out;
Did she contribute to the successful implementation of the project;
Whether the project managers were flexible enough to respond to changes that affected the course of the project;
How correct (erroneous) were the decisions made;
project methodology
Did all methods lead to the achievement of the stated results;
What methods turned out to be the most (least) effective;
project sustainability
Will this project work if it is repeated elsewhere;
Has the project remained relevant, does it correspond to the current moment;
Will the project be viable after the completion of its special funding (as a rule, this issue is of particular concern to foundations);
completeness of solving a problem or meeting a need
To what extent the project solved the stated problem;
Is it possible to say that the problem has been radically resolved, that is, it will not reappear;
Are the needs of the project's clients (target group to which the project is directed) sufficiently satisfied?
Assessment tools
How assessments are made
Regardless of the type of evaluation, there are certain tools used by experts and developers of projects and programs. At different stages of the project life cycle, depending on the purpose of the assessment, different tools are used. Among the most commonly used assessment tools, especially at the stages of situation assessment and long-term impact assessment, the following can be distinguished: questionnaire, survey, observation, interview, conversation, press analysis, legal analysis, text analysis (oral / written), SWQT analysis, profile community/client and others.
Each tool requires certain skills and knowledge of its use. Let's take a general idea of some of them.
Questionnaire This is a written survey. Types of surveys can be probing or express surveys. The questionnaire contains several items of basic information, the demographic characteristics of the respondents, and other characteristics required for each specific case. Questionnaires may take different forms:
Through the press - it is convenient, but very difficult to get completed questionnaires from respondents;
Telephone or by mail (it is expected that the questionnaire will be returned by mail on a prepaid transfer);
On the spot (the surveyor himself collects the completed questionnaires); distribution (at the place of residence, work);
Group - up to 40 people, whom the questionnaire instructs and leaves to fill out questionnaires);
Individualized (individually to the respondent), etc.
Survey- the method of collecting primary information (a set of questions offered to the respondent, whose answers form the necessary information). Polling is an almost universal method, but you need to know: whom to ask; what to ask; how to ask; where to ask; how to process the received data; how to make sure that you can trust the answers received.
Questions can be: direct / indirect; open / closed / semi-closed; personal / impersonal; basic / control; hook questions (to keep interest) / trap questions / trap questions
"filters". Surveys Require Engagement a large number people, they are costly in terms of time, finances, and the data obtained require processing and interpretation.
Interviewing is a direct, oral survey (a conversation conducted according to a specific plan). The interview happens:
Free - a long conversation without a strict specification of questions, but according to the general program;
Standardized / formalized - detailed development of the entire survey procedure, sequence and design of questions (deviations in the wording of questions and the order in which they are asked are not allowed);
Semi-standardized (clarifying questions are allowed);
Narrative - a free story directed by the interviewer, subject to qualitative analysis;
Non-directional - a conversation in which the initiative is almost entirely owned by the respondent;
Group;
Individual.
Observation- a qualitative method of collecting primary information based on individual perception. This method is good in combination with other methods, as it does not always provide a sufficient amount of information. In addition, observation is very subjective, and data obtained through observation require confirmation.
Press analysis / legal analysis / text analysis. Despite the apparent simplicity, this method is extremely laborious and requires a professional approach. However, it is quite cheap, but it takes a lot of time. As a result of using this method, it is possible to obtain high-quality, in-depth information, which will then need to be correctly interpreted.
SWOT analysis. This approach is widely used around the world and is an effective tool to start work on strategic plan or a large project, when serious development efforts are needed to effective solution assigned tasks. It is also used to plan the development of an organization, analyze a new idea, etc. SWOT is an abbreviation for:
"S" - means advantages, "W" - weaknesses of the organization (project, plan, idea). These two characteristics describe the organization's vision (project, plan, idea) at the present moment.
"O" is the opportunities that may appear, "T" - the threats that will have to be faced. Using a SWOT analysis framework, determine what can be done to overcome weaknesses. After analyzing the nature of the benefits, activities are planned to strengthen them. Having understood the threats, they develop measures to minimize them or transform them into opportunities and potential strengths.
Focus group This is a group interview. Moderator, i.e. the person conducting the conversation writes a discussion script ahead of time to focus the discussion on the right problems. The focus group is also interesting for the organizers of the assessment because it involves an active exchange of opinions between people whose attitude to this problem needs to be known. It is also useful to use a focus group when the exchange of views between participants can lead to a deeper understanding of not only the participants' point of view, but also the problem as a whole. For example, in the process of working in a focus group, it is possible to find out whether the leaders of public organizations working on the problem of violence will accept the idea of uniting in a coalition, what obstacles are on this way. If you interview a group of leaders and each of them hears the opinion of the other participants, the leaders will be able to think more deeply about the idea and the possibilities for using it for themselves, which means that the coalition-building process will be more successful. Moreover, the survey will determine the basic technologies for creating and developing a coalition.
Community Profile- contains information about the local population and is used to determine the needs of the local community. It is also called the "social passport". The community profile allows you to:
Get necessary information to determine the necessary services and the level of their provision;
Identify target groups of potential recipients of services;
Give a clear understanding of the people with whom it will be necessary to work in the process of implementing services;
Give a qualitative assessment of the situation;
Explain the aims and objectives of the project social decision;
Choose effective system indicators characterizing the change in the situation as a result of the implementation of the project (plan);
Provide new opportunities for social policy in the local community.
Undoubtedly, a serious issue when using these tools becomes the issue of obtaining reliable information, especially when conducting rather complex types of assessment at the stages of situation assessment and impact assessment. Of course, professional experts will be able to do this without any problems, but if it is not possible to involve them, the following sources of information can be used: statistics (official / unofficial); gossip; official requests; various kinds of databases; MASS MEDIA; Internet, etc. Not all sources of information are reliable, so it is necessary to check the data before interpreting it.
Indicators
It has already been noted above that the assessment allows you to form an opinion, a judgment about the quality, dignity, value, results, etc. project (plan, program, organization). Obviously, in order to conduct an assessment, some criteria are needed, with the help of which it is possible to determine whether the project (program, plan) is being carried out or completed well, whether the organization is working well, whether the declared project is able to change the situation for the better on the stated problem, to evaluate success. The criteria are the so-called performance indicators (indicators). For a more accurate understanding of the rather complex concept of "indicator", let's look at an example. Which indicator allows you to say with exact certainty that a person has normal temperature? This is a thermometer. The thermometer is a reliable, neutral, practical, unambiguous, reliable indicator that allows you to accurately determine body temperature. What indicators make it possible to judge a person's health? Temperature, pressure, pulse, skin color. The baseline values of these indicators are known and can be measured. In the social sphere, defining indicators is not an easy task. However, there is always an easy way to measure performance here.
Measurement is a process of transformation/refinement of information.
The necessary conditions to organize this process:
Determination of what exactly (which parameter) will be measured;
Consistent and clear definition of the measured parameter;
Availability of adequate tools for converting non-quantitative data into quantitative data.
The measurement should provide: reliability, reliability, adequacy, accuracy of the information collected.
A good indicator should have the following characteristics:
Measurability. An indicator is quantifiable at any given point in time;
Result orientation. The indicator characterizes the result, not the process during which this result is obtained;
Reliability. The indicator actually evaluates exactly what needs to be evaluated. This is a true indicator for assessing the intended outcome;
Reliability. The indicator will give the same result when applied again - the next day or a year later - whether it is evaluated by the same person or someone else. The measurement process can be repeated. Any changes in responses are due to changes in the thing being measured and not in the measuring instrument;
Neutrality. The indicator is "objective", not "subjective". Both the proponent and the skeptic will accept the evaluation's answer as correct;
Unambiguity. The indicator has a clear definition that is equally understandable to anyone;
Practicality. The indicator is practical in the sense that it is affordable, i.e. does not require a lot of costs for its application and allows for a real assessment.
Thus, when defining an indicator, it is important to clearly define the basic components of the indicator: definition (formulation of the concept/parameter in such a way that it can be counted); The source of information; periodicity; starting point; target value; unit of measurement.
For example:
The demand for a resource center can be measured by the number of clients; the number of applications for this or that consultation, etc.;
The effectiveness of the grant writing workshop can be measured by the number of applications received for the competition; the quality of projects submitted for the competition (using the average score scored by the participants of the seminar during the verification test), etc.
For a more complete understanding of the concept of "indicator", Appendix 1 of the manual presents the indicators of the Millennium Goals developed by the UN at the Millennium Summit and gender indicators that determine gender development for each Millennium Goal.
Sometimes an expert or developer finds that there are no "measured rulers" for the indicators he wants to measure. Project objectives that focus on the quality of the outcome or how people relate to new service, are tasks for which there may not be a convenient "indicator". When such situations arise, you need to be able to create appropriate "measurements". The most common type of "measurement" that project developers and evaluation experts create when they need new "measurements" are scales, or rating scales.
The quality of the trainings provided within the framework of the project can be assessed as: the highest, good, average, below average, poor quality;
Attitude towards the new health care service:
very satisfied, satisfied, don't know, dissatisfied, extremely dissatisfied.
By counting the number of participants who determined the results of the project in accordance with the rating school, it is possible to quantify the quality of the result or the attitude of people to the new service.
Similar information.
Under expertise in a broad sense means:
» analysis, research conducted by involved specialists (experts), an expert commission, culminating in the issuance of an act, conclusion, in some cases - a certificate of quality, conformity;
» checking the quality of goods, works, services.
Examination (verification, evaluation) is an obligatory stage of almost any activity, since it is designed to assess the compliance of the result of the activity with the planned indicators. Expertise, as a rule, is carried out on the basis of certain rules documented in the form of departmental, regulatory, legislative acts.
All investment projects, regardless of sources of financing and forms of ownership of capital investment objects, are subject to examination in accordance with the legislation of Ukraine before they are approved.
Project expertise is carried out in order to prevent the creation of objects, the use of which violates the rights of individuals and legal entities and the interests of the state or does not meet the requirements of duly approved standards (norms and rules), as well as to assess the effectiveness of ongoing capital investments.
First step implementation of this methodology - the identification of factors that can significantly affect the success of the project. Among the factors that have a primary impact on the effectiveness of an investment project, there may be the characteristics presented above.
Second step The factors are arranged in descending order of priority. To do this, it is determined which of them will most affect the progress of the project. Next, the most significant factor of the remaining ones is established, and so on. The resulting sequence is listed in Table 1.
Third step- assessment of the weight (rank) of each of the listed factors. The sum of the ranks of all factors must be equal to one. In other words, the sum of column 3 of table 1 must be equal to one.
Fourth step– the project(s) or variants of one project must be evaluated for each of the evaluation factors (criteria).
The maximum score for any of the factors for the project is 100, the minimum is 0. For example, if experts recognize that the demand for the project's products will be unlimited, then the value of the factor "demand for the products (services) of the project" for this option project is equal to 100 points.
Fifth step- expert assessment of the influence of each factor (columns 9-13) is obtained by multiplying the weight of each factor by the assessment of this factor for each option (column 3 is multiplied by columns 4 to 8, respectively). Integral expert assessment of the priority of project options is determined as the sum of columns 9-13.
Table 1 - Form for peer review alternative projects
| No. p / p | Characteristic, factor | Weight indicator | Project number (or project variant) | Integral assessment of the project | ||||||||
| ... | ... | |||||||||||
| Total: | - | 1,0 | - | - | - | - | - |
This technique can be used both for the preliminary selection of the most promising options for the implementation of the project, and for the preliminary determination of the feasibility of the project. In the first case, the alternatives that received the highest results remain for further consideration, in the second case, the obtained integral expert assessment of the project is compared with the “lower limit” determined in advance. If the value obtained by expert means is higher than the established limit, the project is recognized as feasible.
If the project is worthy of further consideration, determine the composition of the information that will be required for its development, including:
» detailed marketing;
» engineering and geological surveys;
» assessment of the environment and local sources of raw materials;
» the political situation in the region, republic, country;
» socio-cultural characteristics of the population.
The reader, accustomed to dealing with criteria tables, usually immediately recalls a simple way to order alternatives. In the vast majority of cases, this is the so-called "linear convolution" (weighted sum) - a method of processing a criteria table, beloved by all peoples and at all times. Its essence is simple. First, the weight coefficients of the criteria are selected in some way. Denote them by the vector (w1 , w2 ,... , wm). Then, for each alternative (each i-th row of the table), the following value is calculated
si = ∑ xijwj (the sum is taken for all j from 1 to m). Finally, the rule is adopted: the larger the value of si , the better the alternative ai. That's all!
Unfortunately, this scheme does not always give the correct result! The inexperienced reader is always perplexed by this statement. Statements follow, such as that the given scheme "corresponds to common sense", or "corresponds to an intuitive idea of the comparative quality of alternatives", etc. Here we are faced with a typical situation, which is aptly expressed by the well-known phrase "science begins where it ends common sense". Alas, it is so! At the end of the 20th century, mathematics reached such a level of abstraction that common sense receded into the background. In one of the classic books on SPR methods, namely, in the book of American mathematicians R.L. Keene and X Rife "Decision making under many criteria: preferences and substitutions" (Moscow, Radio and Communications Publishing House, 1981) it is rigorously proved that linear convolution is correct only when all criteria are pairwise independent in preference. criteria, what types of addiction exist, and what follows from this - all this is beyond the scope of our brief introduction.
Go ahead. It turns out that linear convolution is based on an implicit postulate: "a low score in one criterion can be compensated by a high score in another." However, this postulate is by no means true for all models of comparative evaluation of "quality". The simplest example is that a deterioration in the quality of a television's picture cannot be compensated for by an improvement in its sound quality.
But that's not all. Serious problems are related to the criteria. First of all, it is not always possible substantiate the set of criteria that is necessary and sufficient to solve the CPR. It may seem that a set of criteria "naturally" arises in each specific task. But, alas, this is far from the case.
The situation with criteria weights is even more complicated. It can even be said that the criteria weights are the most subtle point in the problem of the criteria ordering of alternatives. Most often, weights are assigned based on an intuition about the comparative importance of the criteria. However, studies show that a person (expert, decision maker) is not able to directly assign the correct numerical weights to the criteria. Moreover, there is evidence (not yet published) that indicates that a person cannot correctly assign weights even on the basis of non-numerical scales. Why do people so often and so willingly manipulate the weighted sum? On this occasion, I cannot resist the temptation to quote an excerpt from the excellent book by Elena Sergeevna Wentzel "Operations Research (Problems, Principles, Methodology)". In the following excerpt, the criteria weights are called "coefficients", the alternatives are called "decisions".
“Here we meet with a very typical technique for such situations - “transfer of arbitrariness from one instance to another.” A simple choice of a compromise solution based on a mental comparison of all the pros and cons of each decision seems too arbitrary, not “scientific” enough. And here maneuvering with a formula that includes (albeit just as arbitrarily assigned) coefficients is a completely different matter. This is already "science"! In essence, there is no science here, and there is nothing to deceive ourselves.
The book was written in the late 70s. Interestingly, around the same time, a scientific approach to the problem of criterion weights was born. Its author is the remarkable mathematician Vladislav Vladimirovich Podinovsky.
In the book of E.S. Wentzel there is a reference to one of the early works of Podinovsky, written by him in collaboration with V.M. Gavrilov: "Optimization according to consistently applied criteria", - Moscow, "Soviet Radio", 1975. It is curious that the analysis of just one approach (successive consideration of criteria ordered by importance) took about 8 printed sheets! Subsequently, Podinovsky managed to give a strict definition of the concept of "importance of the criterion" and publish in this area applied mathematics several monographs and many articles. Vladislav Vladimirovich can rightly be considered the founder scientific approach to the problem of the importance of criteria. To this day, he remains the world's number one authority on this issue. But back to the essence of the issue.
If everything is so complicated, then how do you go about structuring the alternatives presented in the form of a criteria table? This is what we will do now. First of all, we note that the table may contain alternatives that have worse scores for all criteria than other alternatives. It is immediately clear that such alternatives are uncompetitive. They can be safely deleted from the table. After deleting the obviously worst alternatives, only those alternatives remain in the table that, according to at least one criterion, are no worse than the others. The set of such alternatives is called the "set of non-dominated alternatives", or "Pareto set".
1.2 Petition (Declaration) of intent
The investor's idea is realized in the form declarations of intent, as well as tasks (initial data) for the development of pre-project justifications for investments.
These documents are prepared, in addition to the customer, by consultants in the field of project management, as well as experts on special issues. At the same time, an application for preliminary approval of the location of the object is being prepared.
The stage, in addition to the customer, is prepared by the design institute (under the contract), interested legal and individuals(determined by the customer), as well as specialists from a consulting firm.
Approximate composition of the Petition (Declaration) of intent, by example construction project:
1. Investor (customer) - address.
2. Location (district, point) of the enterprise, structure planned for construction.
3. Name of the enterprise, its technical and technological data:
· the volume of industrial production (provision of services) in value terms as a whole and by main types in physical terms;
· the term of construction and commissioning of the facility.
4. Justification of the socio-economic necessity of the planned activity.
5. Approximate number of workers and employees, sources of satisfaction of the need for labor force.
6. The need of the enterprise for raw materials and materials (in the appropriate units).
7. The need of the enterprise for water resources(volume, quantity, source of water supply).
8. The need of the enterprise for energy resources (electricity, heat, steam, fuel), source of supply.
9. Transportation.
10. Provision of workers and their families with housing and communal and social facilities.
11. The need of the enterprise for land resources.
12. Wastewater disposal. Cleaning methods, quality Wastewater, discharge conditions, use of existing or construction of new treatment facilities.
13. Possible impact of the enterprise, structure on the environment:
types of impact on the components of the natural environment (types of violations, name and quantity of pollutant ingredients);
Possibility of emergencies (probability, scale, duration of impact);
production waste (types, volumes, toxicity), methods of disposal.
14. Sources of financing for the planned activities, founders, shareholders, financial institutions, government, commercial banks, supplier loans.
15. Usage finished products(distribution).
1.3 Development of the investment case
Approximate composition of the investment justification:
1.Project summary
2. General characteristics of the industry and enterprise
3. Initial data and conditions, including:
3.1.Goals and objectives of the project
3.2. Characteristics of objects and structures, including:
3.2.1. Capacity of the enterprise, product range
3.2.2.Main technological solutions
3.2.3 Basic building decisions
3.2.4. Location of the enterprise
3.2.5. Providing the enterprise with resources
3.3 Project environment
3.4 Environmental impact assessment
3.5. Current (initial) state of the project
3.6 Personnel and social development
4. Market analysis, including:
4.1. Characteristics of the project product market
4.2. Evaluation of the competitiveness of the project products
4.3. Forecast of the development of the project product market
4.4. Forecast of demand for project products
5.Project management, including:
5.1. Enlarged structure of work
5.2 Project plan
5.3 Project management structure
5.4. Project team
6. Evaluation of the effectiveness of the project, including:
6.1 Initial data and calculation results
6.2.Financial plan
6.3 Risk analysis
7.Applications
The final result of the pre-investment studies is the assignment for the development of a feasibility study. Indicative list the initial data for its development is given in the handout.
An approximate list of initial data for the preparation of a feasibility study for construction
1.Materials of economic and social development targeted scientific, technical and integrated programs of Ukraine, including intersectoral programs.
2. Copies of decisions of state and local authorities.
3. Information about the state of the resources involved in economic activity the future object (enterprise), the natural environment about the infrastructure, about recreational and specially protected areas.
4. Information about the possibility of using imported equipment at the facility (enterprise).
5. Exemplary production program in monetary and physical terms, nomenclature of the main and by-products, requirements for its quality and competitiveness.
6. General characteristics of the construction object (enterprise), information to determine its optimal capacity.
7. Conclusion of the Antimonopoly Committee on the impossibility or inexpediency of increasing production at existing enterprises.
8. Results of previous research and development work on technological processes, equipment, research existing market and trends in its development, etc.
9. Copies of approvals in accordance with the established procedure of design and planning documentation indicating the proposed construction site (sites) and possible places for connection to networks and communications.
10. Data on buildings and structures to be demolished during the construction of the facility, the approximate number of resettled citizens.
11. Other data characterizing the features of the project being implemented.
