Risk identification is the process of identifying risks that could affect the project and documenting their characteristics. Risk identification is performed by the project team members and experts in risk management; it can be participated by customers, project participants and experts in certain areas. It is an iterative process, because as the project progresses through its life cycle can be detected new risks. The frequency composition of the iteration, and each execution cycle in each case may be different. The identification process should involve members of the project team so that they develop a sense of ownership and responsibility for risks and actions to respond to them.
Delphi method is a multi-step method of forecasting and risk identification, providing for the original isolated experts’ judgments and further multiple corrections. They are based on an acquaintance of each expert with judgments of other experts as long as the value spread of ratings will not be within the pre-established desired spacing of varying estimates (Yousuf, 2006). Evaluations obtained by these methods are static and one-time in nature, thus requiring repeated access to experts in the forecast market share, in the subsequent periods. Moreover, the method of internal and external expertise is characterized by a certain degree of prediction of subjectivity.
Reliability of the Delphi method is considered high when forecasting for the period as from 1 to 3 years, and for more distant time (Sackman, 1974). Depending on the purpose of the forecast for expert assessments, there can be invited from 10 to 150 experts.
Qualitative approach makes it possible to evaluate the specificity of each situation. In some cases, a careful study of various specific elements defining the situation may be more important than a systematic quantification. The big disadvantage of this method is the excessive subjective assessments. Old stereotypes of foreign societies can play a fatal role in decision-making.
Scope of expert methods application
Methods of expert assessments are widely used in forecasting and long-term planning, where there are no sufficiently reliable statistics on the issue under study, where there are several possible solutions, and it is necessary to choose the most preferred one (Selten, 1999). Also, these methods are used in the development of new programs in industries subject to the strong influence of new discoveries in the basic sciences.
When analyzing and forecasting economic situation, there are raised a number of difficulties:
- inability to accurately predict the consequences of decisions;
- repeatability and experimental verification of the alleged impossibility of progress and results of the decision;
- presence of factors that are beyond control of the decision-maker;
- there are several possible solutions and a need for selecting one of them;
- incomplete source of information on the basis of which it is necessary to form the problem and make a decision (often the initial information is qualitative in nature and is not quantifiable).
Prerequisites for the use of expertise are (Hsu & Sandford, 2007):
- failure and inadequate information on the status of certain conditions, in which there is creation and development of products;
- stochastic (probabilistic) nature of the information object;
- complexity and novelty of problems.
Organization of examination is carried out at several stages:
1. Determination of goals and objectives of the examination.
2. Choice of the examination procedures.
3. Selection and formation of an expert group.
4. Organization of the examination procedure itself.
5. Processing of information.
6. Making solutions on the result of the examination.
Definition of goals and objectives of expertise
First, the problem is presented - backstory is defined, the arguments are considered in favor of its decision, there is a discussion with all the people involved. The main thing here is to recognize imaginary problems. Therefore, when setting the problem, there is needed publicity and discussion. Once the problem is substantiated, there are defined the boundaries of its existence, the totality of internal and external factors affecting the problem (Yousuf, 2007). For this purpose, there is defined the central issue, which is split into sub-questions. At the same time, experts are trying to limit the field to only those issues, without which it is not possible to answer the central question. Next, there are formulated goals and objectives of the chosen problem implementation. Thus, there are selected major events, factors, central and peripheral issues.
It must be borne in mind that with increasing detail there increases the accuracy of the examination, but reduced the consistency of expert opinion. Organizers of the examination choose the procedure of the selected examination realization. There are various approaches to this issue. There can be carried out individual or group survey; intramural or extramural; open or closed.
Individual survey lies in interviewing an expert and maximizes the use of skills and knowledge of each expert (Landeta, 2006). In the group survey experts can exchange views, can account for the time lost by each of them, to adjust their assessment. Lack of group opinion lies in a strong influence of the authorities on the opinions of most examination participants, in the difficulties of public renunciation of one’s point of view, psychological incompatibility of some participants in the examination.
Methods of group interviews use various modifications of the Delphi method. Delphi methods are characterized by the following features: anonymity of expert opinions; adjusted processing, communications, which is carried out by the analytical group for a number of rounds of the survey, and the results of each round are told to the experts. There is also group response, which is obtained by means of statistical methods and displays the general opinion of the expertise participants.
Delphi method is the most formal of all methods of expert forecasting and is most often used in technological forecasting, data of which are then used in the planning of production and marketing. It is a group method, in which the individual survey is conducted for a group of experts concerning their assumptions about future events in the various areas, where there are expected new discoveries or improvements.
The survey is conducted through special questionnaires anonymously, i.e. personal contacts and expert group discussions are excluded. The responses received are matched by special workers and summarized results again sent to the members of the group. On the basis of such information, members of the group, still retaining anonymity, make further assumptions about the future, and this process can be repeated several times (the so-called multi-round polling procedure). After concurrence begins to appear, the results are used as a prediction.
Application of Delphi method can be illustrated by the following example: a company engaged in marine oil business wants to get information about when it will be possible to use robots instead of divers for underwater inspection of platforms. To start using this method of forecasting, the company should get in touch with a number of experts. These experts should be representatives of various areas of industry, including divers, engineers and technical staff of the oil companies, ship captains, engineers, designers and developers of robots. They are explained the problems the company is facing, and each expert is asked when he believes it will be possible to replace divers with robots. First answers will probably have very large scatter in the data, for example, from 2000 to 2050. These responses are processed and returned to the experts. In this case, each expert is asked to reconsider his assessment in the light of responses to other experts. After repeating this procedure several times, opinions are closer, so that about 80% of the responses give a period from 2005 to 2015, which will be sufficient for the purposes of production planning and implementation of robots.
The Delphi technique is named after the oracle of Delphi in ancient Greece. It was developed by Olaf Helmer, a prominent mathematician of the RAND corporation, and his colleagues and probably, therefore, in comparison with other creative approaches, it provides sufficient accuracy of the forecast.
Delphi method belongs to the class of quantitative methods of group expert assessments. Survey of experts is conducted in 3-4 tours, consisting of a series of questionnaires, questions are specified from round to round. To perform this method, it is necessary to create an analytical group that after each round produces statistical processing of the obtained information.
First of all, analysts define the scope of the preferred quantitative values of objects. Upon such verification, there is carried out another round. Procedure of the expert survey by the Delphi method can be divided into several stages.
STEP 1. Forming a working group. The task of the working group is to organize the procedures of the expert survey.
STEP 2. Expert group formation. In accordance with the Delphi method, the panel of experts should include 10-15 people skilled in a given field. Competence of the experts is determined by questionnaire, analysis of referential level (number of references to the work of this expert), using self-assessment sheets.
STEP 3. Formulating questions. Wording of the questions should be clear and unambiguously interpreted, should assume unambiguous answers.
STEP 4. Conducting examination. Delphi method implies repetition of a few steps of the survey.
STEP 5. Summary of survey. For the first round, the experts are suggested questions. Responses must be submitted in the form of quantitative estimates of the question. The answer must be justified by an expert.
Analytical group conducts statistical processing of the obtained information from all the experts. To do this, there is calculated the average value of the test parameter, the weighted average value of the test parameter, the median is defined as the average member of the general series of numbers obtained from experts and area of trust. Confidence interval is better to be calculated through the index quartile (Savci & Kayis, 2006). Quartile value is equal to ¼ of the difference between the maximum and minimum number of ratings. The confidence area itself is equal to the minimum score minus quartile value, the maximum score plus the quartile value.
Experts should definitely see the results and conclusions of analysts, and then there is held the second (next) round. Experts by the presented results of calculations can see how their opinion corresponds with the opinion of the entire group of experts. They can change their views or leave the same, but in this case present the counter arguments in its favor. In this case, there is observed the principle of strict anonymity. Thus, there are held 2-3 rounds. As a result, there is obtained an accurate assessment of the group.
When using the Delphi method, it is necessary to consider the following:
1. The expert groups should be stable, and the quantity of members should be kept to prudent limits.
2. Time between tours of polls should not be more than a month.
3. Questions in the questionnaires should be carefully considered and clearly stated.
4. The number of stages should be sufficient to provide all participants with the opportunity to get acquainted with the cause of any evaluation, as well as criticism of these reasons.
5. There should be conducted a systematic selection of experts.
6. It is necessary to have self-assessment of experts’ competence on the issues under consideration.
7. There is a need for formula of the estimates consistency, based on data from self-assessment.
Delphi method is applicable to almost any situation requiring prediction, including when there is not enough information to make a decision. There are several modifications of the Delphi method, in which the basic principles of the examination have much in common. The differences relate to attempts to improve the method through more informed selection of experts, the introduction of schemes assessing their competence, improved feedback mechanisms, etc. For the convenience of the information processing, all modifications generally involve the possibility of expressing the response as a number in a quantitative evaluation.
However, it does have drawbacks - such as the subjectivity of the experts’ opinion participating in the survey. It does not allow the dispute between the opinions of experts and for it, there is spent a lot of time. Some of the disadvantages associated with the Delphi method relate to the lack of time, which is devoted to expert for thinking over the problems. In this case, the expert may agree with the majority opinion, to get away from the need to explain what is the difference of his decision from the rest of the options. These defects are eliminated by improving the organization of examinations by creating automated systems for processing the survey results. Technical implementation of such a system is based on the use of computers with external terminals. Computer provides a view of the issues for experts (who can see it through their personal displays), the collection and processing of the responses, request and output of argumentation and other necessary information for the answers.
In addition, some experts believe that the assumption that those who are in sharp contrast with the majority opinion, substantiated their point of view, can lead to greater effect of adjustment, rather than reduce it, as it was intended. Many scientists argue that the Delphi technique is superior to the ordinary methods of forecasting, at least in the development of short-term forecasts.
In most developed countries, to determine the priorities of scientific and technological development in the process of forecasting and decision-making on the financing of large government programs, there are used the following methods: Delphi, establishment of a list of critical technologies, expertise.
Technological forecast based on the Delphi method can be used as an attempt to predict the development of technology for the long term (20-30 years). Developed for the first time in the 50es by the RAND Corp., machinery of the Delphi method was first used for the purposes of national and industrial technological forecasting in Japan (since 1970 already 6 studies were held). Subsequently, largely modeled by the Japanese example, Germany, France, UK, Spain, Austria, South Korea, mainly in the last decade conducted the same experiment.
The Delphi technique lies in the assessment of technology by experts (their number varied from 123 people in Spain to 25,000 in the first stage, in South Korea) on the basis of the proposed schemes, including several positions. It involves a level of research activity in this area, participation in the creation of national wealth, improving quality of life and competitiveness, the expected timing of the implementation of new achievements. Two-, four-stage evaluation procedure allows experts to clarify or revise their point of view, taking into account the views of colleagues and develop, as a result of consistent, real collective position on a whole range of issues raised, the number of which in the first stage, is usually over a thousand.
Forecasting with the use of Delphi method is effective in achieving a number of other, fundamentally important for identification priorities and results. It is cognitive effect, learning and personal enrichment for experts - survey participants, mapping of competencies in individual disciplines, technical areas and countries to develop consensus of the representatives of various sectors of the scientific and technical sphere and, promotion of the wide discussion of the scientific community trends of scientific and technological development of their country and the world.
Japan not only has the longest history of prognostic assessments of technological development of the country and the world, but also the most effective practices using these forecasts to the general orientation of the national scientific and technical sphere, the more interesting, as the state's share in the financing of national science never exceeded 20-25%. Office of Science and Technology, coordinating research programs through strategic basic and applied research in other departments, is responsible for technological forecasting.
Delphi method survey is conducted every five years with a time range of up to 30 years, gradually covering all areas of science and technology. If the first poll had forecast for the period of 1970-2000 was able to cover 5 areas and 644 threads, the latter covering the period of 1996-2025 already included 14 directions and 1072 themes.
Delphi method as an attempt to anticipate the future by collective procedure has certain disadvantages. There is a doubt as to the reliability of the results obtained by the straight aggregation of individual opinions, as the sample group of experts is representing the scientific community. There is blurring the objectives and outputs, as well as a high probability of generating passive and deterministic view of the future, as well as direct uncritical copying of the foreign experience.
So, although the Delphi technique is very popular, its impact on the real structure of priorities in most developed countries should still be considered limited. In many countries, this and other methods to identify priorities often get on barren ground, that is either not provided implementation mechanisms, or given way to other priorities, selected in accordance with any political or lobbying interests.
Other methods of risk identification in projects
Brainstorming. The purpose of brainstorming is to provide a detailed list of project risks (Chapman, 2001). Risk list is developed at the meeting, which involves 10-15 people - members of the project team, often in conjunction with experts from different fields who are not members of the team. Participants of the meeting identify the risks that are considered important for the project, and it does not permit a discussion of risks nominated. Further risks are sorted by category and specified.
Nominal group technique allows us to identify and locate risks in order of importance. This method involves the formation of a group of 7-10 experts. Each participant individually and without discussion lists the visible project risks. Further, there is a joint discussion of all identified risks and individual re-compiling of a list of risks in the order of importance.
Crawford cards. Usually there meets a group of 7-10 experts. Leader reports that the group will ask 10 questions, each of which the participant should answer in writing, on a separate sheet of paper. The question of which one of the risks is the most important for the project, the leader asks several times. Each participant has to consider ten different project risks.
Surveys of experts with extensive experience in projects.
Identification of the underlying cause. The purpose of this process is to identify the most important causes of project risks and group risks by their causes.
Analysis of strengths, weaknesses, opportunities and threats (SWOT analysis). The purpose of the analysis is to assess the potential of the project and the environment. Potential of the project, expressed in the form of its strengths and weaknesses, allows us to evaluate the gaps between the content of the project and its performance capabilities. Evaluation of the project environment shows which opportunities and threats the external environment has.
Analysis of checklists. Checklists are lists of risks, compiled on the basis of information and knowledge that was accumulated during the previous execution of similar projects.
Analogy method. To identify risks, this method uses the expertise and risk management plans and other similar projects.
Methods with using diagrams. Methods for mapping risks in the form of diagrams are diagrams of causality and flowchart of processes that allow us to trace the sequence of events in this process.
The Delphi technique has clear advantages over methods based on conventional statistical analysis of the results of individual surveys. It reduces vibrations at the totality of the individual responses, limiting fluctuations within groups. In this case, as shown by the experiments conducted, the availability of low-skilled experts has a less strong influence on the group assessment than simple averaging results of responses, as the situation helps them to correct answers at the expense of new information from the group.
References
Chapman, R. J. (2001). The controlling influences on effective risk identification and assessment for construction design management. International Journal of Project Management, 19(3), 147-160.
Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: making sense of consensus. Practical Assessment, Research & Evaluation, 12(10), 1-8.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological forecasting and social change, 73(5), 467-482.
Sackman, H. (1974). Delphi assessment: Expert opinion, forecasting, and group process (No. RAND-R-1283-PR). RAND CORP SANTA MONICA CA.
Savci, S., & Kayis, B. (2006). Knowledge elicitation for risk mapping in concurrent engineering projects. International journal of production research, 44(09), 1739-1755.
Selten, R. (1999). What is bounded rationality. Bounded Rationality: The Adaptive Toolbox (Cambridge, MA: MIT Press, 2001, pp. 13-36).
Yousuf, M. I. (2006). The Delphi Technique. Retrieved from http://drimran.articlealley.com/delphi-technique-112396.html
Yousuf, M. I. (2007). Using experts’ opinions through Delphi technique. Practical assessment, research & evaluation, 12(4), 1-8.
