General assessment processes
This page is a encyclopedia article.
The page identifier is Op_en2085 |
|---|
| Moderator:Nobody (see all) Click here to sign up. |
|
Give your opinion to the peer rating of the content of this page. |
| Upload data
|
Contents
Assessment sub-processes
The process of carrying out an assessment can be considered as consisting of several simultaneously on-going sub-processes that are more or less continuous throughout the whole risk assessment R↻ :
- Collection of scientific information and value judgments according to the assessment questions
- Manipulation of scientific information and value judgments to provide answers to the assessment questions
- Synthesis of scientific information and value judgments as an answer to the principal question of the assessment
- Communication of the outcomes of collection, manipulation, and synthesis sub-processes among assessment participants and users
- Management of collection, manipulation, synthesis and communication sub-processes
Underlying all these is the process of observation (and experimentation). This is not, however, considered as being a part of assessment, but rather the basic process of scientific studies that produces the scientific information used in assessments. All these sub-processes are in interaction with each other, and present throughout the whole assessment process, although their roles and appearances may vary significantly during different phases of the process. A sketch of the relationships between these sub-processes (excluding communication) and the object of assessment is given below.
Collection of scientific information and value judgments can be considered as gathering these information types or meta-information about them and bringing it available for the participants of the assessment. The collection sub-process often deals with information that is freely structured, i.e. not formalized as explicit answers to specific assessment questions, or not the same questions as answers are sought for in the particular assessment.
Manipulation takes place as formatting the collected information, as is necessary, so that it addresses the specific questions of the particular assessment as well as possible. The term manipulation should not be interpreted here in its often used negative sense of distorting information, but rather as extracting the essence of the information in terms of the information needs of the assessment.
Synthesis of scientific information and value judgments can then be described as adapting the collected, and manipulated if necessary, information as explicit answers to specific assessment questions. Synthesis takes place on two levels. Firstly, there is the synthesis of different pieces of information as an explicit answer to one specific questions (In the form of a variable). Secondly, there is the synthesis of different question and answer compounds as an overall answer to the principal question of the assessment (In the form of a causal network of variables, constituting an assessment). This sub-process produces formal information in the form of variables and assessments.
Communication is a sub-process that is inevitably intertwined with all the other sub-processes. As an assessment is an activity taking place in social context, there is a need to explicate the outcomes of the intermediate steps within different sub-processes. This is necessary to enable collaborative and iterative development of the assessment. The communication is most reasonably arranged as happening primarily through a shared object of work, the assessment collaboratively worked on.
Management sub-process is managing the processes of collection and synthesis of scientific information and value judgments, and communication. This also includes technical facilitation of the other sub-processes.
Phases of assessment
The risk assessment process is continuous interplay of these four sub-processes that may appear in different forms during different phases of the process. The risk assessment work has six phases. The work during a particular phase is always built on the results of the previous phases. The phases are not clear-cut periods in time, because the work is iterative in nature, and the previous phases must frequently be revisited when the development of the product and feedback brings in new understanding. Thus, the next phase is usually started when the previous phase is still a draft at least in some aspects.
The phases of risk assessment are called:
- Scoping of the assessment. In this phase the purpose of the assessment, the question(s) asked in the assessment, intended use of the output, the temporal and spatial boundaries of the scope of the assessment, and the participatory width of the process are defined and described.
- Applying previous information about the issues being assessed. In this phase the existing information that is available e.g. in the encyclopedia, as existing variables from previous assessments, or as Classes (general properties shared by a group of variables) are sought out and applied.
- Drawing a causal diagram. In this phase the decisions, outcomes, and variables of importance related to the assessment are described in the form of a sketch of a causal network diagram.
- Designing variables. In this phase the variables included in the assessment (the causal network diagram) are described more precisely, including defining causal relations between them. This phase may also include definition of quality criteria and plans for collecting the necessary data or models to estimate the results of the variables. It should be noted that previously this phase was considered to apply to the assessment as a whole. Now we have realised that in practice the design work happens separately for each variable (although usually approximately at the same time). The same applies to the next phase, execution.
- Executing variables and analyses. This phase is actually about collecting the data needed, executing the models described in the Definition attributes of different variables, and storing the results in the result database. Assessment-specific analyses such as optimisation, decision analysis, value-of-information analyses and so on are carried out.
- Reporting In this phase, the results of indicators and assessment-specific analyses are communicated to the users of the outputs. The results are discussed and conclusions are made about them, given the scope of the risk assessment. The communication includes necessary background information.
The phases 1-3 are collectively called the issue framing phase.
Object of assessment
The outcome of the whole process, the assessment product, is a causal network description of the relevant phenomena related directly or indirectly to the endpoints of the assessment, in accordance with the purpose of the assessment. The final description should thus:
- Address all the relevant issues as variables
- Describe the causal relations between the variables
- Explain how the variable result estimates are come up with
- Report the variables of greatest interest and conclusions about them to the users
The greatest improvement, and at the same time challenge, in the new risk assessment method is the explicit emphasis on causality throughout the source-impact chain. Although it may often be very difficult to exactly describe causal relations in the form of e.g. mathematical formulae, the causalities should not be neglected. By means of coherent causal network descriptions that cover the whole source-impact chains it is possible to understand the phenomena and to be able to deal with changes that may take place in any variables in the causal network.
Even if the result estimates for individual variables were come up with by means of measurements, model runs, external reference data or expert judgments, the causal relations between variables should be attempted to be defined simultaneously. The least is to have statements about the existence of causal linkages, although they may be vaguely understood and defined. During the assessment the estimation of variable results should be an iterative interplay between information from data sources and definition of causal relations.
It is quite a long, and not necessarily at all a straightforward, way from creating a general view of the assessment question to creating a complete causal network description of the phenomena of interest. Some of the challenges on this way are:
- What are the all the relevant issues that should be covered in the assessment?
- How are the causal relations between variables defined and described?
- How are the individual variables defined and described?
- What is the right level of detail in describing variables?
- What are the most important issues within the assessment that should be communicated to the users of the output?
Below is a diagram that schematically describes the evolution of an assessment product developing from identification of the assessment purpose to a complete causal network desription. The diagram is a rough simplification of the process, although the arrows between the nodes pointing both ways try to emphasize the iterative nature of the process. The diagram rather describes a gradual transition of focus along the progress of work than subsequent events taking place in separate phases.
In the diagram below, the boxes represent different developmental steps of the assessment product which are representations of the improving understanding about the assessed phenomena and simultaneously the focus of attention as the process progresses towards its goals. The activities are considered to take place in between the boxes. The diagram is made in the form of a workflow description and it does not explicitly address e.g. questions of collaboration and interaction between different contributors to the assessment.
The progress of the assessment process and the development of the product along these phases is described in more detail in the phase-specific descriptions. However, it should yet again be reminded that the above description is a rough simplification and that in practice there are several different iteration loops of defining, refining and re-defining the causal network description as the process progresses.
References
<Reference/>
