Difference between revisions of "Assessment"

From Testiwiki
Jump to: navigation, search
m (→‎Result: disclink -> participants)
(structure updated)
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
[[Category:Universal object]]
 
[[Category:Universal object]]
 
[[Category:Glossary term]]  
 
[[Category:Glossary term]]  
−
{{variable}}
+
{{method}}
 
<section begin=glossary />
 
<section begin=glossary />
−
:'''Assessment''' is a process for describing a particular piece of reality in aim to fulfill a certain information need in a decision-making situation. The word assessment can also mean the end product of this process, i.e. some kind of assessment report. Often it is clear from the context whether assessment means the doing of the report or the report itself. Methodologically, these are two different objects, called the assessment process and the assessment product, respectively.
+
:'''Assessment''' is a process for describing a particular piece of reality in aim to fulfill a certain information need in a decision-making situation. The word assessment can also mean the end product of this process, i.e. an assessment report of some kind. Often it is clear from the context whether the term assessment refers to the making of the report or the report itself. Methodologically, these are two different objects, called the assessment process and the assessment product, respectively.
 
<section end=glossary />
 
<section end=glossary />
−
{{reslink|Clarify the meanings of the words "assessment product" and "endpoints"}}
+
{{reslink|Clarification of terms}}
  
−
==Scope==
+
==Question==
  
−
;The research question about an assessment structure: What is a structure for an assessment product so that it:
+
;What is a structure for an assessment so that it:
 
:* contains a description of a certain piece of reality {{reslink|Rename risk assessment}},
 
:* contains a description of a certain piece of reality {{reslink|Rename risk assessment}},
 
:* the description is produced according to the use [[Purpose and properties of good assessments|purposes]] of the product,
 
:* the description is produced according to the use [[Purpose and properties of good assessments|purposes]] of the product,
Line 18: Line 18:
 
:* complies with the [[PSSP]] ontology,
 
:* complies with the [[PSSP]] ontology,
 
:* complies with decision analysis,
 
:* complies with decision analysis,
−
:* complies with Bayesian networks.
+
:* complies with Bayesian networks?
  
−
==Definition==
+
==Answer==
  
−
These requirements of the structure lead to the following solutions:
+
All assessments aim to have a common structure so as to enable effective, partially automatic tools for the analysis of information. An assessment has a set of attributes, which have their own subattributes.
−
*An assessment product is built in a modular fashion using individual pieces of descriptions of reality, called  '''[[variable]]s'''. The variables are collected with reference to the use purpose that the assessment is intended for.
 
−
*The connections between variables in general, and between the decision and outcomes in specific, are described using causal relations. This leads to the [[Causality|causal network]] as the basic overall structure of the description of reality.{{reslink|Add causal diagram to the attributes of an assessment}}
 
−
*All variables in the [[Causality|causal network]] of a particular assessment must be causally related to the indicators of the assessment.
 
−
* The causal network of variables is complemented with information about the scope of the assessment and assessment-specific analyses and conclusions.
 
  
−
==Result==
+
{| {{prettytable}}
 +
|+ Attributes of an assessment
 +
|----
 +
!Attribute!!Sub-attribute!!Description
 +
|----
 +
!Name
 +
|
 +
|Identifier for the assessment.
 +
|----
 +
!rowspan="7"|Scope
 +
|
 +
|Defines the purpose, boundaries and contents of the assessment. Why is the assessment done?
 +
|----
 +
! Question
 +
| What are the research questions whose answers are needed to support the decision? What is the purpose of the assessment?
 +
|----
 +
! Intended use and users
 +
| Who is the assessment made for? Whose information needs does the assessment serve? How do we expect them to use the information?
 +
|----
 +
! Participants
 +
| Who is needed to participate to make the assessment a well-balanced and well-informed work? Also, if specific reasons exists: who is not allowed to participate? The minimum group of people for a successful assessment is always described. If some groups must be excluded, this must be explicitly motivated.
 +
|----
 +
! Boundaries
 +
| Where are the boundaries of observation drawn? In other words, which factors are noted and which are left outside the observation. Spatial, temporal and population subgroup boundaries are typical in assessments.
 +
|----
 +
! Decisions and scenarios
 +
| Which decisions and their options are considered by the decision maker? Also, if [[scenario]]s (defined here as deliberate deviations from the truth) are used: which scenarios are used and why? In scenarios, some possibilities are excluded from the observation in order to clarify the situation (e.g. what-if scenarios). For example, one could ask which climate change adaptation acts should be executed in a situation where the average temperature rises over two degrees centigrade. In this case, all the scenarios where the temperature does not rise over two degrees are excluded from the assessment.
 +
|----
 +
!Timing
 +
| Description of the work process. When does the assessment take place? When will it finish and when will the decision be made?
 +
|----
 +
!rowspan="3"|Answer
 +
|
 +
|Gives the best possible answers to the research questions of the assessment based on the information collected. Answer is divided into two parts:
 +
|----
 +
! Results
 +
| Answers to all of the research questions asked in the scope and the analyses described in the rationale. If possible, a numerical expression or distribution using clear units.
 +
|----
 +
! Conclusions
 +
| What are the conclusions about the question based on the results obtained regarding the purpose of the assessment?
 +
|----
 +
!rowspan="6"|Rationale
 +
|
 +
| Anything that is needed to make a critical reader understand the conclusions and to convince that the conclusions are valid. Includes all the information that is required for a meaningful answer.
 +
|----
 +
! [[Stakeholders]]
 +
| What stakeholders relate to the subject of the assessment? What are their interests and goals?
 +
|----
 +
! Dependencies
 +
| What are the causal connections between decision actions and endpoints of interest? What issues must be studied to be able to predict the impacts of decisions actions? These issues are operationalised using structured objects. A few different types of objects are needed:
 +
* Endpoints: What are the endpoints of interest for the decision maker? If stakeholders have other interests than the decision maker, what are they? Consider including these other interests into the assessment as well.
 +
* [[Variable]]s: What are the issues that should be looked at to be able to predict the impacts of the decision options? How do they relate to each other (through causation or other ways)? Typically, with health impact assessments:
 +
** What emissions and exposures should be considered?
 +
** What health endpoints should be considered?
 +
** What exposure-response functions should be considered?
 +
** What population subgroups should be considered?
 +
* Value variables: What are people's values and what do they think of as good or bad? Values are processed by scientific means by making observations on people's behaviour and opinions.
 +
* Methods: Methods describe their subject indirectly by answering the question "how can I find the answer to question X?" When using a method, the answer to X can only be found when it is connected to the correct frame of reference. For example, a method may include emission factors for a car and equations to calculate emissions. The emissions of a particular car can only be calculated when case-specific information is known, such as kilometres driven.
 +
|----
 +
! Analyses
 +
| What statistical or other analyses are needed to be able to produce results that are useful for making conclusions about the question? Typical analyses are decision analysis (which of the decision alternatives produces the best - or worst - expected outcome) and value of information analysis (how much would the decision making benefit if all uncertainties were resolved?).
 +
|----
 +
! Indices
 +
| What [[Index|indices]] are used in the assessment and what are cut out as unnecessary and when?
 +
|----
 +
! Calculations
 +
| Actual calculations to produce the result. Typically an R code.
 +
|}
 +
 
 +
== Rationale ==
  
−
{| {{prettytable}}
 
−
|+Table 2. The attributes of an '''assessment product''' (typically a report).
 
−
! Attribute
 
−
! Subattributes
 
−
! Comments
 
−
|-
 
−
| '''Name'''
 
−
|
 
−
| Identifier for the assessment
 
−
|-
 
−
| '''Scope'''
 
−
|
 
−
* [[Glossary#Purpose|Purpose]]
 
−
* [[Glossary#Boundary|Boundaries]]
 
−
* [[Glossary#Scenario|Scenarios]]
 
−
* [[Glossary#Intended user|Intended users]]
 
−
* [[Glossary#Participant|Participants]]
 
  
−
|
 
−
* '''Purpose''' defines the specific information need of the decision-making and the research question that is asked.
 
−
* '''Boundaries''' define which parts of the reality are taken into the assessment and which are excluded within spatial, temporal and other dimensions.
 
−
* '''Scenarios''' define particular conditions that are of interest irrespective whether they describe reality or not (e.g. what-if scenarios). {{disclink|Scenarios}}
 
−
* '''Intended users''' are those for whom the assessment is made.
 
−
* '''Participants''' are those who may participate in the making of the assessment.  The minimum group of people for a successful assessment is always described. If some groups must be excluded, this must be explicitly motivated. {{disclink|Participants}}
 
−
|-
 
−
| '''Definition'''
 
−
| Causal diagram
 
−
* [[Glossary#Decision variable|Decision variables]]
 
−
* [[Glossary#Indicator|Indicators]]
 
−
* [[Glossary#Value variable|Value variables]]
 
−
* [[Glossary#Variable|Other variables]]
 
−
Other parts
 
−
* [[Glossary#Analysis|Analyses]]
 
−
* [[Glossary#Index|Indices]]
 
  
−
|
+
== See also ==
−
* '''Decision variables''': decisions that are considered.
 
−
* '''Indicators''': outcome variables of interest.
 
−
* '''Value variables''': value judgements (usually about indicators).
 
−
* '''Other variables''': any variables that link to the causal network and are within the boundaries of the assessment.
 
−
* '''Analyses''': statistical and other analyses that contain two or more variables, e.g. optimizing.
 
−
* '''Indices''': lists of particular locations along spatial, temporal, or other dimensions. The variable results are calculated for these particular locations.
 
−
Note: ''Causal diagram'' and ''Other parts'' are not attributes but just descriptive subtitles. {{disclink|Appraisal}}
 
−
|-
 
−
| '''Result'''
 
−
|
 
−
* [[Glossary#Result|Results]]
 
−
* [[Glossary#Conclusion|Conclusions]]
 
  
−
|
+
* [[Variable]]
−
* '''Results''' of indicators and assessment-specific analyses. The results of other than indicator variables are described only within the variables. Results can either be a repetition of the results of the indicators and analyses, or a summary of those, or anything between those.
 
−
* '''Conclusions''' are based on the results, given the scope.
 
−
|}
 

Latest revision as of 07:58, 5 March 2015


<section begin=glossary />

Assessment is a process for describing a particular piece of reality in aim to fulfill a certain information need in a decision-making situation. The word assessment can also mean the end product of this process, i.e. an assessment report of some kind. Often it is clear from the context whether the term assessment refers to the making of the report or the report itself. Methodologically, these are two different objects, called the assessment process and the assessment product, respectively.

<section end=glossary /> R↻

Question

What is a structure for an assessment so that it
  • contains a description of a certain piece of reality R↻ ,
  • the description is produced according to the use purposes of the product,
  • describes all the relevant phenomena that connect the decision under consideration and outcomes of special interest (called indicators),
  • combines value judgements with the descriptions of physical phenomena
  • can be applied in any domain,
  • inherits the main structure from universal objects,
  • complies with the PSSP ontology,
  • complies with decision analysis,
  • complies with Bayesian networks?

Answer

All assessments aim to have a common structure so as to enable effective, partially automatic tools for the analysis of information. An assessment has a set of attributes, which have their own subattributes.

Attributes of an assessment
Attribute Sub-attribute Description
Name Identifier for the assessment.
Scope Defines the purpose, boundaries and contents of the assessment. Why is the assessment done?
Question What are the research questions whose answers are needed to support the decision? What is the purpose of the assessment?
Intended use and users Who is the assessment made for? Whose information needs does the assessment serve? How do we expect them to use the information?
Participants Who is needed to participate to make the assessment a well-balanced and well-informed work? Also, if specific reasons exists: who is not allowed to participate? The minimum group of people for a successful assessment is always described. If some groups must be excluded, this must be explicitly motivated.
Boundaries Where are the boundaries of observation drawn? In other words, which factors are noted and which are left outside the observation. Spatial, temporal and population subgroup boundaries are typical in assessments.
Decisions and scenarios Which decisions and their options are considered by the decision maker? Also, if scenarios (defined here as deliberate deviations from the truth) are used: which scenarios are used and why? In scenarios, some possibilities are excluded from the observation in order to clarify the situation (e.g. what-if scenarios). For example, one could ask which climate change adaptation acts should be executed in a situation where the average temperature rises over two degrees centigrade. In this case, all the scenarios where the temperature does not rise over two degrees are excluded from the assessment.
Timing Description of the work process. When does the assessment take place? When will it finish and when will the decision be made?
Answer Gives the best possible answers to the research questions of the assessment based on the information collected. Answer is divided into two parts:
Results Answers to all of the research questions asked in the scope and the analyses described in the rationale. If possible, a numerical expression or distribution using clear units.
Conclusions What are the conclusions about the question based on the results obtained regarding the purpose of the assessment?
Rationale Anything that is needed to make a critical reader understand the conclusions and to convince that the conclusions are valid. Includes all the information that is required for a meaningful answer.
Stakeholders What stakeholders relate to the subject of the assessment? What are their interests and goals?
Dependencies What are the causal connections between decision actions and endpoints of interest? What issues must be studied to be able to predict the impacts of decisions actions? These issues are operationalised using structured objects. A few different types of objects are needed:
  • Endpoints: What are the endpoints of interest for the decision maker? If stakeholders have other interests than the decision maker, what are they? Consider including these other interests into the assessment as well.
  • Variables: What are the issues that should be looked at to be able to predict the impacts of the decision options? How do they relate to each other (through causation or other ways)? Typically, with health impact assessments:
    • What emissions and exposures should be considered?
    • What health endpoints should be considered?
    • What exposure-response functions should be considered?
    • What population subgroups should be considered?
  • Value variables: What are people's values and what do they think of as good or bad? Values are processed by scientific means by making observations on people's behaviour and opinions.
  • Methods: Methods describe their subject indirectly by answering the question "how can I find the answer to question X?" When using a method, the answer to X can only be found when it is connected to the correct frame of reference. For example, a method may include emission factors for a car and equations to calculate emissions. The emissions of a particular car can only be calculated when case-specific information is known, such as kilometres driven.
Analyses What statistical or other analyses are needed to be able to produce results that are useful for making conclusions about the question? Typical analyses are decision analysis (which of the decision alternatives produces the best - or worst - expected outcome) and value of information analysis (how much would the decision making benefit if all uncertainties were resolved?).
Indices What indices are used in the assessment and what are cut out as unnecessary and when?
Calculations Actual calculations to produce the result. Typically an R code.

Rationale

See also

Retrieved from "http://www.opasnet.org/testiwiki/index.php?title=Assessment&oldid=34677"