Testiwiki:Structures of the building blocks of open risk assessments

From Testiwiki
Revision as of 14:11, 26 January 2007 by Jouni (talk | contribs) (→‎Variables: ana code clarification with definition subtitles)
Jump to: navigation, search

Variables

Attribute Question to be answered Comments
Name What is the name of the variable? Two variables must not have identical names.
Scope What is the question to which the variable answers? This includes a verbal definition of the spatial, temporal, and other limits (system boundaries) of the variable.
Description What do you need to know in order to understand the other attributes of the variable? This may include references to relevant literature and indices (dimensions) used in the variable.
Definition How can you derive or calculate the answer? The definition uses algebra or other explicit methods if possible. It also contains all such links from other variables that are necessary to define the variable. This may include formula (how to derive the result based on upstream variables) and data (how to derive the result based on data).
Unit What is the unit of measurement?
Result What is the answer to the question defined in the focus and scope? If possible, a numerical expression.

Subtitles for DefinitionD↷

There can be several definitions for a variable, if there exists several independent sources of information. These are classified under four subtitles to be clear about their role. When the variable is modelled, however, a synthesis must be built on all of these in some way. This is a case-specific issue. If the variable is coded in Analytica, the code itself is located before these subtitles. It should be noted that if there are different kinds of information available, the code does NOT reflect the causal chains alone but has other links as well. (The text under subtitles should be located under Def attribute in Analytica, not in Definition attribute that is for the code only.)

  • Causality: Describes the causal links to other variables (i.e., parents)
  • Data: This is directly relevant measurement of the variable
  • Association: There are data (or estimates) from an analogous or similar situation.
  • Boundaries: There are some boundaries for a group of variables (including this one) that the answer cannot exceed.


In wiki environment, there are additional rules:

  • Each variable is located in the Variable namespace.
    • However, draft variables (when scope has not been fixed) may be parts of other pages.
  • Each variable is one page.
  • The name of the page is the same as for the variable. It is recommended that the name attribute is mentioned within the scope in bold, but it does not need to be in the exactly same format (it can be e.g. more precisely described).
  • Scope starts with the word Scope in the previous line (wiki code '''Scope'''<br>. Subtitles are NOT used with Scope; this way, it locates above the table of contents.
  • All other attributes are described under a fourth-level subtitle (====) that is equal to the name of the attribute.
  • If subheadings are used, they are done with fifth-level =====subtitles=====.
  • Description may contain subheading References (literature references) and Indices (indices or dimensions that are necessary in the description of the variable. An example can be PM emission by sector, which must be indexed by a list or sectors to be meaningful.
  • Definition may contain subheadings Formula (for deriving the result from upstream variables) and Data (for deriving the result from data directly or using inference).
  • Also other subheadings may be used as necessary.

Discussions

Rules about resolving disputes
  1. Any part of any variable (except the fixed attributes) is open for critical discussion about a dispute. A discussion has the following parts:
    1. Dispute is the subject of the discussion. The dispute consists of a list of two or more conflicting statements and a description on how the statements are in conflict in this particular context. If only one statement is listed, there is an implicit statement that the first one is not true.
    2. Argumentation is a hierarchical list of arguments that defend, attack, or comment the statement and other arguments. An argument consists of the argument itself, and signatures of the participants promoting the argument. Variables may also be used within arguments.
    3. Outcome is the result of the discussion. It may be a resolution (see below) or a dispute when resolution has not emerged.


In the wiki environment, there are additional rules:

  • The template Discussion (previously Resolution) is used for structured discussion. The order of the parts is Dispute, Outcome, and Argumentation.
  • It is preceded by a third-level subtitle (===) which consists of the attribute and the topic that are discussed.
  • Templates Defend, Attack, and Comment are used in the discussion.
  • Templates Defend_invalid and Attack_invalid are used for invalid arguments.

Categories

Categories are used to create hierarchical structures. Examples include

  • General variable - variable with the same scope except with a more narrow spatio-temporal borders.
  • General variable - variable with the same scope except with a more narrow subpopulation.
  • Risk assessment - variables that belong to that assessment.

When a variable inherits properties from another variable, there is no need that it inherits the WHOLE variable. Instead, the inheritable pieces can be parts of a variable. In wiki, it is easy to make these inheritable parts as templates (which may have categories as necessary; in this case, also categories are inherited). But what are these pieces then, if they are not variables?

Open risk assessment

To perform an open risk assessment, several methods and tools are needed:

  • Pyrkilo: a structured method for doing risk assessments in a participatory way.
  • Pyrkilo-Wiki: a computer tool (platform) for doing risk assessments using the pyrkilo method
  • Open Risk Assessment (ORA): a risk assessment that has been performed using pyrkilo, pyrkilo-wiki, and a completely open process in the Internet
  • Heande (Health, the Environment, and Everything): a website for performing Open Risk Assessments, hosted by KTL/YTOS

Open risk assessment platform

Error creating thumbnail: Unable to save thumbnail to destination

Value judgement machine

Value judgement machine is a name for a web-based program that collects information about people's value rankings. It is based on a principle that an individual should be able to develop a coherent ranking of issues based on order of preference. Coherent means that A is preferred over B and B is preferred over C, then also A must be preferred over C. There are two alternative ways of performing the evaluation, namely an individual as a decision-maker, and an individual as himself. The former case is more interesting and relevant, because those valuations can be used for ranking outcomes in societal decision-making, while the usability of the latter ranking is not clear. However, both are described for completeness (the former was discovered first).

The idea in both cases is that there is a database of interesting issues that should be ranked for preference. The comparison is performed by individuals using internet interface for this. Issues are sampled (with weighted probabilities, see below) from the issue database two at a time, and the individual is asked to tell which one he prefers more. This is done repeatedly for different issues, until there is an unambiguous ranking for a subset of issues. If the respondent is not coherent, The incoherent answers are pointed out and the respondent is asked to clarify the ranking.

The issues have always a similar form with a few alternative attributes. If an attribute of an issue is changed, it is considered a new issue.

  • An issue is about losing something that existed or getting something that did not exist. This includes a reference "nothing happens". (Can it be something that just happens without being possible to describe that as losing or getting?).
  • The getting or losing happens to
    1. someone whose identity is unknown at the time of decision,
    2. everyone,
    3. the nature or similar non-personal entity, or
    4. the person who is answering
  • The issue happens somewhere
    1. within the family of the respondent,
    2. in the home town of the respondent,
    3. in the home country of the respondent, or
    4. in the world.
  • The issue happens
    1. immediately,
    2. this year, or
    3. after a defined number of years, usually 15 years.

The answers to the comparisons are stored in the database together with the respondents username and time of answer. Respondents are not identified, but the username (and password) are required so that a respondent's answers can be linked together, even if they have been given during more than one session. In the following, we'll describe the exact wording and form of questions that we think should be used. However, it is a slow process to go through pairs of issues, and when the respondent is familiar with the form of questions, he can start putting ranking of issues based on a list of issues where the individual issues can be dragged and dropped into a higher or lower ranking.

In principle, the two issues are randomised from the database. However, it is useful to select interesting pairs for comparison. This can be done by weighting the probabilities for being selected. This is a tentative list of criteria that make an issue or a pair interesting. In brief, the more established ranking, the less there is need for further questions.

  • the issue has been ranked important before
  • the previous answers have been highly varying
  • the pair of this issue has been previously ranked in a similar way than this issue
  • there are only a few answers so far about the issue.


Questionnaire, the respondent as a decision-maker

Suppose you are a decision-maker. Your advisors have been informed about a problem that needs to be solved. They have evaluated the problem and identified two possible solutions. They have also estimated the two outcomes of this decision, and they are convinced that either one will surely happen. However, they have not been able to identify the person(s) who will be affected. They can be anyone (in your town/country/the world; this is defined for the question), but possibly they include someone you know, and it is even possible that one of them is you. It is not known, whether the affected persons in the two outcomes are the same or not.

The decision is made so that those who are affected can find out who made the decision. (Is this necessary, because very often you can indeed find out who made a decision, but you cannot find out that the outcome was due to the decision?) You are authorised to make the decision alone. You can consult anyone if you wish, but the responsibility is only yours.

  • Outcome A: A person loses an arm.
  • Outcome B: A person loses a foot.

Choose between the four options:

  1. I'd rather have A than B.
  2. A and B are equal.
  3. I'd rather have B than A.
  4. It is impossible to answer this question because _______________________

Please provide some background information

  1. Are you personally or is someone you love affected by the outcome defined in A? Y/N
  2. Are you personally or is someone you love affected by the outcome defined in B? Y/N


Instead of making paired comparisons, the tool could provide a list of categories, where the user can select one or more. The contents of the selected categories appear as a list box. The user can drag and drop any issue from the list box to another list box that contains the issues to be evaluated. From here, the user can then drag and drop the issues into a third box, where the issues can be organised into the order of preference. In previous boxes, issues are in alphabetical order. Each issue in the third box can also be tagged "equally important as the previous issue". In this way, a large number of issues can be handled, and the user can rather quickly make his/her own ranking. There should be a selection of obligatory issues that are ranked always. There are then used as calibration points, as everyone has the same in their ranking. One critical issue is "nothing happens", which is the neutrality reference point.

This can be done with a database with the following tables and fields:

Issue

  • issue_id
  • Issue_description
  • category_id

Category

  • Category_id
  • Category_description

User

  • user_id
  • password
  • last_edition_date

Selection

  • selection_id
  • user_id
  • issue_id (user_id+issue_id must be unique)

Assessment

  • assessment_id
  • selection_id
  • preferred_over_id
  • preference ['>','=']

When performing assessment, make query of Selection - Assessment for one user_id. Rules:

  • preferred_over_id must be in Selection/issue_id
  • preferred_over_id must be unique in this set. If it is selected another time:
→ 1) make X the selection_id; 2) find the row where preferred_over_id has its duplicate; 3) replace preferred_over_id on that row with X

Questionnaire, the respondent as himself

Consider the two issues and answer to the following questions.

  • Issue A: The right arm.
  • Issue B: The right foot.

From the table below, select the row that applies to you and select one of the three empty columns.

Select the row that applies to you Outcomes I prefer 1 to 2 I am indifferent I prefer 2 to 1
I have both A and B 1: You lose A; 2: you lose B
I have A but not B 1: You lose A; 2: You get B
I have B but not A 1: You get A; 2: You lose B
I have neither A nor B 1: You get A; 2: You get B


Evaluating the simlarity of things

There is a critical problem in this approach that if/when the number of issues grows, the number of necessary comparisons grows in the second power. It is therefore crucial that only representative and relevant issues are taken into the database. The operator should be critical in what to put in the database. On the other hand, he should carefully listen to what the participants suggest, so that the collection of issues would be as good as possible. Below, there is one suggestion about how to organise issues so that feedback is collected from the users.


There are two outcomes that have been suggested in the outcome database. On the other hand, there are suggestions that there is no need to separate these two outcomes. Please tell what you think.

  • Outcome A: A person loses an arm.
  • Outcome B: A person loses a foot.

Choose between the options:

  1. A is covered by B, and there is no need to describe A separately.
  2. B is covered by A, and there is no need to describe B separately.
  3. A and B are different things, and they cannot be described as a single outcome because _________________
  4. Both A and B could be covered with a single outcome, which is ______________
Retrieved from "http://www.opasnet.org/testiwiki/index.php?title=Testiwiki:Structures_of_the_building_blocks_of_open_risk_assessments&oldid=2158"