Testiwiki:Guidance system development

<accesscontrol>Members of projects</accesscontrol>

Guidance system development contains specification and other information about the information system to be developed for carrying out and managing risk/impact assessments and to storing the products of the assessments as well as information that is useful as input for the assessments. As issues become worked out their implications should be transferred or built-in to the relevant content pages of guidance system, i.e. Guidebook, Toolbox and Data center, or other relevant pages.

Dimensions and hierarchies

The guidance system has two distinct dimensions. It can be confusing to browse through different parts. This is an attempt to clarify these issues. The hierarchy levels are listed from bottom to top for each dimension. The hierarchy list has dramatically changed; the original list was much longer and more complex. However, the old "dimensions" are captured in the new two dimensions. "Class", "real world", and "work environment" are actually examples of structures in the complexity dimension. "Meta" still exists but in a more rich form. "Guidebook articles" about glue (summary) articles, structured (according to the method) articles, and background articles is a practical grouping of objects inside and outside the method. "Resource Centre" is an application of the meta dimension. "Tools/Models" is an application of the complexity dimension.

• Dimension: Meta (or Abstraction level)
  1. Zero meta level is the real world: the real objects and events.
  2. First metal level contains the descriptions of the real world (such as variables and assessments) and the collection and synthesis processes of the first meta level information.
  3. Second meta level contains the general structures of the first-level objects, and the descriptions of these objects (such as process descriptions).
  4. Third meta level describes the development processes and the descriptions of the second-level objects.
  • For objects in the Resource Center (= data center + toolbox)
    1. (Nothing exists from the zero meta level)
    2. Tools/models (software or programs) and core data sets
    3. Descriptions of tools, models, and core data sets, including links to those that are not within the Resource Centre
    4. Descriptions of and links to tool-, model-, and databases

• Dimension: Complexity (or composition level)
  • For product objects
    1. Variable: this level looks at individual product objects (typically variables) that describe particular pieces of reality.
    2. Assessment: this level looks at groups of variables that together form a synthesis of a particular problem at hand.
    3. Policy context: this level looks a particular problem as its relation to the policy context in which it is assessed. This level is usually not described explicitly.
  • For process objects
    1. Method: a procedure for manipulating information (about the real world) into a useful form for a particular place in an assessment.
    2. Assessment framework: A distinct set of methods that, when used together, form a scientifically coherent and efficient way for manipulating all information that is needed to perform a particular kind of an assessment.
    3. Scientific context: this level looks at the assessment frameworks in their scientific context in which they are used. This level is usually not described explicitly.
  • For Classes
    1. Item: an object (typically a variable or a class) that belongs to a particular class. For example, a pollutant concentration in a particular location at a particular time.
    2. Class: a set of all objects that share a key property, defined in the scope of the class. For example a class of "dioxin concentration objects" (in any spatio-temporal location) forms a class that can be useful when utilising the full chain approach for a dioxin assessment.
  • For tools:
    1. Tools/Models that help to specify the result of the variables (e.g. Chimère, WATSON)
    2. Tools that help to perform the assessment as such (so dealing with the process of an assessment). These would be e.g. a tool for stakeholder involvement. D↷
    3. Tools that help to manage the assessment process

What are processes and products in risk/impact assessments?

Fundamentally, a process is an event, or a composite of events, that produces something; a product. Product is an object which has been produced by some process, and has a potential of being used for something built in its structure. So, a process produces a product, which in turn can trigger its use process. Processes thus use products (as their input) and produce products (as their output). Products then have a history of being produced by a process and bear in them the potential of triggering their use process. Processes and products can be seen as different sides of the same coin. The same thing could (often) be considered from the point of view of a product: what is it, what produced it, what can it be used for - or from the point of view of a process: what is it, what inputs does it need/use, what does it produce. A process can produce several products and products can be produced by several processes. Similarly, a product can be used in several processes and a process can use several products (there does not necessarily have to be a 1-1 relationship between processes and products). It is merely a question of the chosen level of observation/description (could be also be referred to as granularity or resolution), that defines how the boundaries of the processes and products are set.

Variable is a typical product object. It can be used as an input of a process, or it can be an output of a process. It can – at least in theory – be validated against reality. In a variable, there is the Definition/Formula attribute, which tells how the result of the variable can be derived or computed. If the formula is a large piece of text, or generally usable in many variables, it might be useful to extract that piece out of the variable and make it a separate process object. This process of creating new objects from within an existing object is called budding. In this way, models can be seen as ancient Formula attributes that were created through budding from a specific variable in the evolution of the system. Also information products called assessments and classes are product objects relevant organizing the information produced in risk/impact assessments.

A risk/impact assessment can be seen as consisting of the process of making the assessment and the product(s) that are produced by the process. The products are produced according to some certain needs (use purpose of the product), which define the intended (desired?) structure (contents, appearance, form, etc.) of the products. The use purpose further defines, or at least sets requirements for, the structure of the process that produces, or is intended to produce, the product (which has the structure that meets the requirements set by the use purpose). In general it can be said that the purpose of the process is to produce the product, whose purpose furthermore is to trigger (become a part of, fulfill its task within, ...) its use process.

When talking about risk or impact assessments the level of observation is such that the term product is usually used to mean the overall output of the assessment process. It is a composite information object composed of smaller chunks of information, which could be referred to as sub-products or product parts. The sub-products or product parts do not, however, differ in their essence as kinds of objects from the aggregate overall product, so they can also be referred to as products. It is merely a question of setting boundaries for objects and choosing the level of observation along the complexity dimension.

The term process, when talking about risk or impact assessments, often refers to the (organized?) compilation of events that produce this overall composite information object. In a similar manner, the process can be disaggregated to different sub-processes such as observation, information collection and synthesis, that do not differ in essence as kinds of objects from the overall process. The process can also be considered to cover also the activities of managing, or organizing, these above-mentioned sub-processes.

The risk/impact assessment product is information. The product and its parts are descriptions of reality, structured descriptions of reality. The risk/impact assessment process is production, collection and synthesis of information and management of these activities. The target of observation is the reality itself. The target of information collection and synthesis sub-processes is information about reality, structured or unstructured. The target of management sub-process is the composite of the other sub-processes. It can also be argued that management is a meta-process of risk/impact assessment process, that observation is a sub-process of risk/impact assessment process, and that only collection and synthesis sub-processes actually belong to the risk/impact assessment process as its internal structural parts. This is merely a boundary definition issue, and in any case it is easy to see that the above-mentioned structural parts of the process are hierarchically related.

Risk/impact assessment is thus (almost?) all about information. It mainly deals with information - descriptions of phenomena, not directly the phenomena themselves - and produces structured information for a certain purpose. A variable (a part of the assessment product, a product object) is a description of a part of reality, not the part of reality itself. The process objects, such as editing a particular variable, on the other hand could be argued to be reality itself, which makes the management sub-process somewhat special, because it manipulates processes, not products.

What does this all mean when thinking of risk/impact assessment information systems and their contents? Naturally, information products can be easily captured within such systems, but processes not (directly). It is obvious that the objects that can be stored within an information system are of the type product and that processes can only be stored in the form of descriptions of the processes or being built-in as functionalities of the system.

Information objects in the guidance system

The content of the guidance system will be composed of:

• pages describing the theoretical and contextual foundations of risk/impact assessment
• pages describing processes relevant to risk/impact assessment
• pages describing the products that these processes produce
• pages containing data and information that can be used in/by these processes, or providing or access to them
• pages providing access to tools
• glue pages, such as lists of other pages according to different purposes and introductory texts

The guidance system contents can be considered to be divided into a guidebook, a toolbox and a data center. The guidebook contains the pages describing the theoretical and contextual foundations of risk/impact assessment and the pages describing the processes relevant to risk/impact assessment. The toolbox contains pages providing access to tools that can be used in/by these processes. The data center contains pages describing the products that these processes produce and pages containing data and information that can be used in/by these processes, or providing or access to them.

The guidance system content will be composed of structured and unstructured information objects. Structured information objects, i.e. those having a pre-defined format, include process descriptions and product objects, all other information objects will be unstructured, i.e. not having a pre-defined format. D↷

Structured information objects

Processes relevant to risk/impact assessment are captured into the guidance system as descriptions of these processes, possibly also including (descriptions of and/or access to) the tools relevant to the process and descriptions of how to manage the process. A process object can be e.g. in the form of a method description (calculation, formal discussion, etc.) describing also the tools, models and other means that are used in applying the method. Process descriptions belong to the Guidebook.

Product objects in the guidebook are pieces of information that are descriptions of reality produced by the risk/impact processes. Product objects belong to the data center.

Unstructured information objects

The unstructured information objects can be of various kinds, after all it includes everything else besides structured objects. Some typical types of unstructured information that are relevant or common for risk/impact assessment can anyhow be identified. One obvious type is data, i.e. unstructured information about reality, which is usable in the synthesis process of producing structured information objects. Another relevant type is encyclopedia-type general information about e.g. the substance relevant to risk/impact assessment or the theoretical/contextual information relevant to risk/impact assessment.

Built-in functionalities

If we wish a risk/impact assessment information system to not only be a storage of information, but to make it functional, we should try to build in some processes into its structure as functionalities. This can, as mentioned above, only be done indirectly, i.e. by providing support of some kind for carrying out different sub-processes of the risk/impact process. In part the guidebook is intended to provide this kind of support, but the system can also have some functionalities that help in carrying out different tasks. The system would also require a workspace where the functionalities can be used and sub-processes and tasks can be carried out.

Article templates

Template: process description (a guidebook information object)

Summary

The summary of a process is a very short overview of the process, and may contain all types of information that are considered relevant for this specific process (max words: 250?)

Name

The name of the process should be unique (there should not be two objects (processes, products or product descriptions) with identical names). The name should be chosen so that it is descriptive, unambiguous and not easily confused with other processes, products or product descriptions. (max words: 20?)

Purpose

(This is the same as Scope in the general object attribute list.) The general purpose of every process is to manipulate information in aim to produce a particular information product. The process-specific purpose describes the intended product of this particular process. This should contain all relevant information needed to distinguish the process from other processes. (max words: 400?)

Structure of the process

(This is the same as the Result (of the process description) in the general object attribute list.) The result section has three sub-attributes. (max words: 3000?)

Input format

The input data, variable or parameter types, their syntax, and other relevant information is described here.

Procedure

The actual process – how it works and how to do – is described here. It consists of e.g. the mathematical formula to calculate the result. The procedure uses algebra or other explicit methods if possible.

Management

A complex process may require that also the management process of the procedure is described. The procedure may for example be a mathematical algorithm. The management process is a computer software that runs the algorithm, so that when you have the software, the management is a trivial task. If you do not have the management process - well, good luck to you and your calculator. So, actually models and software are procedures that have the management process packed into the same neat package. An other example of a management process is a guidance by the U.S.EPA about how to plan and organise for stakeholder meetings. The actual procedure here is to obtain information and feedback from the stakeholders.

Output format

shortly describes the format of the product of the process, which should fulfil the purpose as described in the scope. It furthermore links – if applicable – to the product object.

Rationale

(This is the same as the Definition (of the process description) in the general object attribute list.) This attribute answers to the following questions: What is known about a good process for this purpose? How do you know that the procedure described is good? (max words: 2000)

See also

See also links to pages (both internal guidebook pages and external) which relate to the process subject. All subjects that could be relevant for readers of this page can be listed here. (max links: 20?)

References

All references, as used in the texts above. (max references: 30?)

You can copy and paste the template below to your page. Put the name of the process as the name of your page.

<accesscontrol>members of projects</accesscontrol> {{method}} '''Summary''' ==Purpose== ==Structure of the process== ===Input format=== ===Procedure=== ====Management==== ===Output format=== ==Rationale== ==See also== ==References== <references/>

Template: product (a structured data center information object)

Summary

The summary of a product is a very short description of the product, and may contain all types of information that are considered relevant for this specific product (max words: 250?)

Name

The name of the product should be unique (there should not be two products or processes with identical names). The name should be chosen so that it is descriptive, unambiguous and not easily confused with other products or processes. (max words: 20?)

Scope

The scope of the product gives a research question that this product object aims to answer. The scope includes – if applicable – its spatial, temporal, or other limits (system boundaries). A product object may also be a generic one applying to any spatio-temporal locations. Whether these are just variables, and how they are used in the Guidebook is under study. (max words: 400?)

Definition

The definition describes the data and reasoning that tells us what the answer to the question in the scope is. In the case of a variable, it has four sub-attributes:

Data

What data or observations are available about this object?

Causality

What variables affect the result of this object when they change (i.e., which variables are causally related to this one)?

Unit

What is the measurement unit of the result?

Formula

How is the result derived or computed?.

Definition may link to the process(es) that lead to this product, explaining shortly why these processes are relevant. (max words: 3000?)

Result

Contains the answer to the question presented in the scope. Usually the text is short (max 500 words), but the result tables or figures may be extensive; there is no upper limit.

See also

See also links to pages (both internal resources centre pages and external) which relate to the product. All subjects that could be relevant for readers of this page can be listed here. (max links: 20?)

References

All references, as used in the texts above. (max references: 30?)

Practical information for both templates

--#(number): : General information on how the sentences should be structured etc. (see also my first version and the project that RIVM already have conducted. --Alexandra Kuhn 12:50, 14 January 2008 (EET)

Any section may contain a "more" button to more detailed information that is hidden by default to increase readability. The content is still directly relevant for the section. In addition, if there is a lot of further background information available about the object, e.g. its history, current practice, etc.) which is useful but not required to utilize the object, a background article can be created and linked to from the object. A background article can be freely structured, but an established encyclopedia article structure is recommended.

Intarese outputs as processes/products

Elements of the Intarese method, as developed in SP1, are for the most part process descriptions, but also tools, possibly some products (structured or unstructured descriptions of reality), and information about the theoretical/contextual issues. SP2 provides mainly descriptions of reality (mainly unstructured, but can also be structured). SP3 cases produce descriptions of reality that should mainly be structured, but probably includes also unstructured information. SP4 is responsible for including these types of information in the guidance system and implement the required functionalities (=operationalize the process potential) within the system. For all processes and products, information needs to be provided in the guidance system, and the above templates are to be used for structuring different kinds of information, either about the risk/impact assessment relevant processes or the reality they describe.

As mentioned above, we can always choose the perspective how an issue is being looked at - focusing either on a product or a process. A product description may well include description of the process that is intended to produce it (or where a product is intended to be used). A process description may well include a description of the product(s) that it is intended to produce (or what products it intended to use). Both approaches are fundamentally correct, and the decision which approach is chosen is a practical one. In an information system the practice of describing phenomena should probably be agreed upon, if not for any other reason, then just for the sake of clarity (see argumentation about this issue at: Talk:Guidebook).

When there is a clear 1 – 1 relationship between process and product (e.g. DALY process – DALY product), we can decide upon whether ask for a description of either process or product, in order to avoid confusion. In most cases, the process is the important object that needs to be described, and in practice the methods etc. described in the guidebook will be process descriptions. When there is no such direct obvious 1 – 1 relationship, we can ask for separate descriptions of processes and products. An example of this is meta-analysis (process), and its product exposure response function (ERF). Even though the process of meta-analysis can lead to an estimate of the ERF, meta-analysis can also lead to an estimate of another product (e.g. severity weight). An ERF (product) can also be produced by another process (e.g. expert judgement).

How should an article about disability-adjusted life years be structured? The question is not at all obvious, so the different options were tested in a previous version of this page. Only the conclusions are presented here.

Approach	Purpose	Structure of the process	Rationale
Method (process description object)	DALY estimation is a process for measuring summarized burden of disease adding up increased mortality (years of life lost) and years lived with disability. Purpose: DALY translates the impacts of diseases into life years based on their severity and duration, so that different diseases can be measured using a single currency, the life year. DALYs are based on disease-specific weights. (In contrast, QALYs evaluate the quality of life in a certain health state, not disease.) The research question: What is a good way to estimate DALYs?	The result describes the state-of-the-art method and formulas. Formula: DALY = life-years lost YLL (due to mortality) + life-years with disease YLD = YLL + number of disease cases * severity weight of the disease * the duration of the disease. The severity weights for diseases come from the variable Variable:Disability-adjusted weights for diseases The result is motivated by the content of the definition. Sub-attributes of the result include input (the upstream variables), procedure (the calculations), and output (format of the process output).	The definition contains the reasoning and motivation for calculating DALYs. It compares and discusses different alternatives. It contains links to methodology articles. It is also open for comments and further developments.

--#(number): : Where would we include the question if the calculation of DALYs is a valid approach for estimating the burden of disease? --Alexandra Kuhn 15:41, 4 February 2008 (EET)

Conclusions:

1. The process description (method) seems to be the most suitable perspective to describe DALY. The structure attribute of the DALY process description describes the state-of-the-art procedure for estimating DALYs. If there is not a single best method, several methods can be described. The rationale would then discuss the good and bad properties of each method, and their limitations. Usually a process is a general one, and therefore it will produce a large number of products for specific purposes in several risk assessments. There is no need to describe a "generalized product", as all general information, including the types of output that the process produces, is already described in the process description.
2. The purpose of all process descriptions is to describe a good process for achieving the outcome described in the research question.
3. The rationale of a process description describes the information you need to understand whether some procedure is suitable and good for the purpose.
4. To be precise, a process description is a product object that describes how to actually do the work. The doing itself is the process object, but that is something that vanishes as soon as the work is done and the product has been produced. We try to be precise when talking about the process or process description. However, we do not emphasize their differential nature (process vs. product, respectively), because the difference between doing and talking about doing is probably clear to the reader until someone tries to define that difference using metaphysical terminology.

Framework for describing an IA/RA methodology

When considering an overall methodology of IA/RA, the focal issues to look into are: - the process of making an assessment - the product it produces (for a specific purpose)

The assessment process and the assessment product can be considered as one process-product composite object. The process and the product are thus internal parts (inside the boundary) of this universal IA/RA process-product composite. It is essential to understand (and describe) these internal parts, as well as their sub-parts, their internal relations, as well as their relations with external factors.

In addition to the internal parts there are several external objects (outside the boundary of the IA/RA process-product composite) that need to be understood (and described), in particular in terms of their relations and influence to the internal parts of the IA/RA object, not necessarily so much as themselves as such. These are e.g.: - the target phenomena being assessed (e.g. environmental health - the full chain is an attempt to describe this) - policy context (incl. use process) - scientific context (knowledge and means available) - social context (?)

So, the contextual aspects need to be understood (well enough), and their influences to the assessment process and product need to be defined (considered?). The assessment process and product need to be understood in detail, including the sub-processes, process phases and product parts, given the influences/requirements coming from the (relevant) aspects of their context. This should form the content of the IA/RA method(ology), and subsequently the content of the guidebook. All the details should be considered within this framework.

The information content of an IA/RA guidance system should thus be structured as follows:

(underlying) general assessment framework

• scope (boundary + relevant context)
• assessment objects
  • products (descriptions of base-line reality)
    • assessment
    • variable
    • class
  • processes (descriptions of assessment work)
    • sub-processes
      • observation
      • information collection
      • information synthesis
      • management of assessment process
        • phases of assessment
          • issue framing
          • designing variables
          • executing variables
          • reporting

Guidebook

specific assessment framework

• scope
  • domain description (target area(s) of interest to be scrutinized)
  • scientific context (conceptual means and constraints for assessment)
  • technical context (technical means and constraints for assessment)
  • policy context (organization of the decision making and execution)
    • use process description (intended uses & users of output)
  • social context (organization of the society/ies where assessments are performed and their outputs used)
• assessment objects
  • assessment product descriptions
    • domain specific classes (e.g. steps in causal chain)
  • assessment process descriptions
    • phases of assessment process
      • issue framing
      • designing variables
      • executing variables
      • reporting
• methods and tools
  • per sub-process
  • per process phase
  • per domain specific classes (e.g. steps in causal chain)

Resource center (unstructured information about reality within domain(s))

• collected data within the system
• data descriptions
• access to external data
• access to external data descriptions
• encyclopedia articles
  • glossary

Product warehouse (structured information about reality within domain(s))

• particular assessments
• particular variables
• particular classes

The guidebook is the description of the assessment (processes + products) and the relevant parts of its context in terms of a specific framework (here Intarese framework). It is based on a general assessment framework, which is applicable within all domains. It tells why do you perform an assessment, how do you perform an assessment, and why do you perform an assessment like that. It is essential that the guidebook is comprehensive, detailed and as exact as possible (,but not rigorous and restrictive).

Resource center is merely a collection of information that is, or might be, useful in particular assessment within the specified domain(s). It does not need to be so exact, structured and detailed as the guidebook should be. It is useful, but not crucial.

The product warehouse is a collection of the outputs of specific assessments. Its structure, again, is well defined and strictly organized. This is essential for effective and efficient re-usability of the warehouse contents as well for clarity and understandability of the complex phenomena they describe.

See also (the warehouse is not included as it is not seen as the first priority of the WP 4.2 Intarese):