Difference between revisions of "Joint method development period in Kuopio in 2007"

Revision as of 08:32, 16 January 2007

Joint method development period was an idea from the second annual meeting of Intarese. The idea in short is to get the key method developers from different institutes in one place for a long enough period (several weeks) to be able to produce and finish something during that time. In the end of the page, there is a copy of the WP1.4 plan for months 13-30. Please feel free to make major edits when necessary.

Method development period: a suggestion

We welcome everyone interested in these methods to Kuopio for a week in the end of March. As the best date for the SP1 meeting seems to be March 15, 2007, we suggest that we'll continue immediately after that until March 22. Logistically, it could be practical to have the SP1 meeting in Helsinki, and those who come for this method development workshop would continue to Kuopio directly from there.

The aims of this workshop would be

• to familiarise everyone to the tools that are being used and developed for risk assessment in the participating institutes
• to identify possible overlaps, gaps, and interface mismatchs, and try to find a reasonable solutions to these
• to work together on a specific case study in practice with the existing tools. Some suggestions:
  • ozone levels in Europe (the case in the current demonstrator)
  • alternative methods for urban traffic (a case study in WP3.1)
  • metal risks around a metal industry plant (a case that is most intensively promoted in KTL at the moment)
  • a new case that is started from scratch
  • something else?
• to gain practical experience on the tools and methods and identify development needs.
• to write a report about what we learnt for internal use in Intarese (or even for external use?)

Pyrkilo interface

In this section, we describe the different tools that we have been developing in KTL and within ERAC (Environmental Risk Assessment Center, Kuopio. ERAC is a joint effort of KTL, University of Kuopio, and National Geological Survey of Finland). We have NOT added other programs and tools to this platform, such as the demonstrator of the current toolbox, or the uncertainty program by MNP. This is not because we wouldn't think they aren't important. They are. But we thought it is better to first show what we have and what we don't have in our institute, and only then add other things to the platform.

Overview

The purpose of the platform is to offer all functionalities that are needed to perform an and publish its results. This includes issue framing, drafting the model and variables, collecting data, estimating the values and distributions for the variables, evaluating stakeholder preferences, computing the models, storing the results, and displaying the results to the endusers. Several people and groups of people would use this platform. Depending on his/her role, a person can participate in several different phases of the risk assessment process and contribute to several different ways by offering understanding, opinions, and information. There are many user interfaces to deal with the many tasks that occur during a risk assessment process. The technical details are destribed below in more detail. The methodological issues are described in a manuscript about pyrkilo method, and a draft manuscript about efficiency issues related to the method.

File:Pyrkilo platform.PNG

Programs used in the platform

The platform consists of several programs.

MediaWiki

MediaWiki is the central program in the platform. It is the same program that is used to run Wikipedia, the open encyclopedia. It is basically a content management system with an internet-based, user-friendly interface that allows a number of people to work on a set of documents simultaneously. Its major properties are

• a strict version control
• a simple coding and formatting system
• user identification
• contributions released immediately to everyone to read
• possibility to attach discussion of a page to a related page
• good working environment for text and figures (also tables, although a bit more complicated)
• good categorisation and search tools for pages
• possibility to use templates (a block of content that appears the same way on several pages)

Mediawiki is not especially good at (although it can handle these)

• copy-pasting contents back and forth from one program to another (formatting problems)
• storing ready-made documents (not a file management system)

Alternative solutions could be

• Emailing Word documents around

⇤1: : no version control, people easily drop out of email lists. --Jouni 10:32, 16 January 2007 (EET)

Analytica

Analytica is a Monte Carlo simulation program that has a user-friendly graphical interface. It is a computational tool for predefined risk/decision models where all the relationships have been mathematically defined.

The main properties are

• Monte Carlo simulation for uncertainty propagation
• model structure based on variables and links (similar to DAGs, directed acyclic graphs)
• variable definition using attributes (that are similar to those in the pyrkilo method)
• handling of multidimensional variables in an intelligent way
• hierarchical model structures using modules (i.e. submodels)
• fairly simple file format using XML
• interfaces for importing and exporting data from and to Excel and SQL databases

The main problems include

• not widely used
• commercial program with non-trivial license fees

Alternative solutions could be

• @Risk or Crystal Ball, Monte Carlo simulation programs working as Excel add-ins.

⇤4: : Problems with multidimensional variables --Jouni 10:32, 16 January 2007 (EET)

⇤5: : These are not object-oriented programs. --Jouni 10:32, 16 January 2007 (EET)

• R, S-Plus, Matlab, or other statistical program allowing for simulation

←2: : Some of these have properties that are not in Analytica and could be used as an alternative. --Jouni 10:32, 16 January 2007 (EET)

←3: : R is an open-source program and free of charge. --Jouni 10:32, 16 January 2007 (EET)

Result distribution database

Result distribution database is an idea of an SQL database for result distributions of the variables. The main idea is that the variables and their distributions are program-independent in this environment. If their precalculated result distributions are stored in a database, the risk models can be analysed based on these results independent on which program was used to produce the results. In addition, if the model runs are done in a coherent way, these distributions form a large joint distribution that can be used to conditionalising, backward inference, and optimising, which are usually difficult tasks.

The main properties are

• a large SQL database where each variable forms a table
• each table has a fixed number of rows, which equals to the number of simulations used to calculate the model (in the pilot version, this could be in the order of 5000, but in the real database, it should be in the order of 1,000,000 to 10,000,000.
• tables indexed by predefined fractiles to speed up the performance

Major problems

• needs constant updating and simulation (requirements for the hardware)
• When the structure has been fixed, all models must have the same number of simulations irrespective of model size.

Alternative programs could be

• Netica, a commercial program for handling joint distributions based on sample files.

Other possible programs

These programs do not exist in KTL, and there is not (yet) active development going on related to these. However, they could be potentiallly useful programs to attach to the platform.

File management system could assist the risk assessment by providing a centralised database of original data that could be used in the modelling.

Value evaluator could be a web-based questionnaire where stakeholders could rank different outcomes or events in the order of their personal preference. These could then be synthesised and used in the valuation process of risk assessment outcomes.

Expert elicitator could be a web-based questionnaire where experts could give their best estimates on various variables. The answers could be synthesised based on expert-specific weights that would be determined based on their previous performance in this task. There are existing methods to do this using extensive interviews, but it has never been tried over internet.

Interfaces needed to get the platform running

Mediawiki and the user interface exists already, as it is a major part of the Mediawiki program. It is continuously developed by a large Mediawiki community. KTL has a specialised ICT person to update and develop Mediawiki used in pyrkilo risk assessments. We have several projects that constantly utilise this interface, and experience is increasing rapidly.

Ana-Wiki interface is about 1) translating Analytica model files into Mediawiki pages as variables, and 2) updating Analytica model files based on the updated contents in Mediawiki variable pages. This development is ongoing in the Beneris project and these interface tools should be available during spring 2007. We hope to have a pilot version of Ana-Wiki converter in January.

Database interface is for converting Analytica (or other simulation program, such as R) results into the database. Analytica has functionalities for input/output from/to SQL databases, so this should not be a huge task. However, we do not have practical experience on this. We are actively testing this functionality, and we should have more experience before February 2007.

Result interface is for showing results from the database to the endusers. Although this is a critical thing and should be developed carefully, it can be postponed to a later stage when the other parts are running. However, some kind of pilot interface should be developed rather early so that researchers in the project can test the database and its functionalities.

Data interface, Elicitation interface, and Evaluation interface are only needed if these tools are developed further. These are not crucial thing at the moment.

Developmental needs

Draft Planning for Next 18 Months (1 Nov 06 – 30 April 08)

SUB PROJECT 1.4 START DATE OR STARTING EVENT month 13

OBJECTIVES

- To revise and compile the Risk characterisation methodology report based on the review drafts - To write the Risk characterisation protocol based on the methodological work: Environmental Health Planner (MNP/RIVM), Impact pathway/cost-benefit (USTUTT), and pyrkilo (KTL) - To offer guidance to and collect feedback from SP3 workpackages in their risk assessment work. - To implement the Risk characterisation protocol within the Toolbox (SP4).

DESCRIPTION OF WORK

Risk characterisation methodology report (D18):

WP1.4 has produced draft reports about most of the topics planned. The reviews that are still missing (parts of Health impact assessment (IOM); Policy-relevant indicators (RIVM); Public perception issues (RIVM)) will be finished in the early months of the planning period. Then, the compiled report will be sent for comments within INTARESE (especially SP1 and SP3), revised, and brought together with the contributions of other workpackages in SP1. The report will be finished by month 18. Responsibility: As assigned previously.

Risk characterisation protocol (D17):

Several lines of thought have emerged within INTARESE about possible protocols related to risk characterisation. A comparison of those approaches has lead to a fairly uniform view on how different approaches can be combined in an integrated way, where the best parts of each approach can be utilised. KTL will finalise a manuscript of an participatory risk assessment protocol. This will be improved by causal modelling (by USTUTT) and outcome indicator panels (by RIVM). The method development will be performed with intensive collaboration, as researchers from KTL, RIVM, and USTUTT will join together in Kuopio for a few weeks and apply the methods on a case study around month 17. The current draft of D17 will be revised based on this work. The protocol will be developed in collaboration with another EU-funded project BENERIS (Benefit-risk assessment for food: an iterative value-of-information approach, www.beneris.eu). Food-related case studies will be performed in Beneris, and selected approaches will be tested there. Responsibility: KTL.

Guidance and feedback:

The methodology report and protocol will be distributed to SP3. Each WP3.X should utilise one or more of the methods described in the methodology report D18. Technical guidance will be given to this by WP1.4 when necessary. However, the methods are not novel in the sense that they have been used elsewhere before, but it is important to practice them and gain experience. On the other hand, the protocol described in D17 is a new way of doing risk assessments, and there is both practical and scientific merit in applying it in the policy assessments. Therefore, there will be a strong emphasis on developing, distributing, and applying the protocol in case studies, guiding in its use, and collecting feedback from its applicability within WP1.4 and from WP3.X. This protocol forms the backbone of the risk assessment. The first draft of the protocol, as well as a short practical guide for implementing it are almost ready, and they will be distributed on month 14. To facilitate guidance and feedback, each workpackage in SP3 will be assigned to a partner from WP1.4, who is responsible for the immediate guidance and feedback. When possible, these contact partners are the same as contacts between WP3.X and WP4.1 or WP4.2. A report about the feedback will be produced by KTL, RIVM, and INVS.

KTL: overall guidance; WP3.5, WP3.7 (WP3.1) IC: 3.4 RIVM/MNP: WP3.2 USTUTT: WP3.3, WP3.6 UU: WP3.1

After the work with case studies has really started and the need for immidiate guidance and support diminishes, the focus will be moved to the efficiency of doing risk assessments. Novel ways to combine assessments will be developed and tested within WP1.4 and offered to be used by WP3.X. This will probably be a major effort during the latter part of months 13-30.

Implementing the protocol within the Toolbox

WP1.4 will work closely with WP4.1 and WP4.2 to help develop the INTARESE Toolbox. Especially the protocol developed here will hopefully form an important part of the toolbox. Some of the risk characterisation tools have been created using the Mediawiki program. The interfaces between this program and the programs used in SP4 Toolbox development will be combined when useful and possible. This work will start during the intensive collaboration period in Kuopio around month 17. Responsibility: KTL, UTSUTT

DELIVERABLES

D17 Risk characterisation protocol D18 Risk characterisation methodology report DXX Feedback report D49 Revised risk characterisation methodology

MILESTONES AND EXPECTED RESULTS

M22 Interface and implementation of risk characterisation in the Toolbox (within Interim Toolbox design) MXX 3rd Presentation and agreement on methodology and development strategy