Joint method development period in Kuopio in 2007

From Testiwiki
Revision as of 06:23, 27 February 2007 by Jouni (talk | contribs) (→‎The structure of the workshop)
Jump to: navigation, search

Joint method development period was an idea from the second annual meeting of Intarese. The idea in short is to get the key method developers from different institutes in one place for a long enough period to be able to produce and finish something during that time. The joint method developement period is organized by KTL and targeted mainly to SP1 & SP4 people, although partipcipation is open to anyone interested.

This page contains information on the joint method development period and a brief description of the means and tools that are currently being used in applying the pyrkilo method in environmental health risk assessments in KTL. In the end of the page, there is also a copy of the WP1.4 plan for months 13-30. Please feel free to make major edits when necessary.

Method development period

We welcome everyone interested in these methods to Kuopio for a two-week workshop in March 12 to 23. The participating institutes (and their representatives) are RIVM (Anne, SP1), USTUTT (Alex, SP4), NILU (Sjur, Aasmund?, Hildegunn?, WP4.1), UU (Hanna WP1.3), IC (Clive) and KTL (Risk analysis group WP1.4 + air hygiene lab WP1.2).

The aims of this workshop

  • to familiarise everyone to the tools that are being used and developed for risk assessment in the participating institutes
  • to identify possible overlaps, gaps, and interface mismatchs, and try to find a reasonable solutions to these
  • to work together on a specific case study in practice with the existing tools
  • to gain practical experience on the tools and methods and identify development needs
  • to write a report about what we learnt for internal use in Intarese (or even for external use?)

The structure of the workshop

We will be working on a practical case attempting to make an integrated risk assessment using the methods and tools we have or are developing within Intarese. The case study topic will be airports (air pollution+noise; air traffic+surface traffic).

The workshop participants could be roughly grouped e.g. as follows:

  • core team: Jouni, Mikko, Anne (12.3.-19.3.), Alex (16.3.-23.3.), Hanna (12.3.-19.3.), Sjur (12.3.-15.3.) +Aasmund/Hildegunn (2nd week?), Clive (14.3. evening - 19.3. morning), ...
  • KTL team: Marko, Olli, Anna, Virpi, Eva, Miranda, Markku, Juha V (system support), Aleksi, Einari, ...
  • local leaders: Matti, Juha P
  • remote leaders: David, Erik, Marco, Gerard, Rainer, Aasmund(?), ...

The core team will be working full-time on the case study during the whole workshop period, or at least the time of their presence in Kuopio. KTL team will also strongly takes part in making the assessment and meetings & discussions, but only a proportion of their daily work time will be required to be spent on working on their responsibility areas of the case study. The local leaders will be welcome to attend every part of the workshop, especially the early phases, but no individual particular tasks will be allocated to them.

The remote leaders will be incorporated in the process by preliminary e-mail discussions on the case topic, a teleconference meeting on the second day of the workshop, access to follow-up on the Intarese-wiki, an intermediate report delivered to the SP1 meeting (+ feedback on it) and final reporting on the workshop. The lessons learned will be also presented in the SP4 meeting coming up in late March or April.

The schedule will be roughly as follows:

Daily:

  • morning meetings 9:30 - 10:00 Finnish time, E-wing meeting room
  • documenting/reporting work in Intarese-wiki

Monday 12.3.

  • practical arrangements
  • initial scoping
  • allocation of tasks

Tuesday 13.3.

  • teleconference meeting with remote participants on the content of the case starting 13:00 Finnish time, E-wing meeting room
  • starting the assessment work

Wednesday 14.3. - Friday 16.3.

  • data collection, describing variables, discussions, ...

Saturday 17.3. - Sunday 18.3.

Monday 19.3.

Tuesday 20.3. - Thursday 22.3.

  • finalizing the assessment with respect to SP1 meeting feedback

Friday 23.3

Pyrkilo interface

In this section, we describe the different tools that we have been developing in KTL and within ERAC (Environmental Risk Assessment Center, Kuopio. ERAC is a joint effort of KTL, University of Kuopio, and National Geological Survey of Finland). We have NOT added other programs and tools to this platform, such as the demonstrator of the current toolbox, or the uncertainty program by MNP. This is not because we wouldn't think they aren't important. They are. But we thought it is better to first show what we have and what we don't have in our institute, and only then add other things to the platform.

Overview

The purpose of the platform is to offer all functionalities that are needed to perform an and publish its results. This includes issue framing, drafting the model and variables, collecting data, estimating the values and distributions for the variables, evaluating stakeholder preferences, computing the models, storing the results, and displaying the results to the endusers. Several people and groups of people would use this platform. Depending on his/her role, a person can participate in several different phases of the risk assessment process and contribute to several different ways by offering understanding, opinions, and information. There are many user interfaces to deal with the many tasks that occur during a risk assessment process. The technical details are destribed below in more detail. The methodological issues are described in a manuscript about pyrkilo method, and a draft manuscript about efficiency issues related to the method.

File:Pyrkilo platform.PNG

Programs used in the platform

The platform consists of several programs.

MediaWiki

MediaWiki is the central program in the platform. It is the same program that is used to run Wikipedia, the open encyclopedia. It is basically a content management system with an internet-based, user-friendly interface that allows a number of people to work on a set of documents simultaneously. Its major properties are

  • a strict version control
  • a simple coding and formatting system
  • user identification
  • contributions released immediately to everyone to read
  • possibility to attach discussion of a page to a related page
  • good working environment for text and figures (also tables, although a bit more complicated)
  • good categorisation and search tools for pages
  • possibility to use templates (a block of content that appears the same way on several pages)


Mediawiki is not especially good at (although it can handle these)

  • copy-pasting contents back and forth from one program to another (formatting problems)
  • storing ready-made documents (not a file management system)

Alternative solutions could be

  • Emailing Word documents around
⇤1: : no version control, people easily drop out of email lists. --Jouni 10:32, 16 January 2007 (EET)


Risk-assessment-related functionalities in MediaWiki:

  • Main namespace is an area that contains article-like descriptions of risk-related issues, such as risk assessments, method descriptions etc.
  • Variable namespace contains more structured contributions in the form of. This namespace
    • describes variables and their attributes
    • describes causal links between variables
    • Describes the discussions related to the content of a variable on its Talk (or discussion) page.
    • describes the value judgements related to outcomes or other variables.
    • describes smaller pieces of data (and gives links to larger pieces) and describes how the data was used to derive the estimate of a variable.
    • describes rank correlations between variables (possibly using vine copula method).
    • describes set-item relationships between items. This means that a variable may inherit properties from a more general variable of the same kind.
    • describes other non-causal relationships between variables.
  • Model namespace contains Analytica and other model files. There can be translated into variables in the Variable namespace or updated based on existing variables.

Analytica

Analytica is a Monte Carlo simulation program that has a user-friendly graphical interface. It is a computational tool for predefined risk/decision models where all the relationships have been mathematically defined.

The main properties are

  • Monte Carlo simulation for uncertainty propagation
  • model structure based on variables and links (similar to DAGs, directed acyclic graphs)
  • variable definition using attributes (that are similar to those in the pyrkilo method)
  • handling of multidimensional variables in an intelligent way
  • hierarchical model structures using modules (i.e. submodels)
  • fairly simple file format using XML
  • interfaces for importing and exporting data from and to Excel and SQL databases

The main problems include

  • not widely used
  • commercial program with non-trivial license fees

Alternative solutions could be

  • @Risk or Crystal Ball, Monte Carlo simulation programs working as Excel add-ins.
⇤4: : Problems with multidimensional variables --Jouni 10:32, 16 January 2007 (EET)
⇤5: : These are not object-oriented programs. --Jouni 10:32, 16 January 2007 (EET)
  • R, S-Plus, Matlab, or other statistical program allowing for simulation
←2: : Some of these have properties that are not in Analytica and could be used as an alternative. --Jouni 10:32, 16 January 2007 (EET)
←3: : R is an open-source program and free of charge. --Jouni 10:32, 16 January 2007 (EET)

Result distribution database

Result distribution database is an idea of an SQL database for result distributions of the variables. The main idea is that the variables and their distributions are program-independent in this environment. If their precalculated result distributions are stored in a database, the risk models can be analysed based on these results independent on which program was used to produce the results. In addition, if the model runs are done in a coherent way, these distributions form a large joint distribution that can be used to conditionalising, backward inference, and optimising, which are usually difficult tasks.

The main properties are

  • a large SQL database where each variable forms a table
  • each table has a fixed number of rows, which equals to the number of simulations used to calculate the model (in the pilot version, this could be in the order of 5000, but in the real database, it should be in the order of 1,000,000 to 10,000,000.
  • tables indexed by predefined fractiles to speed up the performance


Major problems

  • needs constant updating and simulation (requirements for the hardware)
  • When the structure has been fixed, all models must have the same number of simulations irrespective of model size.

Alternative programs could be

  • Netica, a commercial program for handling joint distributions based on sample files.

Other possible programs

These programs do not exist in KTL, and there is not (yet) active development going on related to these. However, they could be potentiallly useful programs to attach to the platform.

File management system could assist the risk assessment by providing a centralised database of original data that could be used in the modelling.

Value evaluator could be a web-based questionnaire where stakeholders could rank different outcomes or events in the order of their personal preference. These could then be synthesised and used in the valuation process of risk assessment outcomes.

Expert elicitator could be a web-based questionnaire where experts could give their best estimates on various variables. The answers could be synthesised based on expert-specific weights that would be determined based on their previous performance in this task. There are existing methods to do this using extensive interviews, but it has never been tried over internet.


Interfaces needed to get the platform running

Mediawiki and the user interface exists already, as it is a major part of the Mediawiki program. It is continuously developed by a large Mediawiki community. KTL has a specialised ICT person to update and develop Mediawiki used in pyrkilo risk assessments. We have several projects that constantly utilise this interface, and experience is increasing rapidly.

Ana-Wiki interface is about 1) translating Analytica model files into Mediawiki pages as variables, and 2) updating Analytica model files based on the updated contents in Mediawiki variable pages. This development is ongoing in the Beneris project and these interface tools should be available during spring 2007. We hope to have a pilot version of Ana-Wiki converter in January.

Database interface is for converting Analytica (or other simulation program, such as R) results into the database. Analytica has functionalities for input/output from/to SQL databases, so this should not be a huge task. However, we do not have practical experience on this. We are actively testing this functionality, and we should have more experience before February 2007.

Result interface is for showing results from the database to the endusers. Although this is a critical thing and should be developed carefully, it can be postponed to a later stage when the other parts are running. However, some kind of pilot interface should be developed rather early so that researchers in the project can test the database and its functionalities.

Data interface, Elicitation interface, and Evaluation interface are only needed if these tools are developed further. These are not crucial thing at the moment.

Developmental needs

Draft Planning for Next 18 Months (1 Nov 06 – 30 April 08)

SUB PROJECT 1.4 START DATE OR STARTING EVENT month 13


ACTIVITY TYPE: Research
PARTICIPANT ID KTL IC RIVM MNP UU USTUTT INVS
PERSON-MONTHS PER PARTICIPANT 9 1 4 1 2 3 1
TRAVEL & SUBSISTENCE (IN EURO) 7334 734 2600 334 1067 1800 334
CONSUMABLES (IN EURO) 6800 534 1967 500 700 1267 334
COMPUTING (IN EURO) 1000 1000
EQUIPMENT (IN EURO)


OBJECTIVES

- To revise and compile the Risk characterisation methodology report based on the review drafts - To write the Risk characterisation protocol based on the methodological work: Environmental Health Planner (MNP/RIVM), Impact pathway/cost-benefit (USTUTT), and pyrkilo (KTL) - To offer guidance to and collect feedback from SP3 workpackages in their risk assessment work. - To implement the Risk characterisation protocol within the Toolbox (SP4).

DESCRIPTION OF WORK

Risk characterisation methodology report (D18):

WP1.4 has produced draft reports about most of the topics planned. The reviews that are still missing (parts of Health impact assessment (IOM); Policy-relevant indicators (RIVM); Public perception issues (RIVM)) will be finished in the early months of the planning period. Then, the compiled report will be sent for comments within INTARESE (especially SP1 and SP3), revised, and brought together with the contributions of other workpackages in SP1. The report will be finished by month 18. Responsibility: As assigned previously.

Risk characterisation protocol (D17):

Several lines of thought have emerged within INTARESE about possible protocols related to risk characterisation. A comparison of those approaches has lead to a fairly uniform view on how different approaches can be combined in an integrated way, where the best parts of each approach can be utilised. KTL will finalise a manuscript of an participatory risk assessment protocol. This will be improved by causal modelling (by USTUTT) and outcome indicator panels (by RIVM). The method development will be performed with intensive collaboration, as researchers from KTL, RIVM, and USTUTT will join together in Kuopio for a few weeks and apply the methods on a case study around month 17. The current draft of D17 will be revised based on this work. The protocol will be developed in collaboration with another EU-funded project BENERIS (Benefit-risk assessment for food: an iterative value-of-information approach, www.beneris.eu). Food-related case studies will be performed in Beneris, and selected approaches will be tested there. Responsibility: KTL.

Guidance and feedback:

The methodology report and protocol will be distributed to SP3. Each WP3.X should utilise one or more of the methods described in the methodology report D18. Technical guidance will be given to this by WP1.4 when necessary. However, the methods are not novel in the sense that they have been used elsewhere before, but it is important to practice them and gain experience. On the other hand, the protocol described in D17 is a new way of doing risk assessments, and there is both practical and scientific merit in applying it in the policy assessments. Therefore, there will be a strong emphasis on developing, distributing, and applying the protocol in case studies, guiding in its use, and collecting feedback from its applicability within WP1.4 and from WP3.X. This protocol forms the backbone of the risk assessment. The first draft of the protocol, as well as a short practical guide for implementing it are almost ready, and they will be distributed on month 14. To facilitate guidance and feedback, each workpackage in SP3 will be assigned to a partner from WP1.4, who is responsible for the immediate guidance and feedback. When possible, these contact partners are the same as contacts between WP3.X and WP4.1 or WP4.2. A report about the feedback will be produced by KTL, RIVM, and INVS.

KTL: overall guidance; WP3.5, WP3.7 (WP3.1) IC: 3.4 RIVM/MNP: WP3.2 USTUTT: WP3.3, WP3.6 UU: WP3.1

After the work with case studies has really started and the need for immidiate guidance and support diminishes, the focus will be moved to the efficiency of doing risk assessments. Novel ways to combine assessments will be developed and tested within WP1.4 and offered to be used by WP3.X. This will probably be a major effort during the latter part of months 13-30.

Implementing the protocol within the Toolbox

WP1.4 will work closely with WP4.1 and WP4.2 to help develop the INTARESE Toolbox. Especially the protocol developed here will hopefully form an important part of the toolbox. Some of the risk characterisation tools have been created using the Mediawiki program. The interfaces between this program and the programs used in SP4 Toolbox development will be combined when useful and possible. This work will start during the intensive collaboration period in Kuopio around month 17. Responsibility: KTL, UTSUTT

DELIVERABLES

D17 Risk characterisation protocol D18 Risk characterisation methodology report DXX Feedback report D49 Revised risk characterisation methodology

MILESTONES AND EXPECTED RESULTS

M22 Interface and implementation of risk characterisation in the Toolbox (within Interim Toolbox design) MXX 3rd Presentation and agreement on methodology and development strategy

Retrieved from "http://www.opasnet.org/testiwiki/index.php?title=Joint_method_development_period_in_Kuopio_in_2007&oldid=12732"