Difference between revisions of "Joint method development period in Kuopio in 2007"

Revision as of 12:08, 14 March 2007

Joint method development period was an idea from the second annual meeting of Intarese. The idea in short is to get the key method developers from different institutes in one place for a long enough period to be able to produce and finish something during that time. The joint method developement period is organized by KTL and targeted mainly to SP1 & SP4 people, although partipcipation is open to anyone interested.

This page contains information on the joint method development period and a brief description of the means and tools that are currently being used in applying the pyrkilo method in environmental health risk assessments in KTL. In the end of the page, there is also a copy of the WP1.4 plan for months 13-30. Please feel free to make major edits when necessary.

Check also these links.

• Risk assessment on airports - Kuopio workshop case study
• intermediate report from this workshop
• social activities during the workshop
• description of the pyrkilo method (a manuscript)
• Intarese general method
• Category:Risk assessments on airports
• DALYs, QALYs, Risk-benefit analysis from Wikipedia
• Risk perception – A short introduction in perception issues by Jutta Koehler, Vivianne Visschers, Ric van Poll
• Value of information review by Marko Tainio
• Bayesian hierarchical modeling review by Mika Hujo
• Environmental health planner for policy comments by Leendert van Bree
• HIA and DALYs by Anne Knol
• The Impact Pathway and Cost-Benefit Modelling Review
• Monte Carlo simulation
• File:Third-draft protocol exposure-health effect (1 3).doc

Method development period

We welcome everyone interested in these methods to Kuopio for a two-week workshop in March 12 to 23. The participating institutes (and their representatives) are RIVM (Anne, SP1), USTUTT (Alex, SP4), NILU (Sjur WP4.1), UU (Hanna WP1.3), IC (Clive) and KTL (Risk analysis group WP1.4 + air hygiene lab WP1.2). In addition Mari Vanhatalo from University of Helsinki / EVAHER project will participate in the workshop for a couple of days.

The aims of this workshop

• to familiarise everyone to the tools that are being used and developed for risk assessment in the participating institutes
• to identify possible overlaps, gaps, and interface mismatchs, and try to find a reasonable solutions to these
• to work together on a specific case study in practice with the existing tools
• to gain practical experience on the tools and methods and identify development needs
• to write a report about what we learnt for internal use in Intarese (or even for external use?)

The picture attempts to represent an outline of the available methods and tools to be used in the workshop case study. Comments, additions, corrections etc. are very welcome.

The structure of the workshop

We will be working on a practical case attempting to make an integrated risk assessment using the methods and tools we have or are developing within Intarese. The case study topic will be airports (air pollution+noise; air traffic+surface traffic).

The workshop participants could be roughly grouped e.g. as follows:

• core team: Jouni, Mikko, Anne (12.3.-19.3.), Alex (16.3.-23.3.), Hanna (12.3.-19.3.), Sjur (12.3.-15.3.), Clive (14.3. evening - 19.3. morning), Mari (15.3.-16.3.)...
• KTL team: Marko, Olli, Anna, Virpi, Eva, Miranda, Markku, Juha V (system support), Aleksi, Einari, ...
• local leaders: Matti, Juha P
• remote leaders: David, Erik, Marco, Gerard, Rainer, Aasmund, ...

The core team will be working full-time on the case study during the whole workshop period, or at least the time of their presence in Kuopio. KTL team will also strongly take part in making the assessment as well as meetings and discussions, but only a proportion of their daily work time will be required to be spent on working on their responsibility areas of the case study. The local leaders will be welcome to attend every part of the workshop, especially the early phases, but no individual particular tasks will be allocated to them.

The remote leaders will be incorporated in the process by preliminary e-mail discussions on the case topic, a teleconference meeting on the second day of the workshop, access to follow-up on the Intarese-wiki, an intermediate report delivered to the SP1 meeting (+ feedback on it) and final reporting on the workshop. The lessons learned will be also presented in the SP4 meeting coming up in late March or April. Participation to daily morning meetings via phone is also possible, please inform about your interest in advance on this to the list of meetings below.

organization of the participants

Katiska = student apartment building, Katiskaniementie 6, 70700 Kuopio (Rauhalahti)

Working-hours per person

ADD YOUR OWN HOURS INTO THE TABLE BASED ON WHAT YOU CAN CONTRIBUTE TO THE WORKSHOP

The schedule

Daily morning meetings 9:30 - 10:30 Finnish time, E-wing meeting room

• Tuesday 13.3. 9:30 Minutes

Remote participants: Alex (+49 711 685-87838)

• • Please fill the working hours that you can spend on the workshop
  • What tools exist for scoping?
    • Alex's issue framing tool
    • Something from NILU?
    • Mediawiki categories
  • Based on what we now know, what is the scoping?
  • Tasks for the day: what they are and who does what
  • Anne's causal diagram
    • Methods and tools linked to the diagram

• Wednesday 14.3. 9:30

Remote participants: Alex (+49 711 685-87838)

Discussed issues:

List of methods, tools and programs that we have and what we aould like to have.

• Scoping tool
  • needed
  • Describe functionalities
  • Demonstration
• Stakeholder + communication:
  • Selection of stakeholders?
  • Meetings
  • Mediawiki
• Uncertainty
  • MNP UNC.Guide (reminder for each variable)
  • WP1.3:checklist
  • Monte-Carlo simulation, Bayesian statistics
  • Sensitivity analyses tools (Frey et al. 200? article)
• Perception
  • Online tool
• Presentation and communication
  • Guidance exists (EPA?)
  • Structured reports (also other parts than the final report)
• Expert judgement
  • Selection of experts
  • Expert elicitation (excalibur)
  • WP1.3 (and WP1.5)
• Value of information
  • EVPI, EVPXI and so on
  • Could answer to question:How far you should go?
• Indicators
  • Guidance in developing
  • Need for improved DPSEEA
  • Use of WHO indicators: needs work
  • What is the difference between indicator and variable? (Mikko:indicators are variables with special interest)
  • SP3?
• Combination of data
  • Meta-analysis methods for epi and tox data

• Thursday 15.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
• Friday 16.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE

• Monday 19.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
• Tuesday 20.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
• Wednesday 21.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
• Thursday 22.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE
• Friday 23.3. 9:30 ADD YOUR NAME HERE IF YOU WANT TO PARTICIPATE VIA PHONE (in 2nd floor lecture hall)

Documenting/reporting work in Intarese-wiki

Monday 12.3.

Meeting at 12.30-14.00 E-wing meeting room

• practical arrangements
• initial scoping
• allocation of tasks

Tuesday 13.3.

• Teleconference meeting with remote participants on the content of the case starting 13:00 Finnish time, B-wing meeting room

Agenda:
*Decision on scoping: what are the details to be studied related to airport risks?
*Decision on methods that should be used or tested during the workshop
*Decision on software and tools that should be used or tested during the workshop
*Content of the report from the workshop (draft will be presented in SP1 meeting on Tuesday 20.3.)
*Add other items here as necessary

Teleconference was eventually cancelled due to technical difficulties. We got some comments from Erik which can be found at the case study page along with some discussion.

• Starting the assessment work

Wednesday 14.3. - Friday 16.3.

• data collection, describing variables, discussions, ...

Saturday 17.3. - Sunday 18.3.

• social activities
• freetime
• (individual work on own tasks?)

Monday 19.3.

• preparing an intermediate report for SP1 meeting
• assessment work goes on...

Tuesday 20.3. - Thursday 22.3.

• finalizing the assessment with respect to SP1 meeting feedback

Friday 23.3

• Preparing a report on the workshop

Issues to be resolved

What are the methods that will be recommended for case studies?

What are the tools and software that will be recommended for case studies?

Which methods and tools will be included as parts of the Intarese general method?

Will the Intarese general method and its products be totally open access (General Public Licence GPL)?

Pyrkilo interface

In this section, we describe the different tools that we have been developing in KTL and within ERAC (Environmental Risk Assessment Center, Kuopio. ERAC is a joint effort of KTL, University of Kuopio, and National Geological Survey of Finland). We have NOT added other programs and tools to this platform, such as the demonstrator of the current toolbox, or the uncertainty program by MNP. This is not because we wouldn't think they aren't important. They are. But we thought it is better to first show what we have and what we don't have in our institute, and only then add other things to the platform.

Overview

The purpose of the platform is to offer all functionalities that are needed to perform an and publish its results. This includes issue framing, drafting the model and variables, collecting data, estimating the values and distributions for the variables, evaluating stakeholder preferences, computing the models, storing the results, and displaying the results to the endusers. Several people and groups of people would use this platform. Depending on his/her role, a person can participate in several different phases of the risk assessment process and contribute to several different ways by offering understanding, opinions, and information. There are many user interfaces to deal with the many tasks that occur during a risk assessment process. The technical details are destribed below in more detail. The methodological issues are described in a manuscript about pyrkilo method, and a draft manuscript about efficiency issues related to the method.

File:Pyrkilo platform.PNG

Programs used in the platform

The platform consists of several programs.

MediaWiki

MediaWiki is the central program in the platform. It is the same program that is used to run Wikipedia, the open encyclopedia. It is basically a content management system with an internet-based, user-friendly interface that allows a number of people to work on a set of documents simultaneously. Its major properties are

• a strict version control
• a simple coding and formatting system
• user identification
• contributions released immediately to everyone to read
• possibility to attach discussions of the content. Each page has a discussion page for this purpose.
• good working environment for text and figures (also tables, although a bit more complicated)
• good categorisation and search tools for pages
• possibility to use templates (a block of content that appears the same way on several pages)

Mediawiki is not especially good at (although it can handle these)

• copy-pasting contents back and forth from one program to another (formatting problems)
• storing ready-made documents (not a file management system)

Discussion on content management programs

How to read discussions

Risk-assessment-related functionalities in MediaWiki:

• Main namespace is an area that contains article-like descriptions of risk-related issues, such as risk assessments, method descriptions etc.
• Variable namespace contains more structured contributions in the form of. This namespace
  • describes variables and their attributes
  • describes causal links between variables
  • Describes the discussions related to the content of a variable on its Talk (or discussion) page.
  • describes the value judgements related to outcomes or other variables.
  • describes smaller pieces of data (and gives links to larger pieces) and describes how the data was used to derive the estimate of a variable.
  • describes rank correlations between variables (possibly using vine copula method).
  • describes set-item relationships between items. This means that a variable may inherit properties from a more general variable of the same kind.
  • describes other non-causal relationships between variables.
• Model namespace contains Analytica and other model files. There can be translated into variables in the Variable namespace or updated based on existing variables.

Analytica

See Monte Carlo simulation.

Result distribution database

Result distribution database is an idea of an SQL database for result distributions of the variables. The main idea is that the variables and their distributions are program-independent in this environment. If their precalculated result distributions are stored in a database, the risk models can be analysed based on these results independent on which program was used to produce the results. In addition, if the model runs are done in a coherent way, these distributions form a large joint distribution that can be used to conditionalising, backward inference, and optimising, which are usually difficult tasks.

The main properties are

• a large SQL database where each variable forms a table
• each table has a fixed number of rows, which equals to the number of simulations used to calculate the model (in the pilot version, this could be in the order of 5000, but in the real database, it should be in the order of 1,000,000 to 10,000,000.
• tables indexed by predefined fractiles to speed up the performance

Major problems

• needs constant updating and simulation (requirements for the hardware)
• When the structure has been fixed, all models must have the same number of simulations irrespective of model size.

Alternative programs could be

• Netica, a commercial program for handling joint distributions based on sample files.

Other possible programs

These programs do not exist in KTL, and there is not (yet) active development going on related to these. However, they could be potentiallly useful programs to attach to the platform.

File management system could assist the risk assessment by providing a centralised database of original data that could be used in the modelling.

Value evaluator could be a web-based questionnaire where stakeholders could rank different outcomes or events in the order of their personal preference. These could then be synthesised and used in the valuation process of risk assessment outcomes.

Expert elicitator could be a web-based questionnaire where experts could give their best estimates on various variables. The answers could be synthesised based on expert-specific weights that would be determined based on their previous performance in this task. There are existing methods to do this using extensive interviews, but it has never been tried over internet.

Interfaces needed to get the platform running

Mediawiki and the user interface exists already, as it is a major part of the Mediawiki program. It is continuously developed by a large Mediawiki community. KTL has a specialised ICT person to update and develop Mediawiki used in pyrkilo risk assessments. We have several projects that constantly utilise this interface, and experience is increasing rapidly.

Ana-Wiki interface is about 1) translating Analytica model files into Mediawiki pages as variables, and 2) updating Analytica model files based on the updated contents in Mediawiki variable pages. This development is ongoing in the Beneris project and these interface tools should be available during spring 2007. We are currently testing a pilot version of Ana-Wiki converter.

Database interface is for converting Analytica (or other simulation program, such as R) results into the database. Analytica has functionalities for input/output from/to SQL databases, so this should not be a huge task. However, we do not have practical experience on this. We are actively testing this functionality, and we should have more experience before February 2007.

Result interface is for showing results from the database to the endusers. Although this is a critical thing and should be developed carefully, it can be postponed to a later stage when the other parts are running. However, some kind of pilot interface should be developed rather early so that researchers in the project can test the database and its functionalities.

Data interface, Elicitation interface, and Evaluation interface are only needed if these tools are developed further. These are not crucial thing at the moment.

• Draft Planning for Next 18 Months (1 Nov 06 – 30 April 08) was removed because it was not the final version. To see the draft, click here.