Joint method development period in Kuopio in 2007

From Testiwiki
Revision as of 10:09, 26 March 2007 by Mikko Pohjola (talk | contribs) (links categorized)
Jump to: navigation, search

Joint method development period was an idea from the second annual meeting of Intarese. The idea in short was to get the key method developers from different institutes in one place for a long enough period to be able to produce and finish something during that time. The joint method developement period was organized by KTL and targeted mainly to SP1 & SP4 people, although participation was open to anyone interested. The joint method development workshop was held in Kuopio 12.3.-23.3.2007

This page contains information on the joint method development period and a brief description of the means and tools that are currently being used in applying the pyrkilo method in environmental health risk assessments in KTL. Please feel free to make major edits when necessary. Check also the links below.


Pages created/started in the workshop:


Other related and useful links:


SP1 meeting notes

The SP1 meeting 20.3. notes have been moved to: SP1 Integrated assessment methodology

Method development period

The participating institutes from Intarese (and their representatives) were RIVM (Anne Knol), USTUTT (Alex Kuhn), NILU (Sjur Björndalsaeter), UU (Hanna Boogaard), IC (Clive Sabel) and KTL (Risk analysis group WP1.4 + air hygiene lab WP1.2). In addition we had Mari Vanhatalo from University of Helsinki / EVAHER project and Patrycja Jesionek from TU Delft / Beneris project visiting the workshop for a few days during the first week.

The aims of the workshop

  • to familiarise everyone to the tools that are being used and developed for risk assessment in the participating institutes
  • to identify possible overlaps, gaps, and interface mismatchs, and try to find a reasonable solutions to these
  • to work together on a specific case study in practice with the existing tools
  • to gain practical experience on the tools and methods and identify development needs
  • to write a report about what we learnt for internal use in Intarese (or even for external use?)

Issues to be resolved

  • What are the methods that will be recommended for case studies?
  • What are the tools and software that will be recommended for case studies?
  • Which methods and tools will be included as parts of the Intarese general method?
  • Will the Intarese general method and its products be totally open access (General Public Licence GPL)?
  • What is the most important output for SP 4 and the toolbox?
    • provide a workspace
    • maybe test out methods (e.g. for issue framing) in a draft environment while the design phase is still ongoing
    • We have collected all the necessary parts of an assessment and the corresponding methods and tools - this is a good basis for the needed functionality of the toolbox

methods/tools framework at the start of the workshop

Error creating thumbnail: Unable to save thumbnail to destination

The above picture was made before the workshop as an attempt to represent an outline of the available methods and tools to be used in the workshop case study (and in applying the Intarese method). This idea developed further during the workshop and the results can be seen in the tables at Tools needed in Intarese toolbox page.

The structure of the workshop

We started working on a practical case attempting to make an integrated risk assessment using the methods and tools we have available or are developing within Intarese. The case study topic was chosen to be Schiphol airport (air pollution+noise; air traffic+surface traffic).

The workshop participants could be roughly grouped e.g. as follows (duration of stay in Kuopio in brackets):

  • core workshop team: Jouni, Mikko, Anne (12.3.-19.3.), Alex (16.3.-23.3.), Hanna (12.3.-19.3.), Sjur (12.3.-15.3.), Clive (14.3. evening - 19.3. morning), Mari (15.3.-16.3.), Patrycja (15.3.-19.3.)
  • KTL team: Marko, Olli, Anna, Virpi, Eva, Miranda, Markku, Juha V (system support)
  • local leaders: Matti, Juha P
  • remote leaders: David, Erik, Marco, Gerard, Rainer, Aasmund, ...

The core team was working full-time on the case study during their participation within the workshop period. KTL team also strongly took part in making the case study assessment as well as participated in meetings and other discussions, but naturally were also involved with other issues beside the workshop. The local leaders took part in some of the meetings as were available from other duties and also held some other Intarese related discussions and meetings with the participants during the workshop.

The remote leaders were incorporated in the process by preliminary e-mail discussions on the case topic, a teleconference meeting on the second day of the workshop (which unfortunately failed due to technical reasons), access to follow-up on the Intarese-wiki, an intermediate report delivered to the SP1 meeting (+ feedback on it) and final reporting on the workshop. The lessons learned and other outputs of the workshop will be also presented in the SP4 meeting coming up in mid-April. Participation to daily morning meetings via phone was also possible and was made use of a few times.

Organization and working-hours

The organization table and working-hours table were out-dated and archived to history. You can find them following the link: [1]

The schedule & meeting minutes

Monday 12.3.

Start-up meeting

Tuesday 13.3.

Morning meeting 9:30 Minutes, remote participants: Alex (+49 711 685-87838)

The planned teleconference at 13:00 was eventually cancelled due to technical difficulties. We got some comments on phone from Erik and they can be found at the case study page along with some discussion.

Wednesday 14.3.

Morning meeting 9:30, remote participants: Alex (+49 711 685-87838)

Disussion content: list of methods, tools and programs that we have and what we would like to have:

  • Scoping tool
    • needed
    • Describe functionalities
    • Demonstration
  • Stakeholder + communication:
    • Selection of stakeholders?
    • Meetings
    • Mediawiki
  • Uncertainty
    • MNP UNC.Guide (reminder for each variable)
    • WP1.3:checklist
    • Monte-Carlo simulation, Bayesian statistics
    • Sensitivity analyses tools (Frey et al. 200? article)
  • Perception
    • Online tool
  • Presentation and communication
    • Guidance exists (EPA?)
    • Structured reports (also other parts than the final report)
  • Expert judgement
    • Selection of experts
    • Expert elicitation (excalibur)
    • WP1.3 (and WP1.5)
  • Value of information
    • EVPI, EVPXI and so on
    • Could answer to question:How far you should go?
  • Indicators
    • Guidance in developing
    • Need for improved DPSEEA
    • Use of WHO indicators: needs work
    • What is the difference between indicator and variable? (Mikko:indicators are variables with special interest)
    • SP3?
  • Combination of data
    • Meta-analysis methods for epi and tox data


Thursday 15.3.

Morning meeting 9:30

Agenda/highlights of discussion:

  • Introduction of what has been going on to the newcomers (Patrycja, Clive, Mari)
    • Patrycja to present her BBN model in friday morning meeting
  • MNP uncertainty guide demo (Anne, Sjur)
    • the tool can be adapted to meet Intarese needs
  • Intarese demonstrator demo (Sjur)
    • an updated demonstrator to be done later this year
    • this workshop should provide a rough suggestion of the needed toolbox functionalities
  • tools needed in Intarese toolbox
    • participants to have a look and edit the page - will be discussed later in more detail
  • contents of variable, input and method pages
    • serious effort to making the case required thursday and friday
    • limitations of night-time noise chosen as the only policy option to be considered


Friday 16.3.

Morning meeting 9:30 IN B-WING MEETING ROOM

Agenda/minutes:

  • What to present in SP1 meeting next tuesday?
  • Getting on with the case assessment
    • emphasis on identification of linkages between variables and methods/tools
    • noise policy case as the essential part of the case study assessment
  • Patrycja's BBN model presentation was given in the afternoon 15:00


Weekend 17.3.-18.3.

Monday 19.3.

Morning meeting 10:30, essentials:

  • framework, methods and tools -tables appear to be the most important output of the workshop so far
  • assessment workspace concept seems to capture effectively the primary needs for the Intarese toolbox
  • the case study assessment has been useful in coordinating our discussions about the framework methods and tools, but does not appear to be producing hardly any direct results
  • the main points to be written out intermediate report page for SP1 meeting

Tuesday 20.3.

Morning meeting 9:30, essentials:

  • continue working on the tables
  • contribute to development/description of the assessment workspace
  • case study work can be frozen for the time being at least until we get the SP1 meeting feedback

Wednesday 21.3.

Morning meeting 9:30

Agenda/minutes:

  • feedback from SP1 meeting
    • see the comments by Jouni above
  • decision of focusing efforts on last 3 days of workshop
    • tables still have space for improvement
    • assessment workspace concept needs to be worked on
    • case study can be left to wait for possible further use for the time being
    • let's try to make full use of Alex's presence during this week (valuation presentation, WP1.4 discussions, glossary discussions, ...)

Thursday 22.3.

Morning meeting 9:30

Agenda/minutes:

  • work for the last days
    • improvement of tables (everyone)
    • ideas/comments/anything to be put on the assessment workspace -page (everyone)
    • writing a report: how things happened?, what was good/bad? what did we learn?, etc. (Mikko + everyone)
  • Alex's brief presentation about health impact valuation + discussion

Media:Monetisation.ppt

  • after the meeting we also ended up talking about organising a European risk analysis study program as a spin-off/result of Intarese
  • Glossary discussion (Alex, Mikko)
  • discussion/meeting about WP1.4 deliverables due end of April (Jouni, Alex, Mikko, Marko?, ...)

Friday 23.3.

Morning meeting 10:15 (in Jouni's room)

Remote participant: Clive (+44 1327 843319)

Agenda/minutes:

  • finalizing the output:
    • report to be written with links to the tables, case and assessment workspace -pages
      • description of things worked out (or not)
      • also general lessons learned and individual comments included
      • implications of the output pointed out
      • a pdf-version to be created and distributed within the project (only highlights of the case included)
  • making use of the output:
    • assessment workspace and framework/method description to be included in WP1.4 deliverables due in April
    • assessment workspace and framework/method description to be presented in SP4 meeting in April
    • WP1.4 will suggest assessment workspace concept framework/method description to be included in the training workshop (in May) contents
    • let's try to get some WP3.X('s) to do/present/share/disseminate their work in wiki using the assessment workspace idea
      • this serves the purposes of both guidance and feedback to/from SP1 & SP3
  • We should develop the idea of producing ascientific article about the workshop and its outputs
    • e.g. the process of creating the Intarese method (Clive's proposal)
    • this should be done following the Intarese dissemination strategy

Pyrkilo interface

In this section, we describe the different tools that we have been developing in KTL and within ERAC (Environmental Risk Assessment Center, Kuopio. ERAC is a joint effort of KTL, University of Kuopio, and National Geological Survey of Finland). We have NOT added other programs and tools to this platform, such as the demonstrator of the current toolbox, or the uncertainty program by MNP. This is not because we wouldn't think they aren't important. They are. But we thought it is better to first show what we have and what we don't have in our institute, and only then add other things to the platform.

Overview

The purpose of the platform is to offer all functionalities that are needed to perform an and publish its results. This includes issue framing, drafting the model and variables, collecting data, estimating the values and distributions for the variables, evaluating stakeholder preferences, computing the models, storing the results, and displaying the results to the endusers. Several people and groups of people would use this platform. Depending on his/her role, a person can participate in several different phases of the risk assessment process and contribute to several different ways by offering understanding, opinions, and information. There are many user interfaces to deal with the many tasks that occur during a risk assessment process. The technical details are destribed below in more detail. The methodological issues are described in a manuscript about pyrkilo method, and a draft manuscript about efficiency issues related to the method.

File:Pyrkilo platform.PNG

Programs used in the platform

The platform consists of several programs.

MediaWiki

MediaWiki is the central program in the platform. It is the same program that is used to run Wikipedia, the open encyclopedia. It is basically a content management system with an internet-based, user-friendly interface that allows a number of people to work on a set of documents simultaneously. Its major properties are

  • a strict version control
  • a simple coding and formatting system
  • user identification
  • contributions released immediately to everyone to read
  • possibility to attach discussions of the content. Each page has a discussion page for this purpose.
  • good working environment for text and figures (also tables, although a bit more complicated)
  • good categorisation and search tools for pages
  • possibility to use templates (a block of content that appears the same way on several pages)


Mediawiki is not especially good at (although it can handle these)

  • copy-pasting contents back and forth from one program to another (formatting problems)
  • storing ready-made documents (not a file management system)

Discussion on content management programs

How to read discussions

Statements:

Resolution: Resolution not yet found.

(A stable resolution, when found, should be updated to the main page.)

Argumentation:

← B) is a good choice.

⇤1: : no version control, people easily drop out of email lists. --Jouni 10:32, 16 January 2007 (EET)


Risk-assessment-related functionalities in MediaWiki:

  • Main namespace is an area that contains article-like descriptions of risk-related issues, such as risk assessments, method descriptions etc.
  • Variable namespace contains more structured contributions in the form of. This namespace
    • describes variables and their attributes
    • describes causal links between variables
    • Describes the discussions related to the content of a variable on its Talk (or discussion) page.
    • describes the value judgements related to outcomes or other variables.
    • describes smaller pieces of data (and gives links to larger pieces) and describes how the data was used to derive the estimate of a variable.
    • describes rank correlations between variables (possibly using vine copula method).
    • describes set-item relationships between items. This means that a variable may inherit properties from a more general variable of the same kind.
    • describes other non-causal relationships between variables.
  • Model namespace contains Analytica and other model files. There can be translated into variables in the Variable namespace or updated based on existing variables.

Analytica

See Monte Carlo simulation.

Result distribution database

Result distribution database is an idea of an SQL database for result distributions of the variables. The main idea is that the variables and their distributions are program-independent in this environment. If their precalculated result distributions are stored in a database, the risk models can be analysed based on these results independent on which program was used to produce the results. In addition, if the model runs are done in a coherent way, these distributions form a large joint distribution that can be used to conditionalising, backward inference, and optimising, which are usually difficult tasks.

The main properties are

  • a large SQL database where each variable forms a table
  • each table has a fixed number of rows, which equals to the number of simulations used to calculate the model (in the pilot version, this could be in the order of 5000, but in the real database, it should be in the order of 1,000,000 to 10,000,000.
  • tables indexed by predefined fractiles to speed up the performance


Major problems

  • needs constant updating and simulation (requirements for the hardware)
  • When the structure has been fixed, all models must have the same number of simulations irrespective of model size.

Alternative programs could be

  • Netica, a commercial program for handling joint distributions based on sample files.

Other possible programs

These programs do not exist in KTL, and there is not (yet) active development going on related to these. However, they could be potentiallly useful programs to attach to the platform.

File management system could assist the risk assessment by providing a centralised database of original data that could be used in the modelling.

Value evaluator could be a web-based questionnaire where stakeholders could rank different outcomes or events in the order of their personal preference. These could then be synthesised and used in the valuation process of risk assessment outcomes.

Expert elicitator could be a web-based questionnaire where experts could give their best estimates on various variables. The answers could be synthesised based on expert-specific weights that would be determined based on their previous performance in this task. There are existing methods to do this using extensive interviews, but it has never been tried over internet.


Interfaces needed to get the platform running

Mediawiki and the user interface exists already, as it is a major part of the Mediawiki program. It is continuously developed by a large Mediawiki community. KTL has a specialised ICT person to update and develop Mediawiki used in pyrkilo risk assessments. We have several projects that constantly utilise this interface, and experience is increasing rapidly.

Ana-Wiki interface is about 1) translating Analytica model files into Mediawiki pages as variables, and 2) updating Analytica model files based on the updated contents in Mediawiki variable pages. This development is ongoing in the Beneris project and these interface tools should be available during spring 2007. We are currently testing a pilot version of Ana-Wiki converter.

Database interface is for converting Analytica (or other simulation program, such as R) results into the database. Analytica has functionalities for input/output from/to SQL databases, so this should not be a huge task. However, we do not have practical experience on this. We are actively testing this functionality, and we should have more experience before February 2007.

Result interface is for showing results from the database to the endusers. Although this is a critical thing and should be developed carefully, it can be postponed to a later stage when the other parts are running. However, some kind of pilot interface should be developed rather early so that researchers in the project can test the database and its functionalities.

Data interface, Elicitation interface, and Evaluation interface are only needed if these tools are developed further. These are not crucial thing at the moment.


  • Draft Planning for Next 18 Months (1 Nov 06 – 30 April 08) was removed because it was not the final version. To see the draft, click here.
Retrieved from "http://www.opasnet.org/testiwiki/index.php?title=Joint_method_development_period_in_Kuopio_in_2007&oldid=12798"