Talk:Science-policy interface

Application to the Academy of Finland 25.9.2013

The application written in Finnish was sent to the Kunnallisalan kehittämissäätiö. The basis for the Academy application is similar, but more theoretical work will be done to identify problems in the science-policy interface, and larger practical case studies will be done. And the application should be translated into English.

• Instructions

Metadata about document Avoimen päätösvalmistelun työkalutestaus (rahoitushakemus SA:lle 9/2013) (in Finnish) help

Nimi: Avoimen päätösvalmistelun työkalutestaus (rahoitushakemus SA:lle 9/2013) Tehtäväryhmä: 6.00.01.08 Tehtäväryhmän nimi: Rahoitushakemus Rekisteröinti: Vastuualue: YMAL Vastuuhenkilö: Jouni Tuomisto Luotu: 3.9.2013 Säilytysaika: 15 v Säilytysmuoto: Sähköinen Julkisuus: Julkinen Suojeluluokka: 2 (tutkimuksellisista, juridisista tai taloudellisista syistä suojeltavat asiakirjat) Lisätiedot: Pysyvä linkki: [11] Hyväksyntä: Ei ole

Abstract

There is fair agreement that more participatory approaches to societal decision making would, in general, be a positive thing. It has been seen important especially when including objectives such as environment, health, or wellbeing. Also, more extensive use of scientific information is widely believed to produce better decisions. However, it has been unclear how to improve from the current situation. Widely known problems are that open discussions are often hijacked by extremists and the moderate majority is not heard; careful researchers don't win debates against populists in two-minute time frame; and the public is unable to conclude between competing hypotheses even if a clear majority of researchers are in agreement. So, there is no guarantee that a participatory or evidence-based decision process actually succeeds in a particular case.

There is recent research showing that it is especially the practices of policy making (rather than e.g. better communication by researchers) that should be changed to improve participation and the use of evidence. However, cultural practices of decision making change slowly. Therefore, in this project we implement and study situations where practices are only changed within a limited context, namely an open decision support club, or decision club for short.

The decision club is a joint effort and forum for decision makers, researchers, and all stakeholders. The club focusses on creating shared understanding about a decision situation, the options considered and outcomes expected. The purpose of the club is to produce an open web report including quantitative modelling and recommendations to support the decision. Anyone can join the club, but the members must follow its practices and rules about e.g. structured discussion and criticism based on relevance and facts. The work is organised and done in a web-workspace designed for this task, and it is managed by trained moderators. The overall objective is to produce and recommend better practices for current decision making processes and organisations.

We hypothesise that the practical challenges performed and studied in the case studies of this project will demonstrate that a) the participatory processes are manageable and efficient if implemented within such decision clubs and workspaces, b) the recommendations produced take into account the plurality of views present in the society, c) the conclusions made are systematically consistent with scientific knowledge, and d) the information produced will be practical and applicable in the particular decision situations that are examined. Overall, we hope that such decision club approach will gain popularity due to its good decision support performance, which - importantly - can and will be measured with explicit indicators.

The main practices of the decision club and the technical solutions of the workspace have already been developed, implemented, and tested in pilot projects. For the first time, we will implement these practices and methods in large-scale, real-life case studies with actual decision making processes, participation by citizens, and useful and practical decision support. The research in this project will be about the applicability and performance of the method: Are decision makers willing to use the method in their work and why? Are the rules developed practical enough to actually guide discussions, participation, and research? What improvements are needed to increase the use of decision clubs and the performance of current societal decision making?

--tiivistys voisi olla paikallaan - keventäisin kuvausta vähemmän tekniseksi. Voisi esim. aloittaa siitä, että osallistavuus on viime aikoina noussut esiin järjestötoiminnasta hallituksen linjauksiin asti ja nyt tavoitteena on tutkia mikä osa tästä on kvantifioitavissa avoimella prosessilla ja kehittää nykyaikaista teknologiaa hyödyntäviä välineitä tähän ajankohtaiseen haasteeseen vastaamiseksi ja siihen liittyvien tutkimusongelmien kartoittamiseen; yksityiskohtaisempia kuvauksia voi siirtää myöhempään osaan hakemustekstiä. Hakemuksessa voisi viitata Valtioneuvoston tiedotteeseen 5.9.2013: "Hallitus hyväksyi tutkimuslaitosten ja tutkimusrahoituksen kokonaisuudistusta koskevan periaatepäätöksen" [12] 'Elämme tietoyhteiskunnassa vasta, kun tietoa ja tutkittua tietoa käytetään systemaattisesti päätöksenteon tukena Nyt tehty päätös vie Suomea merkittävästi kohti todellista tietoyhteiskuntaa. Sektoritutkimusjärjestelmän tulee ennakoida yhteiskunnan muuttuvia tietotarpeita. Sen on kyettävä joustavasti muuttamaan tutkimuksen painopisteitä ja tuottamaan tarvittavaa tietoa päättäjille sekä julkisella että yksityisellä sektorilla. Tavoitteena on, että tutkimus toimii yhteiskunnan kehittämisen ja päätöksenteon strategisena resurssina, toteaa pääministeri Jyrki Katainen.' - alottaisin jotenkin muuten kuin "more participatory .. is a positive thing"; sen sijaan voisi viitata suoraan siihen tosiasiaan että uudet teknologiat voivat parantaa demokraattisia vaikutusmahdollisuuksia, ja viimeaikainen kehitys on osoittanut (ks.esim. Avoin ministeriö ymv.) että kansalaiset ovat kiinnostuneita tästä;: --Jouni 17:39, 19 September 2013 (EEST) {{{3}}}

--# : Liikkeelle lähteminen väitteellä "osallistuminen on tärkeää" on riskialtista koska, jos arvioija ei satu itse olemaan osallistamisen asialla, mielenkiinto loppuu ensimmäiseen lauseeseen. Itse olen pyrkinyt lähestymään aihetta 1) vetoamalla tiedon tärkeyteen päätöksenteossa, 2) toteamalla monien näkökulmien ja tietolähteiden olevan välttämättömiä monimutkaisia asioita koskevan tiedon hankkimisessa ja hyödyntämisessä ja 3) vasta sitten nostanut esiin yleisen suuntauksen (monilla rintamilla) kohti osallistamista ja avoimuutta. Tällaista perustelua vastaan on perinteisen umpiotutkijankin hankalampi rimpuilla. --Mikko Pohjola 16:48, 23 September 2013 (EEST)

Research plan

Guidance for the research plan: Finnish English

Project info

• Principal investigator: Jouni Tuomisto
• Title of project: Open dynamic decision support (ODDS)
• Site of research: National Institute for Health and Welfare, Department of Environmental Health, Kuopio
• Duration: 48 months

Background

Significance of the research

Evidence-based decision making is a mega-trend in Finland and in other Western countries. Prime Minister Jyrki Katainen recently said that we live in an information society only when research knowledge is systematically used as a basis of decision making. In the Government plan there is an objective to utilise information about environment and health in all decision making . Enterprise architecture, a management system focussing on information and practices, is in a running-in phase in Finnish administration. In addition, organisational changes are under way to improve the capability of Finnish research institutes to answer societal needs. There is clearly a strong tendency to improve the use of knowledge in the societal decision making, and good research-based solutions are needed.

There are challenges especially in the capabilities of decision makers and decision making processes to actually utilise existing information. This is seen as unhappiness of decision makers about data usability, and also unhappiness of researchers about data use. In this project we will demonstrate, implement, and further develop an open dynamic decision support (ODDS) system that consists of several methods, practices, tools, and web-workspaces. It especially helps to structure scientific information in a helpful format for decision support, and enhances critical syntheses of open discussions on policy issues.

Our experience and also some case reports (Tuomisto, 2013) show that poor applicability of information is a typical bottleneck in decision making. The so called Big Data movement has recently started to improve the situation, but current practices don't yet support its possibilities. (Mervis, 2012)

Indeed, there is a need for systematic decision support especially with issues like environment and health. They are widely accepted as important, but in many decisions they only play a small role among other interests and are easily ignored, if the relevant information is not readily available for the decision maker. Climate emissions, biodiversity, or fine particles from combustion are examples of widely dispersed and crucial issues that rarely dominate decision making.

A decision support system does not attempt to replace actual decision making. However, it can organise information, offer a discussion forum, and spread understanding to the society about what should or should not be done and why. Such a system can be seen as similar to recommendations of evidence-based medicine (käypä hoito) containing the best scientific evidence about how patients should be treated in particular situations. This project attempts to create an ODDS ecosystem for producing evidence-based decision support by utilising open participation of interested people but managing the process by clear and specific rules within the ecosystem.

Even if a particular evidence-based advice is - due to lack of information - so simplistic that it does not help the decision maker, it may still be very useful for other important purposes if it is done openly and shared. First, it may be illuminating and useful to a stakeholder who is interested but not aware of the details of the issue. Second, the possibility to describe and share a large number of distinct decisions may be helpful for other decision makers in similar situations. Third, a scrutiny of multiple decisions at the same time may improve understanding of a bigger picture, leading to better decisions and outcomes for all. Therefore, evidence-based efforts should not be evaluated based on their impact on a single case only.

Because real-life problems are complex and fuzzy, we benefit if more people contribute their knowledge and bring in multiple views and ideas. However, this requires that the information can be received, synthesised, and analysed using quantitative models and otherwise. Methods and tools for such work exist, and one systematic collection of them is called open decision making practice (see Previous work), and the need and capability to utilise them are meeting today. This requires dedicated implementation and research on the possibilities, problems, and new solutions of the implementations. This is what this project is about.

There are specific research needs when ODDS systems are applied with municipalities and national authorities such as AVIs or ELY centres. First, there is a need for large case studies, where impact assessment models are tested e.g. with ecological and health effects of mining. Important aspects are applicability of such models and approaches in general in real environmental impact assessment (EIA) and environmental permit processes. Second, the applicability of existing environment and health impact models should be tested and further developed in areas. Important areas include fine particles and CO₂ from traffic and energy production, and pollutants in soil. Third, practices and models should be implemented and tested in several very small case studies or surveys with a narrow, specific question and only a small involvement from municipalities. This approach helps to identify and serve immediate information needs of municipalities about environment and health in decision making. These research needs are answered in this project.

Previous research

There is active research going on about improving the societal use of scientific results. For example, there are suggestions that the policy relevance of scientific assessments must be improved (Perrings et al., 2011) and that they should better reflect the reality of policy making and include local and non-scientific knowledge (Briggs and Knight 2012, Hulme et al., 2011) . However, the effectiveness also depends on the capability of a decision maker to utilise information as a part of decision making process (Lankinen et al., 2012, Junnila, 2012) . Little attention has been paid to information use, and most related research has focussed on information production (Jones 2009)

A recent study has found that a major problem in the science-policy interface actually lies in the inability of the current political processes to utilise existing scientific knowledge in societal decision making (Pohjola, 2013) . This inability applies also to knowledge about citizens' and other stakeholders' values. Evidence-based decision making requires multifaceted, justifiable, practical information production and effective information use.

This observation has lead to the development of a pragmatic guidance for closer collaboration between researchers and societal decision making. The guidance is called open decision making practice and it was developed by National Institute for Health and Welfare (THL) and Nordem Ltd in 2013 for the Ministry of Social Affairs and Health in aim to improve environmental health assessments in Finnish municipalities, but it is generic and widely applicable. One notice in the work was that knowledge practices should be developed simultaneously in both research and decision making. Otherwise either the information supply does not answer the need or vice versa, and the situation does not improve.

The open decision making practice consists of guidance, practices, and tools facilitating production and use of relevant information for a decision. The practice encourages a decision maker to express the objectives of the decision and options considered, and this information is used to guide all work. A large part of the work is to perform an impact assessment that covers all areas of interest (as defined by the decision maker) and also synthesises participation and contributions from anyone interested. The work is constantly evaluated and managed according to specific guidance about properties of good decision support. All this work and contributions are managed by facilitators, who are knowledgeable about the decision situation, research, and the rules of the practice.

One important part of the open decision making practice is a web-workspace to facilitate decision support, share knowledge and learn from others. In the web-workspace both scientific knowledge and policy alternatives and objectives are systematically represented. It is used to systematically store all relevant information in a structured way. This structure also guides the work of information collection and synthesis in an assessment supporting a decision.

The workspace does not replace the actual decision making or current processes such as debates or committees. Instead, it offers tools and thus facilitates improved knowledge practices to describe decision-related information that is relevant in the eyes of a decision maker, stakeholder, or researcher. A main advantage is that specific rules improving information (such as scientific criticism) can be applied within the workspace even if they cannot be applied in other policy forums.

Open decision making practice follows rules that lead to open and reusable information products. This is done by utilising an open web-workspace as described above, by focussing on topics that are influenced by the decision or influencing the outcomes of interest, and by applying explicit rules about which statements or estimates to reject based on relevance and facts. The practice has been documented in reports (Pohjola, 2013) (Pohjola ym., 2012) , method descriptions (Pohjola et al., 2013) , method testing (Sandström et al, in press) (Pohjola et al., 2012b) , websites , and technical documentation .

A key idea in open decision making practice is to focus on information work, and support the management of that work at the same time. The work is organised in a way that it is easy to obtain the information that is necessary, and also to share the information each participant has. This approach is close to the management system enterprise architecture that looks at four things simultaneously: information, information practices, information systems, and ICT. Enterprise architecture is becoming mainstream in Finnish administration, and therefore there is a clear need for compatible practices that can be extended to new areas such as municipality decision making. Another idea is to collaborate with others sharing similar objectives. Indeed, there are several grassroot activities in Finland about promoting the use of information in decision making (see WP3, WP4).

Open decision making practice is a synthesis of large body of research on different areas, from where we in THL have screened, hand-picked and adjusted excellent ideas into a coherent system of practices and tools. There is no room to describe the underlying knowledge in detail, but the most important are shown on Table 1. To our knowledge, this is the first time when these methods have been combined into a coherent whole and will be implemented in decision support in large, real-life decision situations.

Table 1. Properties needed in open decision making practice and rules or methods applied to achieve the properties.

Property strived for	Method to be used	Description and reference
Participation and contributions
Anyone can participate in decision support.	An open wiki web-workspace: Opasnet.	Interface similar to Wikipedia. Online editing of shared information objects.
Discussions converge to a resolution.	Pragma-dialectic argumentation rules.	Rules define how a statement is accepted or rejected based on defends or attacks by arguments.
Value judgements are expressed and critically evaluated.	Quasi-realistic moral philosophy	Moral statements are expressions of individuals. They can be evaluated with the same tools as factual propositions.
Preferences of several stakeholder groups can to be assessed.	Stakeholder preference elicitation	Stakeholders rank different outcomes. Probability distributions describe the results.
Citizen feedback can be given as maps.	Mapita and other map interfaces	Web tools collect and show data simply by clicking maps.
Criticism and uncertainties
Scientific reasoning is used (not loudest voice winning).	The scientific method of criticism	Falsification of hypotheses, which are treated as possibly plausible a priori.
Descriptions reflect uncertainties in a quantitative manner.	Systematic use of probabilities	Subjective (Bayesian) probabilities and approaches are applied.
Estimates are used systematically even if there are no measurements.	Elicitation of expert judgement	Experts produce probability distributions that can be weighted against experts' performance.
Modelling
Decision descriptions give justifiable guidance.	Decision theory and decision analysis	Probabilities and utilities express decision options, impacts, and valuations.
Discussions, written descriptions, and quantitative modelling synthesised seamlessly.	Structured discussions, ovariables, and OpasnetUtils	A systematic information structure with standardised information objects is used. Further work in this part in WP1., Rintala et al 2013
Evaluation and management of work
The contributions of self-organised stakeholders are managed and synthesised.	Wisdom of crowds and mass collaboration	The work is chopped into small independent pieces in a decentralised way and then synthesised.
The work process must be evaluated and managed.	Properties of good assessment	The guidance consists of evaluation criteria for the current and foreseeable progress, according to the objectives.
Open participation with high scientific quality.	Interactional expertise	Moderators follow and manage contributions using management skills and rules.
Work process management follows national guidelines.	Enterprise architecture	The work processes are looked at from four perspectives: knowledge, practices, information systems, and information technology.
Practice development according to the social and health sector.	Innovillage	Innovillage has guidance about how to develop, implement, and evaluate innovations and useful practices.

Links to other research by the team

The project implements methods that have been developed by the research team in previous research projects about decision analysis, impact assessment, and decision support. Such projects include EU projects Beneris, Intarese, Heimtsa, and Hiwate (integrated environmental health assessment, 2005-2011), Tekes project Minera (environmental and health risks of mining, 2010-2013), and ministry-funded Tekaisu (environmental health assessments in municipalities, 2012-2014). The projects have developed a) methods, models and web tools for impact assessment and b) practices that support integration and use of scientific knowledge and value judgements.

These methods and practices have subsequently been used in many projects. E.g. EU-funded Urgenche (2011-2014) looks at health impacts of climate policies at municipality level and uses Opasnet web-workspace for modelling and project management. Academy-funded CONPAT looks at the sources, behavior and fate of microbial and chemical contaminants and their health and economical impacts. TEKES-funded POLARIS (2009-2012) looked at sustainable water quality management in artificial groundwater production. For previous work about Innovillage or open democracy, see WP3 and WP4.

For illustration, we describe one model developed in previous projects, namely the Wated Guide model, and its use in decision support. The quality and health impacts of drinking water is the responsibility of municipality authorities. The quality control nowadays focusses on the end product, which is always too late to prevent microbial outbrakes. There is a clear need for a tool that enables prediction of impacts and their prevention in different special situations and future investment scenarios. WHO has launched a procedure Water Safety Plan to promote such work, and some countries such as the Netherlands have implemented quantitative microbial risk assesssment in this planning.

Water Guide model (http://fi.opasnet.org/fi/Vesiopas) is a web-based tool for quantitative microbial risk assessment on waterworks level and can be used quickly with little training by the professionals in the municipalities and waterworks. Water Guide has been successfully used in research projects, but it has not made a breakthrough as practical tool for professionals. This project will evaluate the hindances (especially WP2) and solve them (WP1).

Objectives

Research objectives

The ultimate objective is to improve the outcomes of societal decisions involving environmental and health issues by developing and promoting the open decision making practice, which consists of a) practical high-quality environmental information, b) systematic work practices, c) advanced impact assessment methods, and d) modern ICT technology to support societal decision making in general and in Finnish municipalities in particular.

There are some critical research topics that need to be tackled before it is possible to implement the practices described above in a large scale. These objectives can be classified into four groups and presented as research questions.

• Methodological:
  • How can aspects from structured discussions be included in continuously updated assessment models during a policy process without breaking the structure and functionality of the model?
  • How can elicitations of stakeholder valuations be used in a coherent way with multiple stakeholder groups? How can the results be used systematically within probabilistic assessment models?
• Practice-oriented:
  • Which of the current societal decision-making practices are in conflict with the open decision making practice? How can these conflicts be resolved?
  • How to apply the methods of Innovillage within environmental health assessments?
• Communications-oriented:
  • What are the major information or resource deficiencies that prevent the use of open decision making practice in municipalities? How can they be overcome?
  • What are the practical needs of Finnish municipalities or national authorities related to environmental health assessments?
• Work-ecosystem-oriented:
  • How can an ecosystem be developed for societal decision support in such a way that everyone (decision makers, experts, stakeholders, and developers) can effectively participate and want to do it?
  • How should interactional expertise be applied in order to moderate the contributions and work within such an ecosystem?

Hypotheses

The hypotheses of the work are the following:

• The open decision making practice can help participants to focus on the decision options, possible outcomes, and value judgements of outcomes, i.e. the information needs of a particular decision. This will reduce situations where the focus of work is determined based on the availability of data rather than actual policy need and situations where the focus is on authority, power, procedures, responsibilities, or negotiation tactics.
• Practical guidance for including structured discussions by participants in the continuously updating models - based on Bayesian or other online-learning techniques - can be developed. This reduces the need for experts as mediators and thus improves the manageability of assessment processes. The inclusion of discussions can be done without causing large modelling risks such as crashes of the model, memory overflows or non-convergence of estimates.
• Uncertainties about facts can be systematically described based on probabilistic approaches and the contributions from experts and other participants and with the help of moderators. This will help integration of uncertain information from various sources with appropriate weights, and increase the acceptability of the assessment outcomes.
• The quality of decision support can be measured by constantly evaluating the content, applicability, and efficiency of the information production. This evaluation will improve - in a measurable way - the execution of work processes.
• Organisations participating in the case studies of the project will generally find the practices and models as an effective and feasible way to support evidence-based decision making. We also expect that the criticism presented does ruin the foundations of the open decision making practice but rather can be effectively used to improve it.
• Identification of critical communication needs and problems will improve the project communication and facilitate the recruitment of participants to case studies.
• The open decision making practice enables self-organised decision support processes that are independent of this research project. Ecosystems will emerge where municipality decision makers, experts, and citizens launch self-organised activities to support particular decisions of their own interest. This is an ultimate test for the applicability of the method.

Materials and methods

Research methods

Error creating thumbnail: Unable to save thumbnail to destination

Figure 1. An example of a decision diagram, which represents a complex multi-decision, multi-stakeholder decision situation. The main parts and their causal relations are shown according to the open assessment method. D = decisions, C = causal nodes, O = outcomes of interest, S = stakeholders, V = valuations, i.e. preferences by the stakeholders about the outcomes.

The overall method to describe a complex decision situation is impact assessment and typically decision analysis (Raiffa 1997). It consists of a description of the decision (D) and its options considered, a causal network (C) to outcomes of interest (O), and value judgements (V) of the outcomes by the decision maker. However, typically decision analysis looks at a single decision by a single decision maker at a time and involving actual decision makers in the analysis. The approach is extended in this project to cover multiple decisions and decision makers. In addition, value judgements can be expressed by any stakeholder group (S_i) even if they are not in the position to decide. Thus, participants can also learn what other options would be chosen based on other valuations present in the society. A key feature of the method is that all participants share the same causal network, i.e. the description on how things are and how things affect each other. Valuations are expressed as ranks of preference and operated using probabilities (Cooke, 2007)

The causal network is described as a quantitative model. Different methods and model types (such as deterministic or statistical models) can be used in applicable situations, but the paradigm is based on the idea of a Bayesian network, where the issues are described using subjective probabilities, and the relations are described as conditional probabilities. Deterministic and other models can be embedded within the paradigm as necessary. The work starts from a coarse description of all possible options and outcomes, and implausible combinations are rejected as the understanding of the causal chain increases. We have recently developed an approach that enables model descriptions with very coarse and very sophisticated manner (Rintala et al 2013). Thus, an assessment can use the same modelling approach irrespective of the complexity.

Error creating thumbnail: Unable to save thumbnail to destination

Figure 2. Framework for open decision making practice showing the relevant parts of decision making. The practice looks at and manages the whole chain of decision making from support to outcomes. However, the focus of new methods developed and tested in this project is mostly on the decision support.

Because the modelling system conforms to internal consistence required by the probability theory, the apparatus can be used to identify inconsistencies in the inputs: if a stakeholder supports a decision option that will lead to outcomes disliked by her, there is a reason to start looking for explanations for the behaviour: there may be an error or omission in the model, the actor may be unaware of what is known about the case, or she may have a hidden agenda. In any case, there is an alert that directs actions towards increased shared understanding. ⇤# : Tämä kappale pois? --Jouni 00:38, 25 September 2013 (EEST)

The project uses web-workspace Opasnet (http://en.opasnet.org). It is open to be read and used, and it contains implementations of all decision support methods described in Table 1. It also contains several large environmental assessments. Opasnet offers strong support for data management, modelling, and even original research. It consists of a wiki, a modelling software R, a database for small and large data sets, and a web tool developer. Version control and archiving functionality is seamlessly embedded and automatic. Sharing and borrowing assessments, data, and models is made easy.

All of the methods mentioned have been tested and implemented, but this project offers such a unique, coherent combination that has not been implemented elsewhere. However, there are also challenges. Only a small fraction of decision-making problems have been quantified and assessed since (i) uncertainty is often huge and challenging to quantify; (ii) sufficiently accurate and unbiased computational models may not be available for 'objective' evaluation; (iii) values play an important role and any quantification scheme for them will be biased. This project will reduce all these problems by using subjective probabilities, expert elicitation, coarse models for documentation even when quantification fails, and explicit inclusion of value judgements within the models (see Table 1 for more details). Also, the open decision making practice will make all this transparent and subject to criticism.

There are several incentives for decision-makers to use the open dynamic decision support (ODDS) system. Transparency in general is found important in Finland, and collecting feedback from larger stakeholder group gives an opportunity to anticipate public reactions before decisions are made. Also, there is an active international movement of open data and open democracy, so we anticipate new practical tools and methods to become applicable during the project. A main bottleneck is to gather critical masses to participate in the Finnish scale, but looking at many similar decisions in several municipalities at the same time decreases this problem. This will also increase the efficiency: laborious tasks are immediately used by larger groups, thus making it more motivating to accomplish them.

Research material. Research material is obtained from the case studies from the participants (including the project researchers). It will be e.g. scientific literature, descriptions of objectives, minutes from public hearings, web discussions, or new models. Any information in basically any format is uploaded, linked or in other way made available to people reading a page about a case study. All relevant information will be synthesised into a quantitative description (typically a probabilistic model). The material will be used to improve the decision support including quantitative models of that case. Subsequently, the material will be used to improve other similar decision processes and decision practices in general.

Materials management plan. The data will be stored publicly in Opasnet. In cases where the data cannot be published, it is stored either in an encrypted format or in a password-protected area. In rare cases if sensitive personal or other data is used, the rules of THL are obeyed in handling and storage. Daily backup copies are taken from the website. Practices developed will also be stored and published in Innovillage. Opasnet has a built-in version control and archiving functionality.

Because all material is available as open data, most of the typical hindrances of data use are solved by default. In addition, WP2 aims to spread the word about existing data to promote further use. The material is published using CC-BY-SA license, which is in practice does not limit use. However, the merit and ownership of the material stays with the contibutor. All articles, if possible, will be published in open access journals. In any case, the final drafts will be published in Opasnet.

Ethical issues. The project does not involve research on patients, and handling of sensitive data requiring ethical permissions is not anticipated. In any case, the ethical rules of THL will be obeyed.

Risk management. There is a common fear that open decision processes like this project will lead to a huge unmanageable flow of low-quality contributions, and the process will fail in chaos. Our experience in practice has been the opposite: the most difficult part of such a process is to get stakeholders (including researchers and decision makers) interested and involved. In this project, this risk is minimised by (i) applying ready-made models that can produce useful results quickly, (ii) by further developing these models more user-friendly based on feedback, and (iii) using the skills in WP2 to identify and solve critical hindrances of participation.

Another major risk is that the level and complexity of decision support aimed at is, after all, too high. However, this project carefully builds on a foundation with tested and validated methods that have been used in a large scale elsewhere. We have also paid a lot of attention to make sure that the methods used are not contradictory with but rather supporting each other. The risk is actually whether such decision support system is effective enough to be usable with current resources.

A third risk is the amount of interactional expertise needed to facilitate the work. We expect to learn a lot about training needs in this project, but we also anticipate that being an interactional expert is a demanding task that requires a lot of training. Therefore, in a typical decision process e.g. in a municipality, this expertise must come from outside. Thus, there is a need for teaching interactional expertise to a larger group who could be recruited to a decision processes. However, this larger training is out of scope of this project, and other funding will be applied for it from elsewhere.

Implementation and budget

Workpackage 1: Methodology development

Leader: Jouni Tuomisto (adjunct professor). Personnel: Päivi Meriläinen (PhD): quantitative impact modelling, valuations

WP1 work is based on an existing system that can support most parts of decision support work, including modelling. However, there are three main tasks of further research. First, it develops practices to include structured stakeholder contributions in quantitative probabilistic models in a systematic way. This work is based on previous work on pragma-dialectics on discussion side and OpasnetUtils on modelling side (see Table 1.) We need to solve theoretical and technical questions about e.g. inclusion of novel Bayesian techniques. We also need to develop practices and guidance for the users to implement it. Modelling experts in WP4 will participate in this work.

Second, we need to implement methods to include valuations in impact models. This research is based on Cooke 2007, but new research is needed for applying the method to multi-stakeholder situations. Also, tools and guidance are needed to use the improved method in the web-workspace. Third, there is a need to update and build new modules for impact assessment. For example, mining exposure models, life-table models, and energy balance models have been developed in three different projects. These will be updated in such a way that mining model feeds to both others and and energy balance feeds to life table model using standardised structures and interfaces. Similar standardisation is also done to the Water Guide model.

Workpackage 2: Communications and influencing

Leader: Mervi Pitkänen. Personnel: Kaarina Wilskman

WP2 will communicate about the project to the target group of municipality and regional decision makers. The aim is to increase awareness and interest among environmental health authorities but particularly find and recruit participants for small and large case studies (see WP5). This is a challenging task, because especially the large case studies require participants interested in a specific topic from among decision makers, experts, and stakeholder groups. In small case studies with tool testing, the recruitment is targeted to identified groups that would benefit most from using such a tool. The skills of the THL Department of Communications and Influencing are therefore needed and used.

Feedback about the practices and tools will be systematically collected in the case studies, and this information will be used to guide further development in WP1. We will study user experience, usability, and need for the user in all development of tools and practices. The communications work also promotes the development of a stable community of people interested in developing decision support in Finland (see WP3, WP4).

Workpackage 3: Integration of existing practices

Leader: Pasi Pohjola (PhD; Innovillage). Personnel: Tapani Kauppinen (PhD; health and social impact assessment)

WP3 utilizes the existing Innovillage environment for developing the decision making practices and local solutions developed specifically in the selected case studies. Innovillage is a national web-based collaborative development environment for developing, implementing and evaluating methods in social care and health services in Finland. Currently Innovillage contains about 650 models and their local implementations of practices from various areas of social care and health care. The environment is used in national development programs, such as the National Development Programme for Social Welfare and Health Care, run by the Ministry of Social Affairs and Health (http://www.stm.fi/en/strategies_and_programmes/kaste). In WP3 Innovillage works as the environment where the decision-making practices of the case studies are developed and evaluated.

One key area of work is to develop the existing administrative impact procedures (e.g. IVA, SOVA for human and social impact assessments) as an integral part of open decision making practice. Through the use of Innovillage, the outcomes of the research project are disseminated and spread for wider audience. As an open innovation environment, it enables other municipalities and decision makers to utilize the model developed in the project case studies. In this way, Innovillage is developed into a seamless part of open decision making practice.

Workpackage 4: Work ecosystems

Leader: Sami Majaniemi (PhD). Personnel: Leo Lahti (PhD), Mikko Pohjola (PhD)

WP4 work aims to develop a work ecosystem for societal decision support, particularly with regard to decision making with ecological and health significance. The ecosystem is based on existing open-society activities such as Open Knowledge Finland, the Finnish Association for Online Democracy, Sorvi, Avoinministeriö, Kansan muisti and Deliberatiivisen demokratian instituutti. In addition to setting up and organizing a network of actors with interest in participatory decision support, WP4 will study the specific requirements for interactional expertise as well as develop and implement corresponding practices for supporting broadly collaborative decision support within the ecosystem.

WP4 can thus be considered as having practical dimension and a theoretical dimension. The practical dimension focuses on linking the possibilities provided by existing open-society activities with the case studies of WP5 with the purpose of enabling broad collaboration in model-based assessments for decision support. This includes both the arrangement of work by different organizations and individuals around specific assessment/decision cases and solving the technical challenges in fitting different tools and platforms applied by different embers of the ecosystem together. The theoretical side then scrutinizes the needs for interactional expertise arising in the collaborations in the WP5 case studies. It thereby attempts to identify and characterize the most important and crucial aspects of interactional expertise required in collaborative decision support. The organization of collaboration is developed as understanding regarding requirements for interactional expertise increases during the project. The scrutiny of interactional expertise builds e.g. on the periodic table of expertise by Collins and Evans (2007) and the success of collaborative decision support cases is evaluated based on the methods for follow-up and management in the open decision making practice.

Workpackage 5: Management of case studies

Leader: Jouni Tuomisto (adjunct professor). Personnel: Päivi Meriläinen (PhD; management of case studies), Hannu Komulainen (research professor); risks of mining and metals), Ilkka Miettinen (adjunct professor; risks and safety of drinking water)

WP5 manages the case studies and takes care of communication between all WPs, municipalities, other stakeholders, and the open democracy ecosystem (see WP4). This includes regular online meetings and an open project website about upcoming tasks and progress of work.

WP5 also builds and executes impact assessments for case studies. Many of these are existing models (such as the Water Guide) that are, however, originally designed for a narrower use and require development into a more generic and thus more usable form. Also new models are developed for selected priority cases. In addition, research on user experience is performed to guide development. There is also a need for training and support for decision makers and stakeholders about the new tools, and this will be organised by WP5. Another training activity is about assessment methods and interactional expertise within the project assessors; however, larger training for outside need is not in the scope of this project.

Timetable

Table 2. Timeline of the project and tasks.

WP, task	Year 1	Year 2	Year 3	Year 4
WP1
Task 1: Develop practice for including discussions in models	XXXXX
Task 2: Develop practice for eliciting stakeholder values	XXXXX	XXXXX
Task 3: Develop generic impact assessment models	XXXXX	XXXXX	XXXXX
WP2
Task 1: Communicate the practices of the project	XXXXX	x x x	x x x	XXXXX
Task 2: Recuite participants to case studies	XXXXX	x x x
WP3
Task 1: Implement project in Innovillage	XXXXX	x x x	x x x	x x x
Task 2: Compare and merge methods with administrative impact assessments		XXXXX
WP4
Task 1: Create work ecosystem for open decision making	XXXXX	XXXXX
Task 2: Study recuirements of interactional expertise		XXXXX	XXXXX
WP5
Task 1: Develop large case studies (mining)	XXXXX
Task 2: Execute large case studies		x x x	XXXXX	XXXXX
Task 3: Develop and maintain small case studies	XXXXX	XXXXX	x x x	x x x
Task 4: Offer training and support for decision makers, assessors, and other stakeholders	XXXXX	x x x	x x x	x x x

--# : Budjetti tulee osaksi hakemusta, joten tämä on tässä tiedoksi mutta taulukko poistetaan lopullisesta tutkimussuunnitelmasta tilaa viemästä. --Jouni 17:20, 22 September 2013 (EEST)

Budget

• Duration: four years 1.9.2014 - 31.8.2018
• Indirect personnel cost: 55 %
• Overhead: 61 %
• Effective working time: 100 %
• VAT included: No

Table 3. Project budget

	2014	2015	2016	2017	2018	Total
Salaries
Postdoctoral researcher 3200 e/pmo (WP1+WP5)	4	10.5	10.5	10.5	7	13600
Assisting personnel 2700 e/pmo (WP2)	4	5.5	5.5	5.5	5.5	70200
Postdoctoral researcher 3200 e/pmo (WP3)	2	5.5	5.5	5.5	4	72000
Salaries, total	30000	66050	66050	66050	50050	278200
Indirect employee costs, total	16500	36328	36328	36328	27528	153010
Total overheads share	28365	62450	62450	62450	47322	263038
Other costs
Services	6000	15000	15000	15000	9000	60000
Travel expences	2000	2000	2000	2000	2000	10000
Other costs, total	8000	17000	17000	17000	11000	70000
Total costs	82865	181828	181828	181828	135900	764248
Funding plan
Own organisation	24860	54550	54550	54550	40770	229280
Funding contribution from other sources %	30.00	30.00	30.00	30.00	30.00	30.00
Academy funding contribution €	58005	127278	127278	127278	95130	534968
Academy funding contribution %	70.00	70.00	70.00	70.00	70.00	70.00

--# : ** justifications for the total cost estimate specified on the application, by type of expenditure (budget table with justifications). Costs that do not pass through the books of the site of the research shall not be included in total project costs. --Jouni 16:42, 24 September 2013 (EEST)

Research environment

Merits of research team members

Jouni Tuomisto 20 % of working time spent on this project (from the THL budget)

Researcher Päivi Meriläinen, PhD, from the National Institute for Health and Welfare (THL) has a key role in quantitative microbial risk assessment (QMRA) development at THL. Päivi Meriläinen has 10-year experience on risk assessment and has been involved with several EU-funded projects on environmental health research (INTARESE, HiWATE, SecurEau) with special focus on drinking water risk assessment.

Mervi Pitkänen

Hannu Komulainen

Pasi Pohjola (PhD, Social Sciences)is a Development Manager in National Institute for Health and Welfare (THL), coordinating the implementation of the National Development Programme for Social Welfare and Health Care. Previously he has been responsible for developing Innovillage, national open innovation environment for social care and health services. Prior to his career in THL, he conducted post doctoral research in the University of Helsinki on knowledge building and collaborative creativity. His current research interests relate to Science and technology studies, especially on collaborative innovations, the role of expertise in policy and participation, and study of social practices in science and technology.

Leo Lahti (D.Sc. (Tech.); B.Sc. (Pol. Sci.)) is a Academy of Finland postdoctoral research fellow affilited with University of Helsinki, Finland and Wageningen University, Netherlands. He is specialized in machine learning and data analysis, with applications in computational biology and open government data. The development of open source algorithmic tools for these topics has formed an integral part of his research. Lahti is the coordinator of the Finnish Open Science work group under the Open Knowledge Foundation Finland, and a main developer of the sorvi toolkit for Finnish open government data that was awarded the first prize in Apps4Finland competition 2011. His expertise in applied probabilistic analysis and data integration, and his active involvement in the domestic and international open government data community will be a valuable assett for the project.

Sami Majaniemi (PhD, MSc. (Tech)) is a visiting researcher at the National Institute for Health and Welfare (THL) and project manager at Forum Virium Helsinki with 20 years of experience in international research collaboration in the fields of theoretical physics and materials science. More recently, his work has focused on the development of tools and practices of collaborative decision-making and policy analysis through such programs as Action Programme on eServices and eDemocracy and Open Government Partnership Initiative coordinated by the Ministry of Finance.

• • names, tasks and salary costs (with justifications) of persons working within the project. If the names are not known, enter N.N. Also include an estimate of the PI’s working hours on the project.

Mikko Pohjola (PhD, MSc. (Tech)) is a visiting researcher at the National Institute for Health and Welfare (THL) and a research consultant in Nordem Ltd. Mikko Pohjola has made his doctoral thesis on effective decision support by environmental health assessment and is one of the main developers of open assessment method, Opasnet web-workspace as well as open decision making practice. Mikko Pohjola has also been involved in several research projects both internationally (e.g. INTARESE, HEIMTSA, BENERIS, BEPRARIBEAN) and nationally (e.g Finmerac, Tekaisu) with particular emphasis on effective application of knowledge to advance health and wellbeing in societies. --# : jos tällainen teksti minusta tarvitaan/halutaan. --Mikko Pohjola 10:39, 24 September 2013 (EEST)

Site of research

and any tangible support it offers the project, including available equipment

Key national and international collaboration

and distribution of work (“Partners” on the online application)

Other partners

(e.g. Strategic Centres for Science, Technology and Innovation), form of cooperation, description of how the project will benefit from the cooperation (“Partners” on the online application)

Use of international and national research infrastructure

description of how the project will benefit from it (“Infrastructures” on the online application)

Mobility

how the visits or work periods elsewhere contribute to research plan implementation.

Under “Mobility” on the online application, give a detailed description of possible mobility within the project: to and from Finland or between organisations in Finland. The description shall include information on the objectives and duration of the visits and on whether the visits have been agreed.

Training and careers

All researchers in the project are at least postdoctoral researcher, so doctoral training is not anticipated. However, there is a clear training need for all researchers, because they will apply assessment methods in their own respective areas but few of them are actually trained assessors. The same applies to interactional expertise, and facilitation skills must and will be trained within the project (see WP5).

Gender equality is an important thing and a real challenge in this project. This is because almost all of the activists in different self-organised open democracy organisations are male in Finland. Therefore, special attention will be paid to make sure that enough female participants are found to case studies from the municipalities.

Expected results

--# : Tähän osioon uskottavan ja houkuttelevan näköinen listaus tuloksista siten, että se vastaa aiempaan päämäärälistaukseen konkretisoi ilmoitetut päämäärät (ainakin jossain määrin) käsinkosketeltaviksi tuotoksiksi tai muutoksiksi. --Mikko Pohjola 10:26, 24 September 2013 (EEST)

The near-term benefits will occur in local level. Therefore strong emphasis is given in active communication with the real decision makers and users in municipalities, as described in WP2. As a result, municipalities will be able to make more evidence-based decisions about e.g. urban planning or environmental permits.

The work will be done in Opasnet, so detailed information is being published as the work progresses. In addition, case study reports will be published, and several scientific articles are anticipated at least about the open decision making practice, the merging of discussions into models, applicability of the methods, and user reactions about the practices. Active communication with activist groups of open democracy, national organisations (such as Sitra and the Government Policy Analysis Unit) and general public is essential and will be given special attention.

The project develops knowledge practices that are applicable in both research and decision making and enable immediate and effective sharing and learning. It also enables modular development of larger and larger assessments that are internally coherent. This has three major benefits. First, scientific knowledge gets better used in the society. Second, immediate societal information needs are better transformed to research. Third, the use of large assessments hopefully change the way we think about research and practice. The related risks are described in the end of Materials and methods section.

A real scientific breakthrough would be a global model that is used as our target of work rather than the hundreds of thousands of scientific articles in thousands of journals. Although the internet has revolutionised the access to these separate pieces of data, they are still separate pieces. Global models would use a better information structure, namely something that links information objects in a way that mimics how the actual issues are linked in reality. This projects takes a few small steps towards such breakthrough by solving some structural, modelling, and practical issues.

Bibliography

Links to Opasnet tools and models mentioned in the application

• http://fi.opasnet.org/fi/Kohdekohtaisen_Minera-arvioinnin_esimerkkisivu
• http://fi.opasnet.org/fi/Pienhiukkasvaikutukset
• http://fi.opasnet.org/fi/Vesiopas
• http://fi.opasnet.org/fi/Terveysriskin_kuvaus
• http://en.opasnet.org/w/Energy_balance_in_Kuopio

See also

• Quasi-realism
• The detailed example of multidimensional decision space (sudoku) was radically shortened. The original story is archived.