The methods and tools for evidence-based decision making

This page is a variable. The page identifier is Op_en7438
Moderator:Nobody (see all) Click here to sign up.
Give your opinion to the peer rating of the content of this page. {{ #opasnet_rater: }}
Upload data

Question

What kind of operating models, methods and tools exist to improve evidence-based policy in societal decicion-making? Research knowledge in the Finnish Government's decision-making is especially examined. Important are things that help the Implementation plan of evidence-based policy of the Yhtakoytta-project.

Answer

Current practices and improvement needs

The purpose of this overview is to find out what would be good and consistent methods and tools to help improve the processing use of scientific knowledge in the Government's preparation work, decision making and strategic policy. It is also important to understand those principles that are supported by these methods and tools. The recommendation aims for the kind of universality that the methods are good in societal decision-making in general and can thus be taken into use also elsewhere.

It is not examined here how and for what reasons the political decision is finally made. Of course the aim is that the preparation work helps both the decision maker to choose the solution best matching their values, and an outside observer to understand, how the final decision was reached.

The goal is not to give a full and accurate picture of the current knowledge practices. Some kind of general notions are useful to be said in the beginning because they help understand, why and how the current practices should be improved. Some important views are found from op_fi:Tiedolla johtamisen menetelmät ja työkalut#Perustelut (in Finnish).

We especially examine methods, that could be implemented into use already in the near future. However, we do not argue, that there methods should immediately be used in all of the Government's work or that it would even be possible. Rather we offer a justified vision of what kind of methods the methods for improving the use of knowledge in decision-making are, according to research. We hope the stepping stones we suggest are so practical and feasible, that there will be projects and operations where these principles and methods identified here can be tried in practice and implemented into use.

It must be said in the very beginning that the current knowledge practises in Finland are good, and compared to many other countries excellent. When the preparation begins goals, responsibilities and schedules are set, and officials or committees who are familiar with the topic are chosen for the work. There is usually a lot of knowledge for doing to work, and it can be easily found online. The officials and experts are committed to their work and do it with the good of the society in mind.

However, there are important improvements to be made. Especially difficult are decisions about highly complicated matters that have significant long-term effects, like questions about refugees, climate treaties, energy taxation or civil salary. This is when one expert's knowledge is not enough, and even the resources of a diverse group of experts may not be enough to get a proper grip of the complicated situation.

On the whole the activity has been higher in the last years, and progress with open policy practices have been made with a good speed. Transparency and knowledge-based work have become one of the main themes in both the fields of governance and research, and a a large net of people, committees, projects, strategies and corporations has formed around the topic. Open policy practise and open access and wide use of knowledge have been taken into account in almost all strategic policies about the future in ministries, research organisations and the Government (OHRA-recommendations (in Finnish), The improving of knowledge use in political decision making - from goals to reality. (in Finnish)) This has lead to a situation, where many different people and organisations have developed guidelines, methods, tools and services for decision making based on openness and knowledge.

The main idea of this literature review if to go through fresh material especially from Finland but also elsewhere and try to gather a coherent group of the most important principles that can be supported with tools that help the practical work.

Shared understanding as the core goal of new practise

Error creating thumbnail: Unable to save thumbnail to destination

Yhtakoytta-pronciples and their significance in creating shared understanding from observed data.

From scientific and political literature discussing the use of knowledge often stands out many different goals and principles, but usually the thinking is going towards goals based on scientific understanding and involvement. From these megatrends and other goals consistently found in literature we have worked out one core goal that we believe sums everything up to a useful and concrete form:

Decision preparation work has to aim for shared understanding. Shared understanding means a situation where the views of all different involved parties is described thoroughly enough it can be said, of which relevant facts and values there is agreement, of which a disagreement and why. Shared understanding can thus be used to predict, which decision option any party would choose and why.

The idea is of curse radical applied widely, and completely opposite points of view has it's support in today's Finland (that decisions are only spoken about after they have already been made). However, our experience has shown the idea of shared understanding to be a) easy to understand and so an easy guide for thinking, b) an efficient way to find on the whole good solutions in small groups based in trust, and c) an efficient way to identify the mistakes in failed decision preparation processes. In other words if the preparation fails, shared uderstanding helps evaluate and identify those things that could have been done differently to probably avoid the problems.

The methods an tools of decision making have developed so that we can apply shared understanding widely and test if problems are avoided and if the decision making is really made better. This literature review gives tools for this kind of work.

Knowledge crystals create shared understanding

The creation of shared understanding requires many different things and action. However there are three principles above all else, because they are always present when shared understanding has been created and them missing seems to be connected to failure to create shared understanding. These are openness, shared information objects and critique.

Shared understanding

A situation where the views of all parties is described thoroughly enough it can be said, of which relevant facts and values there is agreement, of which a disagreement and why. Shared understanding can thus be used to predict, which decision option any party would choose and why.

Openness

All information is open for everyone to see at all times. Anyone can participate in the work at any point and affect the result. Only the content and quality of the participation are relevant, not the status of the participator.

Shared information objects

The product if written so that the real-time draft can always be found in the same place. Also the changes are done to the same draft so no parallel versions are created.

Critique

The product is always open for comments and critique, and valid (follows the more accurate rules) critique is taken into account in the content.

Knowledge crystal

An object of knowledge built around a specific research question that applies the principles of openness, shared information objects and critique and aims to a shared understanding about the answer by using observed data as arguments.

The wide success of openness, shared information objects and critique requires a specific kind of work among the knowledge. For example a web-workspace is needed, where all relevant information can be put for everyone to see. The workspace has to allow open participation so that the knowledge doesn't distort or break down but can be with reasonable work summarised. The summaries should consistently and inclusively take into account everything said so far. Aside from the workspace also fitting knowledge structures and practises are needed to keep the knowledge organised.

The most important way to keep knowledge organised is to identify exact questions, finding answers to which is the main task of knowledge work. The questions have to a) serve needs of decision-making and b) be research questions i.e. can be answered using scientific knowledge and the drafts of answers can be critiqued with scientific reasons. The scientific requirement also applies to value questions; in these cases the questions are phrased for example "what things does group x find most important considering topic y?" This way also values are seen as research questions and the answers aim to be consistent with observations.

Knowledge crystal is the kind of object of knowledge built around a research question that follows the principles of openness, shared information objects and critique and aims for shared understanding about the answer using observed data as reasoning. The knowledge crystal has indeed been identified as the most important tool or information object for organising knowledge that guides the process of decision preparation work.

The scientific methos produces knowledge crystals

Yhtäköyttä-project suggests a new practice the basic idea of which is quite simple. Because science has proved to be the most efficient way obtain useful information about the world, the same method should also be used in decision making. How is this done?

"Science is the belief in the ignorance of experts," said the Nobel-award winning Richard Feynman. This means that scientific information is based on observation and theories that cannot be disproven with observations despite trying. Believing in experts is not science. Social decision making can follow the same principle. Knowledge can be organised into research questions and attempts at answers that are examined critically based on observations. These collections of a question, an answer and a rationale are called knowledge crystals, if it is the target of open crowd sourcing, meaning it is possible for everyone to participate in building it.

The basic idea of science is that through rational critical thinking some attempts at an answer eventually turn out to be impossible to defend in light of observations. If there is a lack of observations there are multiple possible answers and people amongst themselves have conflicting views. According to Yhtäköyttä-principles this is when must be aimed for shared understanding, understanding the point of view of others. This is different than consensus, which tries to find one option agreeable on by all.

Around the research question is built a group of workers who gather observations, discuss and interpret them to produce an answer or answers that are consistent with all the observed data. This kind of knowledge crystal is then free for anyone to use.

The production of knowledge crystals is a crowd-sourced, communal activity. Thus to work this kind of work requires a few clear principles that help guide and encourage the work into the correct direction and also to prevent the advancement into a wrong direction. By sticking to the principle a practise of coherent way to work is created. The creation of the practise is further helped with suitable tools, so doing the right things would be easy and doing the wrong ones hard. The goal is to make developing and using knowledge crystals as easy and efficient as possible and so make the creation of shared understanding easy.

One of the basic principles of open policy practise is the inclusion (or at least enabling it) of citizens in the decision making process and transparent openness in handling things. Joukkoistaminen demokratiassa (Crowdsourcing in democracy) -book outlines elements tightly to do with crowd-sourcing that must always be present so the crowd sourcing process can work as decision support. There are 1) Defining of goal, narrowing the question and scheduling, 2) communication, 3) simple technical implementation, 4) taking care of the crowd-sourcing process, 5) time limits, 6) event, 7) analysis and process monitoring and 8) commitment to process.

Principles supporting the production of knowledge crystals

In this pondering and previous literature it's possible to find repeatedly several principles that should be shortly described to understand the recommendation. The four first ones already mentioned are aiming for shared understanding, openness, shared information objects and critique. With these are seven other principles that together support the creation of knowledge crystals and shared understanding. More accurate descriptions can be found on the pages of Yhtakoytta-project and Open policy practice. The product in this context means a decision motion, report or other concrete information object. Work means the making of this product.

Intentuonality

The decision maker publishes the effects and values aimed to with the decision.

Causality

The attenders describe the decision's effects to relevant issues as a causation-correlation chain from the decision to the desired effects.

Reuse

The attenders share the product and the information gathered for it so that they are as easy to use as possible for anyone for any purpose.

The skill of summarising

The plaintiff invites to the work enough people who have both substance know-how and the skills to organise knowledge according to these new principles of knowledge work.

Evaluation and management

The attenders describe the progress of the work continuously and clearly. The plaintiff mirrors the situation to the decision maker's goals continuously and guides the work according to that and more accurate guidelines.

Grouping

During the work the plaintiff ensures the attenders feel a part of such a group to which the attender's work is important and which work is socially important.

Respect

The plaintiff gives everyone participating in the work public respect according to the real value of their work. The importance of each task is assessed with crowd-sourcing.

In the end it is important to note, that almost all of the principles mentioned above actualise in modern open source development methods and development environments used as tools, such as GitHub or Bitbucket services. These examples are probably less compelling to people solving humane problems for a living, but they represent well the power lying in the creation of answers found with co-operation for problems quantifiable in code. General knowledge work approaches this paradigm continuously, because solving multi-sector problems requires an exact description and language defining, so the plaintiffs and experts in different fields could at all fit together their views and goals expressed in different languages.

The changes required for the principles to work

Here is a summary of those methods and tools which use should be increased or decreased from the furrent situation, so evidence-based policy could be improved. The base for the recommendation is, how well the method or tool advances the goal principles, that have been identified as important for the success of social knowledge work. Thus all recommendations are examined in relation to every principle in the table.

--# : Taulukossa pitäisi vaihtaa sarakkeiden järjestystä. Oikea järjestys on se mitä tekstissäkin on käytetty. Samalla temppelikuvan pylväät pitäisi päivittää tähän samaan järjestykseen. --Jouni Tuomisto (keskustelu) 16. joulukuuta 2015 kello 14.42 (UTC)

The recommendation for using methods and tools. The table describes the ones whose use should be changed from the current situation.
green: supports goals; red: prevents achieving goals; empty: doesn't affect achieving the goal.

Method or tool and what it produces	Use as a part of a whole	Supports the goal principle?
		Shared understanding	Openness	Shared information objects	Critique	Intentionality	Causality	Reuse	Skill of summarising	Evaluation and management	Grouping	Respect
Open policy practice: a method for evidence-based policy and impact assessment Includes Open assessment and Variable (real-time scientific answer to a relevant question).	To be taken into use in the government's work as decision preparation method and a tool for improving knowledge work.
Opasnet: a web-workspace for doing knowledge work.	Use the workspace to support open policy practice and open assessments.
Respect theory: an incentive for open knowledge work.	The Government begins to reward useful knowledge production with onors according to the respect theory. This requires a web tool for support.
Otakantaa: a web page for presenting projects and discussion.	The projects of decision preparation are described here. Scheduling, questionnaires and discussion forums are used.
Facebook: a community page for discussion about projects.	Parts of projects or questions asked are taken into conversation. Despite its problems reaches almost any target group.
Innovillage: a web-workspace for developing, describing and sharing good practises.	To be used for describing and discussing practices.
Avoin data -portaali: A portal for gathering and sharing open data.	To be used to describe open data everywhere in one place to make it easier to find.
eduuni: A common workspace with Microsoft sharepoint and wiki for cooperation. Access rights can be open or limited to a group.	In use in at least part of public government as a cooperation platform, also possibly coming to use for the Government
Hackpad (or Google Docs): a cloud service for text (or spreadsheet) editing.	Single products are drafted and written openly in for example Docs. However, larger wholes require also other functionalities.
VNK project register: gathers together all Prime Minister Office's projects.	Used to assemble work done in projects
Word and Excel: a personal program for producing text or spreadsheet.	Using separate text and spreadsheet documents is avoided. Instead everything is written in shared information objects with for example Google office programs.
Intranet-programs: Intranet solutions for different preparation work.	The same open source programs used in intranet as in internet, so information can be fluently moved and there is no challenge to learning to use the system.

See also

• The material gone through in the review is listed in an earlier version of this page. (in Finnish)

Yhtakoytta-project

Project topics	Project main page · Task 1. The methods and tools for evidence-based decision making · Task 2. Development needs for evidence-based decision preparation · Task 3. Agile experiments of evidence-based decision making · Task 4. Implementation plan of evidence-based policy · Task 5. The communication and working plan for Yhtakoytta-project · Task 6. The final report of Yhtakoytta-project
Methods and tools	Open policy practice · Open assessment · Open policy practice municipal pilot (in Finnish) · Assessent of environment-caused health problems (in Finnish) · Tekaisu-project · Category:Tekaisu
Publications and other things useful	Yhtäköyttä (Project introduction in Finnish) · op_fi:Avoin päätöksentekokäytäntö voisi parantaa tiedon hyödyntämistä (in Finnish) · op_fi:Ympäristö- ja terveysvaikutuksia koskeva tieto kunnallisessa päätöksenteossa (in Finnish) · All pages in category Yhtakoytta
This page is produced as a part of the Finnish Government's research plan assignment in 2015 (www.vn.fi/teas). The producers of the information are responsible for the contents. The contents do not represent the views of the Government.