Difference between revisions of "Shared information objects in policy support"

Revision as of 09:02, 5 December 2014

This page is a manuscript to be submitted to F1000 Research as a software tool article [1]. The main focus is on variable and ovariable structure and use, but their use within OpasnetUtils and implementation in Urgenche building model are described as well, thus making this valid as software tool article.

We welcome the description of new software tools. A Software Tool Article should include the rationale for the development of the tool and details of the code used for its construction. The article should provide examples of suitable input data sets and include an example of the output that can be expected from the tool and how this output should be interpreted.

Authors

Jouni T. Tuomisto^1*, Mikko Pohjola¹, Teemu Rintala¹, Marjo Niittynen¹, Einari Happonen¹, Clive Sabel²

1 National Institute for Health and Welfare, Department of Environmental Health. P.O.Box 95, FI-70701 Kuopio, Finland.

2 University of Bristol, Bristol, UK.

3 Other?

^*corresponding author

Jouni T. Tuomisto: jouni.tuomisto [] thl.fi

Mikko Pohjola: mikko.pohjola [] thl.fi

Teemu Rintala: teemu.rintala [] thl.fi

Marjo Niittynen: marjo.niittynen [] thl.fi

Einari Happonen: einari.happonen [] soketti.fi

Clive Sabel: c.sabel [] bristol.ac.uk

Title

Shared information objects in policy support

Abstract

Up to 300 words long

There is an increasing demand for evidence-based decision-making and for more intimate discussions between experts and policy-makers. However, one of the many problems is that the information flow from research articles to policy briefings is very fragmented. We developed shared information objects with standard structures so that they can be used for both scientific and policy analyses and that all participants can discuss, develop, and use the same objects for different purposes.

We identified a need for two distinct objects: First, assessment describes a practical decision-specific information need with decision options and preferences about their impacts. Second, variables describe scientific estimates of relevant issues such as exposures, health impacts, or costs. Variables are reusable in several assessments, and they can be continuously developed and updated based on the newest data.

We implemented assessments and variables as wikipages to enable discussions about and public development of their contents. In addition, variables were quantitatively implemented as objects in statistical software R, and assessments used them in object-oriented programming. All functions and objects are freely distributed as R package OpasnetUtils. Several important properties for decision analysis were developed with variables, e.g. flexible indexing of variable results, propagation of uncertainties and scenarios through models, and possibility to write and run models on a web-workspace simply with a web browser.

The concept was tested in several projects with different decision situations related to environmental health. We present here a city-level building model to assess health impacts of energy consumption and fine particle emissions. Technically, the data collection on the wiki pages and the model development using shared information objects succeeded as expected. The main challenge is to develop a user community that is willing to and capable of sharing information between other groups and using these tools to improve shared understanding among policy-makers, experts, and stakeholders.

Keywords

Decision making, science-policy interface, expertise, open policy practice, decision analysis, object-oriented programming, R software, shared understanding.

Main Body

Introduction

• Need for better discussion between experts and decision makers (SA program, VNK money)

Ogp evidence-based decision-making

• However, differences:
  • Slow research studies
  • Experts stay out of value questions
  • Independence
  • Quick policy cycle, reactive approach
  • Different languages, need for more time to read and think but where does the time come from?
  • there is a need for more systematic information flow that is shared by both experts and politicians.

Methods

Shared info objects. Why & requirements

• Continuous flow of information synthesis. You only need the tip of iceberg but its there if you check
• followup of the process
• moderated discussion forum

Implementation

Assessments and variables as the building blocks. Purpose of assessment Purpose of variable Difference in life span and case-specificity For assessment the practical need is important For variable the scientific defendability and resistance against open criticism. Causal connections can be described.

Variable is implemented as a web page in Opasnet wiki web-workspace. A variable page has the following structure.

In addition, it is practical to have additional subtitles on a variable page. These are not attributes, though.

• See also
• Keywords (not always used)
• References
• Related files

All assessments aim to have a common structure so as to enable effective, partially automatic tools for the analysis of information. An assessment has a set of attributes, which have their own subattributes.

Implementation in R

The ovariable is a class (S4 object) defined by OpasnetUtils in R software system. Its purpose is to contain the current best answer in a machine-readable format (including uncertainties when relevant) to the question asked by the respective variable. In addition, it contains information about how to derive the current best answer. The respective variable may have an own page in Opasnet, or it may be implicit so that it is only represented by the ovariable and descriptive comments within a code.

It is useful to clarify terms here. Answer is the overall answer to the question asked, so it is the reason for producing the Opasnet page in the first place. This is why it is typically located near the top of an Opasnet page. The answer may contain text, tables, or graphs on the web page. It typically also contains an R code with a respective ovariable, and the code produces these representations of the answer when run. (However, the ovariable is typically defined and stored under Rationale/Calculations, and the code under Answer only evaluates and plots the result.) Output is the key part (or slot) of the answer within an ovariable. All other parts of the ovariable are needed to produce the output, and the output contains what the reader wants to know about the answer. Finally, Result is the key column of the Output table (or data.frame) and contains the actual numerical values for the answer.

The ovariable has seven separate slots that can be accessed using X@slot:

@name

• Name of <self> (the ovariable object) is a requirement since R doesn't support self reference.

@output

• The current best answer to the question asked.
• A single data.frame (a 2D table type in R)
• Not defined until <self> is evaluated.
• Possible types of columns:
  • Result is the column that contains the actual values of the answer to the question of the variable. There is always a result column, but its name may vary; it is of type ovariablenameResult.
  • Indices are columns that define or restrict the Result in some way. For example, the Result can be given separately for males and females, and this is expressed by an index column Sex, which contains values Male and Female. So, the Result contains one row for males and one for females.
  • Iter is a specific kind of index. In Monte Carlo simulation, Iter is the number of the iteration.
  • Unit contains the unit of the Result. It may be the same for all rows, but it may also vary from one row to another. Unit is not an index.
  • Other, non-index columns can exist. Typically, they are information that were used for some purpose during the evolution of the ovariable, but they may be useless in the current ovariable. Due to these other columns, the output may sometimes be a very wide data.frame.

@data

• A single data.frame that defines <self> as such.
• data slot answers this question: What measurements are there to answer the question? Typically, when data is used, the result can be directly derived from the information given (with possibly some minimal manipulation such as dropping out unnecessary rows).
• May include textual regular expressions that describe probability distributions which can be interpreted by OpasnetUtils/Interpret.

@marginal

• A logical vector that indicates full marginal indices (and not parts of joint distributions, result columns or other row-specific descriptions) of @output.

@formula

• A function that defines <self>.
• Should return either a data.frame or an ovariable.
• @formula and @dependencies slots are always used together. They answer this question: How can we estimate the answer indirectly? This is the case if we have knowledge about how the result of this variable depends on the results of other variables (called parents). The @dependencies is a table of parent variables and their identifiers, and @formula is a function that takes the results of those parents, applies the defined code to them, and in this way produces the @output for this variable.

@dependencies

• A data.frame that contains names and Rtools or Opasnet tokens/identifiers of variables required for <self> evaluation (list of causal parents).
• A way of enabling references in R (for in ovariables at least) by virtue of OpasnetUtils/ComputeDependencies which creates variables in .GlobalEnv so that they are available to expressions in @formula.
• Variables are be fetched and evaluated (only once by default) upon <self> evaluation.

@ddata

• A string containing an Opasnet identifier e.g. "Op_en1000". May also contain a subset specification e.g. "Op_en1000/dataset".
• This identifier is used to download data from the Opasnet database for the @data slot (only if empty by default) upon <self> evaluation.
• By default, the data defined by @ddata is downloaded when an ovariable is created. However, it is also possible to create and save an ovariable in such a way that the data is downloaded only when the ovariable is evaluated.

Decisions and other upstream orders

The general idea of ovariables is such that they should not be modified to match a specific model but rather define the variable in question as extensively as possible under it's scope. In other words, it should answer its question in a re-usable way so that the question and answer would be useful in many different situations. (Of course, this should be kept in mind already when the question is defined.) To match the scope of specific models, ovariables can be modified by supplying orders upstream (outwards in the recursion tree). These orders are checked for upon evaluation. For example decisions in decision analysis can be supplied this way:

1. pick an endpoint
2. make decision variables for any upstream variables (this means that you create new scenarios with particular deviations from the actual or business-as-usual answer of that variable)
3. evaluate endpoint
4. optimize between options defined in decisions.

Other orders include: collapse of marginal columns by sums, means or sampling to reduce data size and passing input from model level without redefining the whole variable. It is also possible to redefine any specific variable before starting the recursive evaluation, in which case the recursion stops at the defined variable (dependencies are only fetched if they do not already exist; this is to avoid unnecessary computation).

• Compute dependencies
• Decision tables
• Data tables in wiki and database

Operation

OpasnetUtils in CRAN. Rcode in Opasnet Wiki working Opbase.data to obtain parameters Objects.store and objects.latest to save and retrieve ovariables, respectively. EvalOutput for evaluating ovariables

Use Cases

We have successfully used shared information objects in several decision support situations. In 2011, we assessed the health impacts of H1N1 influenza ("swine flu") vaccination that surprisingly turned out to cause narcolepsy in children (http://en.opasnet.org/w/Assessment_of_the_health_impacts_of_H1N1_vaccination, accessed Dec 5, 2014). More recently, we assessed health effects and cost effectiveness of pneumococcal vaccines (http://en.opasnet.org/w/Pneumococcal_vaccine, accessed Dec 5, 2014). We have assessed health benefits and risks of eating Baltic herring, a fish that contains dioxin (http://fi.opasnet.org/fi/Silakan_hy%C3%B6ty-riskiarvio, available in Finnish, accessed Dec 5, 2014).

In this article, we will present an assessment about climate and health impacts of building stock, energy consumption, and district heat production. The case study is about Kuopio, A 100000-inhabitant city in Finland.

• Urgenche building model
• HIA

Describe the models Explain the reusability and its importance

Discussion/Conclusions

Other tools Analytica Patrycja bbn Heat website: black box partly, not connected with the rest of the decision support process.

All kinds of DSS but sharing and openness is always lacking as they are designed for single-user work. In addition little support to combine policy need discussion with scientific assessment.

Critical development need to establish a community to share info and open up decision processes.

The Introduction should provide context as to why the software tool was developed and what need it addresses. It is good scholarly practice to mention previously developed tools that address similar needs, and why the current tool is needed.

The Methods should include a subsection on Implementation describing how the tool works and any relevant technical details required for implementation; and a subsection on Operation, which should include the minimal system requirements needed to run the software and an overview of the workflow.

A Results section is only required if the paper includes novel data or analyses, and should be written as a traditional results section. Please include a section on Use Cases if the paper does not include novel data or analyses. Examples of input and output files should be provided with some explanatory context. Any novel or complex variable parameters should be explained in sufficient detail to enable users to understand and use the tool's functionality.

A Discussion (e.g. if the paper includes novel data or analyses) or Conclusions should include a brief discussion of allowances made (if any) for controlling bias or unwanted sources of variability, and the limitations of any novel datasets. Reproducibility: F1000Research is committed to serving the research community by ensuring that all articles include sufficient information to allow others to reproduce the work. With this in mind, Methods sections should provide sufficient details of the materials and methods used so that the work can be repeated by others. The section should also include a brief discussion of allowances made (if any) for controlling bias or unwanted sources of variability. Any limitations of the datasets should be discussed. When antibodies are used, the species in which the antibody was raised, the manufacturing company or source laboratory, the catalogue or reference number, and whether it is a polyclonal or monoclonal antibody should be included. In addition, if the antibody has been previously validated, a reference to the validation study should be included. If the antibody has not been validated, full details of the dilution and use of the antibody should be given in the Methods section.

Data and Software Availability

All articles reporting new research findings must be accompanied by the underlying source data - see our policies for more information. Please include details of how the data were analysed to produce the various results (tables, graphs etc.) shown (i.e. what statistical tests were used). If a piece of software code was used, please provide details of how to access this code (if not proprietary). See also our Data Preparation guidelines for further guidance on data presentation and formatting.

If you have already deposited your datasets or used data that are already available online or elsewhere, please include a ‘Data Availability’ section, providing full details of how and where the data can be accessed. This should be done in the style of, for example:

Source code for new software should be made available on a Version Control System (VCS) such as GitHub, BitBucket or SourceForge, and provide details of the repository and the license under which the software can be used in the article. The F1000Research team will assist with data and/or software deposition and help generate this section, where needed.

Author Contributions

JT and MP developed the idea and information structure of assessments and variables. TR developed their implementation in R with help from JT and EH. JT, MN, TR and CS developed the city building case study and its model. All authors were involved in the revision of the draft manuscript and have agreed to the final content.

Competing Interests

No competing interests were disclosed.

Grant Information

This work was funded by EU FP6 project INTARESE Grant ### (for Mikko Pohjola), EU FP7 project URGENCHE Grant#### (for Marjo Niittynen and Einari Happonen), and National Institute for Health and Welfare.

Acknowledgments

This section should acknowledge anyone who contributed to the research or the writing of the article but who does not qualify as an author; please clearly state how they contributed. Please note that grant funding should be listed in the “Grant Information” section.

Supplementary Material

There are no figure or table limits for articles in F1000Research and most material can be included in the main article if desired. Other types of additional information, e.g. questionnaires, extra or supporting images or tables can be submitted as supplementary material. Please include a section entitled ’Supplementary Material’ at the end of the manuscript, providing a title and short description for each supplementary file. Please also include citations to the supplementary files in the text of the main body of the article. The F1000Research editorial team will liaise with authors to determine the most appropriate way to display this material.

References

• Journal abbreviations should follow the Index Medicus/MEDLINE abbreviation approach.
• Only articles, datasets and abstracts that have been published or are in press, or are available through public e-print/preprint servers/data repositories, may be cited. Unpublished abstracts, papers that have been submitted but not yet accepted, and personal communications should instead be included in the text; they should be referred to as ‘personal communications’ or ‘unpublished reports’ and the researchers involved should be named. Authors are responsible for getting permission to quote any personal communications from the cited individuals.

Figures and Tables

⇤# : Move tables and figures here before submitting. --Jouni (talk) 09:02, 5 December 2014 (UTC)

All figures and tables should be cited and discussed in the article text. Figure legends and tables should be added at the end of the manuscript. Tables should be formatted using the ‘insert table’ function in Word, or provided as an Excel file. For larger tables or spreadsheets of data, please see our Data Preparation guidelines. Files for figures are usually best uploaded as separate files through the submission system (see below for information on formats).

Figure formats: For all figures, the color mode should be RGB or grayscale. Line art: Examples of line art include graphs, diagrams, flow charts and phylogenetic trees. Please make sure that text is at least 8pt, the lines are thick enough to be clearly seen at the size the image will likely be displayed (between 75-150 mm width, which converts to one or two columns width, respectively), and that the font size and type is consistent between images. Figures should be created using a white background to ensure that they display correctly online.

If you submit a graph, please export the graph as an EPS file using the program you used to create the graph (e.g. SPSS). If this is not possible, please send us the original file in which the graph was created (e.g. if you created the graph in Excel, send us the Excel file with the embedded graph). --# : EPS printer2 in Jouni's computer prints EPS files. --Jouni (talk) 09:02, 5 December 2014 (UTC)