Assessments - science-based decision support

This page is a lecture. The page identifier is Op_en2816
Moderator:Nobody (see all) Click here to sign up.
Give your opinion to the peer rating of the content of this page. {{ #opasnet_rater: }}
Upload data

Assessments - science-based decision support is a lecture about assessments as processes and their products that on the one hand are determined by the practical needs of societal decision making, and on the other hand are bounded by the scientific quest for truth. The lecture discusses how these sometimes apparently conflicting goals influence the making and outcomes of assessments.

Scope

Purpose: To describe how the practical needs of societal decision making should be taken account of in assessment and how these needs can be addressed without compromising the scientific quality of the process and its outcomes.

Intended audience: Researchers (especially at doctoral student level) in any field of science (mainly natural, not social scientists).

Duration: 1.5 h + 1.5 h

Definition

• Assessment is a business of creating understanding about reality
  • collection, synthesis, interpretation, organization, communication, use of information
  • driven by practical needs, constrained by the scientific quest for truth

Objectives I:

• learn the basics of an assessment: question/info need, and the three-part structure: decision - factors - outcome.
• learn how an information need is transferred into a research question.
• learn the importance and implications of boundaries in defining an object (assessment, variable)
• learn the roles of participants and intended users.
• learn to participate in discussions about a new assessment (Tapas) and its framing.

Objectives II:

• become familiar with the information structure of an assessment object
• learn how solutions to assessment problems are found (sought for), the parts of assessment
• learn how the assessment solution addresses the need and how need influences the solution
• learn how an assessment consists of a network of individual (independent given causal relations) variables
• learn how the information content of an assessment can be analyzed, organized and interpreted for producing the results

In order to fully understand this lecture it is recommended to acquaint oneself also with the following lectures:

• Open assessment in research
• Variables - interpretations of real-world phenomena
• Science necessitates collaboration
• Evaluation of assessment performance

Result

Part I:

Introduction to concepts:

• Societal needs
• Societal decision makers
• Societal actions
• Indicators: issues of interest
• What is required in order to know what should be done in order to improve the state of issue of interest?
  • Decision → factor → outcome
• Transforming need into an assessment problem → assessment scope - what is needed to know, why, who will use the knowledge (why will they acquire the knowledge)?
  • purpose, intended users,
• Assessment is about finding solutions to the assessment problem
  • probably also as much finding out what is not known in order to solve the problem (consider knowledge management vs. ignorance management)

Example-based discussion:

• what need is addressed, whose need is it?
• who is interested, who is responsible, who is in position to do something about it, who is affected either by the state of affairs or possible actions about them?
• what could be done about it, do we know what should be done about it?
• what is known or not known?
• are there conflicts of interest or differences of perception regarding the needs or possible actions about them?
• if increased understanding about the issue is obtainable, whose level of understanding, and about which aspects of the issue, should be primarily targeted (e.g. scientists, policy-makers, some specific stakeholder groups, common citizens)?
• what are actual specific issues of interest to look at?
• Try to formulate a description of the assessment purpose, boundaries and intended users and sum the assessment problem into a question or a set of questions

Part II:

Introduction to concepts:

• Assessments as collections of variables (scope determined by practical need, definition/result determined by quest for truth given scope)
• Assessments as answers to practical questions
• Assessments as causal networks
• Information structure of assessment objects - scope, definition, result
  • Definition describes the way the assessment problem is attempted to solve
    • decision variables, indicators, value variables, other variables → causal network
    • analyses: e.g. (decision) optimization, conditioning, importance and value of information analysis
• Bayes belief nets - optimization
• Value of information - need for and targeting of extra work?
• Scenarios - conditioning the causal network
• making conclusions and conveying/converging them to use
• appraisal
• indices

Example-based discussion: