Difference between revisions of "Evaluating performance of environmental health assessments"
From Testiwiki
(new plan for abstract) |
m |
||
Line 3: | Line 3: | ||
[[Category:Manuscript]] | [[Category:Manuscript]] | ||
− | This is a manuscript about evaluating performance of environmental health assessments. It discusses different perspectives to evaluating the goodness of assessments and proposes a new more comprehensive approach to considering | + | This is a manuscript about evaluating performance of environmental health assessments. It discusses different perspectives to evaluating the goodness of assessments and proposes a new more comprehensive approach to considering performance of environmental health assessments. |
== Abstract == | == Abstract == | ||
− | + | '''Situation | |
*environmental health assessment is intentional societal activity involving plural actors with diverse perspectives and needs | *environmental health assessment is intentional societal activity involving plural actors with diverse perspectives and needs | ||
Line 16: | Line 16: | ||
*various overlapping perspectives to the issue: uncertainty assessment, quality assurance, model development guidelines, ... | *various overlapping perspectives to the issue: uncertainty assessment, quality assurance, model development guidelines, ... | ||
− | + | '''Problem | |
*all perspectives provide only limited or narrow views to performance | *all perspectives provide only limited or narrow views to performance | ||
Line 25: | Line 25: | ||
**focus either on giving assessment procedure guidelines or considering product as such | **focus either on giving assessment procedure guidelines or considering product as such | ||
− | + | '''Solution | |
*multi-perspective approach to performance | *multi-perspective approach to performance | ||
− | **quality of content | + | **effectiveness |
− | **applicability | + | ***quality of content |
+ | ***applicability | ||
**efficiency | **efficiency | ||
*an integral part of assessment process → applicable also in design and execution (a priori), not only evaluation (a posteriori) | *an integral part of assessment process → applicable also in design and execution (a priori), not only evaluation (a posteriori) | ||
− | + | '''Evaluation | |
− | *focus on the mediator | + | *focus on the mediator of the overall intellectual process - the shared obeject of activity |
**product|use purpose, assessment process|product | **product|use purpose, assessment process|product | ||
**trialogical process, knowledge creation metaphor | **trialogical process, knowledge creation metaphor | ||
Line 44: | Line 45: | ||
− | + | == Introduction == | |
− | + | *environmental health assessment is intentional societal activity involving plural actors with diverse perspectives and needs | |
+ | **provide information on practical decision-making (political, industry, individuals) | ||
+ | **increase awareness and level of understanding on important issues (experts, DMs, SHs, public) | ||
+ | **advance scientific research (experts) | ||
+ | *effectiveness of assessments must be ensured → performance of assessments needs to be considered and evaluated | ||
− | == | + | == Perspectives to performance == |
− | * | + | *various overlapping perspectives to the issue: uncertainty assessment, quality assurance, model development guidelines, ... |
− | * | + | *all perspectives provide only limited or narrow views to performance |
− | * | + | **use purpose of information often not considered |
− | ** | + | **all aspects of performance not covered |
− | ** | + | **evaluation as a separate process, often only after assessment |
− | * | + | **societal aspect often neglected |
− | * | + | **focus either on giving assessment procedure guidelines or considering product as such |
− | == | + | == Properties of good assessments == |
− | + | *multi-perspective approach to performance | |
+ | **effectiveness | ||
+ | ***quality of content | ||
+ | ***applicability | ||
+ | **efficiency | ||
+ | *an integral part of assessment process → applicable also in design and execution (a priori), not only evaluation (a posteriori) | ||
− | |||
− | |||
− | |||
− | |||
*Relation of properties to information structure/content | *Relation of properties to information structure/content | ||
+ | |||
*Evaluation process | *Evaluation process | ||
**a priori and/or a posteriori view | **a priori and/or a posteriori view | ||
Line 80: | Line 87: | ||
== Discussion == | == Discussion == | ||
− | * | + | *focus on the mediator of the overall intellectual process - the shared obeject of activity |
− | ** | + | **product|use purpose, assessment process|product |
− | ** | + | **trialogical process, knowledge creation metaphor |
− | ** | + | **not only collection and use of existing information or learning to deal with new situation, but also creation of new knowledge |
− | * | + | *capable of incorporating the goods from other perspectives into a more comprehensive and coherent approach |
− | * | + | *evaluation of performance can only be done meaningfully against purpose |
− | **data about hypothesis | + | **various intentions need to be identified, explicated and prioritized in all assessments |
+ | |||
+ | *data about hypothesis | ||
== Conclusions == | == Conclusions == |
Revision as of 09:46, 19 August 2008
<accesscontrol>Members of projects,,Workshop2008,,beneris,,Erac,,Heimtsa,,Hiwate,,Intarese</accesscontrol>
This is a manuscript about evaluating performance of environmental health assessments. It discusses different perspectives to evaluating the goodness of assessments and proposes a new more comprehensive approach to considering performance of environmental health assessments.
Contents
Abstract
Situation
- environmental health assessment is intentional societal activity involving plural actors with diverse perspectives and needs
- provide information on practical decision-making (political, industry, individuals)
- increase awareness and level of understanding on important issues (experts, DMs, SHs, public)
- advance scientific research (experts)
- effectiveness of assessments must be ensured → performance of assessments needs to be considered and evaluated
- various overlapping perspectives to the issue: uncertainty assessment, quality assurance, model development guidelines, ...
Problem
- all perspectives provide only limited or narrow views to performance
- use purpose of information often not considered
- all aspects of performance not covered
- evaluation as a separate process, often only after assessment
- societal aspect often neglected
- focus either on giving assessment procedure guidelines or considering product as such
Solution
- multi-perspective approach to performance
- effectiveness
- quality of content
- applicability
- efficiency
- effectiveness
- an integral part of assessment process → applicable also in design and execution (a priori), not only evaluation (a posteriori)
Evaluation
- focus on the mediator of the overall intellectual process - the shared obeject of activity
- product|use purpose, assessment process|product
- trialogical process, knowledge creation metaphor
- not only collection and use of existing information or learning to deal with new situation, but also creation of new knowledge
- capable of incorporating the goods from other perspectives into a more comprehensive and coherent approach
- evaluation of performance can only be done meaningfully against purpose
- various intentions need to be identified, explicated and prioritized in all assessments
Introduction
- environmental health assessment is intentional societal activity involving plural actors with diverse perspectives and needs
- provide information on practical decision-making (political, industry, individuals)
- increase awareness and level of understanding on important issues (experts, DMs, SHs, public)
- advance scientific research (experts)
- effectiveness of assessments must be ensured → performance of assessments needs to be considered and evaluated
Perspectives to performance
- various overlapping perspectives to the issue: uncertainty assessment, quality assurance, model development guidelines, ...
- all perspectives provide only limited or narrow views to performance
- use purpose of information often not considered
- all aspects of performance not covered
- evaluation as a separate process, often only after assessment
- societal aspect often neglected
- focus either on giving assessment procedure guidelines or considering product as such
Properties of good assessments
- multi-perspective approach to performance
- effectiveness
- quality of content
- applicability
- efficiency
- effectiveness
- an integral part of assessment process → applicable also in design and execution (a priori), not only evaluation (a posteriori)
- Relation of properties to information structure/content
- Evaluation process
- a priori and/or a posteriori view
- identification of purpose
- evaluation of quality of content (uncertainty + relevance)
- in principle reality, but in practice golden standard as reference point D↷
- evaluation of applicability
- evaluation of efficiency (effort expenditure)
- overall performance
- potential for effectiveness/effort given purpose
- can be further evaluated retrospectively against realized effectiveness (possibly against redefined purpose)
- potential for effectiveness/effort given purpose
Discussion
- focus on the mediator of the overall intellectual process - the shared obeject of activity
- product|use purpose, assessment process|product
- trialogical process, knowledge creation metaphor
- not only collection and use of existing information or learning to deal with new situation, but also creation of new knowledge
- capable of incorporating the goods from other perspectives into a more comprehensive and coherent approach
- evaluation of performance can only be done meaningfully against purpose
- various intentions need to be identified, explicated and prioritized in all assessments
- data about hypothesis
Conclusions
- There is more to assessment performance than just statistical uncertainty and data source reliability
- Overall performance of assessment can be evaluated systematically and explicitly
- requires consistent information structure
- a priori evaluation should be made an inherent part of assessment process