View source for Assessment of impacts to environment and health in influencing manufacturing and public policy

[[Category:Open policy practice]]
[[Category:Decision analysis and risk management]]
[[heande:Assessment of impacts to environment and health in influencing manufacturing and public policy]]
{{nugget
|authors=  [[user:Mikko Pohjola|Mikko V. Pohjola]]
|reference=  Pohjola, M.V., 2014. Assessment of impacts to health, safety, and environment in the context of materials processing and related public policy. In: Comprehensive Materials Processing, Bassim, N., Ed. Vol. 8, Elsevier Ltd., pp 151–162. {{doi|10.1016/B978-0-08-096532-1.00814-1}}
|urn=  
|ethics=  
|journal=}}

:''This page is a final manuscript of'' Pohjola, M.V., 2014. Assessment of impacts to health, safety, and environment in the context of materials processing and related public policy. In: Comprehensive Materials Processing, Bassim, N., Ed. Vol. 8, Elsevier Ltd., pp 151–162. {{doi|10.1016/B978-0-08-096532-1.00814-1}}.

== Keywords ==

:Chemical safety assessment
:Environmental impact assessment
:Environmental management system
:Health impact assessment
:Health, safety and environment
:Life-cycle assessment
:Materials processing
:Opasnet
:Open assessment
:Participation
:Public policy
:Risk assessment
:Safety engineering
:Shared understanding
:Strategic environmental assessment

== Abstract ==

Industrial processes and products may have severe impacts to safety, health, and environment. Therefore, they are of importance and interest to producers, product users, policy makers, researchers and general public. Health, safety and environmental assessments provide information to support management of these impacts.

Many assessment approaches with different purposes, emphases, and procedures exist. Many of the limitations built in the approaches are unnecessary and even counterproductive. A perspective perceiving assessments and management as a collaborative social knowledge process can be helpful by allowing focusing on relevant issues, and engaging relevant actors to develop shared understanding about actions upon them.

== Introduction ==

Industrial processes, their products and their use may have severe impacts to health, safety, and environment. Therefore, assessments of these impacts are of importance and interest, not only to producers, policy makers and researchers, but also to product users and public in general. Although health, safety, and environmental assessments are most commonly perceived as support to management of the related impacts by either by producers or policy makers, all the above mentioned actor groups may also have relevant roles in both assessment and application of their results in practice.

A great variety of assessment approaches relevant to health, safety, and environment in materials processing and related policy making [1]. There are many commonalities across the range of approaches, but each also have their own peculiarities e.g. in relation to purposes, emphases, and procedures of assessment. Despite that such peculiarities are understandable e.g. in terms of the context of their development and the intended context of their application, many of the limitations built in the approaches can be considered unnecessary and even counterproductive to effective knowledge-based practice in policy, production, as well as everyday life [2]. 

This chapter considers a set of well-known, commonly applied assessment approaches to assessing health, safety, and environmental impacts relevant to materials processing and related public policy making. By means of a specifically designed framework, the approaches are characterized and evaluated in terms of how they address and serve different information needs in this context.

In order to provide means to overcome the limitations apparent in the independent, as well the set of, approaches a novel perspective, perceiving assessment and management of health, safety, and environment as a collaborative social knowledge process, is proposed. Such a perspective fundamentally building on the practices and means for knowledge creation, and the interconnectedness of knowledge and action can be helpful by allowing focusing on the most relevant issues in assessment and management, and engaging all relevant actors to develop shared understanding about the actions to be taken upon them [3].

== Materials processing, public policy and assessment ==

There are several aspects that need to be taken account of in health, safety, and environmental assessment in the context of materials processing and related public policy making. Altogether all the physical, chemical, biological, social, and political factors that play a role in it constitute a complex and challenging field of practice. For instance, materials processing covers a vast variety of different processing activities, resulting products, their use, as well as materials and intermediate goods used in their making. All these may give rise to combinations of impacts e.g. to health, safety, environment, or other aspects of reality. In addition, these impacts are distributed in many ways, e.g. in time and space or among different populations, and may result in a variety of secondary consequences.

The endeavors of dealing with the impacts further add to the complexity. Due assessment of the impacts often requires more than mere interest and will to examine them and their causes. A variety of skills and resources, such as methods, tools and access to information, may be required. Furthermore, sufficient power and social support may be required for taking reasoned actions to deal with the impacts, generally referred to as management. In addition, the regulatory frameworks and conventions in the field of health, safety, and environmental assessment and management bring about constraints to their practice.

It is, however, important to notice the central role of information and knowledge in these processes. Production and use of information and knowledge is at the core of health, safety, and environmental assessment and management. Despite different perspectives, needs and means, essentially the concern for everyone involved, whether in the role of e.g. a producer, consumer, policy maker, or a citizen, is about knowing the impacts and taking knowledgeable actions about them.

In public policy, knowledge is needed e.g. in setting requirements for industrial activity for example in order to secure the safety of those involved in processing or use of products, as well as protect the environment and health of the members of society at large from the impacts from the processing and use of products. Naturally, this must be done based on understanding of not only these specific impacts, but also their relations to other, e.g. social and economic, impacts, as well as different alternative actions and decision options.  Successful industrial management of safety, health, and environmental matters then requires knowledge about the impacts and regulatory requirements regarding them, including guidance of safe use of products, but also e.g. about the needs and perceptions of their customers as well as the members of society at large. The perceptions among potential product users and the public may be crucial for the success of commercial industrial activity. The roles of users and the public are thus essential with regard to health, safety, and environmental assessment and management.

Assessment of safety, health and environmental impacts in the context of materials processing and relevant public policy can thus be ideally seen as an interplay between numerous actors, each representing and promoting their own aims and needs, discussing how things are, how they should be, and how they would be if something happened.

=== Framework for characterizing the settings of assessment ===

Acknowledgement of the multiple roles and needs in relation to health, safety and environmental issues in materials processing and related public policy allows construction of a comprehensive framework for characterizing the settings for their assessment (Table 1). It considers five essential characteristic attributes of the assessment setting, typical categories within each attribute in this context, and examples of questions to ask when addressing each attribute. The framework is an adaptation of the analytical frameworks designed and applied in the BEPRARIBEAN project for studying the state of the art and potential for improvement in benefit-risk analysis within several different fields of practice relevant to environment and health [1,3] and comparing approaches to environment and health assessment with regard to their potential for effective participation [4]. The framework is briefly presented in table 1 and explained below. In the next section it is applied in characterizing different health, safety and environment assessment approaches relevant to materials processing and related public policy.

{|{{prettytable}}
|+ Table 1. Framework for characterizing the settings for health, safety and environmental assessments relevant to materials processing and related public policy.
! Attribute
! Categories
! Questions
|-----
| Impacts
| 
* Environment
* Health
* Safety
* Other
| 
* Which impacts are addressed in assessment?
* Which impacts are most significant?
* Which impacts are relevant to practice?
|-----
| Causes
| 
* Processing
* Product
* Use
| 
* Which causes of impacts are recognized in assessment?
* Which causes of impacts are most significant?
* Which causes of impacts are relevant to practice?
|-----
| Problem owner
| 
* Policy maker
* Producer
* Expert
* User
* Public
| 
* Who has the interest, responsibility and/or means to assess the issue?
* Who actually conducts the assessment?
* Who has the interest, responsibility and/or power to take action upon the issue?
* Who are affected by the impacts?
|-----
| Target
| 
* Policy maker
* Producer
* Expert
* User
* Public
| 
* Who are the intended users of assessment results?
* Who needs the assessment results?
* Who can make use of the assessment results?
|-----
| Interaction
| 
* Isolated
* Informing
* Participatory
* Joint
* Shared
| 
* How does assessment interact with the intended use of its results?
* How does assessment interact with other actors in its context?
* What is the degree of openness in assessment (and management)?
|}

Environment is here seen to cover natural environment [5] excluding humans. Environmental health impacts (health impacts to humans mediated by environment) are thus here considered as belonging to health. Generally, health is perceived in accordance with the definition by WHO [6] as a state of physical, mental, and social well-being, not merely the absence of disease or infirmity. Safety is considered in terms of direct impacts, primarily to health, but possibly also to environment, caused by proper or improper use of functioning or malfunctioning processing equipment, intermediate goods or end products. Although safety thus can be considered as a subcategory of both health and environment, it is mentioned separately due to its importance within the context of materials processing. Also other than health, safety, or environment impacts may exist often are of significant importance to decision making upon the issues in question.

Processing here covers all operations required to realize a product that can be used downstream in the supply chain. A product can either be the end product used by the end user or an intermediate good used for further processing in subsequent phases in the supply chain. With these conceptualizations, use can here be limited to only consider the end use of the end product. Accordingly, e.g. emissions, exposures or impacts deriving from either intermediate goods or end products can be considered in relation to either processing or use, respectively.

Assessments may be initiated and/or conducted by those who have a genuine need or interest to know about the possible impacts of something. Such needs or interests may be derived from concern (e.g. perceived risk of an activity) or foreseen benefit (e.g. competitive advantage). In practice, often the need and interest is generated by regulatory requirements that determine responsibilities and procedures of assessment and decision making upon certain issues. As a consequence it is possible that those with a genuine need or interest to assess, responsibility to assess, and power to decide or take action upon the issue are not optimally involved or taken account of in assessment and related decision making. Policy maker here covers both those involved with regulating e.g. particular projects or operations and those dealing with higher-level policy development. Producer refers to those involved with and in charge of different phases of processing as defined in relation to causes of impacts. Expert means an independent, external to policy making or processing, specialist on the issue, e.g. a commercial consultant or a researcher in the academia. In line with the definition of use in relation to causes of impacts, user here refers to the end user of the end product. Public refers to virtually anyone else who is or may be affected by or is or may be interested in the issue at hand.

The audiences that are recognized and targeted as users of assessment results vary a lot depending on other aspects of the settings within which assessments take place. The intended use may be e.g. in various levels of policy development, different levels of processing operations, practical user guidance, scientific discourse or some of their combinations. The categories of the target attribute are the same as for problem owner.

The interaction of assessment and its context can be roughly described as consisting of two aspects: 1) interaction between assessments and its intended primary use, and 2) interaction between assessment and its other context. Although consideration of problem owner and target of assessment already reveals something about the assessment-use interaction, there are many variations of how the interaction in between may take place. In addition there are multiple combinations how different actors may or may not be included in assessment and/or related decision making, and how interaction with them may take place. For example, results of regulatory environmental impact assessments to support policy decisions on environmental permits on specific projects are often, at least partly, made publicly available. Their applicability may, however, be limited outside their primary use purpose. In contrast, internally initiated and conducted assessments by industrial producers may be classified, although they may contain information that could be very useful in policy making or research. Assessments initiated and conducted in the research context then are often primarily intended for expert audiences via scientific journals, and consequently may be of little use in practice. Despite increasing emphasis on user and public engagement in design, assessment as well as policy making, users and public are still mostly treated as either additional sources of and/or passive recipients of information in assessments.

In situations where assessments are embedded in the processes where their results are used, interaction is best characterized in terms of how the assessment-management complex interacts with its context. Where assessments and the processes where their results are used are perceived as independent entities, interaction is best characterized in terms of how the assessment interacts with its context, including use as an aspect of the context. The categories of interaction are here defined as follows:
* In isolated interaction assessment and use of assessment results are strictly separated. Results are provided to intended use, but users and stakeholders shall not interfere with making of the assessment.
* Informing interaction means that assessments are designed and conducted according to specified needs of intended use. Users and limited groups of stakeholders may have a minor role in providing information to assessment, but mainly serve as recipients of assessment results.
* In participatory interaction broader inclusion of participants is emphasized. Participation is, however, treated as an add-on alongside the actual processes of assessment and/or use of assessment results.
* Joint interaction means involvement of and exchange of summary-level information among multiple actors in scoping, management, communication and follow-up of assessment. On the level of assessment practice, actions by different actors in different roles (assessor, manager, stakeholder) remain separate.
* In shared interaction different actors involved in assessment retain their roles and responsibilities, but engage in open collaboration upon determining assessment questions to address and finding answers to them as well as implementing them in practice.

== Characterization of common approaches to health, safety, and environmental assessment ==

The framework described above is now applied to characterizing approaches to assessing health, safety, and environmental issues according to the impacts the approaches recognize, the causes of impacts they focus on, who conducts the assessment, the purposes the assessments intend to serve, and how they interact with their context. A representative, but by no means exhaustive, set of eight approaches to assessing and dealing with health, safety, and environmental issues is chosen for scrutiny based on all approaches being well-know, commonly applied, and relevant to materials processing and related public policy as well as providing a wide range of perspectives to health, safety and environmental assessment.

Some of the approaches have also been previously scrutinized in related characterizations [1,4], but here the emphasis is particularly on aspects relevant to materials processing and related policy making from the point of view of making use of the assessment results. The approaches are characterized below according to how different aspects are interpreted within each approach, summarized for overview in table 2, and discussed in terms of how they succeed to serve the information needs in their contexts.

=== Environmental impact assessment (EIA) ===

Environmental impact assessment is typically considered as the often evaluation of the effects likely to arise from a proposed major project (or other action) significantly affecting the environment, in order to support decision making upon whether or not a proposal should be given approval to proceed [7,8]. Such project-based regulatory interpretation of EIA is also adopted here, and is in accordance with the European EIA Directive (85/337/EEC, amendments 97/11EC, 2003/35/EC and 2009/31/EC). This characterization primarily considers the national implementation of EIA Directive in Finland (Finnish EIA Act 468/1994, amendments 267/1999 and 458/2006, Finnish EIA Decree 794/1994, amendments 268/1999 and 713/2006), but is complemented with more general descriptions of EIA where applicable. The Finnish regulatory EIA can be considered as fairly representative of the EU Directive and of the mainstream theories and other regulatory implementations of EIA globally [1].

Sometimes also evaluation of the impacts of policies, plans and programs in order to influence higher-level decision making is considered as belonging to environmental impact assessment [9,10]. This approach, often referred to as strategic environmental assessment (SEA), is considered as a separate approach below.

Environmental impacts are of primary concern in EIA. Health and also safety issues may be addressed, but they are mostly given secondary emphasis. In addition, social and economic impacts may be considered alongside other impacts in order to provide a better context for interpretation of the importance of impacts of primary concern. The focus in EIA is mainly on assessing the impacts from processing given that a proposed project (e.g. industrial plant, waste handling site, mine etc.) becomes realized and is in operation according to some of its implementation alternatives.

The responsibility of assessment is given by regulation to the producer proposing the project. In practice, the producers often outsource the assessments from engineering consultancies or other external competent experts. The primary purpose of assessment is to serve policy making e.g. upon permitting and regulation of the proposed project and its operations. EIA is also intended as a mechanism e.g. to support proponents' project design and management, influence higher-level policy making, and promote social learning [10-12]. In Finland, the interest of proponents in making use of EIA results have, however, been found to vary significantly from assessment case to another [11].

Despite being an approach designed for regulatory needs, the linkage between the assessment and related policy processes has been found to be relatively weak, much due to the procedurally oriented development of both theories and legislation related to EIA [4,7,13]. Also, despite the strong emphasis on the role of participation in EIA literature and obligatory public hearings by embedded in EIA legislation, participation is commonly perceived as an additional burden and the influence of participation often remains questionable [4,12,13]. In terms of interaction, EIA is mainly participatory.

=== Strategic environmental assessment (SEA) ===

Strategic environmental assessment is a systematic process for evaluating the environmental consequences of proposed policies, plans, or programs in order for them to be included in policy making on par with economic and social considerations [14]. SEA is often considered as a complement to project-based EIA, but also as an evolved paradigm of environmental assessment moving EIA principles upstream in the policy making processes [15,16]. The international regulatory framework for SEA is mainly set by two legal documents, the European SEA Directive (2001/42/EC) and the United Nations Economic Commission for Europe (UNECE) 2003 SEA protocol, but there are also several other SEA procedures implemented globally, and the characterization below builds on the review of SEA in 12 selected countries by [15].

In SEA, environmental impacts are emphasized and intended to be considered equally alongside economic and social impacts. Health impacts may be considered as instances of social impacts, but not explicitly recognized in many implementations of SEA. The focus of the assessments follows the focus of the proposed policies, plans, and programs being assessed. The policies, plans, and programs may have implications to processing, products as well as their use.

Assessments are initiated and often also conducted by the policy makers from local to national levels proposing the policies, plans and programs. In some implementations it is required that the assessment task is given to an independent expert team in order to ensure objectivity. Assessments feed back to the policy, plan, or program development process. In many implementations assessment results are also made publicly available at some part of the process either as a separate assessment report or as merged into the documentation regarding the proposed policy, plan or program.

In SEA the assessments may be better linked with the use of their results as in EIA, as they are typically embedded in the policy, plan, or program development processes. However, despite similar ideological basis, other participation is often more limited than in EIA, partly due to the abstract and general character of higher-level policy issues, than in EIA. Altogether, the model of interaction in SEA is best characterized either as isolated or informing.

=== Health impact assessment (HIA) ===

Health Impact Assessment is a combination of procedures, methods and tools for judging the potential health effects of a policies, programs or projects across diverse economic sectors on a population, particularly on vulnerable or disadvantaged groups, and improve health and well-being [17, WHO HIA website]. There are many definitions for and variations of HIA, but the interpretation by the World Health Organization (WHO) is probably the most representative and broadly applied and is taken here as the basis for characterizing this approach.

HIA considers health in a broad sense in the context of economic and social impacts, also with the aim to increase equity. Environment is mainly considered in HIA as a potential mediator of health impacts. The focus of assessment follows the focus of the policies, plans and projects being assessed. As regards projects, the focus typically resembles that of EIA and mainly addresses impacts from processing. As regards policies and plans, the focus resembles that of SEA and may well relate also to products as well as their use.

Assessments are intended to be conducted by either policy makers developing the policies and plans, or producers developing their projects, or outsourced from external experts in a somewhat similar manner as is common in SEA and EIA. Assessments are intended feed back to the development of the policies, plans and projects, again in similar manner as in EIA and SEA. However, according to Kemm [18], there is ambiguity among practitioners whether HIA should be in or out of the policy making sphere, and in practice making of HIAs has been much dependent on particular enthusiasts often working in policy making or research on public health, and having access to required resources.

Consequently, the interaction between assessment and use may vary significantly from case to another depending on whether assessments are initiated and/or conducted within policy making, research, or possibly industry. Participation of stakeholders and public is strongly included in the rhetoric of HIA, but the ideas of broad, meaningful and effective participation are often not realized in practice [18,19]. Altogether the model of interaction in HIA can be characterized as participatory.

=== Risk assessment (RA) ===

Most commonly risk assessment (and many other health relevant assessments as well) is perceived to fundamentally consists of the four strictly scientific major steps of hazard identification, dose-response assessment, exposure assessment, and risk characterization, as was suggested already in the classic "Red Book" by the US National Research Council (NRC) [20]. The Red Book dealt with the risk of cancer and other health effects associated with exposures to toxic substances, but later the approach has been applied also to many other areas, e.g. food and nutrition [21], chemical safety (see below) and safety engineering (see below). In addition, the principles and practices of RA have gone through remarkable evolution over the last decades [22,23]. Of all the possible variations of RA, this characterization is based on the approaches described by NRC [20,22,23].

RA is mostly concerned with health impacts. The approach is, however, commonly applied also in terms of safety as well as environmental risks. Fundamentally RA attempts to reveal the intrinsic risk embedded in a single object, e.g. a single substance or a product, but may also consider the risk of an activity. The approach can thus be applied to considering processing as well as use. Typically RA, however, focuses on a product or some of its component (cf. chemical safety assessment below).

In the NRC perspective, assessments themselves are independent scientific expert activities, although they may be initiated by policy makers. Assessments intend to support risk management typically undertaken by policy makers. Also different perceptions regarding problem owners and targeted users exist in various different implementations of RA.

The traditional Red Book RA recognizes virtually no other interaction besides one-way communication of assessment results to specified policy use. This still appears to be a common practice also in many contemporary variations of RA. The evolved versions of the RA approach by NRC [22,23], however, recognize deliberation between assessors, policy makers, stakeholders and public as an essential aspect of the overall process of managing risks. The technical assessment of risk should, however, be conducted by independent experts according to its own standards and guidelines. Interaction may thus range from isolated in traditional RA even up to joint in the evolved interpretations of RA.

=== Chemical safety assessment (CSA) ===

Chemical safety assessment is actually an implementation of risk assessment within the Research, Evaluation, Authorization and Restriction of Chemicals (REACH), a European Community Regulation on chemicals and their safe use. Although not the only one, this is the most dominant interpretation of CSA, and taken here as the basis for characterization. Under REACH, a CSA [24] is required if a substance is manufactured or imported into the EU at 10 tons or more per year per registrant. Comprehensive description, guidance and documentation on REACH can be found from the European Chemicals Agency (ECHA) website. 

CSA primarily addresses safety, but also impacts to health and environment are recognized. As is typical for RA, the assessment focuses on identifying the risk of a chemical as such, in a compound or in a product. In CSA the risk is, however, also considered in relation to the use contexts of the intermediate goods and final products as well as the whole product life cycle in order to define safe exposure scenarios and use conditions. CSA is thus a product-oriented approach, which extends also to both processing and use. 

Based on the regulation, the responsibility to assess and manage the risks is placed on the producer (manufacturer or importer of a chemical containing product). Assessments serve regulatory policy making (by ECHA), but the primary use of the results is in communicating substance properties and safe use conditions to the users of the chemicals, compounds and products downstream in the supply chain.

In making of the assessments, producers collect exposure information along their supply chains and producers of the same chemical are encouraged to collaborate in assessing the risks. The role of others than the assessors, also including the related regulatory agency ECHA, is, however, mainly to only provide and/or receive information. Consequently, informing is the best characterization of the model of interaction in CSA.

=== Life-cycle assessment (LCA) ===

Life cycle assessment is a “cradle-to-grave” approach for assessing industrial systems, beginning with the gathering of raw materials from the earth to create the product and ending at the point when all materials are returned to the earth [25]. LCA enables the estimation of the cumulative environmental impacts resulting from all stages in the product life cycle in terms of energy and material inputs as well as environmental releases and potential environmental impacts associated with identified inputs and releases [25]. LCA is well established and different implementations of LCA follow the same principles. This characterization is primarily based on the interpretation of LCA adopted by the U.S. Environmental Protection Agency (EPA) described by the Scientific Applications International Corporation [25].

LCA is mostly concerned with impacts to the environment. Also health may be considered to some extent, but primarily in terms of environmental and occupational health as impacts mediated by the environment. The focus in LCA is on product [26], but extends to consider the material and energy flows related to all phases of its processing as well as use.

Assessments can be initiated by producers, policy makers, experts and stakeholders (e.g. NGOs), but requirements of e.g. specific skills, tools and access to data limit the amount of those who can actually conduct an assessment. LCA is often referred to as an approach to support policy making [27,28], but depending on the purpose of assessment, its results may be targeted to policy makers (e.g. ecolabelling), producers (e.g. product/process design and management), or users (e.g. ecomarketing) [26].

The needs of the intended use of assessment results as well as different stakeholder perspectives are recognized in LCA, but the approach is mostly confined to specifics of analysis, not interaction with its context. However, participatory strategies have been proposed to enhance the effectiveness of policy-driven LCAs [27]. The common practice of LCA may still be characterized as either isolated or informing.

=== Safety engineering (SE) ===

Safety engineering is actually an approach to engineering design, but is included here because it also includes assessment, and is very relevant in the context of materials processing. The four covering principles in SE, representing different ways of achieving safety, have been defined by Möller and Hansson [29] as: 1) inherently safe design, 2) safety reserves, 3) safe fail, and 4) procedural safeguards. Assessment of risks and effects of safety measures is essential in implementation of these principles [29]. A hallmark example of assessment in SE is nuclear safety assessment [30], with the emphasis clearly on processing. Also assessments of product safety, actually resembling CSA discussed above, may be considered relevant, yet less common, in the context of SE.

As the name implies, safety is emphasized in SE, but consideration also extends to health and environment more generally as potential secondary consequences of undesired events. Focus of assessments related to SE is typically on the impacts of processing, particularly the use of production equipment. In the case of product safety engineering the focus is on the use of products.

Assessments related to process or product engineering are normally conducted by producers themselves. With regard to safety regulation, also policy makers, such as nuclear safety or product safety authorities, may be the initiators of assessments, which are then conducted either by themselves or by independent experts. Correspondingly, assessments may serve either producers' engineering needs or policy makers' needs upon regulation of potentially hazardous processes or products, depending on the assessment purpose. Safety information provided by assessments may also be communicated to users (e.g. product safety) and public (e.g. nuclear safety).

Assessments in relation to safety engineering primarily follow the principles of traditional RA and are typically either considered separate expert activities or embedded in design and management activities. The interaction between assessment and its context or the assessment-engineering complex and its context can be characterized either as isolated or informing.

=== ISO 14000 Environmental management system (ISO EMS) ===

ISO 14000 is a management systems standard addressing environmental management with the aim to minimize harmful effects on the environment caused by activities of an organization and to achieve continual improvement of environmental performance [ISO website]. As such the ISO EMS is not particularly an assessment approach, but includes assessment, and is very relevant to the practice within the context of materials processing. In the ISO 14000 family there are standards also for LCA [31] and the ISO system contains also standards relevant to e.g. safety engineering [32]. Here the focus is, however, on the ISO 14000 family in general, which provides organizations with a management framework helping them to deal with their environmental impacts and responsibilities [33].

The emphasis in the ISO EMS is on assessing and managing impacts to environment. Issues of safety and health are addressed in other standards and not explicitly included in the ISO EMS. Impacts from processing and products as well as their use are addressed in a holistic framework.

Assessments are typically conducted by the producer as belonging to the implementation of the ISO EMS, but assessments may also be outsourced from external experts. Assessments are primarily intended to serve environmental management and internal communications of the producer organization, but also to support the producer's communications to users, public and policy makers.

Assessments are embedded in the environmental management activities of the producer. Policy makers and other stakeholders rarely have other role than as recipients of information provided e.g. by environmental reporting supported by the ISO EMS. The model of interaction is thus best described as isolated, or at best informing.

=== Overview of approaches ===

{|{{prettytable}}
|+ Table 2. Characterization of approaches to environment and health assessment in terms of the impacts they recognize, the causes of impacts they focus on, who conducts the assessment, the information needs the assessments serve, and how they interact with their context. Abbreviations: EIA = environmental impact assessment, SEA = strategic environmental assessment, HIA = health impact assessment, RA = risk assessment, CSA, = chemical safety assessment, LCA = life cycle assessment, SE = safety assessment, ISO EMS = ISO 14000 Environmental management system.
! Approach
! Impacts
! Causes
! Problem owner
! Target
! Interaction
|-----
| EIA
| Primarily environment. Also health and safety. Social and economic impacts provide context for other impacts.
| Mainly impacts from processing in focus.
| Producer responsible for assessment. In practice, often outsourced from external experts.
| Serves policy making (e.g. permits and regulations). Intends to inform also producers and public.
| Participatory. Participation emphasized, but interaction between actors often weak.
|-----
| SEA
| Environment alongside economic and social impacts. Health within social impacts.
| Processing, product or use, according to the policies, program, and plans in question.
| Proponent policy makers. Sometimes outsourced from independent experts.
| Policy makers. Serves development of policies, plans, programs. Results may also be made public.
| Builds on same principles as EIA, but often remains isolated or informing.
|-----
| HIA
| Health, in a broad sense, in context of social and economic and social impacts. Environment a mediator of health impacts.
| Processing, product or use according to to the focus of policies, plans and projects in question.
| Policy makers and health officials. May be outsourced from independent researchers. 
| Policy makers. As in SEA, assessments serve development of policies, plans and projects.
| Participatory. Public participation is strong in the rhetoric of HIA, but not much realized in practice.
|-----
| RA
| Mostly health. Also safety or environment.
| Typically product or component. Also applicable to processing and use.
| Assessment an independent scientific activity. Can be initiated by policy makers.
| Supports risk management by policy makers.
| Ranges from isolated (traditional RA) to joint (evolved RA).
|-----
| CSA
| Primarily safety, also health and environment.
| Focuses on product, but extends to all processing and use.
| Producer (manufacturer or importer of a chemical containing product) responsible for assessment.
| Communicates substance properties and use guidance down supply chain. Supports regulatory policy making by ECHA.
| Informing. Role of others than the assessors is to provide and/or receive information.
|-----
| LCA
| Environment. Also health impacts mediated by environment.
| Focus on product, but extends to consider material and energy flows in all phases of processing and use.
| Can be carried out by policy makers, producers and stakeholders (e.g. NGOs).
| Often considered as policy support, but may address producer, user or stakeholder needs.
| Isolated or informing. Approach mostly confined to specifics of analysis, not interaction.
|-----
| SE
| Safety. Also health and environment as secondary consequences.
| Typically processing. In product safety focus on use of products.
| Producers in process/product engineering. Policy makers or experts in safety regulation.
| Producers engineering processes, or policymakers regulatory processes. May be communicated also to users or public
| Isolated or informing. Assessments often either embedded in engineering or kept mostly confidential. 
|-----
| ISO EMS
| Assessment and management of impacts to environment. 
| Impacts from processing, products and use addressed in a holistic framework.
| Assessments conducted by producer. May be outsourced from external experts.
| Serves environmental management and internal communications of producer. Supports communication to users, public and policy makers.
| Isolated or informing. Assessments embedded in the EMS. Policy makers and other stakeholders as recipients of information.
|}

The summary and comparison of the different assessment approaches across the attributes of the characterization framework (in table 2.) reveals both similarities and differences between them. For example, the differences in what is perceived as being of interest in assessment are significant. In addition, some approaches also intend to take account of what is of interest in management of the issues in question, e.g. in terms of building a context of also other impacts in addition to the impacts of primary interest, while others leave that solely as a question of concern for management.

Also significant variation in the arrangements of assessment, e.g. who assesses, for whom, and how, as well as in the relationship between assessment and management exists. These aspects, as well as constraints to issues of interest, are in some cases determined by regulation, at least to some extent, but often also just seem to derive from the practices and conventions developed and taken up by researchers and practitioners working with different approaches. As a consequence, numerous combinations of relationships between assessment and management, assessment and other stakeholders, as well as models of interaction between them can be identified.

Despite that all the approaches to assessment considered here are intended to support management of health, safety, and environmental impacts, typically either in policy making or management of industrial activity, the linkage between assessment and intended use of its results often turns surprisingly weak (see also [1]). On the other hand, in those approaches where assessment is deeply intertwined with management, the linkages of the assessment-management complex appear weak. It seems that, despite the strong rhetoric of public participation and stakeholder involvement in assessment and management upon issues of health, safety, and environment, the reality of practices seems rather closed and exclusive than open and collaborative. Altogether, in aggregate the assessment approaches appear to be too much focused on processes, procedures and access rather than on purposes and outcomes in serving the needs of management as well as other audiences of assessment information [1,4,7,34,35].

By considering from the point of view of making use of assessment results, particularly if acknowledging the different possible actors and roles discussed in the previous section, all the assessment approaches seem somewhat limited and rigorous, both individually and in aggregate. For example, the constraints regarding the issues of interest may limit the applicability of the assessment results e.g. due to lack of relevance to practice, and the constraints in interaction between assessment, management, and other context may limit the content of quality, applicability, and even efficiency of assessment e.g. due to lack of perspectives, insight, acceptability, and resources (for detailed explanations for quality of content, applicability, and efficiency in the context of assessment, see [35]).

It may well be asked, are these limitations well founded and necessary? If not, a follow-up question then is, what could be done in order to overcome these limitations? The next section will briefly present an alternative perspective, and related examples, to assessment of health, safety and environment, applicable also in the context of material processing and related public policy.

== Towards approaches that serve multiple information needs ==

An alternative approach is to take a contentual, as in comparison to procedural, perspective to assessing issues of health, safety, and environment [4]. Basically it means focusing on the knowledge, its creation, dissemination and application, within the social context of dealing with different actions, the needs they serve, and their impacts and consequences. Such a pragmatic approach, building on the idea of the interconnectedness of knowledge and action, has been recently developed and applied e.g. in computer-supported collaborative learning [36,37], co-development of social and health service practices [38], and environment and health related policy support [2,3,Opasnet in English,Opasnet in Finnish]. Essential enablers for these developments have also been the developments of collaborative technologies, tools and practices for broadly distributed web-based collaboration for various practical purposes [39-44].

The idea of applying a contentual, collaborative approach to assessments relevant to dealing with issues of health, safety, and environment is illustrated in figures 1 and 2 and explained briefly below.


[[File:Shared understanding.png|thumb|center|500px|Figure 1. An image describing assessors (experts on a specific discipline), managers (having the power and responsibility to decide upon certain issues), and stakeholders (anyone interested in or influenced by what is being assessed and/or decided upon) engaging in benefit-risk analysis in order to develop shared understanding about what should be done and how.]]


The framework described in figure 1 was developed as guidance for improving benefit-risk analysis of food and nutrition as well 
as other fields of policy support [3]. It emphasizes increased communication and engagement among all relevant actors in relation to the issue being assessed and managed. The common goal in the process is to develop collective knowledge and shared understanding among the participants to support the different decisions and actions of the different actors involved. Fundamentally the assessors (experts), managers (e.g. in policy or production), and stakeholders (specifically or public at large) are considered as equal participants in the process of creating and applying knowledge. Exclusion of any specific actors from the process should be explicitly reasoned and communicated [2,4]. Of course, all actors involved, however, take or retain their own roles, e.g. expert, producer, policy maker, or citizen, in the shared knowledge process according to ones own interests, needs, responsibilities and capabilities and also take their actions accordingly. The distribution of roles in assessment and consequential decision making is illustrated in figure 2.


[[File:Main page picture.png|thumb|center|500px|Figure 2. Open assessment as a collaborative social knowledge process. A graphical illustration of individuals taking different roles and co-operating in finding out what happened, deciding upon what to do, and taking action accordingly. The strawmen depict the members of a society. The paper sheet depicts an assessment in Opasnet (Q = question, R = rationale, A = answer). Yellow arrows depict observation (here of an undesired event, a toxic liquid spill). Blue arrows depict information flow (from members of society to an assessment in Opasnet, from the assessment to members of society, or directly between members of society). Green arrow depicts knowledge-based action (correctly taking care of the spill).]]


Methods and tools to support such distributed collaboration in assessment and management of health, safety, and environmental issues already exist and are being applied and further developed [2]. For example, Opasnet [Opasnet in Eglish] is an open collaborative workspace providing a wiki-interface, modelling tools, and a database for initiating and conducting assessments and supporting the application of their results in practice. Opasnet has been developed in the context of environmental health, but is applicable in collaborative assessment of virtually any kinds of issues. In principle, initiation of assessments by anyone is thus as easy as opening up a an assessment page in Opasnet, defining the assessment question and attracting the relevant actors upon the issue, e.g. experts, policy makers, producers, and local public, to engage in collaboration to find answers to the question. Of course, in practice the case becomes more complex due to e.g. availability of skills and resources, access to information, raising interest in the issue, regulatory requirements, as well as conventions of practice.

Reasons to take up a contentual and collaborative approach are similar as are often expressed regarding stakeholder involvement and public participation in assessment and policy support [45-47] and relate to overcoming the limitations of common assessment practices identified above in the overview of approaches. Open collaboration between plural participants upon issues relevant to materials processing and related public policy may result e.g. in better understanding of the needs, views and perceptions of users, stakeholders and public, improvements of regulations affecting industrial activity, better understanding of the processing activities and their impacts, and a more well-reasoned basis for public perception of the processing activities. Overall, the potential benefits of open collaboration are based on a better and more broadly shared knowledge-base for managing the interactions between industrial activity, policy regulation, and the society at large.

Naturally, there are also challenges inherent in the approach, as well as a number of hindrances for moving from the current conventions towards the ideal presented above. Suitable systems, methods, tools and data sources need to be developed and made available. Also the regulatory frameworks require updating and adaptation in order to support, not hinder, collaboration among the relevant actors in health, safety, and environmental assessment and management. The biggest challenge probably, however, lies in the currently adopted roles, conventions, and practices that often constrain the ways individuals as well as organizations may engage in such activities. However, the movement towards more open and collaborative societies is taking place in many parts of the world, e.g. in terms of open data, direct democracy, open source programming, and open research, and also the assessment of health, safety, and environmental impacts is bound to develop along with the movement.

Also examples of development towards contentual web-based collaboration upon health, safety, and environmental assessment and management already exist.

The EU-funded URGENCHE research project [Opasnet in English] is committed to developing shared understanding and promoting well-founded actions regarding the climate policy needs of several cities in Europe and China by means of collaborative assessment. In addition, the project produces example assessments, models and data for open use, further development, adaptation and application e.g. to support the climate policy making in other cities. A remarkable aspect of this project, in relation to conventional research projects, is also that the cities themselves are strongly involved in the assessments that address real and practical policy making needs of the involved cities. 

MINERA [Opasnet in Finnish] is a Finnish research project addressing environmental exposures and health impacts from mining activities. Although the project mostly is a relatively traditional research project, it also produces assessment models that are made openly available via Opasnet and are thus available for use e.g. in open regulatory environmental impact assessments upon mining activities by anyone. Also the research on water quality and microbial risks at the National Institute for health and Welfare, engaging both the water works and major water users in the society in collaboration, produces openly available models for water quality and risk assessment.

TEKAISU [Opasnet in Finnish] is a research and development project for improving municipality-level policy making practices, particularly in terms of more effective application of information available from impact assessments addressing impacts to environment and health. It is a pilot project for developing municipality-level decision making in Finland towards open collaboration among multiple actors in joint assessment and management of, in principle, all municipality-level decision making with relevance to environment and health. The founding principles within the project are as described above.

There is also an example of a recent open assessment of the potential and risks regarding jathropa oil and fish waste as alternative sources for biodiesel production [48]. The assessment was initiated by Neste Oil and coordinated by the National Institute for Health and Welfare.

== Summary and conclusions ==

Health, safety, and environmental issues are important in industrial activity, public policy making as well as the society at large. They are also issues of great complexity. Assessment and management of impacts to health, safety and environmental issues are therefore activities that have widespread influence and are relevant to many actors in society, e.g. producers, users, policy makers, experts, and public. Information and knowledge regarding health, safety, and environmental impacts is essential in virtually all activities that people engage in different roles in their lives.

There are many approaches to assessing health, safety, and environmental impacts relevant to materials processing and public policy. They share many similarities, but also differ significantly e.g. in terms of their issues of interest and arrangements of their practice. Characterization and evaluation of some commonly applied approaches according to the impacts and causes of impacts considered, the makers and users of assessment, and the degree of openness, reveals that there are many constraints built in the approaches, individually and in aggregate. These constraints may compromise making full use of their results due to limitations in their quality of content, applicability or the efficiency of their making.

Many of these constraints can be overcome by means of a contentual, collaborative, and pragmatic perspective to production and application of information and knowledge upon issues of health, safety, and environment. Focusing on the methods, tools, and practices of creating collective knowledge and shared understanding regarding virtually any issue of interest is free from many of the limitations of common approaches to assessment and management of health, safety, and environmental impacts. Issues of interest and arrangements of assessment may vary from case to another according to the specific needs of each. Fundamentally, it is only the basic mechanisms of collaborative exploration of questions and finding of answers to them that needs to remain fixed. In this perspective, a plurality of actors is allowed to engage in collaboration, each representing and acting according to their own needs, interests, responsibilities, and capabilities. In principle, no limitations to collaboration should be imposed, unless well reasoned and communicated.

Pragmatic collaborative approaches are being developed within different fields of practice, including health, safety, and environmental assessment. An example of such is Opasnet, which is an open web-workspace primarily for environmental health assessment and policy support, but is applicable to collaborative assessment of virtually any issue, and allows initiation of assessments by anyone. Opasnet is already currently applied in a assessment and policy support in many cases relevant to materials processing and related public policy.

Naturally, there are challenges to implementing such a, at first sight radical, approach in practice. Methods, tools, practices, and regulatory frameworks need to be developed to provide sufficient support to open collaboration upon societally important issues. The biggest challenge probably, however, lies in the current conventions and roles which may prevent individuals and organizations from engaging in open collaboration even if they perceived the related potential benefits.

== List of Relevant websites ==

'WHO HIA website' http://www.who.int/hia/en/

'ECHA website' http://echa.europa.eu/

'ISO website' http://www.iso.org/iso/iso_catalogue/management_and_leadership_standards/environmental_management.htm

'Opasnet in English' http://en.opasnet.org

'Opasnet in Finnish' http://fi.opasnet.org

== References ==

[1] Pohjola, M.V.; Leino, O.; Kollanus, V.; Tuomisto, J.T.; Gunnlaugsdόttir, H.; Holm, F.; Kalogeras, N.; Luteijn, J.M.; Magnusson, S.H.; Odekerken, G.; Tijhuis, M.J.; Ueland, Ø.; White, B.C.; Verhagen, H., State of the art in benefit–risk analysis: environmental health. Food and Chemical Toxicology 50 (2012) 40-55. http://dx.doi.org/10.1016/j.fct.2011.06.004

[2] Pohjola, M.V.; Pohjola, P.; Paavola, S.; Bauters, M.; Tuomisto, J.T., Pragmatic Knowledge Services. Journal of Universal Computer Science 17 (2011) 472-497. http://dx.doi.org/10.3217/jucs-017-03-0472

[3] Tijhuis, M.J.; Pohjola, M.V.; Gunnlaugsdóttir, H.; Kalogeras, N.; Leino, O.; Luteijn, J.M.; Magnússon, S.H.; Odekerken, G.; Poto, M.; Tuomisto, J.T.; Ueland, Ø.; White, B.C.; Holm, F.; Verhagen, H., Looking beyond Borders: Integrating best practices in benefit-risk analysis into the field of food and nutrition. Food and Chemical Toxicology 50 (2012) 77-93. http://dx.doi.org/10.1016/j.fct.2011.11.044

[4] Pohjola, M.V.; Tuomisto, J.T., Openness in participation, assessment, and policy making upon issues of environment and environmental health: a review of literature and recent project results. Environmental Health 10 (2011) 58. http://dx.doi.org/10.1186/1476-069X-10-58

[5] Johnson, D.L.; Ambrose, S.H.; Bassett, T.J.; Bowen, M.L.; Crummey, D.E.; Isaacson, J.S.; Johnson, D.N.; Lamb, P.; Saul, M.; Winter-Nelson, A.E., Meanings of environmental terms. Journal of Environmental Quality 26 (1997) 581-589.

[6] World Health Organization (WHO), Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948. www.who.int/bulletin/archives/80(12)981.pdf

[7] Cashmore, M., The role of science in environmental impact assessment: process and procedure versus purpose in the development of theory. Environmental Impact Assessment Review 24 (2004) 403-426.

[8] Jay, S.; Jones, C.; Slinn, P.; Wood, C., Environmental impact assessment: Retrospect and prospect. Environmental Impact Assessment Review 27 (2007) 287-300.

[9] International Association for impact assessment (IAIA), Principles of Environmental Impact Assessment Best Practice. Fargo, United States, 1999.

[10] Macintosh, A., Best Practice Environmental Impact Assessment: A Model Framework for Australia. The Australian Journal of Public Administration, 69 (4) (2010) 401-417.

[11] Jantunen, J.; Hokkanen, P., YVA-lainsäädännön toimivuusarviointi – Ympäristövaikutusten arviointimenettelyn toimivuus ja kehittämistarpeet (Evaluation of the existing EIA legislation – how EIA procedures function in practice and areas needing improvement). Ministry of the Environment, Helsinki, Finland, 2010.

[12] O'Faircheallaigh, C., Public participation and environmental impact assessment: Purposes, implications, and lessons for public policy making. Environmental Impact Assessment Review 30 (2010) 19-27.

[13] Pölönen, I.; Hokkanen, P.; Jalava, K., The effectiveness of the Finnish EIA system – What works, what doesn’t, and what could be improved. Environmental Impact Assessment Review 31 (2010) 120–128. http://dx.doi.org/10.1016/j.eiar.2010.06.003

[14] Sadler, B.; Verheem, R., Strategic environmental assessment: key issues arising from recent practice. Ministry of Housing, Spatial Planning and the Environment, Hague, 1996.

[15] Chaker, A.; El-Fadl, K.; Chamas, L.; Hatjian, B., A review of strategic environmental assessment in 12 selected countries. Environmental Impact Assessment Review 26 (2006) 15-56.

[16] Pope, J.; Annandale, D.; Morrison-Saunders, A, Conceptualising sustainability assessment. Environmental Impact Assessment Review 24 (2004) 595-616.

[17] World Health Organization (WHO), Health Impact Assessment (HIA), Main concepts and suggested approach. Gothenburg consensus paper. World Health Organization, Brussels, 1999.

[18] Kemm, J., The future challenges for HIA. Environmental Impact Assessment Review 25 (2005) 799-807.

[19] Mahoney, M.E.; Potter, J.L.; Marsh, R.S., Community participation in HIA: Discords in teleology and terminology. Critical Public Health 17 (3) (2007) 229-241.

[20] National Research Council (NRC), Risk Assessment in the Federal Government: Managing the Progress. Washington D.C, National Academy Press, 1983.

[21] Smith, M., Food Safety in Europe (FOSIE): risk assessment of chemicals in food and diet: overall introduction. Food and Chemical Toxicology 40 (2002) 141-144.

[22] National Research Council (NRC), Understanding Risk: Improving Decisions in a Democratic Society. Washington D.C., National Academy Press, 1996.

[23] National Research Council (NRC), Science and Decisions: Advancing Risk Assessment. Washington D.C., National Academy Press, 2009. 

[24] Europen Chemicals Agency (ECHA), Guidance on Information Requirements and Chemical Safety Assessment. Guidance for the implementation of REACH. European Chemicals Agency, 2008.

[25] Scientific Applications International Corporation (SAIC), Life cycle assessment: Principles and practices. Scientific Applications International Corporation. National Risk Management Research Laboratory, Office of Research and Development, U.S. Environmental Protection Agency, Cincinnati, Ohio, 2006. http://www.epa.gov/nrmrl/lcaccess/pdfs/600r06060.pdf

[26] Olsen, S.I.; Moller Christensen, F.; Hauschild, M.; Pedersen, F.; Fred Larsen, H.; Torslov, J., Life cycle impact assessment and risk assessment - a methodological comparison. Environmental Impact Assessment Review 21 (2001) 385-404.

[27] Anex, R.P.; Focht, W., Public Participation in Life Cycle Assessment and Risk Assessment: A Shared Need. Risk Analysis 22 (5) (2002) 861-877.

[28] Fullana i Palmer, P.; Puig, R.; Bala, A.; Baquero, G.; Riba, J.; Raugei, M., From Life Cycle Assessment to Life Cycle Management: A Case Study on Industrial Waste Management Policy Making. Journal of Industrial Ecology 15 (3) (2011) 458-475.

[29] Möller, N.; Hansson, S.O., Principles of engineering safety: Risk and uncertainty reduction. Reliability Engineering and System Safety 93 (2008) 776-783.

[30] International Atomic Energy Agency (IAEA), IAEA safety standards for safety assessment. International Atomic Energy Agency, 2010. http://www-ns.iaea.org/downloads/ni/training/safety_standards_presentations/IAEA%20Safety%20Standards%20for%20Safety%20Assessment.pdf

[31] Finkbeiner, M.; Inaba, A.; Tan, R.B.H.; Christiansen, K.; Klüppel, H.J., The New International Standards for Life Cycle Assessment: ISO 14040 and ISO 14044. International Journal of Life Cycle Assessment 11 (2) (2006) 80-85.

[32] O'Connor, P., Standards in reliability and safety engineering. Reliability Engineering and System Safety 60 (1998) 173-177.

[33] Elefsionitis, P.; Wareham, D.G.; ISO 14000 Environmental Management Standards: Their Relation to Sustainability. Journal of Professional Issues in Engineering Education and Practice 131 (3) (2005) 208-212.

[34] Doelle, M.; Sinclair, A.J., Time for a new approach to public participation in EA: Promoting cooperation and consensus for sustainability. Environmental Impact Assessment Review 26 (2006) 185-205.

[35] Pohjola, M.V.; Tainio, M.; Pohjola, P.; Tuomisto, J.T., The properties of good assessment - a review and synthesis of perspectives to model and assessment performance. Manuscript.

[36] Hakkarainen, K., A knowledge practice perspective on technology-mediated learning. Computer-Supported Collaborative Learning 4 (2009) 213-231.

[37] Paavola, S.; Hakkarainen, K., From meaning making to joint construction of knowledge practices and artefacts – A trialogical approach to CSCL. In C. O'Malley, D. Suthers, P. Reimann, A. Dimitracopoulou (Eds.), Computer Supported Collaborative Learning Practices: CSCL2009 Conference Proceedings. (pp. 83-92). Rhodes, Creek: International Society of the Learning Sciences (ISLS), 2009.

[38] Koivisto, J.; Pohjola, P., Practices, modifications and generativity - REA: a ractical tool for managing the innovation processes of practices. Systems, Signs & Actions 5 (1) (2011) 100-116.

[39] Tapscott, D.; Williams A.D., Wikinomics: How Mass Collaboration Changes Everything. Penguin Group, Toronto, 2006.

[40] Noveck, B., Wiki Government. Brookings Institution Press, 2009.

[41] Surowiecki, J., The Wisdom of Crowds. Anchor Books, 2005.

[42] Oinas-Kukkonen, H., Network analysis and crowds of people as sources of new organisational knowledge. In: A. Koohang et al. (Eds): Knowledge Management: Theoretical Foundation. Informing Science Press, Santa Rosa, CA, US, 2008, pp. 173-189.

[43] Pohjola, P., The Pragmatic Web: Some Key Issues. In: A. Paschke, N. Henze, T., Pellegrini and H. Weigand (Eds.), Proceedings of the 6th International Conference on Semantic Systems and the 5th International Conference on Pragmatic Web, ACM, 2010. http://doi.acm.org/10.1145/1839707.1839745

[44] Schoop, M.; de Moor, A.; Dietz, J., The Pragmatic Web: A Manifesto. Communications of ACM 47 (2010) 75-76.

[45] Fiorino, D.J., Citizen participation and environmental risk: A survey of institutional mechanisms. Science, Technology and Human Values 15 (1990) 226-243. 

[46] The United Nations Economic Commission for Latin America and the Caribbean (ECLAC), Science and Technology for Sustainable Development. A Latin American and Caribbean Perspective. Report on the Latin American and Caribbean Regional workshop on Science and Technology for Sustainable Development, 5-8 March 2002, Santiago, Chile. Series Seminars and Conferences No. 24. ECLAC, Santiago, Chile, 2002.

[47] The United Nations Economic Commission for Europe (UNECE), Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters 25 June 1998.

[48] Sandström, V.; Tuomisto, J.T.; Pohjola, M.V., Evaluating effectiveness of open assessments on alternative biofuel sources. Manuscript.