Archived—Industry Canada - Technology Partnerships Canada
h2 Early Adopters Program
Risk Based Audit Framework (RBAF)
Table of Contents
- 1.0 Introduction
- 2.0 Program and Risk Profile
- 3.0 Risk Assessment
- 4.0 Risk-Based Audit Plan
- 5.0 Reporting
The h2 Early Adopters (h2EA) program is aimed at addressing the urgent need to accelerate market adoption of hydrogen technologies and other hydrogen-compatible technologies that facilitate the transition to a hydrogen economy and attract world-class talent and investment to Canada. Through Technology Partnerships Canada (TPC), support will be given to the establishment of integrated hydrogen complexes such as "Hydrogen Villages", "Hydrogen Highways", and other similar partnerships in locations across Canada.
This support will be targeted at multiple public- and private-sector partners to demonstrate these technologies and showcase Canadian capabilities. These partnerships will involve integrating hydrogen compatible technologies and hydrogen production, storage and distribution technologies with fuel cell and related portable, stationary and mobile applications in a microcosm of the hydrogen economy. Results of this initiative will include real-world experience and expertise; early market adoption of hydrogen technologies and infrastructure needed to support their wide-spread use; and increased consumer and investor awareness.
This Risk Based Audit Framework has been prepared in order to identify major risks which exist or may arise during the design, deployment and operation of this program over the five-year period. This framework provides a formal statement of the risks known or likely at the time of its preparation. It will demonstrate that the h2EA program, Technology Partnerships Canada (TPC) and Industry Canada understand and accept the level of risk in the operation of this initiative and that they have measures and strategies in place to identify and address such risks if they should occur.
In addition to this framework, a Results-based Management and Accountability Framework (RMAF) has been developed to identify the appropriate level of measurement, management and reporting of progress in implementation and operation of the program. These two documents are complementary and should be considered as a coordinated response to the need to demonstrate accountability, manage and report on performance, identify risks to objectives achievement and provide appropriate management responses.
1.2 Definition of Risk
The definition of risk found in the Treasury Board Secretariat (TBS) Integrated Risk Management Framework is the following:
Risk refers to the uncertainties that surround future events and outcomes. It is the expression of the likelihood and impact of unplanned events with the potential to influence the achievement of objectives.
1.3 Integrated Risk Management
The current operating environment in the Government of Canada demands a more integrated approach to risk management than has previously been applied. Organizations are faced with many types of risk, and it is no longer sufficient to manage risk on an ad hoc basis. As described by TBS's Integrated Risk Management Guidelines, integrated risk management is:
a continuous, proactive and systematic process to understand, manage and communicate risk from an organization-wide perspective. It is about making strategic decisions that contribute to the achievement of an organization's overall corporate objectives through the monitoring of the program environment and planned delivery.
2.0 Program and Risk Profile
2.1 h2 Early Adopters Program Background
While Canada is currently a world-leader in the development of hydrogen technologies, its first mover advantage is not secure since other countries are making significant new investments toward a hydrogen economy and are attempting to attract Canadian grown technological leadership. In addition to this international competition, Canadian firms face many technology and marketplace hurdles which they cannot overcome alone. With respect to hydrogen technologies, significant headway must be achieved in reducing cost and improving their performance and safety. On the marketplace side, the most urgent need is to begin testing hydrogen technologies and infrastructure in real-world settings to assess and improve reliability and durability, to support efforts to further reduce costs, and to increase public, consumer and investor awareness and acceptance of hydrogen technologies. Implicit in a transition are necessary steps to an ultimate objective. Thus, emphasis is also required on technologies that are hydrogen-compatible.
Given the importance of social/behavioural change in the acceptance of new technologies, increased consumer and investor awareness and understanding of these new technologies and their uses are critical. There are significant technical, social, institutional and market barriers that cannot be overcome without government assistance.
Substantial progress has been and continues to be made through a variety of federal government initiatives in the areas of technology development and partnership activities, however greater collaboration and coordination is required to meet the next strategic objective - high profile, multiple partner market demonstration projects.
The h2EA program will be managed by Technology Partnerships Canada. However, the program has its own separate Terms and Conditions and is distinct in that it includes contributions to coalitions.
The h2EA program will support public and private-sector partnerships that showcase hydrogen complexes, for example "Hydrogen Villages" and "Hydrogen Highways", to demonstrate hydrogen technologies, illustrate their integration and accelerate market acceptance and take-up. Through demonstration projects, we will learn by doing: lessons learned from both successes and failures will advance the science and pace of technological development in this field. The historical experience has shown that these demonstration projects do not allow firms to rapidly realize main revenue streams, but rather help them to pursue further research and development on next generation technologies. Pursuant to the Treasury Board Policy on Transfer Payments, though contributions under the h2EA program are repayable, little or no repayments are expected.
The h2EA program will be implemented by Industry Canada through Technology Partnerships Canada (TPC) in conjunction with the regional development agencies and other federal departments and agencies and their respective programs. Memoranda of understanding may be developed between TPC and federal departments and agencies to support the implementation of this initiative.
During the period FY2003-04 to FY2007-08, funding will be made available by TPC to the Canadian Transportation Fuel Cell Alliance (CTFCA) to allow it to strengthen or extend its ability to meet the objectives of the h2EA program.
Regional development agencies will be invited to participate in the building of partnership coalitions, considering and monitoring the projects, and communicating the successes of regional initiatives. Other federal departments and their respective programs will also be invited to participate, where their program objectives complement those of the h2EA program.
2.2 Roles and Responsibilities
Consistent with the Treasury Board Policy on Transfer Payments, TPC management will be responsible for determining whether recipients under the h2EA program have complied with the terms and conditions of contribution agreements. Audits of recipients will be undertaken when deemed necessary, and will be risk-based in accordance with this audit framework.
Program Management at TPC is responsible for the ongoing financial and operational monitoring of the program. They are responsible for the audit of recipient's compliance to terms and conditions of the contribution agreements and the reliability of results data.
Regional Development Agencies (RDAs) will assist with the delivery of the program.
Audit and Evaluation Branch
The Audit and Evaluation Branch (Internal Audit) employs a risk-based approach to planning and conducting audits. These audits provide assurance on the adequacy of integrated risk management practices, management control frameworks and information used for decision-making and reporting in the achievements of overall program objectives.
2.3 External and Internal Environment
There are key internal and external factors and risks that will influence implementation of the h2EA program.
External factors that have been identified as potential risks include:
- Political - ongoing political support, unexpected changes in relationships between federal and provincial partners
- Economic - impact of the state of economic activity on the demand for hydrogen and fuel cell technologies and the willingness of developers and suppliers to participate
- Social - levels of social interest in and acceptance of development of the hydrogen economy in achieving social and public good objectives
- Technological - the pace of technological change and the ability of the program to keep up with change; rate of commercialization necessary to maintain a viable industry
- International - changes in international relationships that may affect the willingness of foreign research organizations to collaborate with Canadians
Internally, the following factors are considered relevant to the h2EA program's risk profile:
- overall management approach;
- governance and accountability structures;
- values and ethics;
- operational work environment;
- corporate risk management culture and tolerances;
- existing risk management expertise and practices;
- human resources capacity, and,
- corporate policies, procedures and processes.
These external and internal factors have been given careful consideration in the development of this RBAF.
3.0 Risk Assessment
3.1 Overall Risk Assessment
The following is a summary of the main risks identified in conjunction with deployment and implementation of the h2EA program.
3.1.1 Strategic Risks
These are defined as risks outside the control of the program that can have a significant impact on the overall success of the project.
- Economic: Changes to the Canadian economy could affect either demand for research or the ability of firms to deliver the required services.
- Political: Political changes could affect support for the Hydrogen Initiative or for the h2EA program at the federal or provincial government level.
- Social: Changing attitudes towards and scepticism of hydrogen technology and other non-polluting sources of energy may affect social acceptance of the technology being demonstrated through the h2EA program.
- International: Changes in the international economy, relationships, research environment or investment by foreign governments in hydrogen technology may affect the amount of investment to develop the hydrogen economy in Canada and in other countries - risk that investments by other countries may draw technical expertise and commercial interest away from Canada.
A number of specific risks associated with the technology, the management, design, and deployment of the h2EA program are discussed in the following sections.
3.1.2 Technological Risks
The h2EA program targets integrated models of hydrogen systems - these are considered pre-commercial but later stage demonstration. There is a risk Canadian firms' technology is not ready for this advanced stage of demonstration and program uptake may not be as strong in early years of the program.
3.1.3 Project Level Risks
Canadian hydrogen companies are currently demonstrating technologies and systems mainly abroad and currently have 2-3 years of production of systems planned and committed. Given their current limited production capacity, Canadian program will be competing with foreign jurisdictions for projects and may not get sufficient uptake on the program if its Terms and Conditions are not as or more attractive as those other jurisdictions.
The mandate of the program will require strong partnerships within the industry to deliver an "integrated model of a hydrogen system, such as "hydrogen villages" or "hydrogen highway". The program will more than likely fund one lead organization to partner with other organizations. There is a risk that the program does not receive applications for the types of projects it is targeting or that partnerships that are developed are not maintained over the project period.
In addition, there are risks related to liability. The way the project is designed, where a contract may be solely with one lead organization, TPC will not be able to easily facilitate both financial and technical liability issues between the lead contractor and partners which may lead to non-completion of projects.
The h2EA program plans to establish a firewall between the h2EA program and TPC base program. There is a risk that this firewall is not effective and the lines become blurred between the two programs causing issues with clients on the "fairness" of Terms and Conditions of TPC and of the h2EA program which has a much higher contribution rate.
3.2 Approach to Risk Mitigation
A number of risk areas have been identified above as potentially affecting the success of the h2EA program. With active monitoring, planning and auditing, most of these risks can be adequately dealt with. A number of the key performance indicators in the RMAF report are closely related to the risks identified and the Departmental Performance Report will contribute to the active monitoring of these risks.
In the following sections, Tables 1 and 2 are general descriptions of risk factors, their effects and the required management response. They are used as the basis of assessing the risks listed in Table 3, which describes the risk areas, the likelihood of occurrence and level of impact, the results of the risk, and the response strategy.
Risk likelihood is classified as:
- medium; or,
Risk likelihood depends on the probability of the risk occurring over the period of the initiative.
Risk impact is classified as minor, moderate or severe. These are described in more detail below.
|Table 1 - Risk Impact|
|Causes disruption to specific areas of the program, and potential setback.||Causes disruption to some essential program elements, negative image and media attention, and probable delivery and/or operational setback.||Significant disruption to the overall program, underachievement of objectives, criticism of the initiative and the department.|
Relationship between Risk and Required Management Action
Table 2 outlines the relationship between risk and required management action.
|Table 2 - Relationship between Risk and Required Management Action|
|Low (L)||Medium (M)||High (H)|
|Severe (S)||Considerable monitoring and management required||Must monitor and manage risks||Extensive management effort essential|
|Moderate (Mod)||Risks can be accepted with monitoring||Management effort required||Management effort required|
|Minor (M)||Accept risks||Accept, but monitor risks||Monitor and manage risks|
Matrix of Risks, Likelihood, Impact, Effects and Response Measures
The matrix in Table 3 outlines the key risks and describes the likelihood, impact, effects and response measures.
|Table 3 - Matrix of Risks, Likelihood, Impact, Effects and Response Measures|
(Minor, Moderate, Severe)
(Low, Medium, High)
|Changing attitudes towards and scepticism of hydrogen technology and other non-polluting sources of energy may affect acceptance||Severe||Low||Lower market acceptance of technology, progress negated by entrenched attitudes||Monitor and adjust communication strategy, activities as necessary|
|Changes in the international economy, relationships, research environment or investment by foreign governments in hydrogen technology may affect the amount of investment to develop the hydrogen economy in Canada and in other countries and the level of competition faced by Canadian Firms||Severe||Medium||Other countries move faster than Canada, Canadian capacity committed to foreign projects||Implement program quickly
Establish cooperative arrangements with other countries, particularly the U.S.
|Technological performance does not meet expectations||Moderate||Low||Technology is new and, in some cases, untried||Program staff will perform systematic review and assessment before decisions are made on subsequent stages and investments|
|Canadian firms' technology is not ready for this advanced stage of demonstration||Moderate||Low||Program uptake may not be as strong in early years of the program; failure of technological integration; failure to receive comprehensive proposal; failure to bring the different players to work together||Involve co-delivers, stakeholders and users in implementation and deployment of program
Monitor program delivery and program uptake
|Canadian firms not able to come together to form coalitions (e.g., compete negatively)||Severe||Low||Failure to receive comprehensive proposal; failure to bring the different players to work together||Monitor program delivery and program uptake|
|Coalitions not able to manage project for full term, coalitions dissolves||Severe||Low||Failure of financial viability of one of the player, project delayed or terminated||Due diligence on potential recipients. Monitor program delivery closely|
|Recipients do not comply with Terms and Conditions||Moderate||Low||Improper payments, requirement to recover funds, loss of confidence in program and TPC||Due diligence on potential recipients, Ts&Cs detail conditions for receipt and audit of expenses. Ts&Cs can be amended by mutual consent to reflect unexpected circumstances, monitoring provisions to review recipient performance and expenditures|
|Inability to obtain relevant information to measure program's performance and achievement of outcomes||Moderate||Medium||Lack of accountability, loss of confidence by partners and stakeholders||RMAF identifies indicators for required information
Report on critical performance indicators annually
|Inadequate capacity to deliver the program||Severe||Low||Particularly a risk in the implementation of the program||Re-allocate human resources to this program.
Monitor program delivery
4.0 Risk-Based Audit Plan
This risk based audit plan has a number of objectives. It is intended to ensure that:
- appropriate diligence is exercised with respect to the expenditure of public funds;
- the program is administered in accordance with the Terms and Conditions in the funding agreement;
- relevant legislation and policy of the Financial Administration Act are respected; and,
- the quality of information used by the program, TPC and Industry Canada to monitor and manage the initiative are relevant and available for decision-making purposes.
This risk based audit plan will identify a number of strategies to manage the risks identified in the previous section. These include audits of management practices, operational controls, and contribution agreements, as well as performance monitoring and reporting identified in the companion Results-based Management and Accountability Framework.
This framework will cover the above issues with regards to the management of the h2EA program funding and will also provide the basis for audits of management practices and operational controls. This will give management the ability to ensure the proper management of contributions as well as ensure the proper management of the overall program.
4.2 Risk Based Audit Plan
The risk based audit plan will cover all RBAF objectives associated with transfer of funds, and will also examine the need to audit program and financial controls. This provides the ability to ensure that the funds provided to TPC to deliver the h2EA program are well managed and that the overall delivery of the initiative is well managed by both Industry Canada and TPC.
TPC has certain tools and techniques such as the risk rating scale, detailed guidelines for risk rating and a set of procedures for risk assessments to enable identification of risks, assess the impact of risks as well as to monitor risk migration throughout the project's life cycle - all of which contributes towards accountability and stewardship of public funds.
Program management of TPC will perform monitoring related to the h2EA program in three areas:
- the external and internal environment;
- the indicators set out in the RMAF; and,
- the operational efficiency and effectiveness control process.
Monitoring, in order to reduce risk, will also be conducted on a routine basis as part of standard project management practices in the areas of:
- achievement of expected outcomes;
- threats to achievement of expected outcomes;
- due diligence in the expenditures of funds; and,
- the efficient, effective and economical use of resources.
4.2.2 Recipient Audit
TPC will adopt a risk-based approach to the selection of contributions for audit. The audit risk of each contribution agreement will be assessed annually. The intent is to audit contributions where TPC has identified some particular concerns. In addition, audit selection criteria will be used to help determine the purpose, scope and timing of contribution audits. TPC will undertake to coordinate its audit plans with other government organizations at the federal and/or provincial level, where these are involved in funding the same projects, and will give due consideration to audits of recipients performed by other government organizations. The h2EA program will require auditors to agree that contribution audits will be conducted according to generally accepted auditing standards.
4.2.3 Internal Audit
Audit and Evaluation Branch is responsible for the internal audit of the administration of the h2EA program. The h2EA program will be included in the annual audit universe risk assessment, and will be included in the audit plan based on relative risk.
In its audit of the contribution program, Internal Audit will assess the adequacy of program and financial controls to identify and assess risks over the selection, approval, payment and review of eligible projects or activities and the program's operations: the propriety of transactions; general compliance with Terms and Conditions; the economy, efficiency and administrative effectiveness of program operations and delivery systems; and, any other requirements stipulated under the Treasury Board's Policy on Transfer Payments, the most current version of the Treasury Board publication Guide on the Audit of Federal Contributions, or specific to the Terms and Conditions of the program. Risks to be considered during the audit of the management of the program include: the dollar value of projects; a project's public profile, etc.
The h2EA program will rely on the following reporting strategy:
- The Internal Audit Report will be available as required. Internal audit reports are tabled with the Departmental Audit and Evaluation Committee.
- For each project, the funding recipient will provide an annual report on progress achieved during that fiscal year within one hundred and twenty days following the end of each fiscal year. This annual report shall contain the information listed in Schedule C of each contribution agreement.
- Performance information will also be provided through the Industry Canada reporting progresses for the Reports on Plans and Priorities (RPP) and the Departmental Performance Report (DPR).
- The program's annual progress will be included in TPC's Year In Review.
- The formative evaluation report, to be available in fiscal year 2005, will be submitted to the Department's Audit and Evaluation Committee, to TBS, and an executive summary will be posted on the Department's web site. The full evaluation report will be a public document and, as such, accessible to others upon request.
- The summative evaluation report, to be available in fiscal year 2008, will also be submitted to the Department's Audit and Evaluation Committee, to TBS, and an executive summary will be posted on the Department's web site. The full evaluation report will be a public document and, as such, accessible to others upon request.
- Date modified: