The objective of this audit was to assess the overall procurement management control framework in place to support procurement activities, the compliance of procurement with the suite of government contracting policies, and the efficiency of the procurement function. The scope of the audit covered the period of April 1, 2022, to September 30, 2024.

With increased media attention and public scrutiny, it is important to have assurance that the Natural Sciences and Engineering Research Council (NSERC) and the Social Sciences and Humanities Council (SSHRC) procurement processes are fair, open, transparent, and compliant with Government of Canada policies, and that ensure public funds are spent prudently.

NSERC and SSHRC have established and implemented processes and activities that support the effective management of procurement and to comply with the Treasury Board policies and directives. Some of the processes working well include:

• Communicating and implementing Government of Canada policies and any changes;
• Ensuring business owners with signing authority have completed the mandatory training;
• Approving contracts by the appropriate delegated authority; and
• Testing of internal controls over financial reporting in procurement and accounts payable function.

The audit also identified opportunities for improvement regarding:

• Finalizing and implementing the procurement management framework;
• Ensuring effective document storage and retention strategies;
• Designing and implementing a monitoring and reporting system; and
• Ensuring accurate and complete disclosure of contracts greater than $10,000.

The finalization and implementation of the procurement management framework will enhance the maturity of the procurement function and improve efficiencies in the contracting process.

1. The Vice-President, Common Administrative Services Directorate should finalize, approve, and communicate the procurement management framework, update other key internal documents, and develop an action plan to ensure its effective and consistent implementation.
2. The Vice-President, Common Administrative Services Directorate should implement processes to ensure all documents that support procurement decisions, demonstrate value for money, and confirm the integrity of procurement activities are maintained and easily retrievable.
3. The Vice-President, Common Administrative Services Directorate should design and implement a monitoring and reporting system that:
4. Collects, analyzes, and reports key data to support decisions, identifies trends and opportunities for improvement;
5. Monitors for fraud; and
6. Evaluates and reports vendor performance.

These requirements should be part of the procurement management framework.

1. The Vice-President, Common Administrative Services Directorate should establish additional controls to ensure the accuracy and completeness of proactive disclosure reporting.

This audit was conducted in conformance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing, and as supported by the results of the Quality Assurance and Improvement Program.

Toran Karimi
Chief Audit Executive

Internal Audit Principal: Paapa Abekah

Lead Auditor: Mathieu Turriff

Senior Internal Auditor: Daniel Murphy

Senior Internal Auditor: Samir Harrabi

The Natural Sciences and Engineering Research Council (NSERC) and the Social Sciences and Humanities Research Council (SSHRC), hereby referred to as the Agencies, promote and support research and research training within their specific fields through grants, fellowships, and scholarships. The Agencies report to Parliament through the Minister of Industry and Minister responsible for Canada Economic Development for Quebec Regions.

The Agencies share a Common Administrative Services Directorate (CASD), which is responsible for the Finance division, where the Procurement, Contract and Materiel Management (PCMM) division resides. The PCMM division manages the procurement of the Agencies’ goods and services such as, IT equipment, office supplies, furniture, and professional consulting and contractor services. The last time an audit was completed on procurement at the Agencies was in 2013.

Procurement activities are essential in ensuring the Agencies achieve their objectives. During the audit scope period, there were over 1,000 contracts with a total value of approximately $30.6 million.

Procurement in the Government of Canada is governed by a suite of instruments that includes legislation, policies, regulations and directives. Some of these instruments include the Treasury Board Directive on the Management of Procurement, Directive on Delegation of Spending and Financial Authorities, and Guide to the Proactive Disclosure of Contracts.

Prior to April 2023, the Agencies used FreeBalance to manage financial transactions, business processes, and data, including the procurement activities of the PCMM. SAP (also known as SAP-IRIS) is the Agencies’ new financial system effective April 1, 2023, for which the Treasury Board Secretariat (TBS) is the service provider. SAP-IRIS consists of the SAP software for financial and salary transactions, SAP Web Intelligence (WebI) for financial reports, and the associated technical infrastructure.

SAP provides the Agencies with a suite of new options to help their financial monitoring and business processes as well as additional internal controls. A financial system migration project was completed to transition data from FreeBalance to SAP and evaluate the post-implementation business benefits. This change falls in the middle of our audit period and was considered when defining the scope and criteria.

Since the implementation of SAP, the procurement team has been actively refining their controls and processes. The context of procurement has also changed in the past year, with increased scrutiny from the public and from Parliament, which has added extra pressure and required more controls.

Given the increased attention surrounding government procurement, the President of Treasury Board announced that the Office of the Comptroller General (OCG) will conduct a horizontal audit to assess governance, decision-making, and controls associated with professional service contracts. The Agencies were not included in the scope of this horizontal audit by the OCG, however, for those departments and agencies not in scope, the OCG requested that chief audit executives include a review of procurement internal controls and governance in their organization’s risk-based audit plan if a review had not been completed in the past 12 months.

The authority for this audit is derived from the NSERC and SSHRC’s approved 2024-25 to 2026-27 risk-based audit plan, which was recommended by the Independent Audit Committee and approved by both NSERC and SSHRC presidents on July 15, 2024.

The objective of this audit was to assess the overall procurement management control framework in place to support procurement activities, the compliance of procurement with the suite of government contracting policies, and the efficiency of the procurement function.

The scope of the audit covered the period of April 1, 2022, to September 30, 2024, and assessed whether the Agencies have an appropriate and effective management control framework in place to support contracting and procurement activities. This covered:

• Management of the contracting and procurement function.
• Proper consideration of risks throughout the contracting and procurement cycle.
• Internal controls within the contracting and procurement processes including compliance with relevant policies and procedures.

The audit did not include acquisition cards, employee reimbursements, travel and hospitality expenses, contracts under Shared Services Canada or Public Services and Procurement Canada authority (ex. Tri-Agency Grants Management System Solution contracts), nor Memorandum of Understandings with other governmental departments.

The methodology used to conduct the audit of procurement was as follows:

• Conducted 20 interviews with Agency staff in procurement, finance, and other key stakeholders.
• Review of Agency documentation, guidelines, procedures, and relevant policy instruments and legislation.
• Conducted walkthroughs of key controls and processes.
• Reviewed and tested a sample of 50 procurement files.
• Conducted data analysis to identify trends and to support the sample of files to be tested.

Over the course of the audit, the internal audit team observed key areas within PCMM that are working well and contributing to the enhancement of its maturity. These are as follows:

• The Agencies have established two new oversight committees to establish a formal procurement review and challenge mechanisms.
• The Agencies have established an annual procurement planning exercise.
• The Agencies have developed a Procurement Strategy for Indigenous Business (PSIB).

• Business owners and procurement officers are required to complete the Delegation of Financial Signing Authority (DFSA) training prior to obtaining the delegated authority and are required to re-certify every five years.
• Timely communication of key changes to roles, responsibilities and accountabilities to key stakeholders.
• The procurement team is experienced, knowledgeable, and provide guidance to relevant stakeholders when required.

• Implementation of transaction and entity-level controls.
• Implementation of a quality assurance checklist to ensure key steps are completed during the procurement process.
• Procurement officers ensure the use of appropriate procurement vehicles and use of PSPC tools and procurement vehicles.
• Periodic testing of internal controls over financial reporting in procurement and accounts payable function.

While the Agencies have implemented processes to ensure compliance with Government of Canada policies and directives, there are opportunities for improvement regarding the documentation and publishing of current processes that are available to business owners.

The Treasury Board Directive on the Management of Procurement requires departments and agencies to establish, implement, and maintain a procurement management framework, which consists of processes, systems, and controls. The objective of the directive is that the procurement of goods, services, and construction obtains the necessary assets and services that support the delivery of programs and services to Canadians, while ensuring best value to the Crown.

The Agencies have established and documented a clear structure for the management of procurement. A two-tier committee system was also created and approved in June 2024 to establish and maintain a formal procurement review and challenge mechanism. Procurement planning for professional services has been integrated into existing senior management committees.

The Contract Review Committee (tier I) was created to ensure that contracting activities are conducted in a fair, open, and transparent manner, and consistent with relevant regulations, policies, directives and best practices. The Strategic Procurement Review Committee (tier II) was created to perform a strategic review of the Agencies’ annual procurement planning and high-risk procurement activities to establish linkages to the government’s objectives in terms of procurement.

The governance committees were established recently, and as a result, the audit could not evaluate whether they were effective and provided adequate oversight. However, the audit did ascertain that these committees have established terms of reference that outline the roles, responsibilities, accountabilities, mandates and membership.

The Agencies have a variety of documents available on the intranet to employees. These include the NSERC-SSHRC contracting policy, process for submitting procurement requests to the procurement office, guidelines to evaluators to evaluate bids, among others. Treasury Board directives, policies, and guidance are also available through links on the NSERC and SSHRC intranet.

The NSERC-SSHRC contracting policy was last updated in 2015 and is intended to communicate statutory, regulatory and policy frameworks to employees to effectively support the procurement function.

Since the development and implementation of this internal policy, Treasury Board rescinded their Contracting Policy in May 2021 and replaced it with the Directive on the Management of Procurement (“the Directive”). There have also been additional significant changes to this Directive, such as mandatory procedures for contracts awarded to Indigenous Business (April 2022), the integration of human rights, the environment, social and corporate governance in procurement (April 2023), new regulations for the proactive publication of contracts (June 2023) as well as new regulations regarding values and ethics and professional services (May 2024). When these critical changes have been made to the Directive, PCMM communicated these by email, and through presentations to management and senior management committees. Forms and templates have also been updated to reflect new requirements.

Sections of the Agencies’ contracting policy with notable differences with the Treasury Board Directive on the Management of Procurement are as follows:

• Working with former public servants
• Contracting limits
• Call-ups against standing offers and supply arrangements
• Ethics and professional services
• Termination clauses
• Indigenous contracting
• Proactive disclosures

While the NSERC-SSHRC contracting policy document does not fully reflect the Directive, procurement processes and activities generally comply with the requirements of the Directive.

Outdated policies or other guidance materials increase the risk of inconsistency and non-compliance with Treasury Board directives and agencies’ internal processes, which can lead to operational inefficiencies as it increases reliance on procurement staff to provide guidance on a case-by-case basis. The lack of timely updates may result in confusion, unclear expectations and a lack of accountability.

The Agencies developed a Procurement Management Framework in July 2024, however, it has not been finalized, approved, communicated, or implemented.

This draft framework is based on the Treasury Board’s Guide to Establishing a Procurement Management Framework. It includes detailed activities and outputs for seven major components: organizational alignment and structure, governance, workforce capacity, risk management, service delivery, continuous learning and innovation, as well as results monitoring and reporting. It has been linked to guidance documents, such as the manager’s guide to procuring professional services and the internal contracting policy, which have documented roles, responsibilities and accountabilities.

The Agencies have made significant progress to develop the draft procurement management framework, however, it has not been finalized, communicated, or implemented. While some elements are already functioning, the lack of its finalization and implementation presents several risks, such as inefficiencies, compliance issues and fragmented processes. This could adversely affect resource allocation to critical areas, potentially compromising the effectiveness of procurement operations to manage risks and meet objectives.

The Agencies have controls in place for the delegation of spending authority for contracting authorities and project authorities. Prior to obtaining their section 41 authority or section 32 authority, all procurement officers and business owners must take the in-house DFSA training, as required by the Agencies. There are also further mandatory trainings developed by Treasury Board and administered through the Canada School of Public Service (CSPS) which must be completed by procurement officers. Examples include Procurement Fundamentals and Indigenous Considerations in Procurement.

Position incumbents must also successfully complete the DFSA training provided by CSPS and revalidate every five years, prior to obtaining section 32 authorities. Prior to assigning financial delegations, the human resources division receives a list of participants for the internal DFSA training from Finance. Also, as part of the performance management program, the human resources team provides monitoring reports to directors that include their divisions’ completion rates and employee lists. Once all mandatory DFSA training is completed, MyGCHR is updated.

The Agencies have also developed an internal training for procurement that is optional and provided quarterly, and whose information deck is available via the Agencies’ intranet page. This training presents the basics for the procurement process at the Agencies and is updated as important changes are made to the policy suite.

It should be noted that training available through CSPS or the Agencies do not inform attendees on procurement fraud risks. Procurement officers and business owners are essential to protecting the Agencies from potential procurement fraud. By equipping employees with the knowledge on common procurement fraud schemes and red flags, they can identify any potential wrongdoing and escalate, as needed.

The Vice-President, Common Administrative Services Directorate should finalize, approve, and communicate the procurement management framework, update other key internal documents, and develop an action plan to ensure its effective and consistent implementation.

Management agrees with the recommendation.

Action Plan:

• Finalize and approve the draft Procurement Management Framework.
• Communicate the finalized framework to staff and integrate into the procurement intranet page.
• Update and streamline key internal policies and guidance documents (e.g., contracting policy, templates, procedures) to align with Treasury Board requirements without duplicating them.
• Develop a phased implementation plan including milestones, performance metrics and risk mitigation measures.
• Launch an internal communication and engagement campaign to promote awareness and adoption of the PMF.
• Improve the current "Procurement 101" training course to include elements from the internal policy and the PMF for delivery to procurement officers and purchasing managers.
• Provide ongoing support and refresher sessions periodically to ensure consistent application of policies and controls.
• Establish a feedback process to refine the framework and training materials based on lessons learned and user input.

Processes have been developed and are generally functioning, however there is an opportunity for improvement in document management and retention to ensure completeness and availability of all key documents to support decisions.

Some procurement activities require that the Agencies engage with Public Services and Procurement Canada (PSPC) for service contracts with an expected value of over $3.75M, goods contracts over $25,000 and for certain commodities, and to engage Shared Services Canada (SSC) for contracting of IT software and hardware. For all other contracts, PCMM has established procedures and controls for the appropriate selection of procurement vehicles that is in alignment with the Treasury Board Directive and is based on multiple factors such as dollar value, type of good or service being procured, and complex clauses in the contract.

PCMM have also developed criteria to assess whether a potential contract is high-risk or high-complexity to ensure that any associated risks are addressed and that more experienced procurement officers are assigned to these cases. Criteria for identifying high-risk or high-complexity contracts include factors such as high dollar values, contracts that may attract public scrutiny, or those that are multi-year or with option years.

Upon receipt of a requisition, contracting authorities verify that the project authority has a valid section 32 authority using a DFSA module implemented within SAP that lists current holders of section 32 authority.

Contract processes that are sole sourced are required to be justified using a Sole Source and Limited Tendering certification document. Competitive contracts go through a process to request bids, evaluate them, and award them to the bidder who best meets the evaluation criteria. Business owners develop the statement of work, which identifies deliverables, timelines, and the evaluation criteria. Contracting authorities use the statement of work to develop the requests for proposals, which are shared through the appropriate channels.

For each request for proposal (RFP), three independent evaluators are selected to score each bid. Each evaluator is expected to sign a conflict-of-interest declaration to attest that they have no potential conflicts, and to complete and document the evaluation independently, which is followed by a consensus meeting. The results of the consensus meeting are documented, and all evaluators are required to sign as evidence of their agreement.

While evaluators for competitive contracts are required to complete conflict of interest declarations, this is not a requirement for sole source contracts. Having business owners also sign conflict of interest declarations for sole source contracts would help enhance transparency and integrity in the procurement process.

As part of the audit, a sample of 50 contracts were selected and tested for procurement planning, bid solicitation, bid evaluation, contract approval and award, contract amendments, and overall contract documentation and management. The results were:

What’s working well:

• All contract files include appropriate contract requisitions.
• All contracts were approved by the appropriate delegated authority.
• All competitive contracts had general clauses and conditions that were shared during bid solicitation.
• Most contract files demonstrate that the appropriate solicitation approach was used.
• Most competitive contract files include signed conflict of interest declarations.
• Most contract files had the same standardized structure for document retention.

Areas for improvement:

• In a few instances, the statement of work and task authorizations did not have clearly defined deliverables.
• In a few instances, contracts were missing cancellation clauses.
• In a few instances, it was unclear whether all bidders were provided with the same information in a timely manner.
• In some instances, evaluators did not have detailed notes in the evaluation sheets.
• In multiple instances, amendments were not always clear in demonstrating their justification.
• In multiple instances, files were missing signed consensus evaluation forms.

Note:

• “Most contract files” represents over 95% of files.
• “Multiple instances” represents between 25% and 35% of files.
• “Some instances” represents between 15% and 25% of files.
• “In a few instances” represents between 5% and 15% of files.

Certain areas of the process such as amendment justification and providing bidders with the same information in a timely manner, had misplaced documentation that was found post-audit.

The Treasury Board Directive on the Management of Procurement requires contracting authorities to ensure that accurate and comprehensive procurement records are created and maintained to facilitate management oversight and audit. It is important to ensure that key documents are retained in the appropriate contract files to support decisions, demonstrate value for money, and confirm that the procurement process was completed with integrity. Lack of these key documents could create challenges for the Agencies to respond to any questions regarding the award of a contract.

This observation is not intended to suggest that this evidence did not exist at the time of contract award, but only that it was not available in the contract files at the time of the audit.

“Key controls” refer to the most essential activities that ensure that transactions are fair, compliant with government policies, and align with the objectives of the Agencies. Throughout the procurement process, management has implemented transactional-level controls and entity-level controls as safeguards ensuring greater integrity of the procurement process. Examples of these entity-level controls include modernization of the procurement process to keep up with the constant and important policy changes, the use of tools to streamline the contracting process and avoid errors, the automation of authorizations to help validate DFSA approval, and a risk management process which evaluates contractual risk level. Another control is that only the team at Treasury Board Secretariat (TBS) can create new vendors in SAP, which represents effective segregation of duties.

Transactional-level controls include an ongoing checklist which is used as a quality assurance tool and to ensure that all relevant documents are on file, a peer review process prior to creating a purchase order and a bid solicitation approval process. These controls have been implemented over time and are updated as the procurement function evolves.

Management has not formally identified which controls are considered key beyond those identified as part of the oversight of financial reporting. By identifying which controls are key, the procurement team can streamline processes and ensure that resources are used efficiently.

The Vice-President, Common Administrative Services Directorate should implement formal processes to ensure all documents that support procurement decisions, demonstrate value for money, and confirm the integrity of procurement activities are maintained and easily retrievable.

Management agrees with the recommendation.

Action Plan:

• Improve upon the document management protocol (checklist) specifying required.
• documentation for each procurement stage and ensure it is verified through peer review.
• Establish a framework for evidence of procurement officer acceptance of business owner confirmations.
• Establish and document file naming conventions for the Enterprise filing structure.
• Provide guidance and training to procurement officers and business owners on document retention expectations (signatures, filing, etc.).
• Conduct periodic quality assurance checks to ensure compliance.

The Agencies do not regularly analyze information or data that could help identify trends, support decision-making, review vendor performance, monitor for potential wrongdoing, or identify opportunities for continuous improvement.

In April 2023, the Agencies transitioned from FreeBalance to SAP and are now part of the SAP-IRIS cluster. The cluster is a group of 25 Government of Canada departments which each use a similar SAP platform, standardized functionality but different configurations. Significant configuration change proposals need to be approved by the cluster, which can create difficulties for the Agencies as their CASD model presents a unique reporting requirement that is not needed by most other departments in the cluster.

At the time of change of financial systems, PCMM had identified 135 contracts to migrate from FreeBalance, and 21 additional contracts to enter in SAP (contracts created close to the go-live date, therefore not entered in FreeBalance). The data were entered manually into SAP by procurement staff.

While the procurement team used Microsoft Excel to record new contracts to be created in SAP at go-live, there was no formal comparison between the ending list of contracts in FreeBalance and the opening list of contracts in SAP-IRIS following the implementation. However, the risk of inaccurate and incomplete data in SAP is low as any error would likely have already been detected on the small number of contracts that were manually entered.

The Finance and Procurement teams will continue to use FreeBalance to retrieve historical data of some contracts until 2028-29, at which time it is expected that FreeBalance software and databases will be decommissioned.

The Agencies have established and communicated service standards with target performance for the time to complete each type of procurement, including contracts where PSPC or SSC need to be engaged. These service standards are updated to reflect any changes to Treasury Board policies and directives that could impact the time needed to complete procurement requests. Establishing and monitoring of service standards can be used to evaluate the efficiency of the procurement function, identify areas for improvement, and support decision-making.

Currently, service standard timelines are defined based on the entire procurement process, however there is an opportunity for service standards to be measured by each phase of the process such as, issuance for the request for proposal, evaluation of bids, requesting and confirming security requirements, or negotiation and signing of a contract. Since the standard is not broken down by phases where subsequent client inputs are needed, or involvement from other divisions such as security, it reduces PCMM’s control and its ability to identify improvement opportunities.

While contracting officers enter key contracting dates, such as request date and contract award date into a shared document on SharePoint, there was no formal monitoring or reporting on these service standards during the scope of the audit. PCMM has indicated that they are in the process of implementing Service Manager (intake software) to manage procurement requests. It is expected that this will help with the reporting of service standards by tracking key dates in the procurement process via an automated system.

Procurement reporting needs and the data and information to support decisions and reporting requirements are in the process of being defined, developed and implemented. Currently, data and reports are not collected and analyzed to detect errors and anomalies or identify red flags or potential wrongdoing; this information is also not collected to document and track vendor performance, which is part of the Treasury Board Directive on the Management of Procurement. Currently, the collection of procurement data and information for reporting purposes is limited and primarily used for proactive disclosure purposes. In instances where complaints are filed by business owners on vendor performance, PCMM will include the information in the contract file, but there is no central repository to keep this information to enhance management of vendor performance and improve the detection of potential wrongdoing.

One of the limitations of SAP-IRIS is that while there are standard reports available through SAP and WebI, there is no SAP report available for CASD contracts as the instance of SAP-IRIS is not able to provide reports coming from two agencies; the Agencies manually combine NSERC and SSHRC reports to create their own CASD reports. Management plans to prepare dashboards by utilizing PowerBI and anticipates that these improvements will be implemented in 2025-26.

The Treasury Board Directive on the Management of Procurement requires, federal departments and agencies, to proactively disclose contract details associated with any contracts with a value equal or greater than $10,000, each quarter. Since the migration to SAP, the Agencies rely on an SAP report specifically tailored for proactive disclosure reporting, which identifies contracts with a value greater than $10,000 and amendments that would bring a contract over this threshold. As part of the review of a sample of contracts, it was identified that approximately 8% of contracts that should have been proactively disclosed had not.

Reasons for this include:

• The report provided by WebI does not always include contracts with new vendors immediately after their creation.
• Errors at the time of uploading the information on the OpenGov website.

It is important that the Agencies review their processes to ensure that disclosures are accurate and complete as it could have a negative impact on the reputation of the Agencies and result in the loss of public trust.

The Vice-President, Common Administrative Services Directorate should design and implement a monitoring and reporting system that:

• Collects, analyzes, and reports key data to support decisions, identify trends and opportunities for improvement;
• Monitors for fraud; and
• Evaluates and reports vendor performance.

These requirements should be part of the procurement management framework.

Management agrees with the recommendation.

Action Plan:

• Define key performance indicators (KPIs) and reporting requirements, and leverage Service Manager for monitoring.
• Leverage SAP-IRIS, WebI, Service Manager, SharePoint and Power BI to create dashboards for contract status, timelines and vendor performance.
• Develop fraud indicators and incorporate vendor performance/complaint tracking within our capacity.
• Provide guidance and training to procurement officers and business owners on fraud detection and prevention.
• Use reports to identify opportunities to monitor risks, flag anomalies and recommend process improvements based on data trends.
• Pilot monitoring tools in Q4 2025-26 and adjust as needed.
• Present dashboards and KPIs quarterly to the DCFO and CFO to inform strategic decisions and support governance.

The Vice-President, Common Administrative Services Directorate should establish additional controls to ensure the accuracy and completeness of proactive disclosure reporting.

Management agrees with the recommendation.

Action Plan:

• Review and improve SAP reporting parameters to better capture new vendors.
• Establish and document a verification process prior to uploading disclosures to OpenGov.
• Introduce quarterly spot checks of published data.
• Train procurement staff on proactive disclosure procedures.

The Agencies have established and implemented processes and activities that support the effective management of procurement and to comply with the Treasury Board policies and directives. Some of the processes working well include:

• Communicating and implementing Government of Canada policies and any changes;
• Ensuring business owners with signing authority have completed the mandatory training;
• Approving contract by the appropriate delegated authority; and
• Testing of internal controls over financial reporting in procurement and accounts payable function.

The audit also identified opportunities for improvement regarding:

• Finalizing and implementing the procurement management framework;
• Ensuring effective document storage and retention strategies;
• Designing a monitoring and reporting system; and
• Ensuring accurate and complete disclosure of contracts.

The finalization and implementation of the procurement management framework will enhance the maturity of the procurement function as well as improve efficiencies in the contracting process.