Improving Pass Rates and Learning Outcomes for IUOE Apprentices: Content and Online Software for an Exam Preparation Tool
REQUEST FOR PROPOSALS – RFP
RFP Issue Date: May 17, 2021
RFP Closing: June 30, 2021
Contracting Manager: Emily Arrowsmith
E-mail: emily@caf-fca.org
INTRODUCTION
About CAF-FCA
The Canadian Apprenticeship Forum-Forum canadien sur l’apprentissage (CAF-FCA) is a national, not-for-profit organization working with stakeholders in all regions of Canada. We influence pan-Canadian apprenticeship strategies through research, discussion and collaboration – sharing insights across trades, across sectors and across the country – to promote apprenticeship as an effective model for training and education. Our Board of Directors is comprised of representatives of business, labour, the jurisdictional apprenticeship authorities, education and equity-seeking groups. Through our work, CAF-FCA has shed light on a number of key issues affecting apprenticeship, such as the perceived barriers to accessing and completing apprenticeship and the business case for apprenticeship training. For more information, visit the CAF-FCA website at www.caf-fca.org.
Apprenticeship in Canada
Skilled trades workers are a highly trained and skilled component of the workforce. Apprenticeship training offers advanced technical skills through a combination of practical, on-the-job work experience and intensive technical instruction. Generally, apprenticeship is an agreement between an apprentice and an employer whereby the apprentice agrees to work for the employer in exchange for extensive, supervised, on-the-job experience. As well, the employer agrees to release the apprentice to attend technical training. In all jurisdictions outside of Quebec, the technical portion of apprenticeship, usually taken at a community college, a union-sponsored training centre or at an accredited private training facility, is taken with alternating periods of on-the-job training. In Quebec, all formal technical training for apprentices is done at the secondary school or college level and must be completed before apprentices can begin the workplace on-the-job training. The nominal duration of an apprenticeship is four years. In Canada, the provinces and territories oversee apprenticeship and there are 13 different systems, all with unique regulations and requirements.
Red Seal Program
The Red Seal Program is the Canadian industry standard of excellence for the skilled trades. Tradespersons who meet the Red Seal standard receive a Red Seal endorsement on their provincial/territorial trade certificates. The program facilitates inter-jurisdictional collaboration in support of developing a certified, highly skilled and mobile trades workforce in Canada. There are currently 50 plus Red Seal trades. Red Seal trades with a significant number of apprentices include electrician, plumber, carpenter, hairstylist, automotive service, cook, welder, heavy-duty equipment mechanic, heavy equipment and crane operator.[1]
Below are the Red Seal standards, essential skills information, practice questions and exam counselling sheets for the trades included in this project:
The Red Seal endorsement is acquired by writing an approved examination and attaining a minimum mark of 70 per cent. Candidates for examination must have acquired the pre-requisite training and work experience for the trade before attempting the final certification examinations.
PURPOSE AND OBJECTIVES
From 2021 to 2025, CAF-FCA will partner with the IUOE to pilot an interactive online tool with 2,000 apprentices in heavy equipment operator and mobile crane operator, to track and report on outcomes and communicate results. CAF-FCA is seeking a consultant who can provide content and access to an online platform to support apprentices. Examples of online platforms may include, but are limited to, D2L, Moodle, etc. The pre-existing intellectual copyright for the online platform remains with the consultant or the IT provider, but the content developed during the project is the intellectual copyright of CAF-FCA.
The project objectives are:
- Objective #1 Improve Pass Rates in the Apprenticeship System: Increase exam pass rates for apprentices, including those with learning disabilities, by providing access to an online tool. Tool content and structure will reflect best practices for accommodating apprentices with disabilities. The tool should be able to help apprentices at anytime throughout their apprenticeship prepare for and pass exams. It may have trade specific elements.
- Objective #2 Gather and Apply Best Practices: Share any positive apprentice outcomes based upon the research data and, if the pilot is successful, encourage other unions and employers to adopt exam preparation best practices.
- Objective #3 Help Apprentices with Disabilities Succeed in Apprenticeship: Help apprentices with disabilities complete their training.
JUSTFICATION
Research justifies the need for this project. Apprentices take multiple tests throughout their technical training. They also must pass any provincial/territorial examinations as a part of their training. As previously mentioned, a certificate is obtained when an apprentice completes the number of required hours/levels for their trade, writes a multiple-choice exam and obtains 70 per cent or higher. According to a 2017 Apprentices in Canada ePanel report, the majority of apprentices feel nervous before exams with 47 per cent saying they were “a little bit nervous,” 24 per cent felt “very nervous” and 15 per cent were “extremely nervous.” In a 2020 ePanel survey, test anxiety was a barrier to completion for 18 per cent of respondents. A high prevalence of learning disabilities and ADHD, which affects executive functioning, the apprentice learning style, which does not align with the multiple-choice exam delivery, difficulties with managing stress and shame associated with asking for help are all barriers for apprentices. Not passing the exam leads to low completion rates. Previous Statistics Canada cohort studies indicate the completion rates are 50 per cent. Based upon CAF-FCA's 2020 LMI research, trade specific completion rates vary from 33 per cent to 66 per cent. ePanel apprentices identified online exam preparation strategies (53 per cent), preparatory workshops (43 per cent) and relaxation techniques (30 per cent) as potentially useful resources.
In-person one day workshops that review studying strategies and how to approach a multiple-choice examination have proven to help apprentices pass the certification examination. However, not every apprentice has the opportunity to attend one of these workshops.
IUOE does provide help when studying for the exam during in-class training, but additional supports and supplemental information may be helpful to apprentices.
Various jurisdictions and colleges also offer learning supports for apprentices and implement examination accommodation policies for apprentices with disabilities, but not all apprentices may know about the supports available. In British Columbia, for example, refresher training is offered for trade qualifiers and apprentices. Industry Training Authority advisors create awareness about available learning supports, especially if the apprentice has already failed the exam.[2]
The Nova Scotia Apprenticeship Agency works with apprentices and identifies support services including refresher courses, tutoring, reader-assisted examinations, translators, extended time, private room and learning plan development. The agency’s website provides information about examination preparation and various available resources and also answers frequently asked questions.[3] Tips for writing multiple-choice examinations, videos including Preparing for the Exam, and links to practice exams and sample questions for some trades are also available on this website. Apprentices with special needs are encouraged to self-identify during the application process in order to access suitable accommodations.[4]
The Ontario government launched the Examination Preparation Supports initiative to encourage more apprentices and trade applicants to complete their training and/or certification programs. After several years of delivery, program evaluation statistics show the Certificate of Qualification exam pass rate for Exam Preparation clients is about 13 per cent higher than for non-Exam Preparation clients.[5]
In the Yukon, labour market programs provide assessments and recommendations for apprentices who are seeking examination accommodations. The Learning Resource Centre personnel at Yukon College oversee any accommodations that are granted. In special circumstances, extra time may be informally provided. Most Yukon apprentices go to Alberta for their technical training. Alberta colleges provide support for Yukon apprentices with diagnosed learning disabilities.[6]
Many colleges also offer support services for apprentice learners. Some examples include alternate format course materials, assistive and adaptive technologies, exam accommodations, interpreter and captioning, learning strategies and a reduced course load.
REQUIREMENTS
CAF-FCA is looking for the videos and modules to be created related to the following subjects:
- Training on How to Use the tool
- Study Plan
- Tackling a Multiple-Choice Test
- Math Help
- Document Use
- Breaking Down the Red Seal Training Standard in order to Tackle the Exam
- Strategies to Reduce Test Anxiety
CAF-FCA is also open to feedback from the consultant about additional valuable modules and videos that could be included in the tool.
The consultant must also provide the online platform for the tool. Over the course of the project, 2,000 apprentices will access the platform. Any fees associated with accessing the platform must be included in the budget.
Tool content will reflect standards which are the basis for provincial/territorial and/or Red Seal exams and will align with industry expectations as shared by the IUOE. The tool will better prepare apprentices to pass the exam the first-time saving time and energy and building learner confidence. In most cases, Level 3 and 4 apprentices will use the tool 6 to 12 months prior to the exam. However, certain parts of the tool may be used by Level 1 and Level 2 apprentices to develop good study habits when preparing for technical training exams and to learn how to manage test anxiety at the beginning of their training. In its development, the tool should consider and address the needs of apprentices with disabilities. The tool should be easy-to-use and accessible in English and French. The tool must be dynamic, interactive, engaging and supportive to learners. CAF-FCA is open to feedback from the consultant about the tool set-up.
Privacy must be respected and data must be secure. Consent forms will be integrated into the tool. Tracking mechanisms will also be integrated into the tool to identify the number of users. Overall pass rates will be tracked. Apprentices will be encouraged to provide feedback about the tool through questionnaires. Best practices and lessons learned will be identified. This information will be provided to CAF-FCA annually and will be used for CAF-FCA’s research report. Apprentices with disabilities will have the option to self-identify. Key research questions include:
- Does the tool improve the first attempt pass rates on examinations?
- Are there any patterns related to hours spent using the tool and impact on pass rates?
- What elements of the tool are particularly helpful to apprentices?
- What are the best practices for helping apprentices pass examinations and the lessons learned?
Any annual or license fees associated with using the tool on an ongoing basis must be communicated to CAF-FCA in the proposal.
DELIVERABLES
- 1 Workplan
- 1 Online tool that is interactive and helps apprentices study for and pass examinations. CAF-FCA is looking for the videos and modules to be created related to the following subjects:
-
- Training on How to Use the tool
- Study Plan
- Tackling a Multiple-Choice Test
- Math Help
- Document Use
- Breaking Down the Red Seal Training Standard in order to Tackle the Exam
- Strategies to Reduce Test Anxiety
- 8 Training sessions at a minimum, but more may be required to reach the target of 2,000 users
- 4 Year data gathering from feedback mechanisms and short questionnaires
- 4 Annual Reports summarizing aggregated data and overall outcomes and best practices and lessons learned for each year of the pilot (5 to 10 pages)
FRAMEWORK AND METHODOLOGY
Proponents are invited to bid individually or as a consortium of bidders. The proponent may submit a proposal for the work in part or as a whole. It is also important to note that the development of an appropriate work plan must consider identified project timelines. The team may involve experts involved in areas such as essential skills, preparing for exams and tackling multiple-choice tests. IT support should be available related to the online platform either through the team or a sub-contract with a firm such as D2L, Moodle or another online software technology.
IUOE will invite apprentices to the use the tool and will provide trade expertise when reviewing the tool content. CAF-FCA will administer the project, collect baseline information about current pass rates using provincial/territorial and/or Red Seal data and will be responsible for writing the research and final project report. CAF-FCA will pay for any graphic design or translation costs.
Here is the timeline for the project:
July to September 2021
- A contract will be signed.
- The consultant will develop the content for the tool.
October to December 2021
- The consultant will present a draft of the tool to the Project Advisory Committee.
- The consultant will listen to the feedback from the Project Advisory Committee and will integrate the feedback into the tool.
January 2022 to March 2022
- The consultant will finalize the tool.
- The content in the tool will be translated from English into French.
- The tool will be prepared to be launched in English and French.
April 2022 to December 2024
- The consultant will offer training sessions to introduce apprentices to the tool. At least 8 training sessions will be hosted, but more may be required.
- The consultant will monitor the use of the tool and will collect data for the annual report.
- Adjustments may be required based upon the results and feedback from the Project Advisory Committee. If the tool is not helping apprentices pass the examination, the reasons why will be reflected upon and improvements will be made to the tool. The goal is to have 2,000 apprentices use the tool.
January 2025 to March 2025
- The consultant will provide any remaining data to CAF-FCA.
April 2025 to June 2025
- CAF-FCA will write a draft report.
- The consultant will be available if CAF-FCA has any follow-up questions.
July 2025 to September 2025
- The Project Advisory Committee will review the report and will make comments.
- The consultant will be available if the Project Advisory Committee has any follow-up questions.
DELIVERABLES
The following deliverables will be expected at various stages of the project:
- Work plan
- The contractor will provide a detailed work plan outlining activities, objectives and timelines for the tool creation, data analysis and annual reporting. All plans are subject to revision.
- Presentations and meeting with Project Manager
- The contractor will meet with the CAF-FCA Project Manager virtually to discuss the work plan and establish a schedule of regular meetings and progress reports. All aspects of communication with respect to the ongoing activities will be discussed.
- The contractor will participate in regular meetings with the CAF-FCA Project Manager to consult on project status, agenda items and any matters requiring resolution.
- The contractor will attend all meetings and calls as required, to be scheduled upon the completion of major milestones such as completion of the draft and final tool. Contractors may be expected to present at these meetings.
- Reports
- The contractor will provide monthly progress reports comparing the actual progress with the work plan, re-adjusting timelines if required and highlighting potential issues, problems and opportunities.
- The contractor will provide 4 Annual Reports summarizing the aggregated data and key outcomes. Any best practices or lessons learned will be documented. The reports may be 5 to 10 pages.
- CAF-FCA will write the research report and the final project report. CAF-FCA reserves the right to make adjustments to reports prior to their publication.
CAF-FCA will be responsible for the translation of the tool content and the reports. CAF-FCA will be responsible for the graphic design of the report.
TIMEFRAME
The contract will be administered from July 2021 to September 2025.
BUDGET
The budget for this project is not to exceed $700,000 (excluding HST). This maximum amount includes all professional fees including any expenses associated with travel and the project management of the consulting team. CAF-FCA will sign a firm, fixed price contract that is all-inclusive. Proponents are asked to provide a line-by-line detailed budget in their proposal.
ROLES, RESPONSIBILITIES AND REPORTING STRUCTURE
Contractor
- Develops and delivers online tool
- Reports to the Project Manager
- Liaises and works closely with the Project Manager throughout the course of the project and immediately reports any problems that may alter the timing or scope of the project
Project Manager
- Coordinates with the contractor
- Ensures the contractor completes the work as specified in the terms of reference
- Reports to the Project Advisory Committee, the Research Committee, the Board of Directors and the Executive Director during the course of the project
- Oversees all logistics for meetings
- Provides minutes from meetings
- Maintains all contractual reporting requirements with respect to the Contribution Agreement
- Provides the overall guidance and direction of the research to the contractors throughout the life of the project
- Approves the project work plan, interim and final reports
- Provides feedback and input on the reports
- Writes research report and final project report
- Oversees graphic design and translation of materials and reports
INTELLECTUAL PROPERTY
The pre-existing intellectual copyright for modifying a tool or for the online software remains with the consultant, but the content developed during the project is the intellectual copyright of CAF-FCA and its partners. Copyright includes all tool content, videos, raw data and reports, including all drafts and interim reports produced and it will rest with CAF-FCA. The tool content, data or insights provided for CAF-FCA’s project will not be used for another purpose or shared with another client without CAF-FCA’s express written permission.
PROPOSAL REQUIREMENTS
The successful bidder will be chosen on the basis of selection criteria determined by CAF-FCA.
Bidders must be able to function in both English and French since the proposed project is national in scope.
The successful bidder will include in its team members that can demonstrate direct experience in projects of similar size and scope. The team should be composed in such a way to maintain consistency among members responsible for the tool.
A short-list of contractors may be asked to answer questions via telephone before a final selection of the contractor is made.
The proposal must include the following:
- A maximum of 25 pages in length, using a twelve-point font size
- No additional appendices may be submitted
- The proposal must have an executive summary of no more than 3 pages. The executive summary will present highlights of the proposal and include references to those pages in the text of the proposal on which further information can be obtained.
- A detailed description of the intended approach. An explanation of how the project objectives will be met through the application of the recommended approach.
- A detailed work plan which clearly highlights activities intended and timeframes associated with said activities.
- A detailed line-by-line budget estimate associated with each stage of the work
- Total number of person-days each contractor employee will contribute to the project and the corresponding rates for each member of the consulting team
- Professional roles and activities of each individual.
- Identification of the management structure within the consulting team, including any planned sub-contractors.
- Communication strategies with respect to the Project Manager.
- A brief profile of the organization including:
- the years in business, location and strategic focus of the bidding organization,
- demonstrated experience in working in English and French,
- the names and résumés of all personnel to be involved in the project, and
- references to similar projects completed by the bidding organization.
Bidders are asked to email an electronic file suitable for electronic distribution to
Emily Arrowsmith, Project Manager/Researcher, at emily@caf-fca.org. Proposals are to be submitted on or before 11 a.m. (Eastern Daylight Savings Time) on June 30, 2021.
EVALUATION CRITERIA
Proposals will be evaluated based upon the proven ability of the bidders to satisfy the requirements of this RFP in a cost-effective manner. Specific evaluation criteria are:
- Technical knowledge and experience/Financial 100 points
- Work plan and schedule 40 points
- Team profile and experience 60 points
Proposals will be evaluated and scored based on quality of response to the requirements of this RFP. Selection of the preferred vendor will be based on the highest score. Proposals must achieve a minimum of 75 per cent in each of the categories. Despite the above, in the event that no proposal meets these scoring minimums, CAF-FCA reserves the right to award a contract to the bidder submitting the highest scoring proposal. CAF-FCA also reserves the right not to award the contract if a suitable vendor cannot be identified.
[2] Feedback from provinces for CAF-FCA,
Apprentice Attitudes Towards Learning and Examinations, (Ottawa: CAF-FCA, 2017).
[4] Feedback from provinces for CAF-FCA,
Apprentice Attitudes Towards Learning and Examinations, (Ottawa: CAF-FCA, 2017).
