Analysis provides the core information that is essential in establishing programs that are job related. There are several different types of analyses that each have value when applied appropriately to reflect the complexity and risk associated with job performance. Analysis is used in program development, program revision, and program improvement. Analysis is cost-effective in the long term because it ensures that training resources are used effectively (only job-related tasks are identified and included in the training program).
Factors specific to analysis to consider when selecting an analysis technique include:
These factors supplement the considerations in Appendix A. If the availability of SMEs is limited, a verification analysis (if a task list is available) or an analysis of procedures and other job related documents should be conducted. Lacking the aforementioned resources, job analysis should be conducted using either table-top or traditional techniques. In practice, a combination of table-top, verification, and document analysis techniques has generally proven to be most efficient.
When a performance deficiency is being addressed, a needs analysis defines deficiencies or problems and identifies causes and solutions. Needs analysis is a systematic search for the deficiencies between actual and desired job performance and the factors that prevent desired job performance.
A needs analysis should be performed whenever new requirements are issued, when job performance is below standards, and when requests for changes to current training or for new training are received. A needs analysis helps assure training is the appropriate solution, identifies the training issue(s), and ensures that instruction on the topic is not already included in an existing training program.
Determination of the cause of performance deficiencies and potential solutions is the primary purpose of needs analysis. A needs analysis which addresses a specific performance problem helps management determine whether or not the problem can be resolved by training or if non-training issues must be addressed. Job aids, revised procedures, increased management attention, etc., may be solutions that do not require training. Depending on the application, needs analysis should include:
Needs analysis provides documentation of the determination of needs and recommended solutions. If training is needed, the needs analysis should provide the following information:
When a position that performs a large number of tasks (e.g., management or engineering) is being analyzed, a technique called functional analysis can be used. Rather than conducting a job analysis to identify specific tasks, major functions within the position are identified. After the competencies necessary to perform the major functions are identified, those competencies can be analyzed to determine objectives for training. Functional analysis can be conducted using the techniques described in this handbook.
Job analysis is a process which develops a list of tasks for a specific job or position. Job analysis provides reasonable assurance that tasks essential to safe operation are identified for training. It also identifies tasks that are vital to facility operation and tasks that are of lesser consequence. This information helps management assign resources where they are most needed.
Job analysis involves developing a task list, selecting tasks for training and determining the appropriate level (train and/or overtrain), and validating the results. There are many approaches to accomplishing these three steps. Some techniques (such as those described in DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities) can be relatively elaborate, and while they may provide a somewhat higher level of confidence in the validity of their results, these techniques are both time consuming and expensive. Highly formal techniques should only be used for jobs that are exceptionally hazardous and when other less expensive approaches will not work because of complexity of the task or absence of reference sources (e.g., SMEs, procedures, and other technical documentation). More efficient and less complex techniques (e.g. table-top job analysis) should be used whenever possible. Use of the table-top job analysis technique will generate acceptable results in a short period of time. As a general rule, the lower the impact improper performance of a job has on environmental, safety, and economic factors, the less elaborate the analysis technique should be.
The following are examples of documents that should be reviewed to determine job scope and/or performance requirements when conducting job analysis.
Task analysis is a process of examining each task, one at a time, to identify the task elements or action steps, knowledge, and skills. Task analysis is conducted after the task list is completed and tasks have been selected for training. It is not necessary to identify task elements or action steps, knowledge, and skills for those tasks that do not require training. Task analysis can be conducted using a table-top approach, verification techniques, surveys, or interviews. These techniques determine:
Formal, detailed task analysis can be time consuming and expensive to conduct and should be used only when other less detailed techniques do not produce satisfactory results. Information on conducting detailed task analysis is found in DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities.
After the task list is developed, it is necessary to determine which tasks are important to the facility mission and which are not as important and, therefore, do not warrant formal training. This decision has a significant impact on training resources. If the criteria for determining which tasks are important to the facility mission are too broad, a great deal of resources may be needed to design, develop, implement, and maintain training. Conversely, if the selection criteria are too restrictive, training deficiencies may exist and result in costly personnel errors. The key to an effective balance is to ensure that the correct people are involved in the decisions, and that criteria appropriate to the facility's mission are used. The correct people are the people who know the job, the task, and the associated hazards and complexity of performance.
There are several techniques for selecting tasks for training. The traditional technique involves determining the difficulty, importance, and frequency of each task and applying the results to a decision tree that contains numerical values for each trait. Other techniques, such as SME discussion and consensus, can produce acceptable results. However, it must be understood that the results of any technique used are not absolute and should be challenged, evaluated, and refined as necessary. In all cases, the results should accurately reflect job requirements and should have the concurrence of job incumbents and management.
Regardless of the selection technique used, tasks are identified and grouped as "train," "no-train," and "overtrain/retrain." Train is defined as requiring initial training, no-train is defined as requiring no formal training, and overtrain/retrain is defined as requiring both initial and continuing training. Even though "no-train" tasks do not require formal training, possession of the knowledge and skills associated with the task is still necessary. These knowledge and skills are normally addressed with entry-level requirements or are readily learned as part of working in the position to which they are associated. Some facilities include no-train tasks in their qualification card/checklist.
An additional classification of tasks is referred to as "pre-train" or "just-in-time" training. These are tasks that require training, but will not be taught until that specific knowledge and skill is needed. For example, tasks specifically associated with maintenance of a complex pump could best be taught just prior to scheduled maintenance.
The fundamental elements of the analysis phase are:
Products that typically result from analysis include the following:
If the correct people are not used, the results of even the most detailed analysis can be less than satisfactory. Participants must be well acquainted with the job and must understand the consequences of poor or improper performance of tasks. Facility personnel who should be involved in analysis include experienced SMEs, supervisors, management, qualified training staff, and as necessary, facility engineering, safety, and other technical support staff. The technical support staff can be especially helpful when new systems or procedures are involved. Facility personnel involvement in analysis helps ensure training decisions belong to facility personnel since the results of analysis are used to design training that meets the needs of the job incumbents and line management. The job incumbent should be viewed as the customer and the analysis process as a method for meeting customer requirements.
Task lists serve as the foundation for initial and continuing training programs and must be kept up to date. The frequency of the update can vary from as-needed to every two years, depending on the number of changes that impact training (expected or actual) and training organization procedures. Program-level needs analyses can also be conducted on a periodic basis as part of program evaluation to detect performance deficiencies before they lead to personnel errors.
The practices described in DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities, encourage the completion of task analysis prior to starting the design phase. An alternative approach that combines task analysis with the design and development of training materials has also been found to be successful and efficient. The time and expense of completing the task analysis before design and development is avoided and training material can be developed and ready for use sooner (and therefore more economically). Accordingly, combining task analysis activities with design and development activities is strongly encouraged whenever feasible.
Documentation should be maintained that describes the activities of the analysis phase. Written procedures and forms that support this documentation are readily available within the DOE complex. Task lists and a record of the participants provide adequate documentation in most cases. Reference to existing task lists, guides to good practices, and similar sources of information should be made when such sources form the basis of tasks selected for training.
Training design produces a training development plan that guides the creation of all training materials and strategies. It is important to design an efficient, effective training program that is based on analysis data and reflects job performance requirements.
The techniques for determining training content, sequence, and strategies may vary from highly-formal to less-formal and streamlined. Use of a combination of traditional, table-top, verification, and document/template techniques will usually be most efficient. For additional information regarding technique selection, see Appendix A.
Regardless of the technique selected, the following activities generally occur during training design:
The products that typically result from training design include the following:
Training design products guide the development of all program-specific training materials and strategies. A curriculum outline is developed to establish the basis for the development of a training plan or summary. This is a very important series of activities that provides management with the information necessary to ensure the program will meet the needs of the organization.
A curriculum outline documents the general structure of a training program. The outline is developed by using information from analysis to design initial and continuing training programs. The curriculum outline helps prioritize training and identifies training materials that must be developed. Requisite knowledge and skills, systems, topics, tasks, and testing requirements are grouped within the training program curriculum. Designers determine the most effective strategy for presenting information by determining the sequencing of content presentation, selecting the most appropriate training setting for the objectives, determining testing methods, and identifying the most appropriate approaches to instruction. Instructional approaches should be based on objectives and the setting selected for conduct of the training.
Instructional approaches may include lecture, demonstration and/or practice, discussion/facilitation, oral questioning, role playing, use of case studies, simulation, exercises and walk-throughs, and individualized instruction (structured self-study). The approaches selected will also be influenced by the hazard and/or complexity of the task or job. For example, more structured, formal instructional approaches are appropriate when the hazard or risk consequences of improper performance are greater. Conversely, less formal approaches are sufficient when the consequences are lower. The same holds true with the curriculum outline.
Curriculum outlines may vary from a simple one page outline when the hazard or risk consequence is low to a complex multi-page outline when the hazard or risk consequence is high. After the curriculum outline is developed it should be verified by SMEs. The following techniques may be used individually or combined to establish a curriculum outline.
It is essential that learning objectives accurately describe the required performance for a task. Effective learning objectives clearly state measurable performance the trainee must be able to demonstrate and include conditions and standards of task performance. In most cases, a training designer develops learning objectives and obtains SME and management concurrence. Objectives provide the framework for development of training materials and determination of testing requirements. For detailed information relative to developing learning objectives, refer to DOE-STD-1005-92, Guide to Good Practice for Developing Learning Objectives.
When designing testing specifications for individual units of instruction or for overall qualification, the designer determines performance testing limitations and scoring methods for performance tests. Written and oral test specifications and job performance measures (JPMs), or their equivalent, are then developed. For additional information about testing requirements refer to DOE-STD-1009-92, Guide to Good Practices for the Development of Test Items, and DOE-STD-1011-92, Guide to Good Practices for the Design, Development, and Implementation of Examinations.
A training program description uses the curriculum outline to document training program elements for each position. The information describes training program content (including initial and continuing training) and training program administration requirements.
Details in a training program description may include:
The training program description may be integrated into a document such as the Training Implementation Matrix/Plan or an existing administrative manual such as a Training Users Manual or a Training Management Manual.
Job incumbents (SMEs) and facility management should be actively involved in the design process. Line personnel must accept ownership in the process and concur with the resulting plans and products. Line management is the customer and line management concurrence with the results of the design phase is essential.
Processes used and participants involved in determining curriculum and training content should be documented. Documentation is necessary to track decisions and to provide assurance to the customer (line management) and DOE that the training program design is based on reasonable decisions by appropriate persons.
Several documentation formats can be used, including data bases and computer programs, to provide the linkages needed. At most facilities, the training program description should provide sufficient documentation.
During development, all training, documentation, and evaluation materials are revised, written, or otherwise produced. As training materials are developed, an evaluation to determine effectiveness, cost efficiency, and applicability to the training needs should be made. Training materials should be piloted (tried on a group of trainees or SMEs) and modified to correct any deficiencies. Final materials used in training should be reviewed and approved by line management.
Development of training materials represents a considerable investment and it is important to review and use existing material to minimize development efforts whenever possible. Development may include adopting other materials, revising other materials, or creating new materials. Materials from similar facilities or from facilities that have positions performing similar jobs can often be made facility-specific relatively easily, and can result in considerable savings in time and effort. The table- top and verification techniques are effective in identifying equivalent material and in verifying the applicability of content.
The degree of formality and complexity of training materials should be tempered or influenced by the complexity and hazard of the tasks. For lower-hazard tasks, the training material used need not be as formal or complex as those for higher-hazard tasks. Rather, these materials may only need to include the key points that support the learning objectives. For higher-hazard tasks, more formal training materials may be necessary. Regardless of the complexity or hazard, training materials should contain sufficient detail to ensure consistency.
The fundamental elements of training development are:
Products of training development may include:
The curriculum outline created in the design phase (which supports the training plan) identifies materials that need to be developed. The training setting and learning objectives that are identified in the design phase determine what training strategies are used.
If alternative approaches suggested thus far have been used, the volume and complexity of training materials that require development for lower-hazard tasks should be significantly less than those required for higher-hazard tasks.
The training organization should describe the process(es) used for control and development of materials in a procedure or similar document. This document should also describe how changes to training materials are documented, approved, and tracked.
Program records should be maintained that include (as necessary):
During implementation, training is delivered and trainee mastery of the learning objectives is assessed.
Training should be implemented using written guidance documents (classroom lesson plans, OJT guides, JPMs, written examinations, etc.). Techniques that are used to implement training include:
Training is conducted by instructors who are trained and qualified for the setting in which they perform. DOE-NE-STD-1001-91, Guide to Good Practices for Training and Qualification of Instructors, and DOE-STD-1012-92, Guide to Good Practices for On-the-Job Training, contain guidance relative to instructor training and qualification.
On-the-job training (OJT) requires instructors who are qualified to perform the task or assignment. The use of formal guidance in conducting OJT and evaluations is necessary to ensure consistent implementation. It may be possible to train 2-4 persons simultaneously without sacrificing safety or training quality, however, trainee evaluations must be conducted one-on-one. OJT can be implemented at three levels.
For high-hazard potential tasks, OJT should be conducted as described in DOE-STD-1012-92, Guide to Good Practices for On-the-Job Training.
For tasks that pose a lower-hazard potential, OJT may be conducted using general instructions and task-specific evaluation materials. General instructions guide the process and a brief JPM (or equivalent) is used. These materials are sufficient to consistently conduct and evaluate OJT. Use of general instructions can significantly reduce training development time and lower costs.
OJT for technical staff, oversight personnel, and low-hazard tasks may be conducted and evaluated simultaneously. OJT used for the training of tasks at this level is conducted and evaluated as a one step process (this process is sometimes referred to as structured mentoring). Using general instructions and a JPM (or equivalent), the instructor conducts a qualitative evaluation of the trainee's progress. The instruction and evaluation are repeated until the competency or task is mastered.
During implementation, qualified instructors deliver classroom instruction using lesson plans. Lesson plans should encourage direct trainee participation in the learning process and employ instructional techniques (e.g., briefings, lecture, discussion, case studies, seminars, etc.) that are appropriate to the lesson content and learning objectives. Regular evaluation of the trainee's mastery of the learning objectives is expected during classroom training. Lesson plan detail and format should vary as a function of the complexity and hazard of the task. Lesson plans may consist of less formal guides for low-hazard tasks. These guides should include the key points that support the learning objective(s) and any supporting information determined to be necessary. Lesson plans for high-hazard tasks should contain additional detail and should be consistent in format. The added detail is necessary to address the increased complexity and hazard associated with the task. Regardless of the complexity or hazard, lesson plans should contain sufficient detail to ensure consistent delivery.
Although discussed separately, industry standards consider individualized instruction as a form of classroom training. Individualized instruction is a common technique for training operations, maintenance, and technical staff personnel. For the purposes of this handbook, individualized instruction consists of structured self-study, computer-based training, and interactive video.
Structured self-study can be used to supplement or replace traditional lecture-based classroom training. It is effective in improving knowledge of fundamentals, facility systems, components, and procedures. Instructors and supporting training materials should be readily available during self- study. Following self-study, a written examination or an oral checkout/quiz, based on learning objectives, is usually administered.
Computer-based training (CBT) and/or interactive video can also be used to supplement orreplace classroom training to teach fundamentals, facility systems, components, and procedures. Development of high quality CBT is expensive, but can be cost-effective if large numbers of people are trained on standard topics. To make CBT more effective, assistance (i.e., instructors, proctors, program administrators, etc.) should be available to answer questions.
Laboratory training can supplement and provide a cost-effective alternative to portions of the classroom and OJT program. Training laboratories often use surplus equipment and repair parts that can, in many cases, be used repeatedly. The laboratory allows one instructor to provide training to a greater number of trainees than is possible with OJT. Laboratory training requires approved training materials such as laboratory guides and JPMs.
Laboratory training can be used to provide practice with equipment that is similar or the same as in-plant equipment. It can also be used to provide training in a "job-like" setting without hazards that exist at the job site (e.g., high noise levels, radiation areas, hazardous/toxic environmental conditions, etc.). When laboratory training replicates facility equipment closely enough to provide assurance of task performance, it can serve as a substitute for actual performance on installed equipment for qualification purposes.
A simulator is an apparatus that generates conditions approximating actual or operational conditions. The most widely recognized simulators are those used in the aircraft and the commercial nuclear power industries. Simulators can be either full-scope or part-task. Full-scope simulators are usually large-scale replications, which create a training environment with a high degree of physical and functional fidelity relative to the actual or "referenced" job. A part-task simulator incorporates detailed modeling of a limited number of components or subsystems and demonstrate expected response. Part-task simulators also add realism to training. Simulator training can be highly effective for normal facility operation, anticipated transients, and accident conditions. Because the simulator reflects actual job conditions, OJT may be completed on the simulator in many cases. In addition, this setting lends itself to effective team training.
Elements of implementation include:
Products of implementation may include:
The successful implementation of a training program requires line management involvement. By periodically monitoring on-going training sessions and providing feedback, line management helps ensure training quality. Line management's continued support of training ensures the availability of technically qualified personnel from the operating organizations (e.g., operations, maintenance, technical staff) to the training organization. In addition, the line organization provides in-plant time for qualified instructors to maintain technical qualification. Active involvement is particularly necessary in providing training program funding, facility availability, and support resources for the program.
Process(es) and administrative requirements used for training implementation should be described in a procedure or similar document. The guidance in the Nuclear Information and Records Management Association Guideline for Management of Nuclear Related Training Records, TG-17, should be used to help standardize identification, handling, and storage of training records.
The goal of all training programs is to prepare personnel to safely and efficiently operate in accordance with established requirements.
A training evaluation program should be implemented at every nuclear facility to determine the program's effectiveness of meeting trainee and management needs. Training activities, employee performance, and subcontractor training should be evaluated on a regular basis to determine the effectiveness of training. Changes to the training program that result from the evaluation process should be approved and documented.
Detail in documentation of evaluation can reasonably vary from facility to facility. Post-training evaluation at a low-hazard facility may only require brief interviews with recently qualified job incumbents and their supervisors. Documentation by memorandum to the respective line manager that describes evaluation results and any necessary corrective actions is sufficient. At a high-hazard facility, evaluation may require survey forms or questionnaires from newly-qualified job incumbents and their supervisors. Documentation in this case may include entry of corrective actions into a tracking system, plans and status of training improvements, and approval of all revised training materials. Evaluation activities should be commensurate with the hazard potential, risk, and complexity of job performance.
There are a number of evaluation activities that provide information about the effectiveness of a training program. Common techniques include:
There are several activities associated with evaluation. Preparation for conducting evaluations may include development work, planning, scheduling, and identification of evaluators. The actual implementation of the evaluation process includes preparation, field observation, report preparation, and documentation, including resultant corrective action plans.
Implementation of specific techniques is discussed in section 6.6. A brief explanation of the elements involved in preparation follows.
Table of
Contents
Next Section