The growth and development of people is the highest calling of leadership; Harvey S. Firestone
Performance Culture Strategic Planning Workforce Planning Recruitment, Selection, Succession Planning Performance Management, Rewards & Recognition, Work Environment Career Development, Diversity Management, Knowledge Transfer, Leader Development Retention Metrics
Featured Pages

PROGRAM EVALUATION

Key Question #1

Who is participating in the leader development program? Are we getting the "right" people in our programs, and are they completing program requirements in a timely manner?

Metric sources: Compare numbers between "control" (either current program or no program at all) and "pilot" group on participation indicators such as:

  • Number or percentage of applicants
  • Number or percentage of applicants who fit targeted program profile
  • Number or percentage of actual enrollees
  • Number or percentage of enrollees who complete pre-work
  • Number or percentage of enrollees who complete face-to-face program sessions within a designated time frame
  • Number or percentage of enrollees who complete a final individual skills assessment
  • Number or percentage of enrollees who submit a final written assignment within a designated time period

For those participants who do not complete pre-work, finish face-to-face program sessions, take the final assessment and/or submit the final written assignment, you could contact them on a "follow-up" basis to determine reasons for unsuccessful program completion.

Key Question #2

Did participants enjoy the program? What are participants' reactions to the format/content of the control and of the pilot program designs?

Compare ratings between control and pilot for pre-participation period (projected or anticipated), formative (after each program module) and post-program periods using a streamlined version of a standardized evaluation form.

Key Question #3

Were participants able to demonstrate key competencies during the program period? Which program design, control or pilot, best facilitates demonstration of key competencies during the program period?

Compare participant performance on control and pilot formats for the following:

  • Within-session exercises, which could include:
    • Mini-assessment center results
    • Individual and team performance (process and product) feedback on incremental tasks in learning laboratory (self, learning peers, and facilitator-level check-off using structured format)
  • Number or percentage who complete the final course assessment within a designated time period within benchmark scores

Key Question #4

Are participants able to apply program skills on their job? Which design, pilot or control, best facilitates on-the-job application?

You could compare participant self-reports of their performance on case study pre-work situational responses as compared to written assignment turned in with final assessment, with specific attention to the:

  • Number or percentage of on-time completions
  • Level of detail for completions
  • Frequency of use of program-related competencies in final assessment situations
  • Appropriateness or success of approaches listed in final assessment situations
  • Periodic post-program check-in with instructors and learning peers via methods that could include: direct observation, structured interview, on- line "refresher" discussions, surveys, follow-up mini-lab sessions.
  • 360º pre and post program feedback results on key competencies

Key Question #5

What difference does leader participation in the program make to the efficient, effective functioning of state government organizations?

Identify core work-unit-level outcome measures and compare for leaders who did and did not complete pilot program (e.g., grievance, turnover, absenteeism/ tardiness, work unit productivity). You would need to "control" for situational factors that are not readily affected by leader performance. This could include looking at quantitative measures adjusted to take into account agency-wide norms, as well as in-depth qualitative data collected via pre-post interviews, focus group, observation and/or documentation review. Once you have quantitative measures, you can assign dollar-values to time, work process savings, etc. and extrapolate amounts for work units.

Key Question #6

What are the costs, and the return-on-investment (ROI) for the leader development programs? Are there any differences between the control and the pilot formats?

  • Begin by computing average per participant cost by design format. These costs may include (but are not limited to): program design, materials, facilitator (time and travel), participant time and travel.
  • Then, compute average cost/successful completion by program design/format. This would be reflected by the formula: (Number of enrollees x Average cost per participant) / (Number of successful completions)
  • For example, if the pilot costs were $2660/participant and the average program held 24 participants, with 18 of those successfully completing the program, then the average cost per successful completion would be: (24 x $2,660) / 18 = $3,547.

You could also compute overall successful completion cost by program design/format (see above). Plus, compute a basic ROI by comparing program benefits to program costs. This could be computed using the formula: ROI = (Total dollar benefits for all program enrollees from questions 4 & 5 above) x 100 / (Number of enrollees x Dollar cost per enrollee).