.

Measurement

How we make a difference
TIMpact365 Measurement is second nature

TIMpact365 is our new measurement strategy.

We approach measurement holistically, measuring not only the outcomes and impact of our programs, but also how we offer our programs using a program quality lens. Together, these data provide a deeper understanding of what we do and how we impact youth in our communities.

The TIMpact365 logo pays homage to our outdoor sites across Canada and the US, references the four years of our Summer program with four trees, and communicates our year-round commitment to serving youth, measuring our impact, and improving our programs by “365”.

At the heart of TIMpact365 is our commitment to quality and continuous improvement of our program quality, outcomes and impact. A series of icons has been developed to help us communicate our framework.

Qualitative Data TIMpact365 Qualitative Icon
Quantitative data TIMpact365_Icon_Quantitative_Red

We collect a mix of qualitative (i.e., internal and external program observations, interviews, focus groups) and quantitative data (i.e., survey scores) which provide exploratory insights and useful context to build compelling narratives about our programs.

We track growth and change in each outcome through pre and post surveys and use secondary research as a comparative benchmark to approximate what it would have been like otherwise for each youth by comparing their experience and results with similar youth who did not participate in our programs. We use industry leading frameworks and tools to help us understand how our programs are facilitated, continually identify areas for improvement, and provide a high-quality experience for all youth. Our measurement efforts are all-encompassing as we connect with campers, the Youth Advisory Council, alumni, and staff to fully understand the camp experience.

We firmly believe that measurement is continual. We don’t just measure once, but instead we measure multiple times using a cycle. Through continual measurement we identify what we’re doing well, determine where we can improve, and implement key changes to our programs and staff training.

Program Quality Measurement TIMpact365 How Icon

Program quality is key to understanding how Tims Camps programs are facilitated for our youth. Aligned with many other educational settings and out-of-school-time programs, the fields of positive youth development and camp are prioritizing a program quality lens to ensure that every youth has a quality experience at camp. Through participation in high-quality settings and programs, youth develop greater positive outcomes that are associated with wellbeing and thriving in young adulthood. We seek to measure program quality across all our programs and use program quality data to inform changes and improvements to our programs.

Program quality refers to how programs are facilitated at camp through staff practices that create four environments. The David P. Weikart Centre for Youth Program Quality (part of the Forum for Youth Investment) structures these environments in a pyramid, similar to Maslow’s hierarchy of needs. The lower levels of the pyramid must be satisfied (i.e., the environments must be created) before higher levels of the pyramid can be reached. Health and safety standards are addressed in accreditation models, and therefore program quality focuses primarily on social emotional aspects of the program.

The four environments are:

Pyramid showing pillars of program quality

Copyright 2021 The Forum for Youth Investment

  • Safe: The program is a psychologically and emotionally safe environment for youth. The environment is free from bias and staff actively seek to include all young people and create a space where every youth feels they belong and people with different identities are celebrated.
  • Supportive: Staff create a learning environment in which young people are supported to try different things, develop new skills, and understand and manage their emotions and social interactions.
  • Interactive: The environment is rich in opportunities for youth to work together to achieve goals, independently succeed, and understand others’ strengths, areas for improvement, individual perspectives, and experiences.
  • Engaging: Program participants are encouraged to more deeply engage in learning and learning transfer beyond the program. Participants are also supported in taking agency, ownership, and initiative over their learning through planning and goal setting.

At Tims Camps, we began assessing program quality through our participation in the American Camp Association’s Camp Program Quality Initiative (CPQI). This was a pilot program to understand and develop sustainable systems of program quality in the camp space. We were honoured to be selected to participate in the CPQI pilot.

Through our participation in the CPQI and our initial assessment, we realized the need to prioritize program quality in all that we do and have launched a full-scale continuous improvement cycle focused on program quality assessment. We are using a research-validated measurement tool for internal and external assessments, engaging multiple stakeholders in the assessment process (i.e., staff across the organization, campers, Youth Advisory Council members, school partners), and creating new and unique ways of understanding program quality from the youth perspective. We are using program quality data to reflect on our programs and make changes to better serve youth.

Learn more about the American Camp Association’s Camp Program Quality Initiative (CPQI) and to see what we’ve learned from one full cycle of program quality assessments, click here.

Outcome measurement TIMpact365 What Icon
Our programs are intentionally designed to foster positive youth development in our campers. We know that it’s important we measure their development using a well-established youth development measurement framework. One of the most commonly used is the 5Cs. The 5Cs are:

Competence: Youth develop transferable technical, social, and emotional skills.

Confidence: Youth develop a sense of self-worth and skill mastery.

Character: Youth take responsibility, develop independence, and individuality.

Caring: Youth show empathy and sympathy for others and understand caring for others is important.

Connection: Youth create positive relationships and foster positive interactions.

For more information about the 5Cs model of positive youth development, including theoretical basis, additional definitions, and scientific references, click here.

Youth development of the 5Cs is associated with greater wellbeing, decreases in risky behaviours, contribution to community, and greater success in other life settings.(1)

Twice annually, we ask our summer camp participants to respond to questions about the 5Cs (questions are the research validated PYD-Short Form for practitioners). This information is helpful for us because we can understand how our programs are contributing to campers’ overall development. Learn about what we’ve found so far here.

In addition, we focus on 18 other outcomes, including independence, leadership, appreciation for diversity, responsibility, teamwork, willingness to try new things, being present, and more. Determining youths’ interest in learning each of the outcomes and their actual reported learning is important for us to target our programming to youth needs.

Impact measurement TIMpact365 Why Icon

Our programs have short-term and long-term impacts for youth. Youth experience immediate changes after program participation but also take their camp-based learnings into other life settings, yielding long-term impacts.

Short-term impact:

When young people develop the 5Cs, they also show development in terms of contributions to themselves, their communities, and their futures.(1) We call this the “Big 3” and these impacts are emphasized in our camp-based programming with specific activities to foster this development.

Self: Youth develop a greater sense of self, determine goals for themselves, and learn to modify goals as needed.

Community: Youth understand the importance of community and positively contributing to community and the wellbeing of others.

Future: Youth look to their future with a sense of hope and optimism for what they will achieve.

We ask youth to respond to impact measures prior to camp and after the camp experience so we can understand how our programs impacted them. Survey questions include research-validated measures related to self (intentional self-regulation),(2) community (pro-social behaviours),(3) and future (hopeful future expectations).(4)

To see what we’ve learned from one full year of impact measurement, click here.

Citations
  1. Schmid, K. L., Phelps, E., & Lerner, R. M. (2011). Constructing positive futures: Modeling the relationship between adolescents’ hopeful future expectations and intentional self-regulation in predicting positive youth development. Journal of Adolescence, 34(6), 1127–1135.
  2. Freund, A. M. & Baltes, P. B. (2002). Life-management strategies of selection, optimization, and compensation: Measurement by self-report and construct validity. Journal of Personality and Social Psychology, 82(4), 642-662. https://doi.org/10.1037//0022-3514.82.4.642
  3. Luengo Kanacri, B. P., Eisenberg, N., Tramontano, C., Zuffiano, A., Caprara, M. G., Regner, E., Zhu, L., Pastorelli, C., & Caprara G. V. (2021). Measuring prosocial behaviors: Psychometric properties and cross-national validation of the prosociality scale in five countries, Frontiers in Psychology, 12, 1-13. https://doi.org/10.3389/fpsyg.2021.693174
  4. Schmid Callina, K., Johnson, S. K., Buckingham, M. H., & Lerner, R. M. (2014). Hope in context: Developmental profiles of trust, hopeful future expectations, and civic engagement across adolescence. Journal of Youth and Adolescence. https://doi.org/10.1007/s10964-014-0096-9