Measuring Up: Third-Party Assessment of Four Full-Time-Learning Programs

Independent, third-party assessment that quantifies accomplishments and identifies areas for improvement is what can transform a good program into an exceptional one. And investigating what makes full-time-learning programs work can lead us to that new day for learning, when all kids have access to the people and programs they need to flourish.

Here is an outline of the outside-evaluation strategies employed by each of the four programs in our New Day for Learning, Part Two, coverage.

Build SF

Build SF partnered with Rockman Et Al, a firm that designs and conducts evaluations and research for education-related organizations, to develop a system for assessing the degree to which the Build SF students acquire and master a range of twenty-first-century skills in their program mentorships. The measurement system serves to clearly establish a student's progress in each of several key learning-skills areas, demonstrating how and when students meet benchmarks in workplace skills development, as defined by the Partnership for 21st Century Skills.

Click here to download the full report.

Citizen Schools

Since 2001, Citizen Schools has engaged Policy Studies Associates (PSA), an outside agency that conducts research in education and youth development, to perform a longitudinal study of the Citizen Schools program and its impact on students, comparing students who participate in the program to similar students who do not participate, and following both groups from middle school into high school. PSA's findings demonstrate that Citizen Schools bridges the transition from middle school to high school and improves performance in core academic courses and high-stakes tests well after students graduate from the program.

For example:

• Sixth- and seventh-grade Citizen Schools participants earned better grades than non-program-attending peers in English and math and scored higher on a state English exam during their first year in the program, all at statistically significant levels.
• 59 percent of Citizen Schools participants enrolled in top-tier college-track high schools versus 28 percent of similar peers.
• Students who participated in Citizen Schools in middle school continue to attend school at higher rates in high school. This difference in attendance rates continued into high school.
• In tenth grade, 46 percent of participants earned As or Bs in their English courses to 26 percent of their peers; and 36 percent got As or Bs in math compared to 28 percent of their non-program peers.

Click here for details and the full report.

NatureMapping

In the year 2000, NatureMapping utilized an outside evaluator to appraise its program, assessing multiple grade levels and schools and studying the effects of the program on its student participants, their schools, and their communities. Results were statistically analyzed, finding that the program increased students' awareness and knowledge of their natural environments more than other outdoor education or community service activities. The study also found that participation in the program strengthened the relationship between the schools and their communities.

Here are some examples of the study's findings:

• Students -- who saw their data as useful to others -- exhibited a heightened sense of stewardship; and teachers reported increased effort from students because of this effect.
• Teachers experienced real-life learning in the program because it employed goals and objectives beyond what is expected of in the classroom.
• Teachers observed a positive affect on students who did not traditionally excel in the classroom.

Click here to read more about the benefits of the program.

Click here to download the full report.

PASA

In 2004, the Market Street Research firm explored middle school youth's participation in Providence, Rhode Island, after-school programs. This research revealed both the need for improvement in the quality of Providence's after-school programs and the need for an intermediary organization to assist in providing that quality. PASA became that organization, developing a set of standards and indicators to demonstrate quality improvement.

In a partnership with the High/SCOPE Educational Research Foundation, PASA also developed the Rhode Island Program Quality Assessment Tool, which all after-school program providers can use to assess themselves on specific standards and indicators and to track their improvement over time. The tool is currently used by all program participants across the state and by more than twenty-five of PASA's program partners. Additionally, PASA's programs have access to a citywide data-sharing system that tracks enrollment, participation, and retention to help measure success.