The Outdoor School Program is a program offered by the Saskatoon Public School Division and is currently housed at Marion M. Graham Collegiate. The vision of this program is to offer Grade 11 courses to students in Wildlife Management, Biology, English Language Arts, Geography, and Physical Education. Travelling throughout the province on five major trips, the students enrolled in this program will experience learning in the best possible environment. With this in mind, the mission of the Outdoor School Program is to enable the students to experience learning in the best possible environment, the outdoors. Within these parameters, students will examine their social, economic, civic, and physical surroundings for their impact on the individual and on society as a whole.

Students, parents, and community stakeholders can access public information in one of several ways. A website, registration handbook, and Facebook site are available for a range of information from the general interest seekers to the serious applicant.

Information collected during this evaluation will be goal based with a focus on the theory of “best environment” as the best classroom. All surveys, communication, and viewpoints will be data based in an effort to accurately express how input factors like budget, staff, equipment, and programming can affect output factors such as lifelong learning, competence in an outdoor setting and familiarity and deeper understanding of the inter-connectedness of the physical and social environments. Within this framework, there must will be a commitment from the evaluator that this evaluation will accurately represent the beliefs and attitudes about best practice and the best possible learning environment in the Outdoor School Program. All efforts will be made to represent the opinions of others in a confidential and respectable manner so that each member of this endeavor feels that they are represented accurately and without bias.

Focus of the Evaluation

Purpose

Teaching staff in this program are committed to a high level of academic integrity while facilitating individual learning in the best environment possible. To that end, the purpose of this evaluation is to determine to what extent the Outdoor School Program is enhancing the education of the participants to the effect that they are equipped to have a deeper understanding of the inter-connectedness of the physical and social environments while immersed in a variety of outdoor pursuits. Key areas that will be examined are Staff training, budget (investment and distribution), curricular connection with outdoor pursuits, community involvement, Division commitment and client satisfaction (student, parent, community).

Questions

This evaluation will seek to identify the extent to which students are achieving the learning outcomes within the best possible environment and offer suggestions to re-examine areas that may be adapted or improved. At each stage of this evaluation, major stakeholders such as Division Personnel, School Administration, teachers, and students will be asked the following key questions:

1. Is student learning happening in the best environment possible?

2. To what extent are learning outcomes in each curricular area building connections for the learners to look at their world through a physical, social, and civic lens? ( i.e. Wildlife Management, Biology, English Language Arts, Physical Education, and Geography)?

3. What lifelong learning opportunities are being fostered?

Resources

In this Outdoor School Program, there are a wealth of resources that are used. More specifically, there is an investment of time, money, people, and equipment. This program is offered over the course of one school semester in a students’ Grade 11 year. This equates to approximately

97 days of instructional time. Students are expected to attend their classes and spend approximately 25 days in the school with the remainder of time spent on one of their five major outdoor excursions or out in the community as they access public events and meet with community partnerships. Students are also expected to invest time into an online Math course so that they earn their required Grade 11 Math credit for graduation purposes. These time commitments could not be met by the students if it were not for the time commitment made by the skilled teaching staff. Two staff are hired to facilitate, lead, and manage this program by designing quality learning in the five curricular areas and teaching students to “educate the eye” (McVittie, program teacher) as they experience nature. The art of teaching in an outdoor experience cannot be under-valued. Along with students and staff, there is a commitment by community partnerships to provide leadership and partnership with young people as they strive to understand the interconnectedness of the physical, social, and economic worlds they live in.

Each student must make an investment of $495.00 which includes equipment, facilities rental, expertise, and transportation. Students are also required to purchase some personal gear like long underwear and hiking boots. They also require a bus pass for travel around the city for community events. An overall budget of $25,000 was invested to purchase equipment such as sleeping bags, axes, knives, skis, canoes, tents, kitchen camping supplies, and to cover transportation.

Inputs, Outputs, and Outcomes

Through the use of a Logic Model, this evaluation will show the links between what is invested (inputs), what students, parents, and the community gain from involvement in this program (outputs), and how this program is valued among the participants and the community (outcomes).

Inputs

This evaluation will collect data from these input sources: teachers, training of staff, equipment and specialized gear, tuition, research, technology investment, volunteers, and stakeholder(s) support (Division and school based administration, community, parents).

Outputs

This evaluation will address these outputs sources: student work samples, student feedback, investigation of the Five Major Trips, Nature City Festival and Urban Transport and Design presentations, academic feedback, and outdoor pursuits and level of proficiency.

Outcomes

At the completion of this program evaluation, the outcomes of the Outdoor School Program will be examined. The projected short term outcomes for students are: Increased knowledge in the environmental realm, skills in outdoor pursuits, growth in the understanding of group dynamics and improved relationships with peers, an awareness of social justice issues as well as an awareness of the interconnectedness of the economy with politics, adventure experiences alone and with others, and decision making and the value of trust. Long term outcomes for students include: further educational pursuits in environmental studies, understanding civic responsibilities, increased sense of social justice, parental involvement, and community partnerships.

At the conclusion of this evaluation, one will have a clear understanding of the Outdoor School

Program in terms of its goals, vision, inputs and outputs. Stakeholders will be able to determine whether the investment into this program is feasible, manageable, and most importantly, does it appeal to the citizens of this community as a viable opportunity to educate young people on environmental issues.

Collecting Information

Sources of Information

A combination of qualitative and quantitative methods will be used to gather data in this program evaluation. Paramount to the gathering of any data is the value that must be placed on the validity of the data sources and the data itself. In order to ensure that the data collected is valid, this evaluation will address the following criteria as it is applicable to a “goals-based” evaluation:

Data will be collected on the basis of these questions:

1. How were the program goals established?

2. To what degree is the program progressing toward the goals?

3. Are the goals time-bound and achievable?

4. Are there adequate resources to address the program goals?

5. Are the evaluation tools measuring what is needed to determine program effectiveness?

After identifying the stakeholders in the Outdoor School Program and through consultation with program staff and administration, the tools chosen for this evaluation are:

1. Document review - This type of data collection is a good starting place for any evaluation as it is a means for reviewing material about how a program was implemented.

2. Observation - Gives an excellent overview of what is going on in the program as it was meant to run.

3. Interviews - In this form of data collection, in depth information may be collected from all stakeholders in a confidential format.

4. Focus group - This form of data collection is useful as it can highlight the strengths and challenges.

5. Survey - The confidentiality that comes with the use of surveys is an excellent addition to any program evaluation as many participants and stakeholders can reflect honestly about their practice and participation.

Identifying Stakeholders

Stakeholders are defined as people who share an interest or concern in an organization. The appropriate involvement of these stakeholders will give the evaluator a look at a program from many different perspectives. In the Outdoor School Program, there are many stakeholders involved. In order to identify the stakeholders pertinent to this program see Figure 1.

Using the Information

Data Analysis

The data collection instruments chosen will ultimately help to answer the question, “… to what extent does the Outdoor School Program enhance the education of the participants to the effect that they are equipped to have a deeper understanding of the inter-connectedness of the physical and social environments while immersed in a variety of outdoor pursuits.”

1. Document Review

The document entitled “Best Place” (Archibald, 2002) offers a comprehensive framework of the philosophy of out-of-school education. The advantages of the “best place” to teach and learn are articulated in this document. A concept is put forth for the development and implementation of Outdoor School plus other out-of-school programs. Qualitative and quantitative questions will appear on several surveys in this program evaluation.

1. Observation

Students in the Outdoor School Program spend their time in a variety of locations as they journey through this program. An observation of students within each of these settings will serve to create an impression of how the program operates on a day to day basis. Observation will take place in the classroom as students plan for their trips, learn leadership and outdoor skills, and receive instruction in curricular areas. Indirect observation may take place as one or more of the staff will have access to a video camera to record students on at least one of their 5 main excursions. Direct and indirect observational data will then be collated into themes and / or issues to address.

2. Interviews

Staff and students will be interviewed for their accounts as they are directly involved in the daily operations of the program. Also, valuable information may also be collected in interview format from administrators, parents, and division personnel such as superintendents, and other stakeholders. Interviews will be directed one on one with prepared questions (Appendix A ). The use of a video or voice recording may be necessary in order to have each voice accurately represented. The sensitive nature of information participants may wish to share will be kept confidential.

3. Focus Group

The Outdoor School Program has been in operation for 12 years. In that time, there have been approaches to outdoor education that could be seen as “best practice” and have exemplified the outdoor education experience. There have also been educational choices that have not served the program participants as best they could. It will be through a comprehensive gathering of the key groups involved to extrapolate these nuances. The focus group for this evaluation will consist of Division Board personnel, school administration, community partners, program staff, parents, and students (past and present). Focus group questions will delve into personal experiences and potential issues. (Appendix B)

4. Survey

A survey will be given to a select group of participants and stakeholders in order to collect qualitative and quantitative data in a time bound fashion. (Appendix C)

Data Interpretation

In this outcome based evaluation, each instrument was designed to address one question:

“… to what extent does the Outdoor School Program enhance the education of the participants to the effect that they are equipped to have a deeper understanding of the inter-connectedness of the physical and social environments while immersed in a variety of outdoor pursuits.”

An examination of the data will determine if the objectives of the Outdoor School Program were met and if not, why they were not met. Qualitative data such as interview questions, observation notes, and focus group notes will be examined for patterns and themes that emerge. Quantitative data will be shared in the form of charts, graphs, or percentages.

Data Sharing

The management level of this program will have access to all of the information collected and the recommendations for this program. Students, parents and community stakeholders will have access to the final report and any specific findings relevant to their position.

Managing the Evaluation

This evaluation will be conducted with transparency so that all participants feel as if their viewpoints are accurately represented. It is essential to ensure client security and confidentiality throughout the process. Participants will then feel free to offer candid responses. Through these mechanisms this evaluation will present valid data that will produce recommendations for future programming. The projected budget is outlined in Appendix D as well as a timeline for the preparation, management, and delivery of the program evaluation.

APPENDICES

APPENDIX A – Interview Questions

INTERVIEW QUESTIONS

THIS IS A SAMPLE OF QUESTIONS DIRECTED TOWARD ONE OR MORE OF THE PROGRAM STAFF WHO WILL DELIVER THE OUTDOOR SCHOOL PROGRAM:

1. WHAT SPECIALIZED TRAINING DID YOU NEED TO HAVE BEFORE YOU COULD BE CONSIDERED FOR A TEACHER IN THIS PROGRAM?

2. HOW IS YOUR LESSON PLANNING PRACTICE DIFFERENT IN OUTDOOR SCHOOL THAN IT MAY HAVE BEEN IN A TYPICAL CLASSROOM?

3. WHY DID YOU CHOOSE TO TEACH OUTDOOR SCHOOL?

4. WHAT ARE THE BENEFITS (AS YOU SEE IT) TO TEACHING OUTDOOR SCHOOL [THAN TEACHING IN A TYPICAL PROGRAM]?

5. WHAT TYPES OF TEACHING PRACTICES DO YOU USE THAT SUPPORT “BEST PLACE” FOR THE STUDENTS?

6. DO YOU USE COMMUNITY PARTNERSHIPS TO DELIVER CURRENT ENVIRONMENTAL INFORMATION? IF SO, WHOM DO YOU INVOLVE AND WHY?

7. GIVE AN EXAMPLE OF HOW STUDENTS SHOW ONE OR MORE OF THE FOLLOWING SHORT-TERM OUTCOMES: INCREASED KNOWLEDGE, SKILLS GAINED, TOOK ACTION WITH A CAUSE, IMPROVED RELATIONSHIPS WITH OTHERS, OR BUILD COMMUNITY PARTNERSHIPS.

THIS IS A SAMPLE OF QUESTIONS DIRECTED TOWARD ONE OR MORE OF THE STUDENTS WHO TOOK PART IN THE OUTDOOR SCHOOL PROGRAM:

1. WHAT MADE YOU DECIDE TO ENROLL IN OUTDOOR SCHOOL?

2. WHAT IS YOUR FAVORITE PART OF BEING IN OUTDOOR SCHOOL? LEAST FAVORITE PART?

3. DESCRIBE ONE OF THE LEARNING ACTIVITIES THAT HELPED YOU TO FULLY APPRECIATE LEARNING IN AN OUTDOOR SETTING.

4. IF YOU HAD TO DESCRIBE THIS PROGRAM IN ONE WORD, WHAT WOULD IT BE?

5. GIVE AT LEAST ONE EXAMPLE OF SOMETHING THAT YOU LEARNED IN EACH OF THE FOLLOWING: ENVIRONMENTAL AWARENESS, CIVIC RESPONSIBILITY, SOCIAL JUSTICE.

6. ARE YOU MORE AWARE OF THE INTERCONNECTION BETWEEN YOURSELF AND YOUR ENVIRONMENT AS A RESULT OF BEING INVOLVED IN THE OUTDOOR SCHOOL PROGRAM? EXPLAIN.

7. WOULD YOU RECOMMEND THIS PROGRAM TO YOUR FRIENDS? WHY OR WHY NOT?

8. DO YOU FEEL THAT YOU HAVE LEARNED AS MUCH AS YOUR PEERS WHO ARE IN TYPICAL GRADE 11 CLASSROOMS? EXPLAIN.

Appendix B – Focus Group Questions and Consent Form

FOCUS GROUP QUESTIONS

1. TO WHAT EXTENT DOES THE OUTDOOR SCHOOL PROGRAM ENHANCE THE EDUCATION OF THE PARTICIPANTS TO THE EFFECT THAT THEY ARE EQUIPPED TO HAVE A DEEPER UNDERSTANDING OF THE INTER-CONNECTEDNESS OF THE PHYSICAL AND SOCIAL ENVIRONMENTS WHILE IMMERSED IN A VARIETY OF OUTDOOR PURSUITS?

1. Where is the best place for teaching and learning?

2. Is the traditional approach to the classroom misguided?

3. How does Out of School education enhance the teaching and learning process?

4. Discuss your understanding of the statement, “ Out of School education is good pedagogy”. (Archibald, 2002)

5. Can the responsibilities of the school be accomplished entirely within the school building?

2. INTERCONNECTION

☐	Discuss the statement, “Outdoor education is education in, about and for the out-of-doors.” (Ford, 1992)
☐	What expectations rest on educators to expose ecological crisis?
☐	To what extent does environmental education create informed citizens?
☐	What is the best way to create a connection between humans, other living things, and the physical environment?
☐	What is your understanding of the phrase “new global ethic”?
☐	What is the best place to teach respect for the environment?

3. IMMERSION IN PHYSICAL AND SOCIAL ENVIRONMENTS

☐	What is the value of immersing oneself in a physical environment in order to understand it?
☐	Comment on whether immersion would be more meaningful when experienced in an outdoor setting?
☐	Does one’s sense of social justice grow in relationship with environmental concerns?
☐	Comment on the social responsibilities an outdoor education program must address.

4. CURRICULAR CONNECTIONS

☐	Describe your experience(s) with being in a traditional classroom.
☐	How do you envision a course in Biology being delivered in the outdoors? Wildlife Management, English, Geography?
☐	How could experiential education enhance the learning experience in these subject areas?
☐	Comment on the value of curiosity as a foundation for learning.

Student Centered Questions:

How have you changed since your involvement in Outdoor School?

Would you recommend this program to others? Explain.

To what extent to you see your teachers involved in this experience? Is your teacher a facilitator? Explain.

Are you more (or less) motivated to learn as a result of being involved in the Outdoor School Program?

Focus Group Consent Form

OUTDOOR SCHOOL PROGRAM – MARION M. GRAHAM COLLEGIATE

CONSENT FORM

Marion Graham offers a unique learning experience for Grade 11 students in the Saskatoon Public School Division. Students are enrolled in five classes (Wildlife Management 20, Biology 20, English Language Arts 20, Geography 20, and Physical Education 20) and take part in five major outdoor adventures. Students experience the classroom in a much different way through back-country camping, white water canoeing, backpacking, cycling, wall climbing, skiing, snowboarding and more.

In order to determine the effectiveness of this program, a systematic review of all program facets is being conducted. As part of this review, your participation would be appreciated as a part of a Focus Group that will address the question:

To what extent does the Outdoor School Program enhance the education of the participants to the effect that they are equipped to have a deeper understanding of the inter-connectedness of the physical and social environments while immersed in a variety of outdoor pursuits?

As part of a panel, your voice, including Division personnel, School Administration, community partners, program staff, students and parents will be represented. All opinions and viewpoints discussed will be valued as a confidential part of this reporting process.

I consent to the sharing of information pertaining to my involvement in the Outdoor School Program and will be a part of the Focus Group discussion.

(Signature)

APPENDIX C – Sample Survey

Sample Survey

SECTION 1 BEST PLACE

Determine the BEST PLACE for learning in each of the following categories: O = OUTDOOR SETTING S= SCHOOL SETTING

* SCHOOL SETTING is the traditional classroom

1. Listening to a lecture	O	S
2. Taking notes	O	S
3. Doing a lab experiment	O	S
4. Doing group work	O	S
5. Listening to an Elder speak	O	S
6. Working on a Math problem	O	S
7. Doing a fitness test	O	S
8. Finding a book for research	O	S
9. Writing an essay	O	S
10. Meeting friends for lunch	O	S
11. Learning a new skill for	O	S
survival	O	S
12. Learning CPR	O	S
13. Studying about landfills	O	S
14. Making a model for science	O	S
15. Delivering an oral report	O	S

For each of the following statements, choose the answer that best describes your experience with OUTDOOR SCHOOL:

A = Always U= Usually S= Sometimes N=Never

1. I learn best with lots of examples.	A	U	S	N
2. I love being in the outdoors.	A	U	S	N
3. If given a choice between being	A	U	S	N
indoors or outdoors, I pick outdoors.	A	U	S	N
4. I believe the best learning occurs	A	U	S	N
out in the natural environment.	A	U	S	N
5. Traditional classroom settings are	A	U	S	N
restrictive.	A	U	S	N
6. I learn best when I do what I have	A	U	S	N
been shown.	A	U	S	N
7. It is essential to have social justice	A	U	S	N
taught in schools.	A	U	S	N
8. Care of the environment must be	A	U	S	N
taught in schools.	A	U	S	N
9. Quality education should be delivered	A	U	S	N

in the best environment for learning to	A	U	S	N
take place.	A	U	S	N
10. Traditional school settings are best for	A	U	S	N
teaching and learning.	A	U	S	N

Use the space below to describe YOUR best environment in which to learn.

Tell about at least one activity in which you learned in an outdoor setting. What did you LEARN?

Comment on what you feel would be the best way to teach social justice to young people.

On a scale of 1 to 5 with 1 being the LEAST DESIREABLE and 5 being the MOST DESIREABLE, rate the following items:

1. A high score on a test	1	2	3	4	5
2. Lots of friends	1	2	3	4	5
3. Adventure	1	2	3	4	5
4. Learning lots of facts	1	2	3	4	5
5. Being inspired	1	2	3	4	5
6. Learning new things	1	2	3	4	5
7. Succeeding	1	2	3	4	5
8. Experiencing setbacks	1	2	3	4	5
9. Challenges	1	2	3	4	5
10. Writing about my	1	2	3	4	5
learning	1	2	3	4	5
11. Making a difference	1	2	3	4	5
12. Working alone	1	2	3	4	5

APPENDIX D – Proposed Budget

Source of Project Cost

PROJECT TASKS LABOR HOURS LABOR COST ($) TERIAL COST ($) RAVEL COST ($) OTHER COST ($) OTAL PER TASK

PROJECT DESIGN	Funding Source	1.0	$75.00	$100.00	$0.00	$20.00	$196.00
	Budget limits	1.0	$75.00	$0.00	$0.00	$20.00	$96.00
	Develop Preliminary Design Specification	5.0	$75.00	$100.00	$0.00	$20.00	$200.00
	Develop Detailed Design Specifications	5.0	$75.00	$200.00	$0.00	$20.00	$300.00
	Develop Acceptance Test Plan	5.0	$75.00	$250.00	$0.00	$20.00	$350.00
	Subtotal	17.0	$375.00	$650.00	$0.00	$100.00	$1,142.00

PROJECT DEVELOPMENT	Literature Review	15.0	$75.00	$150.00	$50.00	$50.00	$340.00
	Develop Data Collection Instruments	25.0	$75.00	$250.00	$50.00	$100.00	$500.00
		0.0	$0.00	$0.00	$0.00	$0.00	$0.00
	Development Acceptance Test Package	25.0	$75.00	$150.00	$50.00	$50.00	$350.00
	Perform Unit/Integration Test	5.0	$75.00	$50.00	$50.00	$50.00	$230.00
	Subtotal	70.0	$300.00	$600.00	$200.00	$250.00	$1,420.00

PROJECT DELIVERY	Site rental costs	1.0	$75.00	$100.00	$50.00	$50.00	$276.00
	Cost of print material	5.0	$75.00	$1,000.00	$50.00	$100.00	$1,230.00
	Video Camera	1.0	$75.00	$350.00	$50.00	$100.00	$576.00
	Laptop	1.0	$75.00	$1,000.00	$50.00	$100.00	$1,226.00
	Data storage devices	1.0	$75.00	$100.00	$50.00	$100.00	$326.00
	Focus Group, Survey, Interview	10.0	$75.00	$250.00	$100.00	$100.00	$535.00
	Subtotal	19.0	$450.00	$2,800.00	$350.00	$550.00	$4,169.00

PROJECT MANAGEMENT	Customer Progress Meetings/Reports	1.0	$75.00	$100.00	$50.00	$50.00	$276.00
	Internal Status Meetings/Reports	1.0	$75.00	$100.00	$50.00	$50.00	$276.00
	Miscellaneous Office Expenses	1.0	$75.00	$100.00	$50.00	$50.00	$276.00
	Interface to Other Internal Departments	1.0	$75.00	$25.00	$25.00	$25.00	$151.00
	Personnel Management	1.0	$75.00	$25.00	$25.00	$25.00	$151.00
	Quality Assurance	1.0	$75.00	$25.00	$25.00	$25.00	$151.00
	Overall Project Management	5.0	$75.00	$50.00	$50.00	$50.00	$230.00
	Subtotal	11.0	$525.00	$425.00	$275.00	$275.00	$1,511.00

OTHER COST	Coffee, muffins, cream, sugar, cups	1.0	$75.00	$100.00	$0.00	$0.00	$176.00
	Phone Usage	5.0	$75.00	$100.00	$0.00	$0.00	$180.00
	Postage Costs	1.0	$75.00	$100.00	$0.00	$0.00	$176.00
	Subtotal	7.0	$225.00	$300.00	$0.00	$0.00	$532.00

Subtotals	124.0	$1,875.00	$4,775.00	$825.00	$1,175.00	$8,774.00
Risk (Contingency)	0.0	$0.00	$0.00	$0.00	$0.00	$0.00
Total (Scheduled)	124.0	$1,875.00	$4,775.00	$825.00	$1,175.00	$8,774.00

APPENDIX D – Timeline (Gantt Chart)

The timeline is a suggested time frame for the completion of each phase of work.

REFERENCES

Archibald, K. (2002). Best Place – Out of School Education Concept Plan.

McNamara, C. (1997). Basic Guide to Program Evaluation. Adapted from the Field Guide to Non-Profit Program Design, Marketing and Evaluation.

Board of Regents of the University of Wisconsin System. (2005). Logic Models.

ECUR 809

Thursday 3 April 2014

Project Evaluation Proposal - Final Assignment ECUR 809.3

Introduction