Assignment 5 - Sample Survey

Assignment #5 – Sample Survey

The attached survey was designed and changed multiple times due to a number of items of concern that arose during the administration of the survey.

·         In its early stages the survey was designed primarily with yes/no responses. It was found that this enabled participants to quickly complete the survey however the answers given did not allow participants to accurately indicate their feelings on many of the topics (there was no middle ground).

·         In its second draft (attached) the survey was designed including parents and community volunteers in the same section. This ordering of the survey also did not allow for accurate responses as the participants were not given ample choice and the statements in the section were not generic enough to allow both participant groups opportunity to answer all questions correctly. This caused participants to leave questions blank and/or answer incorrectly.

·         The final draft (attached) the survey allowed participants to differentiate between questions and give fairly accurate answers based on their experience with the PRTI-co-curricular program. Differentiating between parents and volunteers allowed each group to give more honest feedback which could be used to make changes to the program in an effort to improve. Community volunteers, many of whom may be parents of students in the school, really could not answer questions as a parent due to both groups really seeing the program from different sides.

·         It was interesting to note that the sample students saw the program in different ways. Some students saw the co-curricular as an opportunity to be involved in different extracurricular activities during school time and gained from the experience while others saw the option as free time and although they usually participated some did take the opportunity to skip school during the period. Interestingly enough, some students opted out of the co-curricular activity of their own choice in order to finish assignments, or get extra help at times.

·         The questions at the end of the survey allowed each participant group the opportunity to answer based on their own specific ideas and expectations from the program. Answers given here were also seen as beneficial for feedback and further improvements to the program. 

Evaluation Assessment

          Engage Stakeholders

Who should be involved?

The following stakeholders should be involved in the evaluation: ·       High School staff and administration ·       Community stakeholders; parents, District Education Authority(CEC) members ·       Students

How might they be engaged?

Stakeholders may be engaged in a number of ways such as; ·       Completing surveys ·       Asking for specific feedback as to what they see as the benefits of the program ·       Feedback on improvements, or areas of concern ·       Likes, dislikes and advantages of being involved in the program.

          Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

This evaluation will focus on a component of our local high school’s Pyramid Response to Intervention (PRTI) and specifically their Period 2 activities. Period 2 is a scheduled time-block each day in which students are given a range of options based on their specific needs. These options range from guided instruction and tutorial sessions with staff in specific subject areas to time spent learning a co-curricular activity with a staff member or community volunteer.

What is the purpose of the evaluation?

The specific purpose of the evaluation is to determine if the period 2 activities have a positive impact on student achievement. It will also be used to determine areas for improvement as well as if resources in the way of financial aid is sufficient or more is required. Directly influenced by this is to create a better understanding of the Period 2 program for all parties and stakeholders. Deeper understanding will also allow stakeholders to look at data tied to the program as well as understand positive influence co-curricular activities have on student achievement.

Who will use the evaluation? How will they use it?

Who/users	How will they use the information?
Students	To gain an understanding of purpose/advantage of period 2.
Administration and school staff	·     Use to evaluate and tweak the program. ·     Monitor areas that need attention. ·     For funding purposes or to target $ to specific areas ·     To monitor reactions to the period 2 ·     To encourage more involvement and support from community members/volunteers
Community stakeholders	Stakeholders outside the school can also use in order to monitor effectiveness of school programming as well as to gain further understanding of the purpose for co-curricular activities built into the school day. CEC members can use evaluation to inform their constituency members.

What questions will the evaluation seek to answer?

Due to the period 2 being directly linked to PRTI 2 the evaluation may center around two main questions for the school staff to consider: ·         What if the students have learned it (core curriculum)? ·       Participate in co-curricular activities to gain/broaden knowledge and experiences. ·       Take advantage of curricular support to prepare for tests/quizzes/assignments ·         What do we do if the students have not learned it (curriculum)? These questions are at the forefront of their purpose and guide school leadership in determining placement of students.

The evaluation will ask and hopefully answer many questions as linked to the Period 2 such as (but not limited to): ·         How has student achievement increased as a result of Period 2? ·         What are the benefits? For who? How? ·         Is the program cost effective and hence worth the financial resources attached? ·         Are specific skills/ knowledge acquired? ·         How is the community involved? Are volunteers utilized in the implementation?

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
Is Period 2 achieving its goal of increasing student achievement? Do students benefit?	Increase in student grades over previous years. Participation rate – is there 100% participation? Increase in student average attendance over previous years. Students becoming more competent in various co-curricular activities through broader exposure. Students make personal choice to accept extra help in a subject over co-curricular in order to prepare for tests.
Is Period 2 meeting needs of volunteers/community members?	Volunteers eager to provide input to school admin. Volunteer base growing with community members approaching school to be involved. Volunteers provide feedback and offer diverse options. Reactions of the community – positive or negative.
Does school staff see importance of Period 2 as well as successes?	Buy in and willingness to participate. Staff offer further options for students over/above those already in place. They see guided tutorials, and instruction as punishment or for the purpose intended.
Is Period 2 duplicating other efforts?	Attendance levels decrease due to other opportunities outside school.

When is the evaluation needed?

The evaluation is due for March 17, 2012 February 13-17, ·                  Interview school administration to determine program parameters ·                  View school website for further information on Period 2 ·                  Collect photo evidence of program February 20-24, ·                  Survey Students ·                  Interviews with specific students and stakeholders ·                  Testimonials from stakeholders ·        February 27-March 9 ·                  View/analyze available data (attendance rates, achievement indicators such as test scores) ·                  Observation of program in action (extra support as well as co-curricular) ·                   Further interviews with stakeholders ·                    ·        March 10 -17 ·                  Final report prepared for submission ·                  Time to wrap up loose ends and final interviews with stakeholders ·

What evaluation design will you use?

This would be a goals-based evaluation with the main purpose being to evaluate the extent to which the program is meeting it need – that of increasing student achievement.

          Collect the information

 

What sources of information will you use?

Existing information:	School records and student achievement indicators. How others in the community see the program. Look at the program on a national level – do other regions do similar activities? What are there results?
People:	Students and stakeholders will be the best form of human information. Through these members we can determine the success and shortcomings as well as what are seen as important or possible improvement areas.
Pictorial records and observations:	Direct observation of programs in action can help determine the success of the program. This can be seen by level of buyin as well as attendance levels.

What data collection method(s) will you use?

☺	Survey		Document review
☺	Interview	☺	Testimonials
☺☺	Observation		Expert panel
	Group techniques		Simulated problems or situations
	Case study		Journal, log, diary
	Tests		Unobtrusive measures
☺	Photos, videos	☺	Other – School Attendance Records

Instrumentation: What is needed to record the information?

Information will be recorded using paper and pencil with computer software using basic word documents and excel spreadsheets. School electronic attendance management system may be of importance in determining the exact attendance levels during period 2 as compared to other portions of the day.

When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Survey		☺
Interview		☺
Testimonials	☺	☺	☺
Photos/videos		☺
School Attendance Records	☺		☺

Will a sample be used? 

No	☺
Yes		If yes, describe the procedure you will use.

Pilot testing: when, where, how? 

         Analyze and Interpret

 

How will the data be analyzed?

Data analysis methods:	Data will be recorded through note taking (paper and electronic) as well as possible voice recording. Responses will be entered into a computer program and organized using excel spreadsheet as well as word document files.
Who responsible:	Main responsibility will rest within. All data collection will be done by the evaluator.

How will the information be interpreted—by whom?

Interpretation of the data will be done in consultation with my personal school leadership team (Assistant Principal, Literacy Coach, Program Support Teacher). This collection of staff members who are not directly tied to the high school can look at data through the lense of an educator in order to assess the viability of the program and whether or not it is meeting its goals.

What did you learn? What are the limitations?

As indicated, the purpose of the evaluation is to determine if the period 2 activities have a positive impact on student achievement.  It is also important to not allow feelings (from any of the focus groups) to get in the way of whether or not a program is meeting its need. Someone’s dissatisfaction due to their not getting to participate in the co-curricular activity of their choice due to spaces available would not be a good indicator of success.

 

            Use the Information

How will the evaluation be communicated and shared?

To whom	When/where/how to present
School Administration and Staff	A written report directly after evaluation period (mid March) Through key summary statements.
Community Stakeholders	In community CEC meeting through verbal report and final evaluation. Through testimonial submission from students.
Students	Through video presentation. Photo essay Key summary statements.

Next steps?

         Manage the evaluation                Standards

	Human subject’s protection		Utility
	Management chart		Feasibility
	Timeline		Propriety
	Responsibilities		Accuracy
	Budget