Engage Stakeholders
Who should be
involved?
The
following stakeholders should be involved in the evaluation:
· High
School staff and administration
· Community
stakeholders; parents, District Education Authority(CEC) members
· Students
|
How
might they be engaged?
Stakeholders
may be engaged in a number of ways such as;
· Completing
surveys
· Asking
for specific feedback as to what they see as the benefits of the program
· Feedback
on improvements, or areas of concern
· Likes,
dislikes and advantages of being involved in the program.
|
Focus the Evaluation
What are you going to
evaluate? Describe program (logic
model).
This
evaluation will focus on a component of our local high school’s Pyramid
Response to Intervention (PRTI) and specifically their Period 2 activities.
Period 2 is a scheduled time-block each day in which students are given a
range of options based on their specific needs. These options range from
guided instruction and tutorial sessions with staff in specific subject areas
to time spent learning a co-curricular activity with a staff member or
community volunteer.
|
What
is the purpose of the evaluation?
The
specific purpose of the evaluation is to determine if the period 2 activities
have a positive impact on student achievement. It will also be used to
determine areas for improvement as well as if resources in the way of
financial aid is sufficient or more is required. Directly influenced by this
is to create a better understanding of the Period 2 program for all parties
and stakeholders. Deeper understanding will also allow stakeholders to look
at data tied to the program as well as understand positive influence
co-curricular activities have on student achievement.
|
Who
will use the evaluation? How will they use it?
Who/users
|
How
will they use the information?
|
Students
|
To
gain an understanding of purpose/advantage of period 2.
|
Administration
and school staff
|
· Use
to evaluate and tweak the program.
· Monitor
areas that need attention.
· For
funding purposes or to target $ to specific areas
· To
monitor reactions to the period 2
· To
encourage more involvement and support from community members/volunteers
|
Community
stakeholders
|
Stakeholders
outside the school can also use in order to monitor effectiveness of school
programming as well as to gain further understanding of the purpose for co-curricular
activities built into the school day. CEC members can use evaluation to
inform their constituency members.
|
What
questions will the evaluation seek to answer?
Due
to the period 2 being directly linked to PRTI 2 the evaluation may center
around two main questions for the school staff to consider:
·
What if the students have learned it
(core curriculum)?
· Participate
in co-curricular activities to gain/broaden knowledge and experiences.
· Take
advantage of curricular support to prepare for tests/quizzes/assignments
·
What do we do if the students have not
learned it (curriculum)?
These
questions are at the forefront of their purpose and guide school leadership
in determining placement of students.
|
The
evaluation will ask and hopefully answer many questions as linked to the
Period 2 such as (but not limited to):
·
How has student achievement increased
as a result of Period 2?
·
What are the benefits? For who? How?
·
Is the program cost effective and
hence worth the financial resources attached?
·
Are specific skills/ knowledge
acquired?
·
How is the community involved? Are
volunteers utilized in the implementation?
|
|
What information do
you need to answer the questions?
What
I wish to know
|
Indicators
– How will I know it?
|
Is
Period 2 achieving its goal of increasing student achievement? Do students
benefit?
|
Increase
in student grades over previous years.
Participation
rate – is there 100% participation?
Increase
in student average attendance over previous years.
Students
becoming more competent in various co-curricular activities through broader
exposure.
Students
make personal choice to accept extra help in a subject over co-curricular in
order to prepare for tests.
|
Is
Period 2 meeting needs of volunteers/community members?
|
Volunteers
eager to provide input to school admin.
Volunteer
base growing with community members approaching school to be involved.
Volunteers
provide feedback and offer diverse options.
Reactions
of the community – positive or negative.
|
Does
school staff see importance of Period 2 as well as successes?
|
Buy
in and willingness to participate.
Staff
offer further options for students over/above those already in place.
They
see guided tutorials, and instruction as punishment or for the purpose
intended.
|
Is
Period 2 duplicating other efforts?
|
Attendance
levels decrease due to other opportunities outside school.
|
When
is the evaluation needed?
The
evaluation is due for March 17, 2012
·
Interview school administration to
determine program parameters
·
View school website for further
information on Period 2
·
Collect photo evidence of program
·
Survey Students
·
Interviews with specific students and
stakeholders
·
Testimonials from stakeholders
·
February 27-March 9
·
View/analyze available data (attendance
rates, achievement indicators such as test scores)
·
Observation of program in action
(extra support as well as co-curricular)
·
Further interviews with stakeholders
·
·
March 10 -17
·
Final report prepared for submission
·
Time to wrap up loose ends and final
interviews with stakeholders
·
|
What
evaluation design will you use?
This
would be a goals-based evaluation with the main purpose being to evaluate the
extent to which the program is meeting it need – that of increasing student achievement.
|
Collect the information
What sources of information will you use?
Existing information:
|
School
records and student achievement indicators. How others in the community see
the program. Look at the program on a national level – do other regions do
similar activities? What are there results?
|
People:
|
Students
and stakeholders will be the best form of human information. Through these
members we can determine the success and shortcomings as well as what are
seen as important or possible improvement areas.
|
Pictorial records and observations:
|
Direct
observation of programs in action can help determine the success of the
program. This can be seen by level of buyin as well as attendance levels.
|
What data collection
method(s) will you use?
☺
|
Survey
|
|
Document review
|
☺
|
Interview
|
☺
|
Testimonials
|
☺☺
|
Observation
|
|
Expert panel
|
|
Group techniques
|
|
Simulated problems or
situations
|
|
Case study
|
|
Journal, log, diary
|
|
Tests
|
|
Unobtrusive measures
|
☺
|
Photos, videos
|
☺
|
Other – School Attendance
Records
|
Instrumentation: What
is needed to record the information?
Information
will be recorded using paper and pencil with computer software using basic
word documents and excel spreadsheets. School electronic attendance management
system may be of importance in determining the exact attendance levels during
period 2 as compared to other portions of the day.
|
|
When will you collect
data for each method you’ve chosen?
Method
|
Before
program
|
During
program
|
Immediately
after
|
Later
|
Survey
|
|
☺
|
|
|
Interview
|
|
☺
|
|
|
Testimonials
|
☺
|
☺
|
☺
|
|
Photos/videos
|
|
☺
|
|
|
School
Attendance Records
|
☺
|
|
☺
|
|
Will a sample be used?
No
|
☺
|
|
Yes
|
|
If
yes, describe the procedure you will use.
|
|
Pilot
testing: when, where, how?
|
Analyze and Interpret
How will the data be
analyzed?
Data analysis methods:
|
Data
will be recorded through note taking (paper and electronic) as well as possible
voice recording. Responses will be entered into a computer program and organized
using excel spreadsheet as well as word document files.
|
Who responsible:
|
Main
responsibility will rest within. All data collection will be done by the
evaluator.
|
How will the information be
interpreted—by whom?
Interpretation
of the data will be done in consultation with my personal school leadership
team (Assistant Principal, Literacy Coach, Program Support Teacher). This collection
of staff members who are not directly tied to the high school can look at
data through the lense of an educator in order to assess the viability of the
program and whether or not it is meeting its goals.
|
|
What did you learn? What are
the limitations?
As
indicated, the purpose of the evaluation is to determine if the period 2
activities have a positive impact on student achievement. It is also important to not allow feelings (from
any of the focus groups) to get in the way of whether or not a program is
meeting its need. Someone’s dissatisfaction due to their not getting to
participate in the co-curricular activity of their choice due to spaces
available would not be a good indicator of success.
|
|
Use the
Information
How will the evaluation be
communicated and shared?
To whom
|
When/where/how to
present
|
School
Administration and Staff
|
A
written report directly after evaluation period (mid March)
Through
key summary statements.
|
Community
Stakeholders
|
In
community CEC meeting through verbal report and final evaluation.
Through
testimonial submission from students.
|
Students
|
Through
video presentation.
Photo
essay
Key
summary statements.
|
|
|
Next
steps?
|
Manage the evaluation Standards
|
Human subject’s protection
|
|
Utility
|
|
Management chart
|
|
Feasibility
|
|
Timeline
|
|
Propriety
|
|
Responsibilities
|
|
Accuracy
|
|
Budget
|
|
|
Great job Craig. You have addressed all of the important aspects of the program evaluation plan well. I have a sense of where you are going with the evaluation. I believe you will be able to generate useful practical recommendations from a PE such as the one you suggest. The only aspects I wish to know more about are in the management and standards section at the end.
ReplyDeleteJay
Jay, in looking at my gradebook in PAWS I noticed that I had not submitted my logic model when I submitted the Evaluation Assessment. Yikes, I thought it had attached as well. I will send via email as it will not copy/paste.
ReplyDeletePWK-PRTI Evaluation Schedule
ReplyDeleteActivity Responsibility Dates Inputs needed for
Start Finish task completion
Develop evaluation Craig 21/01/2012 28/01/2012 Determine data collection methods that
instrument will bring the best results
Test evaluation Craig 01/02/2012 03/02/2012 Sample students/staff to determine
instruments and collection method is thorough
revise
Determine budget Craig 28/01/2012 28/01/2012 Calculate areas needing funds allocated.
Much will be in-kind HR, with minimal
spending in copy costs.
Set sample groups Craig 21/01/2012 28/01/2012 Determine which groups of parents,
students and staff to interview/survey
Data Analysis Craig 05/02/2012 20/03/2012 Note taking during interviews
Interpretation of data will be done in
consultation with a 3rd party in order to
avoid bias.
Draft Report Craig 17/03/2012 23/03/2012 Complete data analysis
Final Report Craig 26/03/2012 31/03/2012
Thanks Craig. This gives me a great overview of the timeline.
ReplyDelete