An Evaluation of Evaluation

appleIn the spirit of reflection and at the completion of yet another intense course experience, I find that I learned far more than I ever intended. While I can’t say I was giddy at the thought of taking EdTech 505 Evaluation for Educational Technologists, I did recognize that my learning curve would be huge. It was. I entered with not even a working definition of program evaluation and now have the skills to conduct one.

I began the course by creating a Gretel-at-a-glance word cloud explaining who I am and what I hope to gain from the course. My formal experience with program evaluation was nonexistent. My objectives for this course were to learn evaluation techniques that could help me evaluate some of the big picture programs in a school. I feel like I at least have the tools and background I need to begin.

Many of the course assignments were exercises found in the course textbook so I didn’t find the need post them on this learning log. One of the downsides of this course is that it is not really designed to increase my digital footprint.

Program evaluation “enables accountability” (Boulmetis & Dutwin, 2011, p. 38). I enjoyed reading about the various vantage points and considerations that make evaluations meaningful. Everyone, especially in today’s economy, wants to know “what did we get for our money [or time, or effort]? Did it work? Did it do what we hoped it would?” Those are fair and important questions. I appreciate the detailed explanations, both in the module and in the text, of programs, inputs, process, outputs, and outcomes.

Our final project was to conduct a small-scale but real program evaluation. The Explore Nepal program is an extensive school-wide program designed to help students reach out to the Nepali community and gain a deeper connection to their host country. I chose go evaluate the Grade 6 Explore Nepal week-long trip to a local Tibetan monastery to see whether the program objectives laid out for the trip were accomplished. The five objectives include: learning about Nepali culture through community interaction, environmental awareness, service learning, challenging physical activities, and team building. It was a major effort and here is a link to a generic copy of my final report:

Summative Evaluation: Grade 6 Explore Nepal Program

Summative Evaluation: Grade 6 Explore Nepal Program on Google Docs
Summative Evaluation Flipbook

One fun aspect of this course was the option on nearly every assignment to turn in an alternate submission format using some sort of tech tool. Some people created slideshows, videos, infographics, collaborative corkboards, flip magazines, and mind maps. It was fun to see the creativity and it sure was nice to have this option to shake things up. I wish every teacher would do that and allow students to complete an assignment while building their online presence, developing creativity, and taking ownership of their learning.

While I didn’t love everything about this course and would have liked more focus on education and technology, I certainly learned a lot that will serve me well, even if I don’t become a professional evaluator. What a ride it has been!

Other coursework:
Evaluation Design Format
Gap Analysis
Program Cycle
Goal-Based Method, Design, and Type
Top 3 Sites on Data Analysis
Review of Chapters 1-9 in course text
Request for Proposal (fictional)

References:
Boulmetis, J., & Dutwin, P. (2011). The ABCs of evaluation: Timeless techniques for     program and project Managers (3rd ed.). Jossey-Bass.

Authentic Assessments

As a parent of four, I know that kids don’t always “get” it. Sometimes when I think they are listening they are not, or I think they are not paying attention and they remember specific details. It’s no different in the classroom. Teachers spend a great deal of time trying to help students learn and remember what is being taught, often with varying results.

How do we know they really get it? More importantly, how do we ensure that students will remember such content long-term and apply it in their lives? The answer lies, at least in part, in authentic assessments.

I’ve taken a great many tests in my lifetime. Most of them I performed very well on, especially ones that were detailed-oriented and involved memorization. I studied and crammed my brain with information, hoping to remember it all long enough to score well on the end-of-chapter/unit/year test. I usually did. But now that I have been out of school for years, most of what I learned I have long since forgotten.

Most students will not remember everything they learn in school for the rest of their lives. Does that mean we, as educators, give up and don’t try? Of course not. We try, we use different approaches, and we do our best.

Project Based Learning (PBL) involves students throughout the entire process, which can include having them help determine what is going to be assessed and how they will meet such criteria. The more students have a say in what they need to learn – and how they will demonstrate they have learned it – the more they will remember.

What Kids Can Do gives the following criteria for authentic assessments (which align very well with PBL essential elements):

Assessment is for students.

  • Has personal relevance for students (e.g., provides a tangible product they can use).
  • Students become more confident and articulate about what they know.
  • Students feel ownership over the process as well as the product of their work.

Assessment is faithful to the work students actually do.

  • Notebooks, works-in-progress, and routine presentations are basis of assessment.
  • Occasions for reflection and discussion are integrated into ongoing project work.
  • Students are assessed on what they know and do, not what they don’t.

Assessment is public.

  • Students’ goals are solicited and become part of those assessed.
  • Criteria for judgment remain visible and accessible to students from the beginning.
  • Performances are viewed and judged by a broad group of people.

Assessment promotes ongoing self-reflection and critical inquiry.

  • Teachers and students both speak of the qualities of good work, and how to attain it.
  • Standards used reflect those of adult practitioners in the field.
  • Categories and criteria of assessment remain open-ended, subject to challenge and revision.

If teachers would incorporate these four principles into the assessments they offer, even those not technically PBL, I believe students would perform better and retain more. Many teachers don’t actively involve students in designing the assessment, so this may take a little encouragement and practice. Teachers need to see the benefits of allowing more student input during assessment development and how it is a win/win.

For Project: Me, I have designed both formative and summative assessments that will help students “get it” and demonstrate that they “got it.” I will guide the students through a scaffolded brainstorming activity early on that will help them determine what it is they need to know and how they will demonstrate they have reached the standards. For formative assessments, they will self-evaluate their reflective journals and project work reports weekly, create a visual project organizer using a technology tool, and create a project prototype for evaluation. For summative assessments, students will complete a short essay test on what they have learned about themselves and uniqueness, self-evaluate their completed reflective journals, and self/peer-evaluate their presentation both for content knowledge and 21st Century skills. By using a variety of assessment tools, students will be encouraged to set goals and stay on task, track their progress and growth, and produce a culminating multimedia project. Such strategies will help students find relevancy in what they are doing, be assessed on what they are actually doing, communicate with an outside audience, and produce quality work. In the end, they will remember this project and will have truly learned something valuable about themselves.

Now, how can I implement such strategies at home…?

References:

Buck Institute for Education. (n.d.) What is PBL?. Retrieved from http://www.bie.org/about/what_is_pbl

What Kids Can Do. (n.d.). Retrieved from http://www.whatkidscando.org/

Instructional Design Final Project

Wow.

When I started EdTech 503 I really had no idea what was in store. I do now. While I feel somewhat overwhelmed with all I have learned I also feel empowered. I certainly need and want to learn more.

My project for this course was to design three-hour instruction to help 8th and 9th grade students learn to use Google Docs presentations. It is geared for English Literature students who need to do a class presentation on a poet. They will not only gain content area knowledge but learn valuable 21st Century digital skills that they can apply in various real-world scenarios. It’s been fun but daunting.

Here is my 35-page project. There are many supporting documents linked throughout. It has been a grueling process but certainly a learning one.

Instructor’s Guide and ARCS Table

Piece by piece, my instructional design project is coming together. This week I have thought through what the actual instruction will look like. What materials will the instructor use? How will the lessons be organized? How will the instructor motivate the students? I enjoyed this assignment because it was less theoretical and more practical. There are two different models we used and I post them both here. Each image links to Google Docs: