An Evaluation of Evaluation

appleIn the spirit of reflection and at the completion of yet another intense course experience, I find that I learned far more than I ever intended. While I can’t say I was giddy at the thought of taking EdTech 505 Evaluation for Educational Technologists, I did recognize that my learning curve would be huge. It was. I entered with not even a working definition of program evaluation and now have the skills to conduct one.

I began the course by creating a Gretel-at-a-glance word cloud explaining who I am and what I hope to gain from the course. My formal experience with program evaluation was nonexistent. My objectives for this course were to learn evaluation techniques that could help me evaluate some of the big picture programs in a school. I feel like I at least have the tools and background I need to begin.

Many of the course assignments were exercises found in the course textbook so I didn’t find the need post them on this learning log. One of the downsides of this course is that it is not really designed to increase my digital footprint.

Program evaluation “enables accountability” (Boulmetis & Dutwin, 2011, p. 38). I enjoyed reading about the various vantage points and considerations that make evaluations meaningful. Everyone, especially in today’s economy, wants to know “what did we get for our money [or time, or effort]? Did it work? Did it do what we hoped it would?” Those are fair and important questions. I appreciate the detailed explanations, both in the module and in the text, of programs, inputs, process, outputs, and outcomes.

Our final project was to conduct a small-scale but real program evaluation. The Explore Nepal program is an extensive school-wide program designed to help students reach out to the Nepali community and gain a deeper connection to their host country. I chose go evaluate the Grade 6 Explore Nepal week-long trip to a local Tibetan monastery to see whether the program objectives laid out for the trip were accomplished. The five objectives include: learning about Nepali culture through community interaction, environmental awareness, service learning, challenging physical activities, and team building. It was a major effort and here is a link to a generic copy of my final report:

Summative Evaluation: Grade 6 Explore Nepal Program

Summative Evaluation: Grade 6 Explore Nepal Program on Google Docs
Summative Evaluation Flipbook

One fun aspect of this course was the option on nearly every assignment to turn in an alternate submission format using some sort of tech tool. Some people created slideshows, videos, infographics, collaborative corkboards, flip magazines, and mind maps. It was fun to see the creativity and it sure was nice to have this option to shake things up. I wish every teacher would do that and allow students to complete an assignment while building their online presence, developing creativity, and taking ownership of their learning.

While I didn’t love everything about this course and would have liked more focus on education and technology, I certainly learned a lot that will serve me well, even if I don’t become a professional evaluator. What a ride it has been!

Other coursework:
Evaluation Design Format
Gap Analysis
Program Cycle
Goal-Based Method, Design, and Type
Top 3 Sites on Data Analysis
Review of Chapters 1-9 in course text
Request for Proposal (fictional)

References:
Boulmetis, J., & Dutwin, P. (2011). The ABCs of evaluation: Timeless techniques for     program and project Managers (3rd ed.). Jossey-Bass.

Instructional Design Final Project

Wow.

When I started EdTech 503 I really had no idea what was in store. I do now. While I feel somewhat overwhelmed with all I have learned I also feel empowered. I certainly need and want to learn more.

My project for this course was to design three-hour instruction to help 8th and 9th grade students learn to use Google Docs presentations. It is geared for English Literature students who need to do a class presentation on a poet. They will not only gain content area knowledge but learn valuable 21st Century digital skills that they can apply in various real-world scenarios. It’s been fun but daunting.

Here is my 35-page project. There are many supporting documents linked throughout. It has been a grueling process but certainly a learning one.

Instructor’s Guide and ARCS Table

Piece by piece, my instructional design project is coming together. This week I have thought through what the actual instruction will look like. What materials will the instructor use? How will the lessons be organized? How will the instructor motivate the students? I enjoyed this assignment because it was less theoretical and more practical. There are two different models we used and I post them both here. Each image links to Google Docs:

School Evaluation Summary

It’s easy to criticize schools, especially when it comes to the use of (or not enough use of) technology. Parents do it, teachers do it, even students do it. If you have never had to really evaluate your school based on a number of factors, you might want to look closer before voicing your opinion. It’s complicated.

This week in EdTech 501 I surveyed our school’s technology environment. As I became more familiar with the Technology Maturity Model and the Stages of Technology, I realized this was no easy task.

The Technology Maturity Benchmarks and Stages of Technology “link the technology resources to their use in every learning environment. In this way, the existing level of support for students, teachers and support staff can be determined. The premise is founded on the theory that both resource availability and behavioral changes are required to improve educational outcomes.”

The five organizational filters are: administrative, curricular, support, connectivity and innovation. Each of these levels of the organization are addressed with both behavioral and resource/infrastructure criteria.

I began by creating a Google survey form and sent it to a small mix of administrators, teachers, and technology committee members at my children’s school. I received enough responses to give me a good starting point. The results (using a pseudonym) are noted here:

 

As I analyzed these benchmarks, I realized that it takes a village to improve the technology environment in a school. There’s no easy solution, no magic device or gadget. Results are measured on a spectrum and continually adjust. Many intricate factors work together to create the overall picture. A detailed summary of the results are found here:

 

For this assignment, I polished my project management and collaboration skills, information-gathering and decision-making strategies, and formative evaluation technique. I think AECT would be pleased.