Evaluation Course Final Comments

Finally the end of the semester is here! I am glad that I just submitted my final paper. It was really hard, but I have done it! I am really glad I took this class and learned so much about evaluation. I am sure it will help me in the long run. I am not planning on writing evaluation as a career, but still I learned a lot of useful information this semester.

The last few weeks, I read about graphs and data analysis. It was really useful to see how graphs might be distracting sometimes! I had not idea that there are so many “wrong” graphs being used by many people. I really liked this particular article about wrong data displaying. I learned more from that one than from the one showing me the correct way. Well, we learn best by mistakes, right? Before I liked to use all the fancy graphs and charts, but as I read, it is not good at all. Simplicity is the keyword here. Well, I will keep that in mind. (I have to admit that when I saw the titles of the required readings, I thought that the professor is crazy by having us reading this “basic” stuff, but I learned SOOOO much, I am really glad I read it!)

Another useful tip I learned in this class is good timing! By developing the timeline, I was able to keep up with the project. I am glad I did, otherwise it would be almost impossible to do all the work in that time frame. I think I will develop a timeline for everything from now on. It is a great tool and I really enjoyed looking back and forth and checking if I am still on track.

I am grateful for another great semester at Boise State and I look forward to the next one! Thanks, Dr. Perkins for all I learned from you!

Summative Evaluation of ScreenChomp

Last week, I was working on the evaluation of ScreeChomp application for iPad. It is unbelievable how much information I still need to be able to complete the evaluation. For the evaluation, I used a checklist, and a survey. First, I only developed a survey, but as I progressed through the project, I found out, that a checklist would be helpful as well. I decided to collect the data by observing learners and using the checklist, and by asking volunteers to fill out the online survey. I am still waiting for the responses and I have scheduled quite a few observations. . I have already done a few of the observations and found out that people react so differently. I got some surprising answers on my checklist. It is interesting how my predictions were often so wrong! Many people were crazy about the applications, on the other hand, many were not used to using the iPad at all and had problems creating the screencast and working with the commands. I cannot wait for the results to be done. I think I will be surprised myself when I finally analyze all the data I collected.

I learned a little bit more about summative and formative evaluation. For my project, I use summative evaluation – the one which is done after the program or product has been finalized. As I have read, instructional designers are usually good candidates for conducting summative evaluations, especially for the expert judgement stage. Also, I did not have any idea that summative evaluation has so much to do with instructional design.

Evaluation Survey & Understanding Scoring Rubrics

This week I created the evaluation survey for my project. I still hope that I will get enough responses. The survey did not take me to long to create, it was straightforward. I worked in Google Docs and created a new form with all the questions. After reading the papers about creating questions for surveys, I understood how important it is to ask the right questions to get the data I really need for my project.

After I read the papers about scoring rubrics, I had to agree that rubrics are really important for scoring students’ work. I have not seen rubrics until I got to study at Boise State. In my country, the Czech Republic, nobody ever used any rubrics to score students’ work. Sometimes students felt that the teachers does not like them and that is why they received worse grade than their classmates. That is one reason I think that scoring rubrics are helpful. On the other hand, I think that students might get lazy sometimes and “cheat” a little bit on the homework. For example, if the students checks the rubrics and sees that if he does not finish all the requirements, but is still able to get an A, he might choose to do so. In my country, it was much harder to get an A. Everything had to be perfect. Even one small mistake caused a lower grade. But the scope of work was also different, so it is hard to compare. I am glad I learned how to use rubrics because I think it is important to give students know how are they going to be evaluated.

Group Project & Timeline

For the last few days, I have worked on the group projects. I have to say that it was really hard sometimes to work on a group project online. It is challenging for the group to accomplish a task when more people are working on it. I wish we had more time to collaborate on the Far West Labs Project. One of my classmates created a wiki so we can all edit the content. It was my first time using the wiki and I am glad I learned something new again.

We also started to work on the second group project. This one is really tough for me since I am currently overseas and hardly get a chance to be online. I am missing so much! It is very helpful to have group members with different backgrounds since everyone brings something to the table.

I started working on the final project. I created a timeline and EPD. This task was not that hard, but I am sure it will change a few times before I will be able to finish the whole project. When I started writing the timeline, I thought we have a lot of time to prepare the evaluation project, but as I look at it now I can see that there is not that much time anymore. I am planning on spending the whole next week on working on my project.

I have to tell that I already learned so much that my knowledge from this course can already be used for a good cause! Every Saturday I go with my daughter to Czechoslovak nursery school. The program started last year with just a few kids. This year, there is almost 30 kids enrolled! The non-profit organization needs help to evaluate the program since they did not count on so many new members. I offered help to evaluate their program and I am really excited that I can use the knowledge I learned in this class already. It will definitely help me with this small project. Maybe they will even be able to get some needed funding.

Problem Analysis and First Evaluation Project

When I was reading the required chapters about problem analysis, I was not sure if I completely understand the whole concept. To me it was always as easy as finding a problem and solving it. I did not have any idea it is such science to first discover that there is a problem, which can be a problem of the problem, or problem of the solution. Next, it is another problem to solve the problem, and more problems will show up after the problem has been solved. Does anybody understand that? I do not. After reading the three chapters, I was more confused than before. I will have to go back and reread them eventually. I just did not connect the dots. On the other hand, when I was reading, I was thinking: “Of course, I have to use intuition and experience”. I think that the way it was written just confuses me. A lot of the points the author made were just clear and I did not find it necessary to write three chapters about it. But I might be wrong, and probably I did not read between the lines.

This week I was working on the first evaluation project. It was challenging, but I definitely learned from that assignment. I am looking forward to work with my classmates on the final version of it. One of my team members created a wiki so we can collaborate. I have never used a wiki before, so it will definitely be something new and interesting for me. I did not have a chance to read my team members’ projects yet, but I would like to see how they differ from mine, if they used the same evaluation method as I did, and what kinds of data sources they recommended.

I still have to work on the evaluation test this week. I am learning something new every day, I love it!

Evaluation or Research?

I have to say that before taking this class the differences between evaluation and research never occured to me since I have a little experience with both. I would not think that people might have problems interchanging the two. Well, as I read the book and Dr. Perkins’ notes about the differences between evaluation and research, I started seeing why some people might get confused. In the beginning, I was confused as well, but after rereading the information, it became clear that research is not evaluation and vice versa.

This week I also reviewed a lot of terms and definitions. The way we learned the definitions was by writing our own definition of a term and reflect on it. This is the first time I have used this approach and I still do not know if it worked. I usually studied definitions by drilling, so this is much more engaging activity and I really appreciated the positive change. On the other hand, it was really hard for me to reflect on many of the terms since they were totally new to me. Moreover, in my culture, relfection is something that was never introduced in school, so I am having a little hard time to put my thoughts together and make connections while I learn. I hope that this task will become easier as I practice because sometimes it makes me really frustrated. Wasn’t I talking about this in my previous blog post somewhere…?

I really enjoyed reading about the project proposals of my classmates. I appreciate the diversity of the class and how each person takes different approach. What I missed last semester was that we were sharing ideas about project through the semester, but at the end, I was not able to see the final products. I was so looking forward to read those projects! I would be able to learn from my classmates and compare my project with theirs. I would also like to get feedback from my classmates since they have different views and it is great in the discussion forums. So hopefully we will be able to post the final evaluation projects and I will have a chance to see what my classmates created. I like that some of them are evaluating products, or applications, and others have other creative ideas, such as Joshua who will be evaluating how teachers are using lunch breaks to learn from each other. Isn’t it really smart idea? Well, I really learn from my classmates every day.

What I enjoyed was learning about the different types of sampling. It is interesting by how many methods we can choose a sample from a population for the project. I probably will use the judgment sample for my project since I will be glad to find people who are available to be a part of it. For the survey, I will most likely use google forms. It is easy to create and access.

Well, I am done reading the book. It was reallly fast, I think that I will go back many times to remind myself every aspect of evaluation as I evaluate the application I chose. I really enjoyed the reading, the book was very useful and full of great easy-to-read information. It is definitely a keeper.

Evaluation Models, Data Sources and Analysis

This week I found out that there is more than one model used in evaluation. To the most common models belong the discrepancy evaluation model, the goal-free model, the transaction model, the decision-making model, and the goal-based model. As I was reading about the models, I came across one question – if the person who performs the goal-free evaluation should be a subject-matter expert or not. In my opinion, it really depends on the situation and the actual program evaluated. Sometimes it is really necessary for the evaluator to be a subject matter because important information might be overlooked. It is definitely an area where I would like to do a little bit more research to see what others think and how to deal if such situation ever comes up.

I got a smile on my face when I started reading chapter 7 about data analysis. Well, there was nothing funny, but I finally found material that I am familiar with! When I was taking the two statistics courses at college, I thought that I will never need to know what standard deviation is, and here you go! It seems like it might become handy. Even thought I already knew about this topic, I was glad to refresh my memory about the most frequent item – mode, the average – mean, and the middle point – median. The authors of the book suggest using a statistics software to do the calculations. I wonder if we will be using such educational technology.. (Here I just connected my knowledge to my other class material where we just talk about the educational technology definition 😉

For my evaluation project, I decided to review and analyze a screencasting program for iPad. I will probably use the Bates’ ACTIONS model for my evaluation. It seems like one that would be most appropriate for this type of evaluation. Since I do not know what is the next step in the assignment, I just downloaded the application, but have not started using it yet. I am a little afraid that I will not have enough time to conduct the needed observations. Hopefully the program will be easy to grasp and I will find enough volunteers for my project.

Well, I have two more chapters to read. I cannot believe that I almost finished the book by now. I enjoy reading it, but I look forward to start with the project. It seems it will be fun…

An Introduction to Evaluation

Second week of school is over and I am already overwhelmed with a lot of new information. So far I learned what an evaluation is and why evaluate in the first place, what are some benefits and limitations of an evaluation, how to make sure that an evaluation will be successful, and how to use the results of an evaluation. I learned how evaluation fits into the planning process and how to develop an evaluator’s program description. All the information is new to me, so right now I am trying to digest it.

I have never thought that an evaluation can have some limitations. As I read the chapter about benefits and limitations of evaluation, I remembered one of my teacher who acted so differently when she was observed in our classroom during her evaluation. Since I do not have much experience as a teacher, I did not figure out that it is a connection until my classmate made a point about it in the discussion forum. Now I understand that evaluation can really have some limitations.

I found interesting that for evaluation to be successful, an evaluator needs to have experience and be credible. I was thinking how this work when an entry level evaluator creates an evaluation. Could this possibly be successful even though the evaluator is a beginner? How an evaluator gains the needed creadibility to start creating succssful evaluations?

So far it seems like an interesting topic to study. Sometimes I feel that the lack of my experience in the education field is a limitation to my learning. But as I read the discussion forums, I learn so much from my classmates. I appreciate that we have this opportunity since this helps me to connect the dots.

Another Semester Just Started!

It is the end of August and another semester is here. I am a little afraid, because I feel like I do not have ANY time to study. I hope it will get better soon.

This semster I am taking the 504 and 505 courses. First, I wanted to take three classes this semester, but my baby girl is so active, I thought it would be too hard to keep up with everything. So I decided to take only two classes. After reading the introductions for the classes, I feel like I should have only taken one class… Hopefully it won’t be that bad. Right now I am getting ready to start and learn. I am looking forward to it!