Standard 5: Evaluation

Candidates demonstrate knowledge, skills, and dispositions to evaluate the adequacy of instruction and learning by applying principles of problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.

  • Camtasia Captioned Screencast

    Artifact: Camtasia Tutorial Website &
    Captioned Screencast

    Context: The tutorial website and supporting development wiki was created in Spring 2011 as a part of EDET 703 Design and Development Tools II. This website is a self-contained instructional module designed to teach university level instructors on the use of Camtasia Studio screen capture and editing software.

    Conditions: The Camtasia tutorial was a product of a group. One of the interesting challenges was using the Camtasia screen capture software to capture the use of Camtasia itself. This made for confusing editing to say the least!

    Scope: The tutorial and development wiki is available on the internet. It was implemented in Spring 2011 with a group of instructors in the College of Education at USC.

    Role: This was a group project. As the designated narrator, I was responsible for creating, capturing, and editing the training videos themselves. This included adding closed captioning for the videos. I also developed the self-assessment used by our target learners.

    ADDIE: The steps of the ADDIE process were followed directly, as shown on the companion development wiki site. The tutorial site that was designed and developed grew out of learner analysis. After the target learners utilized the Camtasia tutorial, they responded to an online survey as evaluation.

    Performance indicators:

    5.1.1 Identify and apply problem analysis skills in educational technology contexts (e.g., conduct needs assessments, identify and define problems, identify constraints, identify resources, define learner characteristics, define goals and objectives in instructional systems design, media development and utilization, program management, and evaluation).

    A comprehensive analysis of the instructional goals, learner entry states, and the context was used to direct the creation of the tutorial.

    1.1.2.b Create instructional plans (micro-level design) that address the needs of all learners, including appropriate accommodations for learners with special needs.

    Though not required by the assignment, I decided to include closed captioning in each of the training videos. As seen in the survey responses, users of the tutorials (who were not hearing impaired) appreciated having closed captioning available. This shows how captions can work as Universal Design and benefit all learners.

    1.1.5.a Utilize a variety of assessment measures to determine the adequacy of learning and instruction.

    1.1.5.b Demonstrate the use of formative and summative evaluation within practice and contextualized field experiences.

    5.2.1 Develop and apply criterion-referenced measures, formative and summative evaluation strategies.

    Assessments used in the evaluation stage include both formative assessment in the form of a survey and summative assessment in the form of a sample task. Survey results showed that the design of the site and videos were easy to follow, but that the differences in the Mac vs. PC version of Camtasia needed to be addressed.

  • Top
  • Artifact: Basic Principles of Waveforms

    Context: Created in Fall 2009 as a part of EDET 722 Instructional Design and Assessment, this wiki shows the stages of the instructional design process applied to a single module of material. That module covered audio waveforms and their basic parameters such as amplitude, frequency, polarity, and phase.

    Conditions: This was my first course in the EdTech program. The fundamental instructional principles are solid, but the delivery of what is the slide portion of a traditional chalk ’n’ talk lecture does not translate well to a slide show video.

    Scope: The development wiki is available on the internet. The instructional content (slideshow and audio examples) that was developed throughout the project has been incorporated into the first module of MUSC 365 and it currently utilized each spring semester.

    Role: I am the sole creator of this content.

    ADDIE: The wiki explicitly steps thru the ADDIE process, beginning with a review of the Instructional Design process itself and a comparison of several ISD models. The Evaluation used former students of MUSC 365. They viewed the slideshow video and completed an online survey. Based on their feedback, slight modifications were made before incorporating the material into regular classroom use.

    Performance indicators:

    5.1.1 Identify and apply problem analysis skills in educational technology contexts (e.g., conduct needs assessments, identify and define problems, identify constraints, identify resources, define learner characteristics, define goals and objectives in instructional systems design, media development and utilization, program management, and evaluation).

    Extensive analysis of needs, learners, content, and objectives was used in the design and development stages. Problem analysis skills were also required with the media utilized in this example. I was faced with the problem of how to present a slideshow with embedded audio examples, since SlideShare (and other slideshow hosts) cannot include audio. To overcome this limitation, I captured a QuickTime video of the slideshow and audio.

    5.2.1 Develop and apply criterion-referenced measures, formative and summative evaluation strategies.

    As shown on the wiki evaluation page, formative and summative evaluation are addressed. The results of the summative survey with a group of students was used to adjust the delivery of the content.

  • Top