.

This document is currently

UNDER DEVELOPMENT

Core concepts are diagrammed, but not connected and described. NOT READY FOR REVIEW. Expected completion date: Feb 2018.

Low-Stakes, High-Value Assessment Systems

Explorations in computer fundamentals and Java programming

This document describes an assessment system that expands the traditionally test-centered approach to include informal, frequent micro-assessments and expand the use of formative tools.

Informal Formative Summative
Focus: Single skill
Reference: Activity /exercise
In-person & Non-screen
Conversational
Low-stakes
Iterative/Cyclic
Minimally documented
Highly individualized
Focus: Bundle of skills
Reference: Lesson
Mostly In-person & Low-screen
Quickly evaluated & returned
Rarely "graded" for points
Group-checked when possible
Often lesson-based
Focus: Mastered skills
Reference: Unit
Minimized stakes
Cumulative
Revealing of mastery
Exploration-inducing
Interesting to students
Multi-rigor

Example: Circulating the room, inviting students to explain various elements of an in-process flow-chart of a computer motherboard. Offering micro-feedback as one walks around. Could be called: Assessment By Walking Around (ABWA) Example: Asking students to write 1-3 lines of Java code which capture the essence of a lesson on method calls on a 1/4 sheet of paper. Scanning those slips as they're turned in. Offer group and individual feedback at the start of the next class. Example:

Jump to a section:

Thinking framework

System design rationale: Critique of test-centered assessment
Diagrammatic model of assessment system
Case study module overview
Role of system diagramming in technical fields
Links to companion documents
Comparing summative-driven and formative-driven assessment models

Assessment tools explored:

Informal assessment profiles & student work artifacts
Formative assessments profiles & student work artifacts
Summative assessments profiles & student work artifacts

Supporting sections:

Ideas for future system development
Theoretical influences and additional reading
In defense of documentation
Author background


System design rationale: Critique of test-centered assessment

Many teachers root their assesment systems in a schedule of high-stakes, class-standardized quizzes, tests, and exams. While potentially a useful tool for certification and formal skill evaluations, exam-based courses often induce unproductive stress as well as a cycle of binge-studying and information dumping.

Class-standardized instruments administered in introductory courses also tend to either frustrate lower-proficiency students or fail to challenge higher-performing students.

Model 1: Balanced coverage of course streams with assessment tools
diagram of assessment tools
This model constructs a course at a post-secondary level from three streams of attention: curricular,

The following document outlines an assessment system built around low-stakes assessment tools which yield frequent, high-resolution data that can actively influence how we can best guide our students' learning process. By centering assessments around a handful knowledge and skill sets which are central to our course, we can draw insight from our assessment data cumulatively as we "spiral up" the rigor of practice.

Diagrammatic model of assessment system

Model 1: Balanced coverage of course streams with assessment tools
diagram of assessment tools

Case study module overview

Module Objectives

Our discussion of assessment tools is based on the development of a course module (i.e. a mini-unit) which was scheduled over three 1.5-hour class periods. The objectives of the lesson module are included in the opening of the module guide and are listed below:

  1. Describe the function of each core computer component in terms of the role it plays in routing and processing digital data
  2. Illustrate the relationship between each core computer component using a system diagram (components and flow relationships)
  3. Discuss the value of determining system “bottlenecks” for improving how a system works.

Discussion of objectives

This module took final form through a few attempts at teaching the function and relationships of computer components less successfully than described in this document.

Informal assessment data gathered during a more traditional instruction-at-board approach suggested that fundamental thinking skills in terms of system diagramming and component relationships was not strong enough to support the modeling of computer components with which the students were not at all familiar.

Objective 1 demands an understanding of both what computer components do and how their basic input --> output structure allows them to connect to other components in the overall system. The requirement to frame the component's role in transferring digital data requires a general understanding of what kind of tasks are actually underway inside the components being modeled.

Objective 2 is the core objective discussed in this case study: students are asked to use a system diagram to describe their knowledge of computer components. We could describe this objective as a "composite" aim because students must combine knowledge of component functions with the skill of conceptually modeling relationships between components.

Objective 3 asks students to synthesize their knowledge of a computer system and its constituent components and their skill of system diagramming to generate an explanation for a computer behavior, known as bottlenecking. This third objective is assessed directly in the module's summative assessment.

Diagram of module flow and applied assessment tools

The following diagram depicts the planning and execution sequence of the computer component module alongside the assessment scheme used during the module. The sequence flow of the module is depicted in rounded boxes down the center of the diagram. Those sequence items are grouped into more traditionally known lesson phases that roughly correspond to the three classic lesson phases: teacher presentation ("I do"), supported/guided student practice ("We Do"), and independent student exploration ("You Do").

Model 2: Assessment tools applied throughout the learning module
module sequence flowchart
This case study discusses how informal, formative, and summative assessments were implemented during a learning module designed to teach students knowledge of computer components and the skill of system diagramming. Informal and formative assessment tools are used frequently and iteratively throughout the lesson and allows the teacher to guide skill development at a much finer resolution than traditional summative-heavy assessment approaches.

informal assessment model

Section 2: Informal assessment profiles & student work artifacts

2A: Summary of Informal Strategies

Tool Use Example Summary of data Impact on course execution
Example 1: Identifying flows in system diagrams
Circulating the room as students designed their system diagrams. Asking the question: What components is this part connected to? What stuff is flowing between them? How can we show the flow on the diagram?
Most students were comfortable connecting components in a diagram with a line, but many fewer had declared what, exactly, was flowing between them. Additionally, many students who were diagramming a car were not isolating a particular sub-system in their diagrams: brakes were linked with a line to the engine, for example. Designed a system diagramming exercise using my rainwater catchment system which involved explicit flow of water and electricity. Water flows in pipes, and electricity flows in wires. Students watched a series of videos of me showing and describing various subcomponents of the rain water system. Students diagrammed each sub-system during the videos and revised their diagrams at the conclusion of the exercise. Check out the system diagramming guide used in this activity.
Example 2: Method calls in Java
Responding to a student question in Java about why the code they had just written was not being executed by the compiler. I turned this question into a mini informal assessment with the following questions:

"What is the only method that the compiler runs without any further instruction from the programmer?"" (the main() method).

"If the compiler only runs main(), how can we direct the compiler to run this other block of code you placed in its own method?"
The student was familiar with the concept of a method call, but had not internalized the relationship between the concept of a method call and the sequential execution of java code through various methods. I had learned through previous informal assessment that the student gets overwhelmed by non-linear program flow. Designed a diagram for mapping method calls and flow of execution that attempts to show the transfer of data from method to method through method parameters and return values. Check out the method diagram designed in response to this assessment data.

2B: Instrument examples and results

These abbreviated examples show evidence of the student work assessed during informal assessment tools and, where possible, include examples of revised student work based on the immediate feedback of the informal assessment.

2Biii: Car system diagram: informal assessment and feedback

One exercise in the module sequence requested that the students draw a system diagram of a vehicle. We started with an open-ended task to simple "diagram a car" and iteratively moved into diagramming a single subsystem of a vehicle with flows between components.

I provided individualized feedback to the student who created this car diagram which encouraged the labeling of what, exactly, is flowing between components in the car?

Student work: Cary subsystem diagram and flows between components
Diagram of car subsystem
The exercise underscores the value of exploring concepts on paper through diagramming and flow charting: as this student was adding components to the model, I was able to both conduct an informal assessment and provide feedback on the model in one fell swoop. My guidance was to include a flow label on each line that had previously been drawn. "Gas" was added which I later boxed in green for emphasis.

Based on this feedback that was given during an informal, one-on-one assessment, we can see that the student internalized the principle of including what is flowing in a flow chart diagram. The ultimate goal of this lesson was to create a flow diagram of the core components of a computer system. The artifact below shows the same student who drew the car correctly and thoroughly labeled the flows between computer components

Diagram of car subsystem

arrow_upwardup to contents


Formative assessment profiles & student work artifacts

formative assessment model

Formative assessments are handy tools for "snap-shotting" student progress at the conclusion of a discrete lesson activity. In our case study, I reviewed student work produced during various exercises in the module to tweak how I implemented the next item in the sequence. This section discusses formative assessments theoretically and then walks through student data I gathered using formative assessment tools.

The good ole' exit ticket corrupts formative assessments: A example from high school

Quite a few secondary lesson models involve the collection of formative data in the form of end-of-lesson snap shots, often called "exit tickets". This is an apt name and probably hails from the shared teacher experience of confronting a rush of high schoolers trying to turn in their mini-quizzes to earn their permission to exit the classroom and scurry off to their next period class in the alloted three minutes and zero seconds.

I share this bleak anecdote from the navigation of packed high school schedules to illustrate how perfectly wonderful assessment strategies can be corrupted by their institutional and instructional context.

If students feel "rushed" in working through an assessment tool, the experience for the student was probably unpleasant--which is toxic to meaningful learning. Since this was a consistent experience of my students in my and many other classes, I needed to have revised my approach. (This document showcases that now-tardy revision.)

From a data perspective, my instrument validity plummeted when my students completed them while hurried and distracted.

Model 3: Eroding formative assessment value with a ticking clock
What happens when formative assessments are squished
Due to over-runs on time allotted to core lesson material and practice, students are often asked to "take" an assessment instrument without enough time to give thoughtful effort to each of the items. A similar pattern also occurs when longer tests or exams are administered and a fair chunk of students do not finish--either the test instrument should be shortened or more time should be given. Asking students to rush destroys assessment validity and annoys students.

My formative assessment system also was administratively broken: By requiring students to turn in their exit tickets at the end of each class session, I accumulated (in a very ugly pile) upwards of a hundred of these one-page quizzes over the course of a single day of teaching. The documentation process alone was tedious and breathtakingly time consuming.

At the request of the school administration, I was requested to score and document each student's proficiency rating in a spreadsheet, which was later scrutinized by my manager. This meant the quizzes had become high stakes for the students and for me--which unnecessarily strains the organic process of learning and teaching. Finally: the graded instruments were (rightfully, indeed) supposed to have been passed back to students the very next class session. Rarely could I or my many of my colleagues keep this boat afloat. The tools were reduced to the rubble of administrative annoyance.

After transitioning out of secondary education, I vowed to never again lay my eyes upon another exit ticket. This document is, in effect, about the revival of the formative assessment's spirit, leaving behind my poor implementation of this tool while teaching at the secondary level.

Formative assessment strategies for the post-secondary teacher

We can check on the progress of students on a lesson-by-lesson basis in a way that yields useful data and does not frustrate us or students. The following table summarizes a few such strategies.

Summary of Formative Strategies

Tool Use Example Summary of data Impact on course execution
Example 1: Baking formative assessment tools into lesson activities

Ask students to create a system diagram based on the information gleaned from watching the video-taped system tour. Students submitted their diagrams through their portfolio folders.
Student proficiency ranged from unable to differentiate system diagram component names (input, choice, output, etc.) from physical components to correctly diagramming each sub-system using the diagram components classifications. Impact 1:

Request & support the labeling of sub-systems within the core computer component set

Impact 2:

Impact on teaching: SLOW DOWN! Introduce the component types (Decisions, inputs, etc.) with simple example

Impact 3:

Impact on teaching:Increase support of more systematic student taking habits
Example 2: Under development

Instrument examples and results

I designed the formative assessment tools in this system diagramming module as the core output of our discrete module activities. The following work samples were produced by students after an hour-long series of steps to learn about system diagramming of components that are tangible (cars and water control systems).

After reviewing all the student diagrams, I can sketch out a range of skills present in the class. The distribution of the results range of skills becomes the spectrum of response proficiency

Overview of student work samples diagram

Student work sample A: Lower-proficiency response

Student work showing basic/developing proficiency in system diagramming
System diagramming student work A

Student work sample B: Median-proficiency response

Model 3: Eroding formative assessment value with a ticking clock
System diagramming student work B

Student work sample C: Higher-proficiency response

Model 3: Eroding formative assessment value with a ticking clock
System diagramming student work C

arrow_upwardup to contents


Summative assessment profiles & student work artifacts

summative assessment model

Summary of Summative Strategies

Tool Use Example Summary of data Impact on course execution
Example 1: Identifying flows in system diagrams
Example 2: Method calls in Java

Instrument examples and results

Diagram of car subsystem Diagram of car subsystem

arrow_upwardup to contents


  • Represent a system through a symbolic process chart which shows each component and the various flows (data, water, electricity, etc.) to connected components
  • `