Designing Economics of the Environment – Part 3 – Assessment Strategy

~2000 words, ~10 min reading time

Part 1, Part 2
So, in Part 2, we learned that my students will have lots of ‘splaining to do, as the verb “understand” in my course learning objectives more closely align with the facet of understanding called “explanation”. So, that should play some role in my assessment strategy.

Elements of Assessment

A few elements that I’ve decided need to be part of this, for one reason or another:

(1) “Chapter Assignments” – to convince students to read, they have to, in some way, respond to that reading. So, I’m going to have my students do Chapter Summaries. Based on the course learning objectives, I’ve decided to make these written assignments where students select some combination of economic models, problems, and solutions, and explain them. The main purpose is to get students started thinking about these, and convince them to at least skim the chapter with enough attention paid to write a summary of it. Graded for completion.

(2) “Article Responses” – this is an opportunity for students to go a bit deeper into topics they find interesting. They will read papers from academic journals (or similar sources). Totally their choice. The purpose is to get students to engage with the material a little beyond what we do in class. Graded for completion.

(3) “Case Studies” – the department declares that “problem sets” are part of this course. The professor who has taught this before has provided me with the details for 5 case studies that they assign. These are reasonably advanced assignments (he noted that his students needed help with these), and require some of the more technical skills that we’ll be talking about. This hits the “Applying” aspect of “understanding”. Here, I will want to grade for quality rather than just completion.

(4) “Term Paper” – the department declares that a “research paper” is part of the course. So, here I’m doing that. I’m forcing students to go through a 4 step process – proposal, annotated bibliography, rough draft, and final draft – with a response to comments and a reflection.

(5) Reflections – every assignment has a required “From this assignment, I learned that…” section. This is to encourage reflection. Also, the form of this statement is important. “I learned that” forces a sentence to follow. “I learned…” doesn’t. Example: “From this assignment, I learned about elasticity.” That’s just a word. Only tells me that you saw the title of the chapter. Not nearly as good as “I learned that elasticity is how much quantity responds to a change in price.” In addition, I have built in “Midterm Reflections” every few weeks, to encourage students to keep track of their progress in the course. These, however, are not required.

(6) Final Exam – I’m going to follow Linda Nilson’s advice and give them the final exam early in the course – ideally, on the first day, if I can get my act together. Final exams of some variety are required by the University, and I think they are a good way to see what students actually absorbed. Because this can’t be revised, I do grade this one with partial credit.

(7) Revisions – Students are allowed to revise any unsatisfactory assignment except the final exam, as long as they submit a “Revision Form”. (This is a Word document where the student has to answer four questions: which assignment they’re revising, why they didn’t meet specs originally, what they changed to meet specs, and what they plan to do to ensure they meet all specs in the future.)

Specs Grading – The Core Concept

The core concept of specs grading is grading assignments on a pass/fail, satisfactory/unsatisfactory basis on the basis of clear “specifications”.

What these specifications look like is up to your goals for the assignment. A good rule of thumb: look at the rubric you use now. Write your specs to be somewhere around the top 2 levels of your rubric. Naturally, you don’t want to be too mechanistic about doing it this way – take the opportunity to think about WHY things are in the rubric and what is really acceptable and what would lead you to hand the paper to the student and say “do it again”.

For my purposes, I write up two sets of specs, depending on the assignment: format specs, and content specs. Format specs lay out things like length, file type, work cited pages, numbers of sources. Content specs say what should be in the paper. If I’m grading for quality, the content specs will include descriptors like “correctly” or “reasonable”. Otherwise, they won’t.

Just as a couple examples: for my chapter assignments, students must meet these specs:

A satisfactory submission will:

(1) Be at least 300 words.

(2) Be submitted as a Word document.

(3) Contain a summary that deals with at least one of these elements: (a) economic models, (b) problems in the economics of the environment, or (c) solutions to these problems, and for each of these you must include:

(a) For economic models – describe the economic model, explain its assumptions, uses, and limitations.

(b) For problems – describe the problem, and the conditions that lead to it occurring. Examples are encouraged, as appropriate.

(c) For solutions – describe the problem being solved, the solution, why the solution could alleviate the problem, and any limitations to the solution.

(4) End with a paragraph starting “From reading this chapter I learned that…”

Remember, this is a “completion” grade. For these, I can use “power grading” – or what I like to think of as “grading at a glance”. Import the file to word, check the word count. Check the last paragraph. Skim the middle to make sure it’s relevant. Done. Doesn’t need to take more than a few seconds – but as a written assignment, I can get some “deep” data about what students think is worth including and what they think they are learning.

For the final draft of their term paper, my students will have this set of specs:

To be satisfactory, the final draft must be at least 1,000 words long, be

Designing Economics of the Environment

~700 words, ~ 3 min reading time

Part 1 here.

So, in the previous part, I laid out my course learning objectives for Economics of the Environment, putting the verbs in bold.

1. Students will understand economic concepts, models and tools for analyzing environmental and natural resource issues and problems.

2. Students will understand problems that arise in the efficient use of depletable and renewable resources, and understand potential solutions to these problems.

3. Students will understand problems that arise from the use of environmental resources such as air and water, and understand potential solutions to these problems.

Normally, the verbs in learning objectives provide a great clue for what kinds of assessments are appropriate for the course. If a learning objective says that students should “explain” something, you better ask them to “explain” it!

The verb “understand”, though, leads to…

The Problems with Understanding

There are two main issues with the verb “understand”:

(1) It can have multiple meanings, which vary by context.

(2) “Understanding” is inherently internal, which makes it impossible to observe – let alone assess.

Good news! There is a solution! Thanks to the work of Wiggins and McTighe on “Understanding by Design”, we have a way to translate the internal, unobservable “understanding” into observable activities that we can actually assess – the trick is actually to recognize the different meanings of “understand”, and then to assess based on those.

Wiggins and McTighe suggests there are 6 “facets” of understanding:

(1) Explaining – that is, the ability to connect “cause” and “effect”.

(2) Interpreting – that is, recognizing the meaning or importance of a concept.

(3) Applying – that is, being able to take a concept and use or recognize it in a different context.

(4) Shifting Perspective – that is, the ability to approach an issue from multiple points of view.

(5) Empathy – that is, the ability to imagine being in the place of someone else.

(6) Self-knowledge – that is, being aware of one’s own mastery and limitations.

Now, not every one of these “facets” is present in every learning objective that uses the word “understand”. For example, it would be silly to think that “understand” in “Students will understand economic concepts” means “empathize with”. However, “apply” and “explain” could pretty easily fit in there.

Translating “Understand”

So, a little rewriting, then. I’m going to take out “understand” and put the relevant verbs in…

1. Students will explain and apply economic concepts, models and tools for analyzing environmental and natural resource issues and problems.

2. Students will explain problems that arise in the efficient use of depletable and renewable resources, and explain potential solutions to these problems.

3. Students will explain problems that arise from the use of environmental resources such as air and water, and explain potential solutions to these problems.

Much better! Now, when looking at it this way, it’s clear that my students will have “some ‘splainin’ to do”! Also, I should be asking students to apply the various concepts, models, and tools in a variety of relevant problems. This, naturally, should have an impact on the kinds of assessments I give – specifically, this is not a class where multiple choice questions are going to do well assessing the learning objectives, unless I spend a lot of time crafting very good multiple choice questions (which is possible!). Probably just easier to have the students write – especially since I use Satisfactory/Unsatisfactory grading with revisions, which makes grading writing MUCH easier. Given the nature of the tools we’re using “applying” will require doing some problems, as well.

The Next Task

Now that I know what types of assessments I’m going to be giving (lots of writing), the next step is to figure out the content – in the learning objectives, there are three types of content mentioned:

(1) Concepts, models, and tools – I group these together, because, in this context, there is very little point in distinguishing between them.

(2) Problems

(3) Potential solutions to those problems

So, this provides a framework for me to start putting content into. Now, I have quite a bit of reading to do before I can really complete this… So, I’ll stop here for now.

Designing Economics of the Environment – Part 1 – General Principles, Backward Design, and Course Learning Objectives

~900 words, ~4 min reading time

Part 2 here.

So, next semester I’m going to be teaching Economics of the Environment for the first time. Because I think some people might find it interesting, I think I’m going to post some of my thinking process as I design this course over the break. So, I’m starting here today with some “initial thoughts”

General Principles

(1) I’m going to use Specs grading – I’ve been very happy with this so far. So, I’m going to stick with that.

(2) The Final is going to provide a “grade modifier” – where performance on the final will adjust the course grade based on other assignments by +1/3 of a letter grade to -2/3 of a letter grade. This gives the final a reasonable weight, but also ensures a clear “floor” to a student’s grade going into the final.

(3) The assignments are probably going to be a combination of reading responses, problem sets, and a research paper (for students who aim for an A). Also, need to ask metacognitive questions for all of these. (Even a simple “what did you learn while doing this assignment?” would work.)

(4) Deadlines scattered throughout the semester.

(5) I’m debating about giving the students the Final Exam on Day 1. When I heard Linda Nilson speak this year, she suggested this. For essay-based finals, I think this is pretty brilliant. As she pointed out, if you write a good final, you’re going to have students having the exact discussion that you want them to have over the course of the semester if you do this.

Planning Materials

While I believe very much that each of us needs to make a course “our own” in some fashion, I also believe that there needs to be enough commonality across instructors for a course that it can meaningfully be considered the “same course”. To that end, I have the following…

(1) Our Department has Course Learning Objectives for all courses. These have to be listed in the syllabus, and provide a good way too focus the course.

(2) Our Department also has “Basic Course Sheets”. This includes the Learning Objectives, the description in the catalog, and also a description of what in-class and out-of-class activities are used in the course.

(3) For courses that have been recently created or revised, we have “Course Information Sheets”. These add to the mix an example textbook or two, and a schedule with the amount of time spent on each topic in the course. Unfortunately, this course has not been updated recently, so getting one of these is extremely difficult.

(4) When I start teaching a course that someone else teaches or has taught, I often get whatever they’re willing to share with me – I usually just ask for a syllabus and thoughts on textbook choices, but often get more than that. The professor who teaches this course at the Kent campus was EXTREMELY generous, and provided syllabus, assignments, lecture notes, etc. I have LOADS of material here. Obviously, I’m not going to just use everything I was given – I have to adapt things to my style. But, editing a pre-existing course is much easier than reinventing the course from scratch.

Backward Design

“Backward design” is kind of the “state of the art” in course design. The idea is simple enough. While “forward design” starts with course activities/content, and then tries to figure out how to assess that, “backward design” starts with learning objectives, then moves to consider assessment, and finally designs the course activities with those assessments in mind. Now, the reality is that it’s a recursive process. Sometimes, when you’re teaching you realize that there are hidden learning objectives that you hadn’t thought about. Or, you realize that the assessments don’t *quite* assess what you wanted them to. Or that there are missing or extraneous course activities that don’t reinforce the assessed learning objectives. Alignment is a bit of a dance, and it takes a few iterations to get it right. (Example: when I taught stats this semester it was RADICALLY different than the last time I taught it.)

But, using a backward design, it turns out, works well as a first pass on the more realistic “recursive design” that actually happens. So, let’s take a look at my learning objectives.

Course Learning Objectives

Based on what I got from the Department, these are the three course learning objectives:

1. Students will understand economic concepts, models and tools for analyzing environmental and natural resource issues and problems.

2. Students will understand problems that arise in the efficient use of depletable and renewable resources, and understand potential solutions to these problems.

3. Students will understand problems that arise from the use of environmental resources such as air and water, and understand potential solutions to these problems.

When moving from learning objectives to assessments, the key is to look at the verbs in the learning objectives, as they hint at the kinds of assessments you should do. “Identify” is a different kind of assessment than “explain” which is certainly different than “apply”. (“Identify” often lends itself to a matching/multiple choice assessment strategy – though you can always frame these as short answer. “Explain” often lends itself to short answer or essay – but a well-crafted multiple choice question can do the same thing.)

As you can see, there’s a strong consistency in the learning objectives for this course – students will understand. It turns out that this is a particularly difficult verb to assess – but more on that in the next installment!