Ah, rubrics. Ah, humanity.
Discussion of Creating Rubric
Tasked with developing a rubric for an assignment that was already completed, and applying it to content created by collaborative colleagues, rather than developing a rubric prior to the assignment and using it for content created by students who are in a more hierarchical position, I first thought about what the tool should do. I decided that it should be a generic set of questions that advanced thinking about the theory and its application, and set up a framework for a true assessment, which, as authors in Digital Writing Assessment and Evaluation and elsewhere have noted, should be hyper-local to reflect the exigencies of a particular assignment, the culture of the institution in which it is situated, and the population being assessment. Since the requirement was for it to be a rubric about applying a theory, and not a theory of networks per se, I did not feel that I could start at the logical place, with the questions asked of us on the first case study, reflecting the parameters of the assignment:
- How does the theory define your object of study (as a whole, broken into pieces)?
- What and/or who is a network node?
- What types of agency are articulated for various types of nodes?
- How are different types of nodes situated within a network?
- What are the types and directions of relationships between nodes?
- What happens to content or meaning as it travels through a network?
- How do networks emerge, grow, and/or dissolve?
While normally I would have turned to assignment objectives and guiding questions, such as those above, because so many of the questions were directly related to networks, and not application of a theory, I took the first question: “how does the theory define your object of study, as a whole and broken into pieces?” and used that as the basis, breaking that question down into component parts that I divided into two main categories: articulation of the theory and its context, and application of the theory to the object of study.
Daniel and I collaborated on a rubric as we thought it would make sense to both develop a rubric and apply a rubric developed by another (both of which are required in teaching). I began with the categories and a draft of the questions; Daniel and I discussed and tweaked questions/attributes, and he added a third category regarding local instance of the OoS. I then added the Gold, Silver and Bronze categories below, as a way of rating each category, while he used them as the binary “Yes” or “No”. We posted the link on our Facebook group, and Amy and Jenny also visited the rubric to offer some comments.
Blank Rubric:
Rating: (with figure skating analogies. I will not be the Russian or Ukrainian judge.)
Gold — Clear, sound, complete, cogent, says something new. You had the difficulty and landed the jumps.
Silver — Mostly clear, some gaps or rough patches, tends to repeat what is known but may have surprising insights at places. Possible two-footed landings and moments of stumbling, but the crowd loves you and the overall impression is positive; took some risks to earn reward.
Bronze — More nascent view, larger gaps in explanations, reasons; ideas are sound but could be improved with more “fleshing out”; you’re at the games and at the right competitive level, you have the moves, but this particular performance doesn’t demonstrate your full potential.
|
Theory Clearly Articulated and Contextualized |
Rating: |
|
|
|
Theory Clearly Applied to Specific OoS and Explained |
|
|
|
|
Theory Mapped to Local Context (Praxis) |
|
|
Rubric Applied to Summer’s Case Study of MMO Guilds
I felt a bit awkward using the rubric with a classmate’s work, as rubrics connote “assessment” rather than “feedback.” I don’t have a problem giving critical constructive feedback that may point out that the item is at the “silver” or “bronze” level, but giving it that label implies a grade that I don’t feel qualified to give, and I don’t want to risk a collegial relationship by appearing to be superior (the position from which assessment generally comes). Thus, I qualify that these are my impressions, and that the levels of Gold, Silver and Bronze are all “top finishers” who are on the podium, distinguishing themselves from the field. My attempts to identify areas of where further attention might be given may point out flaws with the reader and her understanding rather than the writer and hers.
|
Theory Clearly Articulated and Contextualized |
Rating: |
Discussion: |
|
Bronze | Brief mention of the theory and a single quote. Mentions again at the end but most discussion is of the guilds themselves. Concepts such as felicity, genre, typification, or Bazerman’s overall take not explained or contextualized. Thus, it is difficult to know what Bazerman is saying and how Summer is considering Bazerman’s premises (her interpretation of them). |
|
Theory Clearly Applied to Specific OoS and Explained |
||
|
Silver | Summer does a great job explaining what guilds are, what they do, how they operate, and the difference between game-global and game-local (although I think that distinction is lost some in the discussion). I remain uncertain how the various parts of a guild, such as a perk, an application, the bank, the discourse, the mentors, correspond to parts of Bazerman’s theory and I am not sure how Bazerman helps me understand guilds in a different way. |
|
Theory Mapped to Local Context (Praxis) |
||
|
Bronze
N/A |
This is part of our rubric, but not necessarily part of the assignment, so it is understandable if it would not appear. Summer hints at the idea of “game-local” in the beginning, so I hoped I would have Bazerman’s concepts illustrated with an actual instance of game-play in a guild, where I could see the concepts in action. In such a short case study, though, this is impossible, and thus would be something for continuation if this approach were expanded. Screen shots and embedded videos helped with demonstrating a local instantiation of the game and guild activity. |
Discussion of Applying Rubric
I found it somewhat difficult to apply the rubric to the Case Study #1, since the assignment was not for a full application of a theory (which is the rubric I developed) and was more of a “sandbox” attempt at moving toward a full application of a theory. Thus, it doesn’t seem it *could* have scored Gold, since the writer wasn’t asked to do all that the rubric asked. However, applying the rubric did help me identify and quantify some gaps in Summer’s Case Study, which did a fantastic job explaining the Object of Study (which would have to be done in any article or research piece about it) but spent less time in the theoretical lens being applied, which was the object of the assignment. Summer did a great job using hypertext to extend her text without impinging on her word count parameters, so that guilds could be defined and examples of applications provided. She contributed to my understanding guilds and how they affect play in WoW, but I was unable to learn how these guilds are a genre system, and I left the case study still hoping for a discussion of this very interesting premise: “Bazerman’s theory of speech acts and systems of human activity can define the local level of MMO guilds through interactions between players and the cohesion and disruption felt once those interactions begin to collect into trends and movements” (emphasis mine). What Summer has set up in this discussion is the clear proof that the WoW guilds are an object worthy of study, that they are a network of people articulated primarily by speech acts, and that this network influences game play and player affiliation. I still want to hear HOW. I hope she explores this further.
Conclusion
Lastly, I think the rubric that we developed is helpful in thinking about the components required in applying a theory, and striking the balance between enough context and explanation of both the theory itself and the OoS, and spending enough time tying the two summarized and contextualized pieces (the theory and the OoS) together. It helped me clarify what I need to do with my own Case Studies.
Image from: http://booklovingfool.wordpress.com/2013/05/22/oh-bartleby-oh-humanity/
One Response to Theoretical Application Rubric –> Summer’s MMO Guilds