[Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment

[Joseph Suggs]

My comments/thoughts (apologies if any of these are duplicative of others'). And I'll also note that a large-district perspective may be quite different on the utility of this tool than smaller districts.

  1.  Is the purpose of the ORA clearly stated?
Yes
  2.  Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)?
I think so, although perhaps some things should be stated even a little more explicitly. Specifically, if I understand correctly, students must complete at least 20 items on the ORExt to count for participation regardless of whether the ORA is used.

What needs more clarity is whether this will be a required component or will be an optional tool. For Portland, unless this is mandated, I think it is extremely unlikely that any teacher will use it as we already have similar tools in place. In addition our teacher's union will likely see this as a workload issue if required and will want to take away one of our other assessments.

One more question: is the decision to use the ORA tied to student abilities on specific ORExt domains? I'm guessing most teachers would be able to figure this out, but may be worth making more explicit in the instructions.
  3.  Does the ORA appear to be a useful tool for the intended population (i.e., face validity)?
Yes, except that in Portland we already use similar (and slightly more comprehensive) tools that we would likely not replace with the ORA. Per comment above, if ORA is mandated for these students this will put us in a difficult position whether we have to drop something else (per union negotiation) that we would prefer to use over the ORA.
  4.  Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales?
The only thing not clear to me here is whether there is another (secure) document that includes prompts or interaction scenarios that would be used. The instructions give a fairly concrete example (using a beverage) but the scoring instructions are more generic. Is there a set of prompts that will be used for the ORA? If not, is this assessment time bound/on demand or can teachers rely on (recent) past experience to complete ratings? Most of the content on the rating is already being attended to in the classroom. I'm rambling a bit; I guess my suggestion is more clarity is needed as to whether this is a constrained assessment or an looser, non-time-bound observational tool.
  5.  Are the ORA scoring instructions clear?
Yes
  6.  Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt?
Maybe not useful for these severely impacted students. As previously mentioned, in Portland we already do additional assessments for this group of students and couple that with how closely teachers know and work with students already, the utility for the teacher seems limited.
  7.  Do you feel that clear directions regarding the use of the results from the ORA have been provided?
Yes
  8.  Do you have any recommended edits/changed to the ORA?
What is the anticipated PD addition for this assessment? The instructions seem relatively clear so that additional training would be limited; but in some cases, may be needed.
Joe

From: Assessment_Advisory_Committee [mailto:assessment_advisory_committee-bounces at listsmart.osl.state.or.us] On Behalf Of HEIDE Jordan
Sent: Friday, September 25, 2015 7:48 AM
To: assessment_advisory_committee at listsmart.osl.state.or.us
Subject: [Assessment_Advisory_Committee] Oregon Extended Assessment Observational Rating Assessment

We'd like to request your review of the attached proposed Oregon Extended Assessment Observational Rating Assessment (ORA)(see attached Administration Instructions and ORA) designed to provide instructional and functional information for teachers and parents in in four domains: attention, math concepts, receptive and expressive communication.  The ORA is designed to be administered to students who are not able to access the academic tasks presented on the Oregon Extended Assessment (ORExt), even when extensive, universal supports are provided. The ORA will be incorporated into the Extended Assessment beginning 2015-16.
Given the description and purpose of the ORA, we are seeking feedback on the questions below by 9/30/15:

  1.  Is the purpose of the ORA clearly stated?
  2.  Is the intended population who will participate in the ORA clear (i.e., is the decision rule used to determine whether to administer the ORA clear)?
  3.  Does the ORA appear to be a useful tool for the intended population (i.e., face validity)?
  4.  Are the ORA administration instructions clear, particularly in the areas of describing the LOI and COM scales?
  5.  Are the ORA scoring instructions clear?
  6.  Do you feel that IEP teams will be able to use this information for students who are so low functioning that they cannot access the academic demands of the ORExt?
  7.  Do you feel that clear directions regarding the use of the results from the ORA have been provided?
  8.  Do you have any recommended edits/changed to the ORA?

Please submit your feedback to Brad Lenhardt (brad.lenhardt at state.or.us<mailto:brad.lenhardt at state.or.us>). Finally, we would appreciate it if you would keep this information confidential at this time. Once we finalize the ORA we will distribute it through the typical ODE communication channels.

Sincere thanks for your expert consideration and assist
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omls.oregon.gov/pipermail/assessment_advisory_committee/attachments/20150929/36fe654e/attachment.html>