[R2R-OR] What to write on my application about evaluation?

Katie Anderson katie.anderson at state.or.us
Wed Jul 30 08:57:51 PDT 2014


Hi! I'm starting to get questions from some of you about the 2014-2015 Ready to Read Grant application which is must be post-marked August 31, 2014. Most questions are about the evaluation so I'm resending the outcome based evaluation guidance I sent you on June 18th.

In addition, here are a few reminders:

*         It's okay to select only one outcome for each project. Only select the one, two, or three outcomes you really want to work hard at achieving. It may be appropriate for large libraries to select more than three outcomes depending on their projects.

*         You only have to do one thing to evaluate for each outcome. For example, if increasing family participation in summer reading is the outcome you select, then you might evaluate that through family reading logs. That's the only evaluation you need to do for that outcome. You don't have to also collect feedback from families and take photos during events-you can, but you don't have to.

*         Think about what evaluation method(s) will work best for staff and their workload.

*         Think about whether or not you can use the data you collect in your evaluation(s) for other things in addition to your Ready to Read report.

Please contact me if you have any more questions.

As my colleague Ann Reed says, "Call early, call often!"

Thanks,
Katie


Katie Anderson, Library Support and Development Services
* Youth Services Consultant * Oregon Center for the Book Coordinator *
Oregon State Library, 250 Winter St. NE, Salem, OR 97301
katie.anderson at state.or.us<mailto:katie.anderson at state.or.us>, 503-378-2528

[Fizz Boom Read][Girlandcloud]
Summer Reading 2014 at Oregon libraries<http://libdir.osl.state.or.us/>!
Find a summer food site<http://www.summerfoodoregon.org/>.

From: R2R-OR [mailto:r2r-or-bounces at listsmart.osl.state.or.us] On Behalf Of Katie Anderson
Sent: Wednesday, June 18, 2014 3:46 PM
To: r2r-or at listsmart.osl.state.or.us
Subject: [R2R-OR] 2014-2015 Ready to Read Grant application now available; due August 31st

Dear library directors and Ready to Read key contacts,

Hardcopies of the 2014-2015 Ready to Read Grant application have been mailed to all library directors and key contacts, please look for them in your snail-mail. Digital copies of the application are available to download from the State Library website in both word format and PDF (http://www.oregon.gov/osl/LD/Pages/youthsvcs/aboutready.aspx). Remember, the list of grants which tells you how much money your library may apply for is not available online because it is subject to change between now and when you receive your grant in December 2014. The grant list is attached and is  included with the hardcopies you should receive this week or next week via snail-mail.

Last year I provided coaching to the libraries that needed the most help with outcome based evaluation (OBE) on their 2013-2014 Ready to Read Grant application, and I promised I would send out OBE guidance to all of you with the 2014-2015 application. That guidance is below. I highly recommend that you read it thoroughly and have it in front of you when you fill out the application because the guidance walks you through the application step by step. I've created two examples of completed applications that reflect the examples in the guidance below so take a look at them to see how it all fits together. In addition, I've attached 3 scanned copies of real applications that demonstrated OBE successfully that look different than my two examples-we all have different ways of thinking and communicating so I thought these might be helpful too.

Please contact me if you have any questions or need help.  As Ann Reed always says, "call early, call often!"

Thank you,
Katie

PS: Remember, the proposed changes to Ready to Read still have to go through the legislative process and, if approved, Reading for Success will replace Ready to Read starting with the 2015-2016 grant cycle.

Katie Anderson, Library Development Services
* Youth Services Consultant * Oregon Center for the Book Coordinator *
Oregon State Library, 250 Winter St. NE, Salem, OR 97301
katie.anderson at state.or.us<mailto:katie.anderson at state.or.us>, 503-378-2528

[Fizz Boom Read][Girlandcloud]
Summer Reading 2014 at Oregon libraries<http://libdir.osl.state.or.us/>!
Find a summer food site<http://www.summerfoodoregon.org/>.

Step-by-step instructions for filling out your Ready to Read Grant application with OBE:

Your library programs, services, and activities can change people's knowledge, attitudes, skills, or behaviors. Outcome based evaluations (OBE) is a way to show how the library helps people make those positive changes.


1.      Select 1-3 outcomes listed in the grant application.  Choose the outcomes you want to focus staff time and R2R money on to change your patron's knowledge, attitudes, skills, or behaviors.

a.      Summer reading example (SRP):

                                                              i.      Youth will increase their reading/listening comprehension and ability to effectively communicate their understanding of and opinions about what they are reading/listening.

                                                            ii.      More families with youth will participate in the summer reading programs together as a family.

b.      Early literacy example (EL):

                                                              i.      Young children will increase their print motivation, vocabulary, print awareness, narrative skills, letter knowledge, phonological awareness, and writing/drawing skills.


2.      Describe the activities you plan to implement to help your patrons achieve each outcome you select.  Here are two ways of thinking about this:

a.      (SRP) I want to use R2R to hire summer reading performers and buy giveaway books.

                                                              i.      Who benefits from performances and giveaway books? Kids and families.

                                                            ii.      How can performances and giveaway books change kids' and families' knowledge, attitudes, skills, or behaviors? Performances can bring families together more than other library activities and conversations with adults about the performances and books kids are reading or listening to can develop comprehension skills.

                                                          iii.      What activities can I do to get more families to attend performances together? Families attending the performance events together will get to enter a drawing to win a set of read-aloud books that appeal across ages.

                                                           iv.      What activities can I do to help adults in families and library staff engage kids in conversations about the performances and books that develops comprehension skills? At each event library staff will distribute a handout listing ideas for fun comprehension building activities families can do together at home and set up a display of related library material so families can easily grab and check them out as they leave the library. When kids turn in their reading logs they will be asked a comprehension question by library staff or volunteers and be given a free book.


b.       (EL) I want to use R2R to get young children to increase their print motivation, vocabulary, print awareness, narrative skills, letter knowledge, phonological awareness, and writing/drawing skills.

                                                              i.      What fun activities provide kids an opportunity to learn and practice the six early literacy skills? Reading, singing, writing, talking, and playing.

                                                            ii.      How am I going to engage young children in those activities? Present weekly preschool storytimes at the library. Each storytime will focus on one or two of the six early literacy skills and include reading, singing songs, using the dialogic reading method to talk about at least one of the books, and one play or writing activity. Activities might include things like creating shapes out of play dough and talking about them to develop vocabulary or using shakers while singing songs to develop phonological awareness.



3.      Reread the activities you just described in this application. Did you describe conducting early literacy training to parents, childcare providers, other early childhood professionals, or library staff? Did you describe taking library materials, programs, or services out of your library and providing them elsewhere in your community? You/your library director will have an opportunity to report early literacy training your library provides with other funding on the Public Library Statistics survey. This application is only about how your library uses R2R funds, no additional information needs to be provided.



4.      You must evaluate and report on each outcome you select.  How do you know when you have changed people's knowledge, attitudes, skills, or behaviors?  Some evaluation methods may be personal observation, photographs, surveys of participants, comments on books in a blog, and parents' observations. Just remember to document and file the results so you can include them in your R2R final report.

a.      (SRP) This example has two outcomes, therefore there will be two evaluations

                                                              i.      How will I know if the performances get more families with youth participating in the summer reading programs together as a family? If I see more people come to performances as part of a family compared to the number of people who come individually or kids who come without an adult caregiver. How will you document this? Take photos.

                                                            ii.      How will I know if kids increase their reading/listening comprehension and ability to effectively communicate their understanding of and opinions about what they are reading/listening? I'll ask kids a question when they turn in their reading logs to get their free book. I will know whether or not my outcome was achieved if more kids' answers demonstrate comprehension.  How will you document this? Create a checklist for library staff or volunteers to quickly and easily check-off whether or not each kid is able to communicate their understanding of or opinion about one book they read/heard

b.      (EL) How will I know if storytimes helped the children increase their print motivation, vocabulary, print awareness, narrative skills, letter knowledge, phonological awareness, and writing/drawing skills? At the end of the last storytime each month I'll ask parents to raise their hands if they think their child has increased a particular early literacy skill as a result of attending storytime. How will you document this? I'll write down the question I asked and the survey results after storytime.



5.       Reread the activities you just described in this application. Are you partnering with an organization, agency, business, or school to implement the activities you described? If so, what are the names of your partners?



6.       Reread the activities you just described in this application. How many children 0-14 do you think will participate between January 1, 2015 and December 31, 2015? How many teens and adults 15 years and older will participate between January 1, 2015 and December 31, 2015? Calculate these numbers by:

a.       (SRP) Estimate how many people will attend each performance. For example, based on last year's performance attendance I think about 25 children and 20 teens and adults will show up on average. We plan to have 3 performers next year so that will be (25 x 3) 75 youth ages 0-14 and (20 x 3) 60 adults ages 15 and older.

b.      (EL) Estimate how many people will attend each storytime. For example, we plan to do storytime for 40 weeks and, based on storytime attendance this year, I think about 6 kids and 4 parents will show up every week. That's (40 x 6)  240 youth ages 0-14 and (40 x 4)  160 adults ages 15 and older



7.       Items on your budget that are paid for with R2R funds must be mentioned in your activities or evaluation. The whole purpose of filling out this application is to tell the State Library how you plan to use R2R funds to achieve outcomes.

a.      (SRP)  Contract with performers ($600) to present events that will be fun for the whole family. At each event library staff will distribute a handout listing ideas for fun comprehension building activities families can do together at home and set up a display of material related to the event and activities on the handout so families can easily grab and check them out as they leave the library. Families attending the performance events together will get to enter a drawing to win a set of read-aloud books ($100) that appeal across ages. When kids turn in their reading logs they will be asked a comprehension question and be given a free book ($300).

                                                               i.      But don't forget the other things necessary to implement these components of your summer reading program that also cost money such as staff time ($600) to book performers, create handouts, promote performers, set up displays, host performances, take photos at performances, order books, and ask questions and distribute books; printing photos ($10) for the evaluation; and printing fliers promoting the performances to post in the library and around town (promotional materials $10).

                                                            ii.      Are any other funders contributing to these components of your summer reading program? The Friend of the Library will pay for one of the performers ($300).

b.      (EL) Contract with a storytime presenter ($716) to plan and present weekly preschool storytimes at the library. Each storytime will focus on one or two of the six early literacy skills and include reading books, singing songs, using the dialogic reading method to talk about at least one of the books, and one play or writing activity. These activities might include things like creating shapes out of play dough and talking about them to develop vocabulary or using shakers while singing songs to develop phonological awareness (activity supplies $284).

                                                              i.      But don't forget the other things necessary to implement storytimes that also cost money such as staff time ($60) to promote storytime and storytime fliers to post in the library and around town (promotional materials $10).



8.      If successful, do you plan to continue implementing this project in future years?



9.      If you plan to continue this project in the future, where will the funding to pay for it come from?



10.  Please double-check your work before resubmitting your R2R Grant application. Reread it-even better, have someone else read it if possible-and answer these questions:

a.      Does each activity explain how it addresses one or more of my outcomes?

b.      Is there at least one activity for each outcome I selected?

c.       Do all of my activities include at least one thing paid for by R2R?

d.      Are all of the items in the R2R column of my budget mentioned in my activities or evaluation?

e.      Will my evaluation really tell me whether or not each outcome is achieved?

f.        What will success look like for each outcome I selected?



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 3127 bytes
Desc: image005.jpg
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 2117 bytes
Desc: image006.jpg
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0005.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 3127 bytes
Desc: image007.jpg
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0006.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.jpg
Type: image/jpeg
Size: 2117 bytes
Desc: image008.jpg
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0007.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SampleApSRP.DOC
Type: application/msword
Size: 155648 bytes
Desc: SampleApSRP.DOC
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0001.dot>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SampleApEarlyLit.doc
Type: application/msword
Size: 153088 bytes
Desc: SampleApEarlyLit.doc
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0001.doc>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RealApSmallLibrary.pdf
Type: application/pdf
Size: 174564 bytes
Desc: RealApSmallLibrary.pdf
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0004.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RealApMediumLibrary.pdf
Type: application/pdf
Size: 176412 bytes
Desc: RealApMediumLibrary.pdf
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0005.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RealApBigLibrary.pdf
Type: application/pdf
Size: 393002 bytes
Desc: RealApBigLibrary.pdf
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0006.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: R2RgrantList2014-2015-Correct.pdf
Type: application/pdf
Size: 119595 bytes
Desc: R2RgrantList2014-2015-Correct.pdf
URL: <http://listsmart.osl.state.or.us/pipermail/r2r-or/attachments/20140730/24e4ebb6/attachment-0007.pdf>


More information about the R2R-OR mailing list