In any educational scenario, the key measurement is not what was taught, but what was learned. Information literacy is no exception. As schools and libraries make an effort to teach information literacy, they are also ramping up their efforts to assess this type of knowledge.
Assessing information literacy ranges from the informal to the formal, and it varies based on the institution doing the assessment. This post will cover some of the efforts made by school and academic libraries to measure their information literacy programs.
Why Assess Information Literacy?
When institutions make plans to measure the information literacy of their students, one of the first questions they must ask themselves is why exactly are we assessing this information. Reasons can and do vary. Some want to track what students are learning; they want to use the assessment tool to determine what concepts students understand and what areas need more instruction and practice. Another reason to assess is because librarians want to improve their instruction. “Without assessment, program weaknesses cannot be easily identified or corrected.” (Oakleaf & Kaske, 2009, p.277) Additionally, librarians need to be accountable to the larger institution, such as the university or the school district. (Furno & Flanagan, 2008) Scores on information literacy tests can even affect accreditation. (Oakleaf & Kaske, 2009) The reason for the assessment will help determine what measurement tools are used.
What Needs to Be Assessed
Creating an assessment program begins by determining what skills need to be measured. The skill sets measured will vary, of course, depending on the education level of the students; high school students will be asked to be proficient at different things than college students. Some of the skills assessments try to measure include the following:
Ability to use Boolean search strategies
Understanding of controlled vocabulary
Ability to find appropriate print and electronic articles
Creation of key search terms
Critical evaluation of web sites
What Assessment Tools to Use
One of the most complicated areas for library instructors is determining what tools to use for assessment. The options are seemingly endless. Libraries can create their own tools, or they can implement one of the tests already available. Furthermore, libraries need to decide what type of test to create.
There are a multitude of ways to evaluate student learning. While there is not space here to detail the benefits and disadvantages of each type, it is important to delineate the options available for librarians.
Fixed Answer Assessments
One format most of us are familiar with is the fixed answer, often known as multiple choice, test. (Oakleaf, 2008) Here, students select from a set of predefined answers. Libraries can create these on their own, even tailor them to the specific instruction session, and for the most part they are relatively simple to create and score. For example, James Madison University requires all freshman to meet standards on the Information–Seeking Skills Test (ISST) that they developed themselves.
In addition to creating their own assessments, libraries can also use some of the standardized options available. Two possibilities include the following:
Created in 2002 at Kent State University, Project SAILS (Standardized Assessment of Information Literacy Skills)is intended to assess college students. (Seymour, 2007) The web-based exam is primarily used to measure groups of students; in fact, data is not available on a per-student basis. Even so, it can give colleges and universities an idea of how their students are performing compared to students at other schools. (Project SAILS)
TRAILS (Tools for Real-time Assessment of Information Literacy Skills), also developed by Kent State, is similar to Project SAILS except it is tailored for high school students. Here, librarians and teachers can see how each student performed. The tests are administered online. (TRAILS)
Performance Assessments
Performance assessments attempt to mimic real-world experiences in an evaluation, letting students display their skills in an information-seeking environment. These types of evaluations can take on a number of formats. For example, librarians or instructors can simply assign students a task, watch them complete it, and evaluate their success. (Oakleaf, 2009) Another option is to review the actual bibliographies of student assignments and critique their value. University of Connecticut tested a research portfolio assessment, giving students a research topic and having them keep detailed notes of their search strategies and results. (Sharma, 2007)
Rubrics
Rubrics are a set of criteria used for judging a project or assignment. Different categories are outlined, with student expectations listed for each; students are then scored based on how well they meet each objective. (Oakleaf, 2009) For information literacy, an example assignment would be for students to create a bibliography of resources on a certain topic. Categories to be measured may include quality of sources, use of search terms, proper citations and the like.
What Is In Store for Information Literacy Assessment
In 2004, Educational Testing Services (ETS) began a pilot program for its latest test called iSkills, a test designed to measure information literacy. (Katz, 2007) The 75-minute test used a performance-based model, presenting test-takers with different scenarios, then asking students to solve the information need using software that mimics what students see in the real world. There are currently two different testing levels; core is aimed at graduating high school seniors and first- or second-year college students, while advanced is intended for upper-level college students. (Educational Testing Service)
The test is not intended to determine college placement. Instead, it is “an assessment tool that will assist schools in determining whether their students have information and communication technology skills and their level of competency.” (Kenney, 2006)
Knowing that ETS, the same organization that develops other high-stakes assessments such as the SAT and the GRE, is participating in the information literacy arena indicates that information literacy assessment is gaining ground. As our society continues to be saturated with news and research, libraries will continue to make plans for information literacy assessment.
References
Ann Jason Kenney. (2006, March). The final hurdle? School Library Journal, 52(3), 62-64,9. Retrieved July 17, 2009, from Research Library. (Document ID: 1010873521).
Educational Testing Service. Retrieved July 17, 2009, from http://www.ets.org.
Furno, C., & Flanagan, D. (2008). Information literacy: Getting the most from your 60 minutes. Reference Services Review, 36(3), 264-271. doi:10.1108/00907320810895350.
Information-seeking skills test.Retrieved July 17, 2009, from http://www.lib.jmu.edu/gold/isst.htm.
Katz, I. R. (2007). Testing information literacy in digital environments: ETS's iSkills assessment. Information Technology and Libraries, 26(3), 3-12.
Oakleaf, M. (2008). Danger and opportunities: A conceptual map of information literacy assessment approaches. portal: Libraries and the Academy, 8(3), 233-253.
Oakleaf, M., & Kaska, N. (2009). Guiding questions for assessing information literacy in higher education. portal: Libraries and the Academy, 9(2), 273-286.
Project SAILS. Retrieved July 17, 2009, from https://www.projectsails.org/.
Seymour, C. (2007). Information technology assessment: A foundation for school and academic library collaboration. Knowledge Quest, 35(5), 32-35.
Sharma, S. (2007). From chaos to clarity: Using the research portfolio to teach and assess information literacy skills. Journal of Academic Librarianship, 33(1), 35-127.
TRAILS. Retrieved July 17, 2009, from http://www.trails-9.org/.
7 comments:
I am curious how test results would compare with other standardized test scores for the same individual. If student A might score poorly on math, science, writing, etc. but score high on using the Net for research, or score high on being able to analyze web content and determine its usefulness. As the 3 R's of traditional education give way to more complicated, subtle methods of learning and demonstrating learned skills, it would be interesting to see if crops of students are coming out of schools knowing how to research and cite a paper, but not being able to coherently assemble the information into a traditional research paper. Maybe the term papers and theses of the future will be more like multimedia Powerpoint presentations and less like traditional papers. Just something I was thinking about while reading your blog. Good job Greta!
We hear a lot of rhetoric on Information Literacy, but we don't often hear about assessing these tools. Assessments would be extremely helpful in measuring how the IL is working in your institituion and how changes might improve your results. Thanks for bringing up this important topic.
Assessing information literacy is something that public schools are beginning to be required to do. Each year our 8th graders have to take a state mandated information literacy test. The problem is that this test is multiple choice scantron. As you mentioned in your blog, a task oriented test would better display a students ablilities.
Very interesting blog. I had no idea that there was a growing movement for standardized testing of literacy skills. The very fact that Educational Testing Services is involved does hint that this area of assessment is "gaining ground." (Grond, 2009)
I must admit that I'm a bit apprehensive about adding more standardized testing to schools. Especially in light of all of the testing children are already required to take. More importantly, these tests do take away class time from children because schools have to dedicate portions of their years to standardized testing prep.
I remember when I was in school we had very few standardized tests and so there was a lot more freedom for teachers to create unique and memorable lessons. I understand the need to evaluate, but I do wonder how much creativity we take out of the classroom in the years with mandatory standardized testing.
Perhaps there is a middle ground that we could find between the need for this type of testing and the benefit that students can gain from learning for the purpose of learning and not just to pass some test so that your school can get more funding.
I also did not realize there was assessment testing being conducted in schools. It seems as though the performance and rubrics models would provide the most accurate results out of the assessments you listed, but the fixed model is probably perceived as more convenient and less time consuming to most test providers. However, the performance and rubrics testing could be successfully integrated with daily assignments and lessons so as not detract from valuable class time, as noted by Joseph (Miller, 2009) in his post. It may cause those giving the test to spend more time reviewing the final results, but it would be worth it considering how important information literacy is to future students or generations. Thanks for presenting such an interesting topic Greta!
Greta - I also found your post to be really interesting. My boss was a reference librarian at Notre Dame when the library began to actively work with the professors to incorporate information literacy skills into their courses. The library has a Web page of resources for the instructors, and lots of material on information literacy and how to assess the instruction of it. My boss "fondly" remembers SAILS from her library days.
ND's instructors' Web page:
http://www.library.nd.edu/instruction/resources/index.shtml
Very thought provoking and well researched blog, Greta.
I think assessment is important, but can it be overdone? My kindergarteners are assessed so much in June. That is basically all I do for that month. Yes, they can read; however they are only 5- 6 years old. We forget to be concerned about whether they can play and get along w/ each other. We wonder why some are burned- out by nine years old.
Post a Comment