A. Q&A

Q. There is no doubt of the debt our field owes Mary Gaver. The study you describe is often referenced as the basis for funding that came in the late 1960s to establish thousands of elementary school libraries. Gaver’s work also lead to creation of hundreds directorships for coordination of district-level school library media services and helped to support efforts to establish a school media director at the state level in all fifty states. Dramatic results indeed.

There are several issues related her studies on the "Effectiveness of Centralized Library Service in Elementary Schools Phase I and II" (Gaver 1960, 1963) that I ask you to consider and for the reader to keep in mind as they review the content of the rest of our discourse. Several questions are also posed for your response.

1. The Iowa Test of Basic Skills, one exam used to determine the standard measure of student performance for her study, contained a specific section on library skills. While it is true that elementary students scored higher if they were from a school with a centralized library, they were also the students who had received sessions in learning about library-use skills similar to those on the exam. Students not exposed to such skills, or without access to a school library, were not likely to score as high on the exam. Based on your recent experience with standardized exams around the country, do they lack content that adequately tests the information skills we expect school library media programs to teach and enhance? What information skills could be added to standardized exams to make them more relevant to the instructional goals and purpose of school library programs?
2. To a large degree in her 1960 pilot study and then to some degree in the follow-up study, Gaver tended to draw the greatest amount of responses from the emerging, growing, and wealthier suburban schools from the sample invited to participate. Although there were some rural and inner-city schools added, the sample in both phase one and phase two was dominated by new schools with predominately white populations. Did the suburban school districts continue to dominate the data in the recent state studies? Are there examples you can describe of extensive date from inner-city schools? Was it clear that the school library media program was having a reasonable impact on student learning in these disadvantaged schools?
3. Gaver saw the need to measure not just the number of holdings, but to try to determine the quality of the collection and the quality of the library staff service. She went into a great deal of detail, far beyond just certification or amount of time the librarian was available. She established very specific definitions for ratings of accessibility, collection quality, and reading levels, and defined membership for local review panels to make evaluation judgments on local collections. Quality, in her studies, was much more than quantity. How did your team learn to define quality and translate such measures so that they could be compared against numeric measures of student learning?
4. While it is true that Gaver did not have access to the modern computer technologies, there is an amazing amount of statistical analysis in her studies and the richness of her investigations can be seen by the learning environment descriptions she established along with the numeric data she collected. Many of the state studies you have conducted have included descriptions of quality learning situations related to the school media program. Describe the value these descriptions have added to your reports.
5. Gaver’s phase one and phase two studies in the early 1960s were focused on emerging elementary school library programs. Such a focus was very reasonable for what she wanted to accomplish, from what I understand. Measures of the impact of school libraries in high schools, however, seemed to be even more complex and in studies conducted by researchers other than Gaver, the impact on learning in secondary schools has not been clear. Did you find measuring the impact of school library media programs in secondary schools to be more difficult than in elementary schools? What, if anything, might account for the additional complexities in measuring the impact of school library programs on student learning in high schools?
6. Gaver’s studies were conducted in the early 1960s. One justification she offered for establishing central library collections in each elementary school was the lack of children’s departments in many of the public libraries. Today, the justification might need to be slightly different. Nearly all public libraries, including branches, have a children’s department and a growing number now address young adult services with special collections and full-time professionals assigned to youth services. Some might argue that where public libraries have increased services to young people, the need for school libraries in elementary schools has diminished. Often, youth collections in both fiction and nonfiction in public libraries have surpassed the schools in currency and objectivity. What have you seen in your studies that would help us continue to make the case for elementary school media centers regardless of the services at public libraries? What new data should we be gathering to show the value of those in youth services at the public library collaborating with local school library media specialists?

A:

1. The extent to which standards-based exams adequately test information literacy skills is certainly an issue, but not one which I can address based on a thorough review of the tests used in the states we have studied. Indeed, the validity and appropriateness of standards-based testing generally, while controversial from the start, were not within the scope of our studies. A thorough critical review of state tests on this point would be a tremendous service to the field; but, if one has been conducted, I am unaware of it. It would not be surprising if such a review revealed inadequacies. Several years ago, at least one state exam included a question about using a card catalog—certainly not an up-to-the-minute question. Obviously, we would prefer to see questions focused on critical thinking skills and appropriate information-seeking strategies, including use of both print and non-print resources and electronic access to information, specifically licensed databases.
2. Our studies were not designed to compare inner-city, suburban, and rural schools, though schools from all three settings were included. Random samples of schools were invited to participate in each study, and volunteer participants replaced nonrespondents. Therefore, other things being equal, schools in those three types of settings should have been represented in our data in proportions approximating reality in each state. Of course, other things were not equal. Most fundamentally, the likelihood of our receiving a survey response from a school depended on the presence of someone to complete and return the questionnaire. It seems reasonable to suppose that we were likelier to receive returns from schools with credentialed librarians than those with other types of staff, and from those with any type of staffing than those with none at all.
3. We made no attempt to assess collection quality in the ways done by Gaver. In addition to collection size, in several states, we also found predictable correlations between test scores and average copyright dates for selected materials (e.g., categories such as reference, or subject areas such as astronomy).
4. The sidebars in our reports were just that, not reports of the sort of qualitative analyses conducted by Gaver or, lately, by Todd and Kuhlthau. Our intent was to provide first-person accounts of examples of our findings in the voices of teacher-librarians, classroom teachers, and principals from the states under study.
5. The more problematic experience for us in studying high schools was one of uniformity rather than complexity. The study of correlations involves identifying two or more variables that vary together. That assumes that both variables do, in fact, vary in the data being studied. Very rarely—fortunately—we found that virtually all responding high schools reported one full-time credentialed librarian. In that circumstance, it was impossible to correlate a lack of variation in librarian staffing with dramatic variations in test scores. (Conversely, we sometimes found that virtually no responding elementary schools reported the presence of a librarian, creating the same problem in reverse.) In the Colorado-style studies published since 2000, we had test score data for only one grade per level—elementary, middle, and high school—but, in the original Colorado study we had test score data for six grades. Notably, these data revealed that the impact of libraries during the course of a student’s school years increased dramatically during the early elementary years (first and second grades), peaked during middle school (seventh grade), and tapered off dramatically during high school (tenth grade). Considering the tremendous impact of social and economic factors on student performance, this pattern was interesting, but not surprising. One would expect that the impact of libraries in particular and the school experience generally would be somewhat cumulative over time, and that, as students approach the end of their required schooling—during the years also characterized by social pressures in non-academic directions—the impact of schools and their libraries would diminish.
6. None of our school library impact studies has attempted to examine the relative merits of children’s (or youth) services in public libraries and school library programs; so, they provide no findings comparing the relative impact of the two on students’ school performance. Beyond that, the case for library programs in elementary schools is supported by the majority of the studies that moved beyond mere correlation to controlling for other school and community differences. In the Colorado, Iowa, Massachusetts, Michigan, Oregon, and Texas studies, correlations between test scores for an elementary grade (third, fourth, or fifth, depending on the state) and indicators of school library development (usually a composite measure combining data on staffing, collection size, and expenditures) remain, even when other school and community factors are taken into account. The absence of similar findings in other states is attributed to data limitations, or, more pointedly, reality. All empirical research on this topic is limited by what is actually happening. It can only analyze relationships between variations in school library development and variations in test scores where such differences actually exist. Unfortunately, in some states (e.g., Alaska, New Mexico), the level of school library staffing—especially at the elementary level—was so uniformly low (e.g., virtually no credentialed librarians and/or library aides) that, statistically, the data performed more as a constant than a variable. Simply put, the value of having credentialed librarians in elementary schools could not be demonstrated statistically in states where such staffing was extremely rare. As the number of states in similar circumstances grows, a different research design, focusing more intensively on the rare cases of exemplary staffing, will be necessary.

[Back to Article]

B. Q & A

Q: Are you aware if Professor [William L.] Bainbridge ever made such a statement again or reinforced this idea in any of his many writings? Based on letters from you and others dissatisfied with data from SchoolMatch as reported in American Libraries (Lynch and Weeks 1988), it seems his follow-up was less than satisfactory. Bainbridge has been a very vocal critic of research studies based on just correlative evidence. The strong correlation he mentioned in 1988 was based on a finding by Steven Sundre, the vice president for operations at SchoolMatch. Sundre found the strongest association of the factors measured at the time to be the amount of money expended annually on the resources for the school’s library media program and the level of achievement of students in that school on the National Merit Exam.

The SchoolMatch service (www.schoolmatch.com) continues today to provide data to school administrators, real estate agents, parents and others for a price. They continue to list "Library/Media Resource Expenditures" as one of the factors in the generic "School System Report Card." Other factors include "Teacher Salary Expenditures," "Facilities Expenditures," "Vocational/Technical Education Expenditures," and "Guidance and Counseling Expenditures," similar to 1988 factors. SchoolMatch clearly states, however, "While there is no direct correlation between the amount of money a school system spends and the quality of its education, many parents prefer a system that has more money to spend." Given this information, shouldn’t we expect expenditures on library materials to be a stronger predictor than counseling, vocational education or facilities expenditures in prediction of national merit performance? "Strongest" predictor does not tell us how strong the correlation was or continues to be, and SchoolMatch now discounts any of these correlations to have much meaning, especially any one factor standing alone. Did your dissatisfaction with the SchoolMatch data spur you to establish your own process for measuring the more specific characteristics of the school library media program against the more general standardized exam scores? How have you and your research partners improved on the SchoolMatch measures?

A: I have not monitored Bainbridge's later comments, if any, regarding his original comments during the National Public Radio interview in the late '80s. It seemed obvious at the time that he was speaking "off the cuff" in response to an equally spontaneous question from the NPR interviewer. The serendipity of that moment was, as we and others regularly acknowledge, the catalyst for the first Colorado (Lance et al., 1997) study, and, by extension, its successors. What many of us questioned, in the immediate aftermath of the Bainbridge NPR interview, was whether or not the asserted link between school library spending and National Merit Scholarship test scores was controlled for overall school spending, community wealth/poverty, or any other indicator that might have rendered the claim a spurious one. In other words, was it really school library spending that explained the test scores, or might it have been that schools that could afford to spend more on everything had higher scores? Because the documentation provided at the time did not lay this matter to rest, I wrote the letter included in the "SchoolMatch Revisited" article that appeared a few months later in American Libraries. The fact that bivariate correlations can often be explained away by antecedent or intervening variables was one of the assumptions underlying our studies. The research design we developed in Colorado, and that was later applied in Colorado and other states, moved beyond bivariate correlations alone by adding into the methodological mix control variables reflecting many of the school and community conditions that might have explained away any link between test scores and school libraries. The news in the first Colorado study that bolstered the SchoolMatch claim—and that has been so often confirmed by succeeding studies—is that, usually, the relationship between school library development and test scores cannot be explained away by other school and community conditions. In the studies completed since 2000, several refinements were introduced. More detailed data on what the first Colorado study called simply the "instructional role" of school librarians has made it possible to identify links between test scores and hours spent by school librarians on specific activities (e.g., planning collaboratively with classroom teachers, teaching information literacy skills to students). For some states, that list of activities was modified—sometimes new activities were added; sometimes the list was simplified. More recent studies have also generated more precise evidence of the value of flexible scheduling. In several states (e.g., Colorado, Iowa, Oregon), correlations between test scores and individual visits to school libraries suggested that scheduling flexible enough to permit larger numbers of individual visits was efficacious. But, in more recent states (e.g., Michigan, Illinois), correlations between test scores and the number of weekly library hours available for flexible scheduling have made this case more explicitly.

[Back to Article]

C. Q & A

Q: When the first Colorado study was presented to a group of reviewers (1992 American Library Association Annual Conference, San Francisco), observations were made that the study was useful as a foundation, but not conclusive in that it did not give evidence for a cause-effect relationship. As a baseline, it provided some idea of how item inputs such as number of resources, number of staff hours, or amount of technology reported for school library media programs related to student achievement as measured on standardized tests. For my part, as one of the reviewers, there was the request that more attention be given to measures of action inputs and possible relationships to student achievement. In other words, did actions taken by the school media specialist to teach something, to promote something, to plan with the principal, to demonstrate greater confidence in teaching and leadership, or even basic actions to weed and discard old materials have any relationship to student performance? As your team developed the process for the studies that followed the first examination of Colorado, describe how you have attempted to show closer indications that actions taken by the school library media specialist professional to improve the school library media program tend to relate to improved student performance. How have you revised the factors for action performance over recent years in the additional state studies to explore possible relationships with student achievement and specific levels of performance by the school library media specialist?

A: While I would agree that our first study was not conclusive, I disagree that it was merely correlational and presented no evidence for a cause-and-effect relationship. What distinguished that study from its predecessors was its taking into account school and community variables that might have explained away links between libraries and test scores as mere correlations. Controlling for competing causes is the only means by which statistical modeling can support a claim of cause-and-effect; there is no statistical test of causation. In the intervening dozen years, the repeated identification of links between libraries and test scores that cannot be explained away by other school and community conditions seems to me to offer considerable additional evidence for a cause-and-effect argument. If the correlation is spurious, why has it been found in multiple states over time? If there are other school or community conditions that consistently explain away library-test score links as spurious, I am unaware of them. Granted, however, the availability, or lack thereof, of data required to test some other possible explanations is a serious concern.

The cause-and-effect argument is further bolstered by the array of data we have usually analyzed and the interrelationships revealed: it is not just that schools with more librarian staffing have higher test scores. Schools with more librarian staffing are more likely to have library staff who are involved in a variety of leadership and collaboration activities that have positive consequences for collection size, the accessibility of library resources through computer networks, and the extent and types of library usage. And these indicators of the school library’s characteristics and activity, in turn, are associated with higher test scores. Thus, I believe it is inaccurate to dismiss the findings of these studies as merely "correlational." To be sure—especially in the present political climate—it is essential that we establish the conclusiveness of these claims experimentally, and initiating the conversation needed to chart that course was the point of the presentation that gave rise to this article.

In the second Colorado study and its successors, we did indeed attempt to respond to recommendations that we identify more specific actions that comprise the "instructional role" of the school librarian. That was a great challenge, particularly the first time out, for the second Colorado study. School librarians were asked to report how many hours during a typical week that they or their staff spent on various activities. As the list included a couple of residual items, the total for this list was required to match the total number of staff hours reported. Thus, the list of activities had to be sufficiently concrete and specific that respondents could determine for which activity to report each hour or fraction thereof. In addition to the activities listed in the previous answer, this list usually included time spent identifying materials for teachers, teaching collaboratively with classroom teachers, providing in-service training to teachers, meeting with the principal, attending faculty meetings, serving on school committees, and attending meetings with other librarians at building, district, or state levels. In later states, time spent developing collections was added to the list.

In a third Colorado study, tentatively titled "How School Librarians Teach Critical Thinking," we will be taking this type of question to a new level. Rather than utilizing general reading, writing, or language arts scores, we will be employing building-level summary results for specific sets of test items corresponding to the information literacy objectives embedded in our state’s reading and writing standards. A new, more specific list of staff activities has been created, and it will be targeted as precisely as possible to the information literacy objectives for which we now have item-level scores for each school. We are hopeful that this refinement will generate findings that will have a stronger claim to cause-and-effect status. While, in the first year, we will be doing the same types of statistical analysis we have done in the past, we are hoping that, in a second year, we will be able to marshal the cooperation of a sufficiently large number of schools to attempt a controlled randomized trial—that is, the type of study that will be eligible for inclusion in the U.S. Department of Education’s WWC.

[Back to Article]

D. Q & A

Q: The data from these "state studies" have been useful to address the question "What impact does the school library media program have on student learning?" And that is important, of course, but explain why this was the focus and not a more general, "How does the impact of the school library media program on student learning compare to other factors and other instructional interventions?"

It seems the school library media program impact was isolated, and not examined as an interrelated in some manner to other factors that may impact student performance. In each study, the measure of the school library media program was identified as lower than social-economic factors. Conclusions were offered that such major factors, however, could not explain all of the influences on student performance and the school library media program factors were often described as significant, but rather small. Can there be a stronger case made, as Dianne Oberg (2002), Professor at the University of Alberta, has suggested if we could place the positive school library media program factors in combination with other positive factors influencing student academic performance. Can we test such in at least two directions? First, do the positive factors from school library media programs enhance and make even stronger other positive factors found in favorable learning environments? Second, do the positive factors for school library media programs when found in disadvantaged learning environments compensate in some manner for the lack of other positive factors such as dynamic teachers and administrative leadership? It seems that in some environments positive factors can feed off of each other to create an even more powerful series of interactions to support student learning. Consider this observation from Oberg (2002, 11):

 . . . the claims now presented in the school library literature [are] that school library factors alone can account for 2 percent to 9 percent of student achievement. Improving student achievement, however achievement is measured, is a complex and challenging task. Gains in student performance can best be achieved through a combination of factors. School library programs properly equipped and staffed, can make a difference in terms of measurable gains in student achievement but, in combination with other factors such as teacher efficacy and teacher learning opportunities, they can help to achieve even greater gains in student achievement. The leadership role of the teacher-librarian, when focused on teacher learning and on other factors related to increased student performance, can contribute to increased gains in student achievement.

Often I hear from school library media specialists who have read summaries of the state studies (seems they seldom read an entire study, but simply accept the conclusions they want without question) and believe that if one or two of the findings they select are implemented this will somehow change things dramatically at their local school. For example,"If only we had flexible scheduling, students will achieve higher test scores." Or, "It says test scores rise as the number of staff hours rises, so if we’d hire an additional clerk next year, surely this will help test scores." The selective application of an isolated factor tends to not have much impact unless many other constructive actions are also taken within the school, the district, and perhaps the general community. What are your observations, as I’m sure you have heard from many who have attempted to take steps to use your studies to justify changes?

Another concern about the level of impact of school library media programs suggested from several of the state studies is, " Does that level of possible impact justify the cost needed to generate that level of impact?" For those of us who have always been believers in the value of libraries, librarians and multiple-resource inquiry learning, the conclusion is obvious. We see and accept much more impact taking place on the learner than just some improvement on standardized exams. To an administrator, for example, who must view a much broader field in time of great restriction of the financial resources he or she has to use, effectiveness and efficiency are both taken into account. And if we limit the definition of student achievement to gains on standardized exams, then there may be more cost-effective means to increase scores at least to the level that local parents are satisfied. This seems to put the school library media program into a very difficult position.

Conference and workshop presentations based on the evidence from these state studies tend to promise a 10 to 20 percent gain in test scores IF the school will dramatically increase investments in books, technology, and staff, perhaps tens of thousands of dollars beyond what is currently committed to the school library program. This is without mention of the investment that is probably needed to improve other factors in association with the school library media center in order to move toward such dramatic increases in student scores. Some school budgets can allow for purchase of resources or capital items (books, computers, facility renovation) from budget lines that can increase for a short term, but then recede again. Investment in additional staff and additional school media specialists is a long term one and administrators need to be convinced that such investments are wise choices when they also have pressures to employ more science and math teachers, more reading teachers, and to keep classroom size in elementary schools reasonable. Anyone making a substantial outlay should expect reasonable returns. Do the numbers from the state studies provide such assurance? Can it be said for certain that more investment in school library media program resources and personnel will guarantee higher student achievement? Do we know that such an approach is more efficient and effective than other options—greater investment in special reading teachers, math tutors, more science labs, and leaving the investments in books and librarians to the local public library? What if we find that investment of an additional $100,000 per year in classroom book collections and two special reading teachers results in higher average student achievement on standardized tests than investing that money in the school library media program as implied from the state studies? Could it be that there are several combinations of factors that can result in as high or higher student performance on standardized exams than the range predicted from the state studies with additional investments in the school library media program and personnel?

Correlation studies give us indications of relationships and may help us make some estimated predictions—if we know one condition exists, another condition is also likely to be found. Surely in these studies there have been "outliers" found where students are achieving well on standardized exams without the level of school library resources we believe to be enriching. What do we say in response to such school programs that have found a way to deliver acceptable student academic performance without investing in extensive school library resources and certified school library media specialists?

A: Not only is "improving student achievement . . . a complex and challenging task," so is assessing the relative merits of the factors that contribute to it. The complexities and challenges are of at least two distinct types. The first has to do with the basic reality. Education—or, for that matter, any human enterprise—is complex. It is also organic. The number of potential factors impacting any given student’s academic achievement is exceedingly large. Which factors are in play for any given student is yet another wildcard. What is wrong with the current U.S. Department of Education efforts to dictate the types of research questions and methodologies that researchers may ask and employ is that the question ("Is there a systematic effect?") and methodology being dictated (CRT) suggest an alarmingly oversimplified view of the educational enterprise. This is what might be called the "magic bullet" approach. The implicit underlying question is "What one thing can we change to make the biggest possible difference?" If only it was that simple! The strong influence of this simplistic view is perhaps not coincidentally friendly to the many brand-name solutions being offered up by the private sector. (The most cursory review of what is currently listed on the WWC Web site reveals a surprising preponderance of brand-name interventions.) The WWC requires—which it should—that scientific research must involve objective observation and must consider competing causes, besides the one that is the focus of a given study. By employing available data from sources that are as independent as possible from the observed—state test scores, census data, state department of education statistics—and by considering the influence of other school and community conditions (e.g., teacher-pupil ratio, per pupil school spending, adult educational attainment, poverty), we have consistently attempted to meet these two generally accepted criteria of scientific inquiry. The relevant data available for all schools in a state is usually limited to the data states are required to collect from schools and districts and to report to the National Center for Education Statistics. This type of data does not include data on the multitude of different interventions with which an investment in a school library might be deemed to compete. Theoretically, such a study, assessing the relative merits of various interventions—including, perhaps, some indicators of collaboration between librarians and teachers—is a terrific idea. But, I am hard-pressed to imagine how one marshals the effort required to amass such data on a sufficiently large scale—that is, for one or more states, or perhaps for all poorer districts in a state—to gain the endorsement of the WWC. The cost of such an endeavor, not to mention the degree of voluntary cooperation required at school and district levels, would be truly staggering, cutting across disciplines and other interest groups as it inevitably would. The political viability of the large research team required for such a study would also be challenging to establish and maintain, as I suspect every stakeholder group involved would want its own "expert" (or "advocate") on that team.

"Can it be said for certain that more investment in school library media program resources and personnel will guarantee higher student achievement?" Of course, the answer is no. On the occasions I am fortunate enough to address audiences comprised primarily of teachers, administrators, or other education decision-makers, I always start the speech with this ostensibly teasing, but deadly serious, warning:

And, I believe that position is supported by the findings of the so-called Colorado studies.

[Back to Article]

E. Q & A

Q: Site-based management has left many local decisions at the building level. School library media specialists have had to demonstrate their value directly to their local principal. Some have and many have not. Could one result over the past decade be a growing gap between schools who have adequate school library media resources and staffing and those who do not because the school library media specialist has not fully communicated and demonstrated an effective instructional role? Are there schools where decisions have resulted in less investment in school library staff and resources but more in reading programs and classroom collections and the results have been more direct access to books and teaching of reading for less money leading to student performance that is as good or better than when all funding went into the school library media program? School libraries work, but do they work well enough? As well as other program investments?

A: There seems to be little question that site-based management has been bad for school library programs. There was no direct investigation of this issue as part of our studies, but the declining fortunes of school library programs since the advent of site-based management make the relationship so obvious that it has become conventional wisdom. While many accomplished teacher-librarians have managed to thrive under this system, the perception among many practitioners seems to be that other accomplished teacher-librarians have suffered under it. And, certainly, the fortunes of school library programs without talented leaders have waned (perhaps deservedly). As stated earlier, our studies were not designed to provide evidence regarding the comparative value of school library programs versus particular reading or other interventions.

[Back to Article]

F. Q & A

Q: I am glad you emphasized current collections and access. These aspects were also emphasized by Gaver and have shown in many other studies. Again, actions are necessary to make items provided more useful and hopefully more effective. There is no doubt of Krashen’s support for libraries and professional librarians (Krashen 2004). He has concentrated over the years on showing how more funds are needed to purchase new reading materials of all kinds and to place them in all learning environments, especially in homes and in disadvantaged schools. He has approached this from at least two ways. First, making the argument that funds spent on regimented reading programs and texts should be shifted over to building general book collections with lots of diversity in challenging and interesting material and for all to read and share, including teachers with teachers, teachers with students, students with teachers, parents with teachers, and on and on. Establish a reading culture and, although not all will become high-scoring test takers, you will increase the proportion of the population that will be "ok"—they will gain intellectually and contribute to the culture as good citizens. Second, where is the national commitment? Krashen often draws attention to efforts to establish a greater national investment in reading materials. Some of his work along with Jeff McQuillan (1998) gives light to the lack of attention across the nation to addressing the resources needed to help students and families in poverty gain access to reading materials and literacy assistance. What have you found in the state studies that would suggest investment in school library media programs and personnel in the poorest schools might have a greater influence on the degree of student learning than what we tend to find in the affluent and middle-class environments we so often research? Just recently, Krashen repeated his message (School Library Journal, June 2005), and over the AASL Forum (July 3, 2005), " . . . the real problem: Those who come from low-income families have little chance to read, because of the lack of books in the home, the community, and their schools."

A: The message from our studies—at least, the majority with successful regression analyses—is that, regardless of how rich or poor a community may be, improvements in school library programs are associated with increases in test scores. When the question of the school library’s impact on poor students in particular is posed to me, I cite the studies by Krashen and colleagues before our own, as they address the issue more directly. It seems relevant to remind readers at this point that the Colorado studies employ schools rather than students as the units of analysis.

[Back to Article]

G. Q & A

Q: The Todd and Kuhlthau (Whelan 2004) studies in Ohio are similar to many of the evaluations conducted in the 1960s and 1970s of "demonstration" library media centers. Identification of the "best" and then analysis of what seemed to work at those programs. From that exercise came descriptions that could be adopted for state and national standards or models toward which other schools could aspire. The Ohio sample, thirty-nine high-rated school media programs, is very suburban with more than a 90 percent white population base. Clearly there will be some important characteristics of the effective professional information agent for future national guidelines, but is this also an example among the state studies where the disadvantaged schools have not been examined? Do you have examples of how the state studies have illustrated the special problems and potential solutions of the inner city schools? Your reports indicate that social-economic factors were controlled in your data. Can you explain how that was accomplished and what it means in the data interpretation?

The qualitative approach for most of the Todd (2003) and Kulthau work in Ohio is different from many of the interviews and testimonials gathered in the past in that it is client-based. What do students and teachers have to say about the value of the school library program and media specialist, rather than just hearing from the library media specialist alone. This approach is especially important because it moves impact on student achievement away from a narrow definition associated only with standardized tests and opens an array of valuable services provided by the school librarian. In most of your state studies testimonials were offered as well. As you have mentioned in your presentations, these help to illustrate the context for actions taken by school library media specialists to improve their local learning environment. Please elaborate on what you see as the value of the testimonials gathered for your studies.

Only positive testimony from teachers and students was accepted for examination in the Ohio study so that conclusions could emphasize the constructive role the library could play in providing services with a staff of qualified librarians as "information agents." Again, this is a useful model, from selected exemplary programs, but when testimony has been gathered from a more generalized sample there is a clear message that parents and students value the idea of a school library, but find, in many cases, the person they identify as the school librarian to not be engaged with meaningful learning efforts (KRC Research 2003). Often, they characterize the school librarian as a person who does little and perhaps even gets in the way of learning. How can we ignore such data? Shouldn’t we be experimenting with models of library media center learning environments that can have potentially even greater impact on student learning than the state studies have tended to indicate?

A: Our findings relating test scores and selected library staff activities—particularly clusters of activities that manifest the leadership skills of library program administrators and the engagement of credentialed librarians in collaborative teaching and learning with educator colleagues—have already had some demonstrable impact on standards and guidelines at the state level in Colorado. Those documents have been used as models by some other states.

Of course, the current edition of Information Power was written before the second Colorado study or any of its successors was published. But, the findings of the first Colorado study are cited in the current IP. It was not the purpose of any of our studies to compare library programs in inner-city, suburban, and rural schools.

The Colorado methodology controls for other school and community conditions through regression analysis, a statistical technique that weighs the separate effects of multiple independent variables (i.e., library, other school, community factors) on a single dependent variable (i.e., test scores), while holding each of the other independent variables in abeyance. Regression analysis accomplishes through statistics what would be virtually impossible to do experimentally—namely, identify a sufficiently large number of schools that could be matched closely on multiple control variables.

While data on students’ perceptions of how libraries and librarians help them—data such as that collected in Ohio and other states—can be very informative, I doubt that it would be deemed objective by WWC reviewers, and objectivity of observed measures is a major WWC criterion for recognizing a study as scientific. From that perspective, it seems highly unlikely that students’ self-assessments would be taken as seriously as test scores—regardless of how flawed or narrowly designed the tests may be. I hasten to add that I find lamentable the U.S. Department of Education’s implied inclination to discount qualitative research as unscientific.

[Back to Article]

H. Q & A

Q: Perhaps a major benefit from the recent state studies is that the evidence presented has given a larger number of school library media specialists confidence to initiate arguments for local changes in scheduling, resource support, professional development, and additional clerical support. Responses from practicing school library media specialists through LM-Net seem reasonable. This probably does represent some serious effort to apply findings locally. Perhaps practices in local action evaluation will help give further evidence if some of these steps make a difference. Are there any indications that the state studies will have an impact on the "larger picture," such as major new funding for resources or mandates for additional professional staff? Will the state studies help to make a convincing case for more funding to improve school library collections through federal title initiatives? Will these studies help to make the case to regional school accrediting agencies of the need to maintain or increase the number of and quality of school library media professionals during a time when we see, especially at the high school levels, a decline in full-time school media specialists in the secondary schools?

A: The Colorado and other state studies were cited frequently by Senator Jack Reed and others as evidence for passage of the Reed Amendment, which is providing the first federal funding dedicated to school libraries in several decades. Needless to say, it was by no means the only evidence, and the successful passage of that amendment is owed most of all to the lobbying efforts of many energetic and articulate advocates for school libraries. Thus, by that assertion, I mean only to claim that our studies provided some of the evidence they marshaled so persuasively. Likewise, this line of research by us and others was one of the factors contributing to the 2002 White House Conference on School Libraries. That event brought this and other research, and the testimony of principals and librarians from schools with exemplary libraries, to over two hundred representatives of schools and districts, as well as funders and opinion-shapers (e.g., Carnegie Corporation), from across the nation.

The Library Research Service is collecting data on local (school and district) outcomes associated with the school library impact studies through an ongoing survey on its Web site at www.lrs.org/impact.asp. Some of the results of this survey to date were reported in this article. Because this survey is being done on a shoestring budget, no attempt at a statistically representative sample of U.S. schools is being made. This online questionnaire is simply enabling us to systematically gather more data that would otherwise have been purely anecdotal. What the evidence to date from this survey suggests is that improved working relationships with administrators or teachers (about two-thirds), increases in collection size and electronic access to information (almost half), and more flexible scheduling (almost two-fifths) are more likely outcomes than increased numbers of librarians or support staff (8 and 14 percent, respectively).

How these studies are regarded or used (if indeed they are) by regional accrediting agencies is unknown to this author.

[Back to Article]

I. Q & A

Q: Your team has gathered a remarkable amount of data and the reports for various states are filled with correlations that are tagged as significant. As you explain to the reader in each report, significance is an indication that we can assume your sample is representative of your overall population, and that the findings are not merely by chance. Some interpret the levels of the correlations you report as significant. Some statisticians (Stephen and Hornby1997) state that when a correlation is shown to be significant, its value to the researcher remains a matter for the researcher’s own judgment, and judgment will include consideration of the size of the sample, other possible causes of weight, what the researcher intends to do as a result of the finding.

Should we not also consider the level of the correlation?

Usually correlation levels are reported as positive or negative and then range from zero at no correlation up to 1.0 as very strong or perfect correlation. Most of the correlations with student achievement reported in the state studies have been at +.3 or lower. Most of your populations have involved several hundred schools, so strength of correlation can vary in interpretation depending on how large the population examined, but +.3 seems moderate at best, not strong and certainly not highly strong. Texts on education research disagree on this subject, but some state as follows:

Yet we often hear that these state studies have shown consistently strong positive correlations between collections, funding, services, and staff compared to student achievement. From state to state the correlation ranges seem to be fairly close. In most refereed education journals, correlations in the .3 to .4 levels tend to be described as moderate, not strong. What is your interpretation? Is this a series of strong correlations, or a series of studies that show us consistently there are positive relationships, but consistently moderate and seldom, if ever, strong. It seems there is a great deal of room for improvement of school library media program impact on student achievement and strong correlations are not really present?

If several of the states were to replicate their studies in five years to determine what, if any, changes have taken place, what sort of data would be most beneficial? Would higher correlations indicate improvement by most schools or perhaps indicate a continued growth of the gap between the affluent schools and the disadvantaged schools?

A: The authors of the Lance and Rodney studies are diligent about characterizing the correlations we report as "positive" and "statistically significant"—where that is the case—without characterizing the strength of the relationship as weak, moderate, or strong. While familiar with the standard "breaking points" for using those terms, I consider them problematic for the reasons discussed earlier. The educational enterprise is a decidedly complex and complicated business. By the usual statistical standards, it is a rare finding indeed that could be characterized as "strong" in traditional terms. We have opted to leave it to individual readers to interpret the strength of the reported relationships by presenting detailed tables reporting correlation coefficients for individual bivariate relationships. For the record, I have corrected a number of other authors—before publication whenever given an opportunity to review—who have "over-interpreted" the results of the Colorado studies.

Web Resources

SLMR readers may read further about scatter plots and examine the normal interpretations of correlation strengths at these Web sites:

• Scatter Plots, general definition
• See changes from weak to strong correlation scattered data, and adjust input data to see the changes in line of best fit at Scatter Plot and Line of Best Fit.

[Back to Article]

J. Q & A

Q: With or without the standards of the federal research clearinghouse, wouldn’t such a progression in the development and testing of measures of the impact of school libraries be expected? There is a systematic progression in such investigations conducted over many years and by many different researchers. The correlation studies have helped to identify some relationships that need deeper examination before we can draw clearer conclusions. To do this we should build on previous work, refine the methods, and draw on, eventually, strong measures to test the validity of findings. Thus, movement toward application of experimental methods is not only expected, it is essential if any findings of the relationship between school media programs and student achievement are to be validated. Those who practice such a progression are certainly "skeptical" along the way. It is a fundamental characteristic of a scientist in any field to doubt. Caution should be used to not misrepresent evaluation studies targeted and funded to find positive influences alone as research studies that also identify problems and shortcomings in school library media programs. The state studies have tended to look at data from the education system as it is. This is fine, but we should keep in mind that they do not test for new approaches to delivery of information services and skills, teaching of inquiry and information literacy, and measures of student performance beyond standardized exams. Staying with data that represents limited actions of the school media program and measures of achievement limited to standard exams seems to place restraints on the thinking we should engage in to establish more authentic learning environments in the Information Age.

From the beginning it is unlikely that many in the school library media research circles believed the correlation studies to be headed toward any conclusive cause and effect statements. Such does not mean there is a resistance to the findings, but there should be a great deal of accepting such studies for what they can and can not do. All along it has been clear that such correlative data are welcomed to help us identify relationships and then hypothesize as to why these relationships exist or not, and why they are not stronger. Regardless of the barriers and limitations, steps toward more controlled and experimental studies need to be taken. Your call for a new direction is important, although perhaps it could have been made a state or two sooner. There are at least two more states conducting studies as of this writing, both similar to the Colorado model, and by the end of 2005, nearly $750,000 will have been spent on such studies. Has it been worth that much investment? Do the state studies establish enough foundation and raise enough questions that similar funding or more can be attracted to conduct the more experimental studies? What do you see as the potential for attracting the necessary funds and researchers in order to conduct the empirical studies needed to gain the more conclusive evidence we need?

A: The school library impact studies published to date were conceived, if not completed, before the full ramifications of the No Child Left Behind establishment’s control of federal education research policy had become clear to many of us. So, it is little surprise that none of these studies pursued a large-scale controlled randomized trial design. Indeed, the Colorado methodology was originally conceived as a successor to the vast majority of small-scale, quasi-experimental studies that preceded it. That more rigorous experimental studies would be politically valuable for the foreseeable future seems clear. I am optimistic about the prospects for such studies because the vast majority of the relatively small cadre of researchers in this field are well aware of this imperative. There are, however, two efforts that could distract from, if not actually discourage, that course of events.

The action research movement is promoting what might otherwise be called "self-assessment." While it is clear from a host of anecdotal accounts that the data yielded from such efforts are being used persuasively by many teacher-librarians with administrators and classroom teachers at the local level, it is difficult to imagine it passing muster with the NCLB establishment as being objective and considering competing causes. Practically, it also seems doubtful that such research is ever going to be pursued on a large scale, especially in a time when fewer and fewer teacher-librarians have support staff and, indeed, fewer and fewer school libraries have teacher-librarians. For the foreseeable future at least, might the field be better off channeling this energy into encouraging local participation in the type of large-scale controlled randomized trials we need to be doing? It is difficult to argue with the potential value of action research at the local level, but it does require a substantial investment of time to be done well. One has to wonder: can we afford to do both?

I will suggest an answer to that question in my response to your question about whether or not the Colorado studies have been worth their cost. The answer to that question depends entirely on one’s perspective. It is no mystery why the first Colorado study was never replicated on a statewide basis (at least, to my knowledge), whereas the second one has been replicated, to greater and lesser degrees, more than a dozen times. What changed between the first and second Colorado studies? Standards-based testing roared in like a lion. When the first Colorado study was done, only a very small subset of respondents to our statewide school library survey could be studied, because they were the largest group among the respondents who utilized the same standardized test—the Iowa Tests of Basic Skills. By the time the second Colorado study was done, students in selected grades in every elementary and middle school in the state were required to be tested under the Colorado Student Assessment Program (CSAP). High-stakes testing has been the engine driving the demand for these studies; make no mistake about it. Politically, it does not matter in my state if all the studies have been done in other states, utilizing their state-mandated tests. I need a study that demonstrates the impact of school libraries and librarians on my state’s test scores. Every one of the studies done by the Lance and Rodney team was initiated by school library advocates in the state in question, commissioned by the state library agency or a state library or educational media association, and funded, directly or indirectly (in whole, and occasionally in part) by the state library agency. In other words, the vast majority of the studies have been conducted, not as purely academic endeavors pursued by LIS faculty, but as policy studies commissioned by practitioner-oriented organizations with obvious advocacy interests and pursued by hired consultants. Thus, both political and market forces, as well as academic ones, have been at work in the remarkable proliferation of these studies. The good news in the situation is that, given sufficient incentives, state library agencies and associations do appear willing and able to muster the funding and other support needed to commission research. The challenge is the one to which this discussion calls attention: the imperative that we marshal such resources in new directions that will increase useful knowledge and enliven advocacy efforts on behalf of beleaguered school libraries and librarians.

The second potential distraction from the pursuit of the types of studies that can gain the endorsement of WWC is a new wave of qualitative studies modeled on the Ohio study. This is most regrettable, for, like you, I believe there is much to learn from qualitative studies. I also believe they make a valuable contribution to the literature; but, the fact remains, studies replicating a purely qualitative methodology are not likely to be approved by WWC or embrace by the NCLB establishment—unless their criteria for scientific research change, which seems most unlikely. Fortunately, as was the case for the Colorado studies, funding for such studies does not emanate from the U.S. Department of Education. So, anybody who wishes to pursue a qualitative study for their state, district, or school has several viable funding options (chiefly LSTA grants from state library agencies, perhaps National Leadership Grants from IMLS). The risk of focusing on this type of study, however, is that we will continue to be open to charges of "preaching to the choir." If the larger strategic goal of school library advocates is to engage the NCLB-driven education establishment, resources should be focused on studies that meet the narrow WWC definition of scientific research. Indeed, if we are wise, we will seek to communicate with the federal Institute of Education Sciences, which is responsible for WWC, about proposed research designs in advance of future studies. Why not invest our time, money, and effort wisely by seeking advance approval of proposed designs? There is likely no single authoritative voice on that issue, but surely some dialog with WWC representatives is possible. In the absence of that, we should study carefully the types of studies that are and are not approved for whatever clues we can derive about what is and is not acceptable to WWC reviewers.

[Back to Article]

K. Q & A

Q: There certainly needs to be a greater effort on the part of professors in higher education to find ways to integrate information literacy, resource and inquiry-based instruction, and the general philosophy of the modern school media center into more areas of training for those who will serve our schools as administrators and teachers. Such can certainly help the case for what an effective school library media specialist and his or her staff can do for improving the school’s learning environment. I’m not certain, however, that it rests completely with "educating" future and current principals.

We still have a long ways to go with educating future library media specialists and certainly a very long ways to go with the majority of the practicing school media specialists to understand how they can play the full role needed in order to really have some influence on student learning. As Kathleen Craver (1986) documented and concluded nearly two decades ago, it takes a generation or two for dramatic changes to take place. To move a large majority of school media specialists to the level that they are academically able and are fully willing to accept an expanded instructional role may still be a decade away. Many "perception" studies have been conducted over the past forty years (McCracken 2001) and have consistently shown the principal’s perception of the instructional role to be greater than that expressed by most school media specialists. Even as recent as five years ago, such studies continued to show a large proportion of practicing school media specialists resisting the instructional role because it was too time consuming, was not supported or respected, or was not part of the profession they expected. Too often those in professional roles find other duties to fill their day than placing a priority on instructional interventions that might logically make a difference in improving student performance. Could you tell from your studies if the instructional role is increasing in frequency and quality?

Certainly, not all of this resistance is in the older ranks of practice, but retirements over the next decade may help to bring on a larger portion of the field who are both willing and able to assume an effective instructional role. Defining and demonstrating effective and efficient instructional interventions is high on the educational methods research agenda for several in our universities. Our certification programs, undergraduate and graduate, will hopefully continue to raise and enforce instructional role expectations for future school media specialists. State credentialing agencies will also need to maintain high demands.

How have your studies helped us further define the most important aspects of the school library media specialist’s instructional role? If these state correlation studies have suggested how better instructional engagement by the school media specialist makes a positive impact, isn’t the opposite also likely and are there not steps to be taken to remove school media specialists who may be actually contributing to poor student academic performance by not doing their job adequately? How should professional librarian associations, credentialing agencies, higher education, and local school districts work together to establish new job descriptions, annual review of school media personnel, professional development opportunities and rewards? Such is a tall order and it may be that the state studies have given us several snapshots of what needs to be done, but we have yet to establish a reasonable agenda for necessary change.

A: The quality of practicing teacher-librarians is as variable as the quality of administrators and teachers. When speaking about these studies to practitioners, I usually introduce the topic of leadership like so:

From there, I report the second Colorado study findings that leadership by teacher-librarians leads to collaboration with other educators, not the other way around. Many years ago, the Colorado Library Marketing Council—notably, a group made up almost exclusively of special librarians—decided to do something about the generally poor marketing—especially self-marketing—skills of librarians by offering a series of workshops. They did not have the resources to address this need as broadly as they would have wished so they chose one sector to focus upon: school librarians. Why school librarians? Because this group of expert self-marketers (as special librarians, their survival depended on it) could see what was happening to school librarianship under site-based management. And what they saw was that far too many school librarians had no idea how to ensure their own and their profession’s survival. This problem is precisely why I find some cause for optimism in the huge wave of school librarian retirements that has already begun. Yes, sadly, some of those jobs will disappear with the retirees. But, the jobs that are filled are likely to be filled by younger people, just beginning their careers, and hopefully emerging from LIS or educational media programs that have equipped them to survive in the "brave new world" of No Child Left Behind.

What should be done about under-performing incumbent practitioners? It seems impertinent to offer an opinion, considering that I have never occupied such a position and, indeed (and dare I add, ironically), am not qualified to do so myself. But, it did not stop me when visiting with a memorable conference participant after presenting a session a few years ago. A sixty-something woman introduced herself and said "That was a very thought-provoking session." "How so?" I asked. "I’m a classroom teacher planning to retire in a couple of years, and, knowing that, my principal sent me to this conference with the suggestion that I consider taking our school’s librarian position so I can "coast" into retirement." As politely as possible, I replied:

She looked a little stunned, but surprisingly unoffended, and said thoughtfully (as much to herself as to me, it seemed) "I've got to seriously reconsider this offer."

Hopefully, under-performing incumbents will either see the necessity of doing a better job to stay employed or retire. And hopefully, the principals of under-performing teacher-librarians will understand that just because the last person did not do a good job does not mean the job isn’t needed. As I ask when speaking to principals, can you imagine abolishing the teaching of math or science on that rationale?

Perhaps school library consultants in state library or education agencies—where those positions still exist—or state library or educational media associations should consider their options for strengthening state certification programs for teacher-librarians in ways that discourage poor practice. Is this likely to happen? Perhaps not. But, if a profession does not police itself, one way or another, who else is going to do it?

[Back to Article]

L. Q & A

Q: What are the plans for future dissemination of the findings from the state studies? Describe the likely paths to publications in trade and refereed research publications outside of the school library media field? How accepting do you believe these journals will be to the methods used and findings reported from the state studies?

A: As the Illinois study is the last such study for the Lance and Rodney team, it seems the proper time to sum up our contribution to school library impact research. Actually, such a summary and overview has been in progress for some time. The original purpose was to summarize the findings of the studies in three separate chapters—elementary, middle, and high school—for another author’s book about school libraries aimed at principals and other administrators. The fate of that book is somewhat up in the air, for reasons that are not germane here and which I am not at liberty to discuss. Suffice it to say that, if that book does not materialize, the chapters will likely be reworked into articles for submission to appropriate journals, preferably journals read primarily by administrators or teachers rather than librarians. As there is nothing unique about the methodologies employed in our studies, I would anticipate the reception such articles would receive to be determined almost entirely by how they are written—specifically, what findings are asserted, how acceptably they are asserted based on standard scholarly practices, and how well the detailed evidence for the claims is presented—and by whom such articles are read.

To be sure, all of us who do research on school libraries need to make more regular, timely, and diligent efforts to report our work through established channels for publication and presentation—in the library and information science field, to practitioners, and to other educators and education decision-makers. Doing so is another of the WWC criteria for endorsing research with which it is difficult to disagree.

Special challenges will likely have to be surmounted in order to publish in other fields. Practitioners and even academics in most fields tend to focus on keeping up with the literature of their own fields, but the pressures to read ever more widely are certain only to become stronger in the future. We may find that, if we want to be published in other fields, we may have to modify our research questions or methodologies somewhat—at least in perspective—to be considered relevant, and therefore publishable, beyond the library and information science literature. Publishing more widely might also be made easier by collaborating on research projects with colleagues from the other fields whose attention we seek.

[Back to Article]