• Main
• Acknowledgments
• Table of Contents
  
  
  
  
• Chapter 1
• Part 1
• Part 2
  
  
  
  
• Chapter 2
• Part 1
• Part 2
  
  
  
  
• Chapter 3
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6
  
  
  
  
• Chapter 4
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6
• Part 7
• Part 8
• Part 9
  
  
  
  
• Chapter 5
• Part 1
• Part 2
  
  
  
  
• App. A

2.Study Methodology

The Texas study followed the methodology used by Lance, Rodney, and Hamilton-Pennell in the Colorado, Alaska, and Pennsylvania studies while also attempting to expand upon those procedures.

2.1Questionnaire

The questionnaire used in the Texas study was similar to the questionnaire used in the Colorado, Alaska, and Pennsylvania studies with some modifications. The questionnaire consisted of nine sections.

The first section-Identifying Information-requested the name of the school and district, school level, grade levels, whether the school has more than one library, and in case it does the library for which information is being provided.

Section II-Library Management-dealt with library size (i.e. seating capacity), availability of a district library coordinator, existence of a summer reading program, and the relationship with the local public library as manifested in on-going communications with the local public library and in working cooperatively to promote summer reading programs. This section also inquired into the existence of a library policy and procedures manual and board approved policies relating to copyright and to collection development, especially in regard to materials selection, weeding, and reconsideration of challenged materials.

Section III-Library Staff-addressed the staffing levels of the library including paid professional staff and paid support staff as well as adult and student volunteers. Information sought for each staffing category involved number of staff and volunteers, number full-time and part-time, person hours per week, and the highest education and certification of paid staff members.

Section IV-Service Hours-inquired into the number of hours in a typical week the library is open during school hours, before school, and after school, as well as in the summer.

Section V-Staff Activities-listed 18 different types of activities librarians are likely to perform. The activities were grouped into three categories: teaching and learning, information access and delivery, and program administration. Librarians were asked to record the number of hours in a typical week that all paid library staff spend on each activity.

Section VI-Library Use-addressed a range of library uses in a typical week. Uses included scheduled and unscheduled visits to the library by individuals and group of students, staff, administrators, and parents and the percent of regularly scheduled and flexibly scheduled visits. Data were also requested on the number of books and materials checked out and used in the library and the number of materials loaned to and from other libraries in the district and outside the district.

Section VII-Library Technology-inquired into the availability of computers with different types of functionality located in or under library supervision. Categories of computers included, for example, those with Internet connection, with access to the library catalog, library databases, and with networked access to CD-ROM resources. Information for those categories of computers was also requested for computers in school from which networked library resources may be accessed. Questions in this section also addressed the number of PCs and Macs by processor speed, speed of fastest Internet connection, Internet policy, and type of technology equipment in the library.

Section VIII-Library Collection-revolved around the characteristics and size of the library's print and non-print collection and the type and number of purchases made in 1999-00. Questions inquired into the size of seven categories of library materials: print volumes, print subscriptions to newspapers and magazines, electronic subscriptions, encyclopedias and reference titles on CD-ROM or laser disc, video materials, and software packages. Data were also requested on the type of online licensed databases library has, teacher and student access to these licensed databases from their home computers, and library participation in a system for the evaluation of print and non-print materials.

Section IX-Library Operational Expenditures and Capital Outlay-asked for 1999-00 budget information pertaining to books, newspapers and magazines, electronic format materials, non-print materials (e.g. audio, video, and microform), electronic access to information, other operating expenditures, and the library's capital outlay including equipment and capital purchases such as furniture and shelving.

2.2 Sample

According to the Texas Education Agency (TEA), there are 7,467 schools in Texas.

The sample selected for this study was stratified by educational level. The sample consisted of four strata, as listed in the adjacent table. The size of each stratum was proportional to its relative size among the total number of schools in Texas. Within each stratum, schools were selected at random.

A total of 600 schools were selected. The schools selected included:

Questionnaire packets were mailed to 600 librarians on August 29, 2000. The questionnaire packets included a cover letter, a copy of the questionnaire, and a self-addressed and stamped return envelope. A reminder postcard was mailed out to non-respondents on October 10, 2000. In addition, library administrators were contacted via e-mail and asked to communicate with librarians in their district who were in the sample to encourage them to respond. Similarly, regional Educational Service Centers contacts and librarians subscribing to the Texas Library Connection (TLC) listserv were asked to encourage sampled librarians in their region to complete and return the questionnaire by mail, fax, or e-mail. During the last week of October and the first week of November 2000, each of the non-responding librarians was contacted by telephone and asked to respond.

Responses were obtained from 503 librarians, yielding an 84 percent response rate. As shown in the table below, the responding libraries were highly representative of both the sample and the universe of schools in Texas.

Of the 503 schools that responded to the survey,

• 267 (53.1 percent) had Grade 4
• 104 (20.7 percent) had Grade 8
• 129 (25.6 percent) had Grade 10

These schools were included in the analysis.

2.3 Texas Education Agency Data

In addition to the survey data provided by school librarians, school and district data were obtained for the schools in the sample from the Texas Education Agency (TEA) through its Public Education Information Management System (PEIMS). Data were obtained in four areas:

• District and school financial characteristics
• Staff characteristics
• Student characteristics
• Academic performance

The data obtained from the Texas Education Agency is detailed in the table depicting the conceptual design of the study.

2.4 Community Data

Data were also obtained from the Federal Reserve Boards' Federal Financial Institutions Examination Council (FFIEC) web site on community economic characteristics for each of the schools in the sample. The FFIEC web site has Census information on income, population, and housing at the Census Tract/BNA level. The site allows the user to extract respective data for a particular geographic location, in this case a school, through a two-stage process. First, by entering a street address, town/city, state and zip code data for the school, the user is able to identify the specific Census tract and BNA. Entering the tract and BNA information generates data on

• Median family income
• Percent of population below the poverty line
• Percent of minority population

Data not available through the FFIEC web site on some of the schools were extracted from the U.S. Census American Factfinder module at www.census.gov. Median family income data and percent population below poverty line data were based on the 1990 Census. These data were adjusted to the year 2000 using an index multiplier that accounts for the intervening changes in the Consumer Price Index (CPI) and the Cost of Living Adjustment (COLA) used by the Social Security Administration.

2.5 Study Design

The conceptual design of the study is three-tiered. It consists of:

• Spheres of influence

  The study has three spheres of influence:

  • The library itself
  • The school where the library is located
  • The community in which the school is located
    
    
    
    
• Indicators

Indicators are elements that stipulate whether the respective sphere of influence should have a positive effect on students' academic performance. For example, a well-staffed library is an indicator of a good library program that would be more likely to encourage and facilitate academic performance than a poorly staffed library.

The indicators delineated for the library sphere of influence include:

• Policies and procedures
• Facilities
• Hours of operation
• Staff
• Staff activities
• Collaboration with public library
• Library usage
• Library material resources
• Technology resources
• Library budget

The indicators defined for the school sphere of influence include:

• School and district financial characteristics
• School staff
• Student characteristics
• Performance on TAAS

The indicator specified for the community sphere influence include:

Economic characteristics

• Measures

Measures are objective elements describing the indicator. For example, staffing levels, types of staff, hours of work, and staff qualifications are measures of a well- staffed library. The study consisted of over 200 measures.

The spheres of influence, indicators, and measures are presented in the following table.

2.6Statistical Analysis

The data were analyzed separately for elementary, middle/junior high schools, and high schools. A variety of statistical analyses were used.

Tests of Statistical Significance

Tests of significance are conducted to determine whether the results are representative of the entire universe (i.e. all school libraries in Texas) rather than of the sample of libraries. Tests of statistical significance are reported as probability (designated by a "p"). Typically, probability is reported as p<.05, p<.01, and p<.001. A probability of p<.05 refers to a 95 percent certainty; p<.01 refers to a 99 percent certainty; and p<.001 refers to 99.9 percent certainty that the results are representative.

Tests of significance are reported in association with Pearson product-moment correlation coefficients in a bivariate correlation.

Bivariate Correlation

A bivariate correlation examines the strength and direction of a relationship between two variables. The bivariate correlation coefficient shows the degree to which variation in one variable is related to variation in the second variable. Pearson's correlation coefficient (designated as "r") ranges from -1.00 to +1.00. A negative r value points to a negative relationship between the two variables: that is, as one variable increases the second variable declines. A positive r value points to a positive relationship between the two variables; that is, an increase in one variable is associated with an increase in the second variable. The report of the r value of the relationship between two variables also includes an indication of its statistical significance.

In addition, r-squared, which ranges from 0 to 1.00 measures the proportion of variance in one variable that is explained by the other variable.

Partial Correlation

Partial correlation is a single measure of association describing the relationship between two variables while adjusting for the effects of one or more additional variables. Partial correlation helps identify variables that may affect the relationship between two variables and allows us to remove their effect from the relationship between the two variables. The assumption underlying partial correlation is that the relationship between these variables is linear. By identifying such intervening variables, partial correlation helps to determine causality.

Factor Analysis

Factor analysis is used to (1) to explore and detect patterns of variables; (2) to confirm hypotheses about the structuring of variables; and (3) to construct indices for use as new variables. Factor analysis is useful for establishing the relationship among groups of related variables that were measured on different scales (for example, percentage and dollars).

Factor analysis reports the loading of each variable on a factor and its direction. A high loading signifies the weight given to a variable. In addition, factor analysis determines the percent of variance accounted for by each factor.

Regression

Multiple regression analyzes the relationship between a dependent variable (e.g. percent of students who met minimum expectations on TAAS reading) and two or more independent variables (e.g. library print resources, electronic resources, hours of operation, staff activities). Multiple regression helps evaluate the contribution of a specific variable or set of variables, find a structural relationship, and provide an explanation for complex relationships among multiple variables. Through multiple regression we can obtain a prediction equation; find out how accurate it is; and determine what percent of the variance in the dependent variable is accounted for by each of the independent variables. It also helps us simplify the prediction equation by deleting those independent variables that do not add substantially to prediction accuracy once certain independent variables are included

2.6.1Analysis Limitations and Difficulties

The study attempted to establish the strength of the relationship between library resources and activities and students' performance on TAAS and identify those library variables that contribute most to students' TAAS performance. Students' performance on the TAAS, the study's dependent variable, was measured by the percent of students who met minimum expectations on TAAS reading.

Little Variance on TAAS. A significant limitation identified in the study involves the limited variance in the performance of students on TAAS (i.e. percent of students who met minimum expectations on TAAS reading). Basically, in 13 to 16 percent of the schools in the three samples, 80 percent or less of the students met the minimum TAAS expectations. In 46 to 54 percent of the schools, 91 percent or more of the students met minimum expectations. For example:

• In the sample of elementary schools:

  In seven percent of the schools, 70 percent or less of the students met minimum TAAS expectations;

  In eight percent of the schools, 71 to 80 percent of the students met minimum TAAS expectations;

  In 31 percent of the schools, 81 to 90 percent of the students met minimum TAAS expectations; and

  In 54 percent of the schools, 91 to 100 percent of the students met minimum TAAS expectations.

• Similarly, at the middle/junior high school level:

  In three percent of the schools, 70 percent or less of the students met minimum TAAS expectations;

  In 13 percent of the schools, 71 to 80 percent of the students met minimum TAAS expectations;

  In 37.5 percent of the schools, 81 to 90 percent of the students met minimum TAAS expectations;

  In 46 percent of the schools, 91 to 100 percent of the students met minimum TAAS expectations.

• In the high school sample:

  In four percent of the schools, 70 percent or less of the students met minimum TAAS expectations,

  In nine percent of the schools, 71 to 80 percent of the students met minimum TAAS expectations,

  In 34 percent of the schools, 81 to 90 percent of the students met minimum TAAS expectations, and

  In 52 percent of the schools, 91 to 100 percent of the students met minimum TAAS expectations.

The low degree of variance in the percent of students who met minimum expectations on TAAS at all educational levels significantly limits the usefulness of this variable, the study's key dependent variable. In addition to the limited/lack of variance, the percent of students who meet minimum expectations on TAAS constitutes a "unrefined" measure, since it gives all students who passed minimum requirements the same weight without differentiating among them by the extent to which they surpassed minimum expectations.

Little Variance in Library Variables. The relationship between a variable and TAAS performance can vary depending on the value of the variable. For example, if all libraries operate either with insufficient staffing or with excess staffing ranges, it will appear as if there is no relationship between library staffing and TAAS performance when, in reality, there is no relationship at current staffing levels. If school libraries were staffed at the effective staffing range associated with the number of students, then a strong relationship between the two may emerge.

Little variation in a variable can affect the analysis of results also in another way. If the variation in a variable is small, small variations in student TAAS performance that result because of changes in that variable may be difficult to distinguish from the noise resulting from measuring TAAS performance. A statistical analysis of the variable may indicate a statistically insignificant correlation when a correlation does exist. The potential for this type of difficulty can be diagnosed by reviewing the distribution of the individual variables and comparing that distribution to the range of feasible variation.

For example, all elementary schools in our sample reported the presence of at least one professional librarian. Consequently, this survey does not support testing of whether the presence of a librarian has a positive effect on TAAS performance.

Establishing Cause and Effect. Establishing cause and effect through statistical analyses is a difficult and multi-step process. A first step in this process is to establish that certain variables are correlated. The following steps are to determine the effect these variables have on TAAS performance; provide a theoretical, research-based or intuitive basis for these relationships; and from this infer the likelihood of a cause-and-effect relationship.

For example, a statistical analysis can indicate that performance on TAAS at a school is positively correlated with the average number of items per student checked out from the library. This correlation, however, can not determine whether a high percent of students who meet minimum expectations on TAAS at a school leads to greater library usage (because highly performing students enjoy reading more) or whether greater library usage results in a higher percent of students meeting minimum TAAS expectations (because students are more exposed to an environment that encourages greater levels of academic performance). This difficulty of interpreting the reason for a correlation is compounded by the fact that the correlations themselves can be interpreted in different ways.

Similarly, students' TAAS performance may be positively correlated with the number of adult volunteers working in a library either because

1. Adult volunteers augment the library staff thereby allowing the librarian to devote more time to teaching and training activities that lead to student learning and contribute to TAAS performance, or
2. Adult volunteers in the library are more likely to be present in communities that are actively interested in education, and academic performance is encouraged in the home.

If the first explanation is accepted, then establishing programs to encourage adult volunteers (or to raise staffing levels in other ways) will result in a higher percent of students meeting minimum expectations on TAAS. If the latter explanation is accepted, the involvement of adult volunteers in the library will have little effect.

The primary tool we have used to address this issue, was the a priori breakdown of the measures that might affect TAAS performance into the sphere of influence-indicator-measure hierarchy. This breakdown helped us examine a variety of different factors that could result in erroneous conclusions.

Other difficulties with a cause and effect analysis involve situations of the following nature. Principals in low performing schools may respond to the situation by instituting measures to improve TAAS performance by increasing library staff, enriching the library collection, installing technology in the library and throughout the school, and encouraging collaboration between librarians and teachers. From a statistical point of view, such actions mean that measures to increase TAAS performance correlate with low performance, because these measures are in place at schools with low TAAS performance levels as well as at schools with high levels of TAAS performance. Situations like these can result in analyses that are less discriminating and more difficult to interpret.

Uncorrelated, But Complementary, Measures. Most of the library measures included in this study do not have a direct effect on TAAS performance. Instead, these measures have a direct effect on one or more indicators, which have a direct effect on performance. For example, good library staff, which is an indicator of a program that encourages higher performance, is expected to have a direct effect on performance, while staffing levels and qualifications, which are measures of a good staff, have an indirect effect by helping build a good staff.

Masked Correlations. The influences considered in this study were identified on the basis of previous research and recent studies. The library is the primary focus of this study. The school and community spheres of influence are used primarily to control for non-library factors that could mask the influence of the library factors. For example, the school size and TAAS performance are not significantly correlated. However, school size correlates with limited English proficiency, which has a strong negative influence on TAAS performance. By controlling for limited English proficiency, the correlation between school size and TAAS performance becomes significant.

Non-linear, Single-variable Correlations. Some variables may affect TAAS performance in a non-linear manner. For example, library staffing that is below the level required for general library maintenance could result in librarians with little time to work with students and teachers. For staffing levels below this cutoff, further staff decreases will have little affect on TAAS performance because the library staff is already unable to contribute directly to performance. As staffing levels increase beyond this cutoff, librarians have more time to work with students and teachers on activities that affect TAAS performance and increases in staff should result in greater levels of student performance. Above a certain level, the effect of staff levels on student TAAS performance will decrease because the library staff already has sufficient time to implement important activities, and any additional staff will have trouble finding new activities that will impact student performance. In fact, the resources spent on additional library staff at these levels could result in decreased student performance because the resources allocated to excess library staff were probably taken from other resources that might positively affect student performance.

Non-linear, Multi-variable Correlations. It is possible that some variables have little effect on TAAS performance by themselves, but do have an affect on performance when taken together with other variables.

Continue on to Chapter 3