Updated June 1998
This part of the Texas State Records Management Manual is divided into two sections. Section I deals with Micrographics, the oldest of the document imaging technologies. Assessing the feasibility of microfilming is discussed as are the procedures and technical aspects of the micrographics conversion process. Section II deals with Electronic Document Imaging, the newest imaging technology. The discussion in this section parallels that of Section I, exploring the feasibility of electronic imaging and then dealing with issues concerning the media conversion process.
Pertinent to this part of the Texas State Records Management Manual are two sets of administrative rules adopted by the Texas State Library and Archives Commission: Microfilming Standards and Procedures (13 TAC §§6.21-6.35) and Electronic Records Standards and Procedures (13 TAC 6.91-6.99).
The principal uses for microfilm have changed in recent years. Prior to the emergence of electronic document imaging, film was used to manage active records as well as to preserve records. Film is still used in this manner, but the focus has begun to shift.
With the advent of document imaging systems, active document management is better suited to the newer technology. These systems offer benefits such as rapid retrieval, mass storage, concurrent access, and ease of use. Manufacturers of optical disks may guarantee their optical media for 50 or even a 100 years. The problem is that even though the disks may physically survive 100 years, the supporting technology may not; i.e., the software and hardware needed to read today's optical disk may not be capable of doing so many years from now. That is why state agencies and local governments are required to develop a migration strategy for upgrading equipment as technology evolves.
Microfilm, on the other hand, has long been and continues to be the medium of choice for preserving long-term records. The life expectancy of microfilm can be up to 500 years if processed and stored correctly. Not only is it a stable and proven medium, it is eye-readable. Film requires only the most simple technology (a light source and a lens) to be read.
A state agency's requirements for recordkeeping will dictate the methods it chooses. If it has a need for fast retrieval of active records, the use of an electronic imaging system should be explored. If, on the other hand, it needs to preserve records for long periods with little or no activity, then microfilming would be more appropriate.
Throughout the section, terms will be used that are specific to micrographics.
Micrographics systems are information storage and retrieval methods that utilize the special advantages of microfilming in the areas of space saving, reproducibility, durability, file integrity, and retrievability.
Micrographics is the creation, use, and storage of microforms.
Microform is a generic term for any medium which contains miniaturized images.
Microfilming is the photographic process of creating miniaturized images of records on film.
Source document microfilming is the creation of filmed images directly from paper documents, maps, and drawings.
Computer Output Microfilm (COM) contains filmed images created from electronic data, either from magnetic tape or from direct transmission from the host computer.
Microfilming has advantages and limitations which must be considered before making a decision to microfilm. Remember: a micrographics system will not change poor records management into good records management. The following is a brief discussion of the pros and cons.
Advantages of microfilm include:
SPACE SAVINGS - Microfilm can be stored in less space than the corresponding paper records.
LOW-COST DISTRIBUTION - Film duplicates are inexpensive to produce and can be mailed at a lower cost than the corresponding paper records.
INFORMATION RETRIEVAL - Access and retrieval can be improved due to the concentration and ease of manipulation of microfilmed information.
FILE INTEGRITY - Once records are microfilmed, their order cannot be changed, and the chance of losing or misfiling individual documents is eliminated.
SECURITY - The original film or security copy is kept off-site to protect against loss of information. Any working copies of the microfilmed records will be made from the protected master copy.
ARCHIVAL PRESERVATION - Microfilm copies of important records can be used for reference instead of the original document. Also, if filmed properly and if the film is processed and maintained according to required standards, the life span of historically valuable records can be increased by microfilming.
REVERSION TO PAPER - If necessary, a paper copy of the original document can be produced from the microfilm image.
LEGAL VALIDITY - Texas law provides that microfilmed records, if filmed and processed in accordance with the state Microfilming Standards and Procedures (13 TAC §§6.21-6.35) are original records and shall be accepted as such by any court or administrative agency.
COST SAVING POTENTIAL - If well planned and maintained, a microfilm program can produce cost savings through decreased storage costs and increased retrieval efficiency.
Limitations of microfilm include:
LACK OF ACCEPTANCE - Even though microfilm is a legally accepted medium, many people are hesitant to accept filmed records in lieu of paper. Some people will resist using microfilm, especially if they have not been trained or acquainted with the technology. This is a limitation of attitude, not of the microfilm, but it still must be considered.
QUALITY CONTROL - Preparation and filming must be closely monitored so that documents are not filmed out of order or in any other violation of file integrity. Film processing must also be monitored because it is a complex, chemical process that requires specialized equipment, trained staff, and process controls.
TRAINING - Personnel must be properly trained to detect errors in filming and levels of film quality. The cost of training can be substantial.
SPECIALIZED EQUIPMENT - Since microfilmed records cannot be read with the unaided eye, microfilm readers and reader-printers are needed to access the images and to make paper copies of the filmed documents. This equipment can be expensive to purchase and maintain.
COST - The cost saving potential of microfilm is substantial, but the process of microfilming is expensive. Document preparation, filming, processing, testing, duplicating, and indexing are all costly procedures. A poorly planned microfilm program can spend much more than it saves in storage costs.
STORAGE MAINTENANCE - Microfilm requires a more carefully monitored storage environment than does paper. It is more sensitive to variations in temperature and humidity.
Space. The most visible advantage of either microfilming or electronic document imaging is space savings. For example, microfilmed records can require as little as two percent of the space occupied by the same records on paper, but space savings alone is not a reason to microfilm an agency's records.
A decision to microfilm records should only be made after a feasibility study has been conducted to determine whether microfilming is functionally and financially appropriate for managing information within the agency. Careful consideration must be given to the effect of micrographics on agency staffing, funding, costs, and operations. An agency's records management needs often change due to growth, new legislation, or changing internal policies. The feasibility study should take into account both present and projected needs and may have to be done again when conditions change.
Staff of the State and Local Records Management Division are available to advise agency records management officers in conducting a microfilming feasibility study.
A needs assessment gives direction to the feasibility study. Microfilming can be a very efficient storage and retrieval method, but only if your agency can benefit from its use. An agency should analyze its records to determine which would be candidates for filming. In addition, a cost analysis of the possible application of micrographics in your agency must be carried out.
RECORDS RETENTION - Microfilming of records must be in compliance with a records retention schedule that has been approved by the agency records management officer, the State Auditor, and the Director and Librarian of the Texas State Library. See "Records Scheduling" (Part II, Texas State Records Management Manual) for more information on the records retention schedule approval process. By using the retention schedule, which is based on the inventory and appraisal of your records, you will be able to make informed decisions about which, if any, of your records to microfilm.
What is the retention period listed on your agency records retention schedule? When can the records be destroyed? Storing inactive hard copy records in the State Records Center or at another off-site records storage facility rather than in active office space can result in significant cost reduction. Because of this, it is usually not cost effective to base a decision to microfilm records with short-term retention periods solely on the space savings.
VOLUME - High volume may suggest that a records series be microfilmed for convenience of handling, but volume should not be used as the only consideration. For example, if a records series has a high volume and a low retrieval rate but a retention period of only three years, it is usually a waste of personnel time and tax money to film the series. Some agencies may be tempted to base their decision to buy microfilming equipment on a backlog of records which should be filmed. The danger in this approach lies in justifying the expense and upkeep of the equipment once the conversion from paper to film has been made. The feasibility study should incorporate long-term as well as short-term projections of micrographics needs.
USE - How frequently are the records used, by whom, and for what purpose? How much would access to the records be improved if the records were on microfilm? What would be the resulting cost savings or cost avoidance?.
QUANTITY - How quickly do the records accumulate? How much space and filing equipment could be saved during the retention period if the records were filmed and the paper originals destroyed? To what extent would the cost savings in space and equipment justify the cost of microfilming?
PHYSICAL CHARACTERISTICS - What is the shape, size, and condition of the original records? How will these aspects of the records affect the costs of document preparation and filming? Will the information be legible on film? Dimensions, degree of fragility, and even color of the records can make a difference in deciding if and how the documents can be filmed.
COST CONSIDERATIONS - If the agency decides to contract with a private service bureau for the source document filming of records, the cost is usually calculated on a per image basis. For example, the Imaging Services Unit of the State and Local Records Management Division currently charges on a per image basis, with the cost being calculated by the characteristics of the records, the type of camera required, and other variables. In many cases this is more cost effective to an agency than establishing an in-house micrographics operation.
If an agency decides to establish an in-house micrographics operation, there are also fiscal considerations that must be included in the feasibility study. Each agency will have its own manner of conducting an economic analysis, and the following elements of analysis should be considered.
LABOR - Several questions can be asked about the labor requirements of an agency's current records management system.
- What are the present steps required to access a record or to enter a record into the system? How might this be changed if the records were microfilmed?
- To what extent is the present file accessed simultaneously by more than one person and to what extent does this result in delays in retrieval, access, and refiling?
- What is the present turnaround time for manually retrieving a specific document? How does it compare with what can reasonably be expected if the record were in a microfilm format? For example, how much less walking time would be involved for a file clerk? Turnaround time on requests can affect user productivity and clerical labor requirements.
- What is the present time requirement for refiling? How does it compare with what could reasonably be expected if the same document were filed more conveniently in microform?
- To what extent are retrieval and refiling delays incurred by previous misfiling of documents under the present system?
- What are the possibilities that microfilming would permit more convenient locating of the file? What might this mean in terms of personnel or mail travel time, within or between locations, by those who must access the file regularly?
- In what ways will the adoption of micrographics be likely to affect the training of personnel and the skills and responsibilities required of them?
CAPITAL OUTLAY - The initial cost of the filming and processing equipment can be substantial. Capital outlay involves the costs of cameras, laboratory machinery for the processing and quality testing of film, film editing devices, and any installation charges.
MAINTENANCE COST - There will be maintenance costs associated with the use of any micrographic equipment. The equipment contains mechanisms that are subject to wear and misadjustment, and components that are breakable or have limited life. As a general rule, the anticipated annual maintenance cost may be as much as 10 to 15 percent of the purchase price of the equipment. When equipment is rented or leased rather than purchased, the annual cost may be between one-half and one-third the purchase price, depending on the vendor. Maintenance is often included in the rental charge.
MATERIALS AND SUPPLIES - Depending on the type of microfilming done, the necessary supplies and related materials may include:
For document preparation:
- Paper, pens.
- Staple removers.
- Tape, sealing material, iron.
- Production logs.
- Camera brushes, cleaning supplies.
- Splicing tapes.
- Reels, film boxes, cartridges.
- Control strips.
For editing/quality control:
- Magnifying devices.
- Tracking logs.
PERSONNEL REQUIREMENTS - In addition to anticipating how the use of microfilm will affect current staff, hiring and training of additional staff to operate and maintain micrographics equipment must also be considered. Staff positions may include:
- Document preparation clerks.
- Camera operators.
- Quality control editors.
- Processing technicians.
FLOOR SPACE - If an agency decides to establish its own micrographics operation, the cost of having the equipment in the agency offices must be considered. You must calculate the cost of the space taken up by the equipment itself as well as any tables or stands on which the equipment will be placed, and the operating space required for use and handling of the machinery.
STRUCTURAL ALTERATIONS - It may be necessary to undertake structural modifications to rooms to be used for microfilming, especially film processing. Processing labs must be properly vented because hazardous chemicals are used.
The microfilm program must be reviewed yearly by the records management officer for compliance with the Government Code, Chapter 441 and the state Microfilming Standards and Procedures (13 TAC §§6.21-6.35). The following review checklist may be useful:
- If you have installed an in-house system, evaluate the equipment's performance and maintenance records. What percent of the time is the equipment in use? Is the equipment appropriate for the volume and the speed required.
- If your filming and processing are being done elsewhere, evaluate the quality of filming overall, the turnaround time for filming and processing, and the frequency and severity of problems with any aspect of the filming system.
- Examine the documented operational policies and procedures. Are they sufficient to ensure an effective microfilming operation, or do they need revising or expanding?
- Evaluate the microfilm staff's skills. Are they performing tasks for which they are properly trained? Is additional training warranted?
- Are costs running within budget and at the projected rates?
- Review the system as a whole. Is it meeting agency goals?
When considering the microfilming of records, a comparison between contract services and the establishment of an in-house operation should be a prime consideration. It is clear that an agency must carefully analyze the decision to set up an in-house micrographics operation. An alternative to an in-house system is the use of services offered by the State and Local Records Management Division of the Texas State Library.
An imaging service bureau is administered by the State and Local Records Management Division, which is authorized by statute to provide micrographics services to state agencies and local governments on a cost-reimbursement basis through interagency contracts and other agreements. The service bureau at the division meets or exceeds state standards and requirements for producing microfilm originals and convenience duplicates.
General procedures for contracting with the State and Local Records Management Division for these services include the following steps:
- An agency contacts the division for information regarding the feasibility and advisability of microfilming records of the agency.
- An imaging consultant from the division visits the agency to examine the records and determine whether a records series can be reproduced on film.
- The consultant works with the agency to determine the appropriate filming format and specifications.
- The division provides cost estimate an estimation of the duration of the job.
- A contract or agreement for services between the agency and the State and Local Records Management Division is negotiated.
Whether you contract with the State and Local Records Management Division or with a commercial service bureau, or determine that it is cost justified to do all or part of your micrographics in-house, standard procedures must be followed in order to produce an acceptable product. All film must be produced in accordance with the state Microfilming Standards and Procedures (13 TAC §§6.21-6.35). The following are procedures for converting hard copy records to microfilm.
The actual micrographics process begins with document preparation. Each file and its documents are made "camera ready" before they are filmed. The document preparation process involves removing all bindings, clips, fasteners and staples, repairing paper tears, flattening bent pages and inserting indexing documents.
Improper document preparation can slow the filming process, damage documents as they become caught in equipment, and even damage the camera equipment. It is essential that proper care is taken to make the documents ready for smooth and efficient progress through the filming stage.
After documents have been properly prepared, they are filmed using the appropriate equipment. Camera type is determined by a number of factors such as user requirements, condition of the documents to be filmed, and final microform format preference. There are three types of cameras most commonly used in the source document filming of government records.
The planetary camera is used for large, brittle or old documents which require care in handling. It is also used to film books and pamphlets which must remain bound and for other projects requiring the highest quality of filming.
In using a planetary camera, the operator places each document on a stationary copy board for filming. This minimizes the chance of mechanical or operator error and usually increases the resolution quality of the image on film.
The rotary camera films same-size documents which can be automatically fed into the camera, as well as other documents of a condition and size to permit carriage through the camera's mechanism. Filming is faster than with a planetary camera, but there is a greater margin for error.
The step and repeat camera is a large computer-interface camera which produces microfiche. Using this device, the camera operator can film various sizes of documents at an increased speed and can program indexing information directly onto the film. This camera is ideally suited to micropublishing.
After filming, the next step is microfilm processing. Proper processing of the film is critical to the overall quality and archival longevity of the film. The processing equipment and chemicals must be continuously monitored to ensure an acceptable microfilm product.
The State and Local Records Management Division maintains a fully equipped micrographics laboratory, staffed by technicians using industry-standard technology.
After the microfilm is processed, it must be tested on a densitometer to ensure that image and background have proper density. The resolution of the film, which is the quality of sharpness of filmed images, is inspected through a microscope. To evaluate the image against a standard, a resolution target must be filmed at the beginning and end of each roll of film.
Another required test, the methylene blue test, must be performed weekly on each processor to determine the amount of sodium thiosulphate left on the film after processing. Sodium thiosulphate affects the long-term quality of the film storage. If the reading from this test is not satisfactory, the documents must be refilmed and processed. This service is available to all agencies that either process their own film or have it processed by a bureau other than the State and Local Records Management Division. The test is done on all film processed at the division.
No matter where or by whom documents are microfilmed, all film must be properly edited. This is a key element of the quality control procedures that must be in place in any micrographics system. Editing reveals if the documents are accurately represented on film, if there are errors in the filming by the camera operator, and if the processing measures are satisfactory. See below for a discussion of commonly encountered problems and defects. Under no circumstances should the original documents be destroyed until the film images have been edited.
Editing must be conducted in a clean environment to avoid contamination or damage to the film. No smoking or food should be allowed in the area. Editors should wear lint-free cotton gloves when handling all types of film. During editing, attention must be given to indexing, organization, and any evidence of operator error or equipment malfunction.
Standard editing equipment includes a light box or light table, which provides diffused illumination evenly dispersed under the viewing area, and a microfilm reader, which is a projection device for viewing an enlarged microimage with the unaided eye.
Light box editing is performed by loading a reel on one side of the light box and manually passing it across the box to a reel on the opposite side. Technicians can spot problems with images, and then make a more careful inspection with a magnifying device.
Microfilm reader editing uses a roll reader for detailed editing of the film. Each document image is inspected to make certain it is legible. If an editor detects errors, the corresponding documents are sent through a retake cycle and refilmed. The new film is then processed and inspected. The refilmed documents are spliced to the beginning of the original roll of film to maintain file integrity.
Editing and retake procedures must be strictly adhered to in order to produce a product meeting minimum state quality standards.
Defects are classified as either major or minor. If important information is obliterated, the defect is classified as major. The probable cause should be noted on an inspection report and the identification of the affected documents should be recorded on a refilming log. If information is not obliterated, the defect is classified as minor. The probable cause should be noted on the inspection report.
There are a number of common problems or defects that can occur during filming or processing that leave characteristic marks on film. Editors should be aware of these marks and conditions.
BLANK FILM - No images of any kind, usually a result of the film not advancing or shutter failure.
CONTRACTION - A shortened image containing a dark bar across the width of the image on film exposed in a rotary camera. This is usually caused by a failure of the film drive.
DARK VERTICAL STREAK - A defect that appears as a darker density line running parallel to the edges of the microfilm.
1) In a rotary camera this is caused by:
- A foreign object, located between the documents and the lens which has reflectivity greater than that of the documents being filmed.
- Improper positioning of the lamps or mirrors.
- Lamps not evenly matched.
2) Defect in film manufacturing.
3) Created during processing.
DOUBLE EXPOSURE - Two distinct sets of documents will be superimposed on the film when double exposed. The overlapped sections of the documents will appear darkened, although the outlines of each set of documents are still distinguishable. The possible cause is that the operator has re-exposed a previously exposed roll of film, or the film has failed to advance.
EDGE FOG - Dark margins along the length of the film. This is the result of light leaking between the flange of the spool and the film. This condition is aggravated by:
1) Prolonged handling of the unprocessed microfilm in room light.
2) The flexing of the flange of a plastic spool during unloading.
3) Use of a spoked reader reel in place of a camera spool for take-up.
4) Camera spool flanges out of tolerance, too far apart or bent.
FINGERPRINTS - Darkened or black areas on the microfilm. Probable causes are:
1) Improper camera loading.
- Taking too long to load camera in room light.
- Failure to observe subdued-light loading recommendations.
- Failure of camera operator to provide sufficient leader and trailer before and after document exposures.
- Camera door ajar or accidentally opened.
2) Defect created during film manufacturing.
3) Camera light leak.
4) Fog from processing.
- Improper or contaminated developer.
- Failure to observe safelight recommendations.
5) Use of outdated or improperly stored film.
FOLDED DOCUMENTS - A defect in which the document being filmed has folded over on itself, causing some information to be blocked out. This is caused by improper feeding or transporting of the documents through the camera. It is more likely to occur when documents are in poor condition.
FRILLING - A puckering and peeling of a photographic emulsion layer from its support during processing. Usually caused by excessive temperature or improper compounding of the chemical baths, poor adhesion qualities of the emulsion to the base, improper hardening of the gelatin, the use of very soft wash water, or a combination of these circumstances.
JAM - This will appear as parts of documents followed by a dark streak on the film. In rotary cameras, a jam is caused when one or more documents are caught in the exposing plane.
LIGHT VERTICAL STREAK - A defect that appears as a lighter density line running parallel to the edges of the microfilm.
1) In rotary cameras this is caused by:
- An obstruction of some type between the documents being filmed and the lens which allows less light to be transmitted to the lens.
- Scratched or dirty guides.
- Improper positioning of the lamps and mirrors.
- Lamps not evenly matched.
- Lamp or lamps burned out.
2) Defect in film manufacturing.
3) Created during processing.
MOTTLE - Cloudy or blotchy appearance, uneven density, generally caused by insufficient agitation during processing, storage conditions, or defective emulsion.
OVERDEVELOPMENT - Images or D-min (the lowest density obtainable in a processed film; occurs when there is no image on film), or both, are darker than normal. This is caused by the film being developed too much because of:
1) Excessive time in the developing chemicals.
2) Excessive temperature.
3) Over-strength solution.
4) Excessive agitation.
5) A combination of these circumstances.
OVEREXPOSURE - Images are too dark, but the D-min remains normal. This is caused by one or more of the following:
1) Improper response of exposure control.
2) Light density too high.
3) Aperture too large.
4) Exposure time too long.
PRESSURE MARKS - A defect found in processed film that may appear as an area of reduced or increased density. An abrasion or a striking of the emulsion may cause the formation of a latent image or the destruction of an existing latent image.
RESIDUAL DYE-BACK - Residual dye-back is indicated by black particles or dark streaks remaining on the microfilm. It is caused by incomplete removal of the back-coating material.
RETICULATION - A processing defect affecting gelatin layers on photographic film which, upon drying, shows an irregular surface due to the formation of small, irregular scaly patterns. Sharp differences in the temperature or pH of successive processing solutions are the usual causes of reticulation.
SCRATCHES - A dark or light linear groove which damages the base (also called the sensitized side) of the film and which is usually caused by faulty equipment or improper handling.
STACKED OR OVERLAPPED - A defect in which one image or document partially covers and obscures another. In planetary cameras, this is caused by improper film advance, causing the image to overlap. In rotary cameras, overlap is caused by improper setting of the document stop or improper clutch adjustment.
STATIC MARKS - Black spots, streaks, or tree-like forms produced on microfilm by the discharge of static electricity, generated in the film by friction and made visible by developing.
STRETCHED - An elongated image which is caused by the document stopping, hesitating, or slowing down while the microfilm continues to advance in the rotary camera.
SYNCHRONIZATION, OUT OF - An out of sync condition will cause the image to have blurred bands across the width of the film. In rotary cameras this is caused when the speed of the film transport is not synchronized with the speed of the document transport.
UNDERDEVELOPMENT - The images will appear too light on the microfilm. This is caused by insufficient development due to:
1) Developing for too short a time.
2) Use of a weakened developer.
3) Too low a temperature.
UNDEREXPOSURE - The images will appear too light on microfilm, but the light-struck areas at the beginning and the end of the roll appear at high density. Insufficient exposure of the sensitized material is due to:
1) Improper response of exposure-control device.
2) Light intensity too low.
3) Lens aperture too small.
4) Exposure time too short.
WASHBOARD - A defect that appears as alternate bands of greater and lesser density across the width of the film. This may be caused by:
1) Fluctuating illumination.
2) Faulty document transport.
3) Faulty film transport.
WATER SPOTS - A defect that usually appears as dots or rings and may be caused by:
1) Deformation of the gelatin layer in an irregular spot pattern. This is caused by water drops on the surface during drying, due to improper pressure.
2) Residue from materials in the wash water.
The state Microfilming Standards and Procedures (13 TAC §6.23) prohibit the use of original microfilm as a working copy. Therefore,unless the original film is meant to serve simply as a backup to records in hard copy or in electronic format that will not be destroyed after filming, the microfilm must be duplicated and the duplicate used as the working copy. Original microfilm must be stored in a separate building from that in which duplicate copies, if any, or the original records are housed.
After the film has been processed, edited and duplicated as necessary, it must be properly stored. For detailed information regarding the minimum requirements for storage of original microfilm, please refer to the state Microfilming Standards and Procedures (13 TAC §6.26).
State Microfilming Standards and Procedures (13 TAC §6.27) require that an inspection of stored microfilm must be conducted every two years, except if the microfilm has been stored under temperature and/or humidity conditions other than those specified in the standards, it must be inspected yearly. The microfilm should be examined for discoloration, blemishes, fogging, fungi, buckling, brittleness, image fade, adhesion, or other signs of degradation or deterioration. See the state standards and procedures for more detailed information on inspection of stored original microfilm.
The State and Local Records Management Division has vault space for 330,000 rolls of microfilm, which is equivalent to 528,000 cubic feet of hard copy records. The division maintains an index of the film's content to provide efficient retrieval, adherence to retention periods, and periodic inspection to insure archival longevity.
The decision to automate all or part of a governments records is one of the most significant decisions a government makes, not only because can involve a great deal of money, but also because it is a highly visible undertaking. Electronic document imaging is too complex and costly to undertake without first engaging in extensive investigation, discussion, and decision-making. These systems can be vulnerable to undetected alteration, loss, or unauthorized disclosure of information. Comprehensive and detailed planning is needed before these systems are established. In addition, the maintenance of electronic records requires the adequate management of policies, procedures, and equipment to ensure the accuracy and availability of the records. This section will focus on electronic document imaging through the use of scanners. Staff of the State and Local Records Management Division are available to consult with agency records management officers on other electronic document imaging technologies.
ELECTRONIC DOCUMENT IMAGING(EDI) - a technology used to convert and process paper documents and/or microfilm to digital electronic information. It describes processes that store copies of documents in digital format (computer readable) and their associated character-coded index data for retrieval. These systems are pictorial, and originate from scanned documents. Scanned images can be stored either on optical media or magnetic media.
OPTICAL STORAGE - an optical disk in a direct access storage device that is written and read by laser light. All optical disk systems use binary digital coding to store the information, a high powered laser light beam is used to record information and a low powered laser light beam to read the information. Optical systems use optical disks to store information and a computer to manage (access and retrieve) the information. Two types of optical discs are in use today; 1) WORM -write once read many times and 2) rewritable or erasable optical disk.
MAGNETIC STORAGE - recording of information (analog or digital) on a medium of specially prepared grains of iron oxide. Examples of magnetic storage are magnetic tape, hard drives, floppy disks and some hybrid optical disks known as magneto optical disks. Magnetic media may be reused by erasing data or writing directly over previously recorded information.
SCANNER - a device that optically senses a human-readable image and contains software to convert the image to bitmapped, machine readable code.
Electronic document imaging has advantages and limitations which must be considered before making a decision to microfilm. As with a micrographics system, electronic document imaging will not change poor records management into good records management. The following is a brief discussion of the pros and cons.
Advantages of imaging include:
SPACE SAVINGS - Optical disks and magnetic media offer greater storage potential than corresponding paper filing cabinets.
RAPID RETRIEVAL - Faster access to records reduces labor costs, thus eliminating significant personnel overhead. Faster retrieval also increases customer service.
CONCURRENT ACCESS - Two or more users can access data at the same time over an imaged enabled network.
IMAGE ENHANCEMENT - Poor quality originals can be enhanced by software designed to remove noise and increase brightness and edge sharpness.
OPTICAL CHARACTER RECOGNITION - Scanned documents can be converted to machine readable text by the use of optical character recognition software, thus eliminating the need to index the documents in some cases.
WORKFLOW - A workflow system can automate and control certain business processes. The system can schedule processing, route documents automatically among departments and can track each transaction's status.
Limitations of imaging include:
COST - Consideration must be given to the cost of hardware, software, system maintenance and administration and conversion costs such as document preparation, indexing, and quality control.
COMPLEXITY - While computer systems are becoming more "user friendly", a certain amount of technical skill and experience are still needed to adequately operate and maintain an imaging system.
LACK OF STANDARDS - Successful retrieval of images on imaging systems depends on the availability of appropriate retrieval hardware and software. Images recorded on one vendor's imaging system may not be accessible on another vendor's system due to lack of standardization of media, hardware, image compression, image format, and retrieval software. National and international standards associations are currently addressing these issues.
RISK OF OBSOLESCENCE - Due to rapidly changing technology, images recorded today's imaging system may not be accessible many years from now if the system is not maintained properly. Proper management of optical and magnetic media as well as computer operating systems, software applications and hardware is essential to preventing records inaccessibility due to technological obsolescence.
Electronic document imaging (EDI) systems offer new opportunities to reduce costs, increase productivity, and improve customer service. However, as is also the case with microfilming, the decision to employ electronic document imaging should also be based on a feasibility study. A feasibility study is the analysis an agency must perform to determine if information imaging technology will solve a problem, and if so, in what way. Is EDI possible and suitable for the agency? Will the cost of imaging justify the acquisition of this new technology? Imaging alone may not yield the benefits promised. Quite often a thorough examination and redesign of business processes may be needed before adequate results are achieved. The feasibility study should reflect the agency's present and projected needs. It should identify direct and hidden costs of implementing an imaging system as well as identify and measure direct and strategic benefits associated with an imaging system.
The needs assessment should help you determine whether your agency needs this new technology. What you should ask of your agency's recordkeeping practices and the records it creates and maintains are similar to those asked in reaching an informed decision to microfilm. Questions to ask during a needs assessment include: Has a records inventory been completed? Does the agency have an approved retention schedule? Are records purged on a regularly based on the retention schedule? Are the records being considered for imaging active or inactive? What is the activity rate of the records in consideration? How fast is the information needed when called for? Who are the users of this potential system and what are their requirements? Is there inexpensive, appropriate storage available to inactive records? Have alternative methods for filing and retrieval been considered to solve the identified problems with active records? In addition, what is the size and condition of the paper records to be scanned? Are there identified difficulties with using the records in paper or microfilm form, such as excessive retrieval time, high cost of distributing multiple paper copies to different locations, or the need for simultaneous multiple access within the agency?
Staff of the State and Local Records Management Division are available to advise agency records management officers in conducting a electronic document imaging feasibility study.
The central word of caution to those considering electronic document imaging: potential users of imaging systems too often confuse their records retention requirements with their retrieval needs. Imaging systems are designed to provide rapid retrieval and distribution for highly active documents. However, given their limited stability and dependence on particular hardware and software configurations, it is highly unlikely that a single set of imaging components will be able to satisfy retention and access requirements for long term and permanent government documents. Officials need to be aware that maintaining records exclusively in electronic format takes a serious and ongoing commitment of financial resources. Constant monitoring of the hardware, software, and documentation for the imaging system as well as implementing specific steps to protect the electronic records medium are required. All storage or maintenance of state government records electronically must adhere to the state Electronic Records Standards and Procedures (13 TAC 6.91-6.99).
An analysis of costs associated with any existing recordkeeping system as well as any proposed electronic imaging system must be conducted to determine if the new system is economically feasible. Costs to include for analysis of the present system should be direct costs such as labor involved with storage, retrieval, processing, refiling, and distribution. Current costs of materials and equipment such as filing cabinets, filing supplies, floor space, paper, copiers, and printing supplies must also be included. An agency should be able to calculate the cost to maintain the current system for a period of several years. Indirect costs such as agency administrative support, fringe benefits, facility amortization, and utility charge back should also be factored into the total cost of the present system and any projected future system.
Costs for the new system will include one time costs such as hardware, software, training, installation, application development and back file conversion. If the imaging system will be networked be certain to include the cost per seat to install a network or upgrade the existing network. Ongoing direct costs such as personnel, system maintenance. And supplies need to be included in the economic analysis. Finally, ongoing indirect costs such as agency administrative, facility and utility costs should be included. Backfile conversion can sometimes be accomplished by a service bureau that specializes in scanning documents instead of your agency attempting it in-house. This can result in significant cost avoidance during the conversion process because service bureaus have the edge in necessary equipment, personnel, and experience to handle the task.
Any decision to resort to electronic document imaging in an agency must be reached in a judicious and thoughtful manner; otherwise tax dollars will wasted and staff time forfeited. What follows is a discussion of key conversion issues that must be examined and considered by an agency before a final decision is made to convert any records of an agency to a document imaging medium.
Before conversion begins, detailed planning must be done. Some questions to be answered during the planning stage are: What is the volume of the project? What types of documents are to be scanned? How many record series will be involved and what are their retention periods? How many scanners and indexing workstations will be needed? Will OCR be needed? How many people will be needed to scan and index? Will this be a "day forward" conversion or will a backfile conversion be needed? What is the disposition of the documents after scanning?
Document preparation is often overlooked during the planning stage. Document preparation can be very labor intensive and costly. An agency must develop procedures for receiving and tracking documents as well as preparing them. Any problems with files (missing pages, out of sequence, poor quality) are better resolved during document preparation before it gets to the scanning station. Some steps involved in document preparation are purging of files, unbinding, page repair, and staple removal.
A scanner converts documents to computer processible digitized images for storage on optical discs or magnetic media. There are many types of scanners on the market today, from hand held scanners to large engineering drawing scanners. Paper scanners include flatbed types and automatic document feed types. Flatbed scanners are used for delicate or hard to handle documents and generally considered to be the slowest type of scanner. Automatic document feed scanners are typically high speed scanners capable of converting large volumes of standard size documents into digital format. Film scanners are also available and vary in characteristics from small volume "scan on demand" types to fast, efficient production models. Film scanners can scan roll film (both 16mm and 35mm) as well as microfiche and jackets.
An index is a computer searchable database that acts as a finding aid for images stored on an imaging system. Proper indexing is critical to the successful implementation of an imaging system. If the index is poorly planned, document images will be unretrievable and system performance will be degraded. Defining user needs and requirements for indexing elements is the first step in the planning stages of the new system. First, find out how users typically ask for and use documents in a particular application. This process may include interviews or discussions with potential users of the imaging system. Secondly, design an indexing scheme. This would include such items as how many indexing fields there will be, how long the fields will be and what characters will be included in the field. Next, design the indexing procedures such as manual data entry from the hard copy or entered from the displayed scanned image. Optical character recognition (OCR) can sometimes be used in indexing to eliminate manual data entry, thus eliminating significant personnel costs. Finally, run a test on actual documents and analyze what the cost will be to index the documents.
The purpose of quality control is to ensure that the quality of the digitized images and associated indexes meet the minimum standards required by law and by those standards set by the agency. The successful implementation of the imaging system depends on effective quality control procedures that are properly documented, administered and maintained. These procedures include, but are not limited to image inspection, scanner testing, and verification of image and index data recording.
IMAGE INSPECTION- Images may be displayed for inspection either immediately after scanning or just prior to index entry. Documents may be scanned in batches for inspection at some later time. The combination of image inspection and index data entry can be very effective. If the displayed document serves as a source for indexes, the operator must examine the images closely. The quality control operator must look for whether the document was scanned right-reading, the document passed through the scanner without skewing or folding edges, the document is obscured in any way. The operator must confirm that the small print, light pencil marks, and faded areas are legible. State standards require that every scanned image is visually inspected.
SCANNER TESTING - Scanner testing provides a means of quality control from document input to output. The use of ANSI/AIIM MS44-1988 is required for paper scanners and ANSI/AIIM MS49-1993 is required for film scanners. The purpose of these procedures is to allow the operator to determine that the scanner is properly setup before scanning documents. Secondly, the procedures give the operator knowledge of what the scanner is capable of and finally to provide the user with information needed to set up criteria for quality control procedures. Verification of Image Recording- In order to confirm that optical disk drives and other system components are recording information properly, a sample of recorded images should be retrieved from optical disks and displayed or printed for operator inspection and comparison with the source documents. This should be performed daily during the first several weeks of system implementation and periodically thereafter. Original documents should not be discarded until reliable, consistent optical disk recording is confirmed and all state standards are met.
VERIFICATION OF IMAGE RECORDING - In order to confirm that optical disk drives ad other system components are recording information properly, a sample of recorded images should be retrieved from optical disks and displayed or printed for operator inspection and comparison with the source documents. This should be performed daily during the first several weeks of system implementation and periodically thereafter. Original documents should not be discarded until reliable, consistent optical disk recording is confirmed and all state standards are met.
INDEX DATA VERIFICATION - Since many retrieval operations involve an exact match of a specified character string, or numeric value, a data entry error involving a single character in an index record can render its associated image unretrievable. Two methods of index verification can be used. Visual proofreading is one method in which the operator visually verifies each index. This method is less costly than the second method, but can result in errors. The second method is known as double keying the index. This method assures index accuracy because two different operators are required to input the same index information. The computer then checks for discrepancies between the two entries and alerts the operator if the indexes do not match. While this method assures an accurate index, it also doubles the time and cost of data entry of index material.
Backup procedures must be established to protect against loss of access to imaged records. State standards permit the use of paper, microfilm, or electronic media (optical or magnetic) to backup the information. If the user chooses to backup images on electronic media, procedures must be established to create security copies of digital records and any software or documentation that is necessary to read the records. For essential records, the backups must be stored in a separate building from where the records are maintained.
IMAGE BACKUP - images are typically stored on optical disk for current retrieval. The user may elect to create a duplicate optical disk or backup the images on magnetic tape.
INDEX BACKUP - indexes are generally stored on magnetic disks for online retrieval. Indexes can be backed up on either optical or magnetic tape.
SOFTWARE AND DOCUMENTATION BACKUP - software will probably be backed up on magnetic media, but can also be stored on optical disk. Documentation may be printed out for a backup.