This post is partly the sixth recap in our multi-part series of the 2013 NAGARA E-Records Forum (presentations available here) and partly a recap of the ARMA Houston 2013 Conference & Expo (presentations available here).

One recurring theme emerged from two conferences we recently attended: there’s no single out-of-the-box solution that will handle all aspects of electronic records management and digital preservation.

Having said that, we recently heard three different case studies of organizations using cloud services in innovative ways to meet various requirements of ARMA’s Generally Accepted Recordkeeping Principles (“The Principles,” formerly “GARP”) and ISO 16363 (the Trusted Digital Repository standards we recently wrote about).

Using “The Principles” to Select a Cloud Provider

Thoughts on “The Principles: A Needs Assessment Case Study — Selecting A Cloud Provider,” John Isaza, ARMA Houston 2013 Conference by Marianna Symeonides.

Cloud computing is here to stay, and there is no denying the enormous effect it will have on RIM programs. John Isaza, a lawyer with the Rimon PC law firm based in California, has helped many organizations, including Fortune 100 companies, develop information governance and records management programs. He gave a presentation about helping the legal department of a large non-profit organization select a cloud provider that would adhere to ARMA’s Generally Accepted Recordkeeping Principles. (These principles are Accountability, Integrity, Protection, Compliance, Availability, Retention, Disposition, and Transparency.)

Isaza and the client embarked on a needs assessment to discover the maturity of the client’s RIM program, which would help them identify weaknesses. For the assessment process, Isaza used ARMA’s Principles Assessment Tool. (At around 100 questions, this is an incredibly thorough document that is very time consuming to read and digest – distribute only to key personnel!) Isaza then conducted 1-1/2 hour interviews with people who would be invested in the process of implementing a new cloud solution and who represented a cross section of the entire organization–HR, marketing, legal, payroll, and other departments. The results were tabulated and the team made recommendations. In all, the assessment and interviews took about three or four weeks.

Findings

Following the assessment, Isaza and the team identified the particular strengths and weaknesses in the organization’s current practices. We’re sure our readers can relate to a few of these points!

The biggest challenges that the client and Isaza faced in implementing a RIM program included:

• Outdated organization-wide retention schedules were forced on the client department
• Volunteers, who were spread out all over the world, were not networked. They would provide information and documents from their own computers and flash drives were given away “like candy.”
• There was no remote access option available
• Several shared drives
• No protocols in the event of a legal hold
• Bare bones budget

On the bright side, Isaza and the client had several factors in their favor:

• Support of CIO and IT
• General buy-in of the Principles-based needs assessment results
• No immediate heavy litigation to stall the program
• The client implemented the #1 recommendation: They hired a records manager. He did not have any formal training in records management, but he was willing to learn.

Turning the Findings into an RFP

Isaza and the client then drafted a RFP (Request for Proposal) for a cloud-based records management program. Some of the functionalities that the client requested are:

• Easy-to-use interface for non-technical users and advanced functionality for sophisticated users
• Scalable as additional departments and employees would become users and as the quantity of stored files increases
• Files are stored in their native format
• High availability of files
• 24/7 support
• Hard copy documents can be loaded into the system using multifunction copier-scanners
• All file types that possess searchable text needed to be searchable, including documents within zip files

The RFP was submitted to ten vendors, and the client received 5 responses. Isaza noted that ultimately, the decision about which vendor to choose was probably quite subjective, and that sales techniques can have a big effect. However, he warned that one should not be “too charmed,” and should “check under the hood,” so to speak.

Experience with the Cloud Vendor and Lessons Learned

The client has been using the selected vendor for over a year now, and unfortunately there are some pretty huge weaknesses with the service. For cloud-based projects in general, records management functionality is often added on as an afterthought. This particular product, for example, lacked a way to tag each record with a record series number from the schedule, a creation date, or other key words to use for searches.

Isaza shared the lessons that he learned from this experience, which included:

• A Principles-based needs assessment is key
• Not a single vendor was DoD certified
• A records manager trying to implement these programs should insist on reviewing the vendor contract. There are many traps in the contract for a novice. Keep in mind that the client in this particular case was the legal department – but they were not records management experts. Isaza illustrated this with the example of an organization that wanted to extract their data from an ediscovery company – but according to the contract, it would have cost $15 million.

Overall, there were some successes – but there were also five slides’ worth of shortcomings. For example, that 24/7 support that the client wanted so badly? It was only live people from 9 AM-3 PM New Zealand time – the rest of the day, it was email support that promised a response within 24 hours. The good news is that Isaza has been able to take those lessons with him as he helps other organizations with their own assessments and and selections of cloud providers.

Using ISO 16363 to Select a Cloud Provider

North Carolina and DuraCloud

Thoughts on “Archives and the 3rd Party Cloud,” Kelly Eubank, NAGARA e-Records Forum 2013 by Angela Ossar.

Kelly Eubank (Digital Services Branch, North Carolina Department of Archives and History) talked about using DuraCloud for backup storage of their archival digital materials at the NAGARA e-Records Forum 2013.

The archives has 142 TB of information in local storage at their headquarters in Raleigh, 90 TB in offsite storage in Asheville (4 hours away; they get the data there by putting it on hard drives and shipping it), and is now using DuraCloud to back up an additional 25 TB of material. DuraCloud serves basically the same function as their Asheville storage — an off-site, far-away place to store copies, and little else.

How many items are they storing in the cloud? Oh, just 1.4 million. The files comprise digitized and born-digital materials, government records and also special collections like maps, video, audio, email, and, soon, their archived web pages (archived using the Archive-It tool) and possibly Facebook pages. Their main concern is their born-digital material, though, which they consider more important since there’s no paper backup for those materials.

The content is uploaded to DuraCloud in BagIt “bags.”  BagIt is a Library of Congress file transfer tool that North Carolina uses for authentication and verification purposes. A “bag” is basically just a container for digital files, like an envelope is for paper documents.

Using DuraCloud for their backups helps them meet the Archival Storage function of the OAIS model (ISO 16363). Third party storage is only one small piece of their preservation strategy, but it’s a vital part.

Kentucky and Preservica

Thoughts on “Digital Preservation Comes of Age: Reports from the Field,” Mark Myers, NAGARA e-Records Forum 2013 by Angela Ossar.

Kentucky is a forerunner of state digital preservation programs. They were the only government archives who participated in the (pretty terrifying-sounding) ISO 16363 test audit, for example, and seem (to me anyway) to be on track to becoming the first state archives that is truly “ISO 16363 compliant.”

Kentucky purchased the Preservica system in 2011 to ingest digital materials. To an outsider, Preservica seems to be much like DuraCloud, the system North Carolina uses. Both use Amazon S3 for storage. And both, unsurprisingly, were not designed specifically for use by state archives — so a lot of adjustments have had to be made for the programs to work.

Kentucky uses Preservica for its “dark archive” — it’s not intended to be a public interface, for now. It does more than store materials, though. It can also perform normalization, the process of converting digital files to more “archival” formats. For example, when Attorney General opinions are accessioned as Microsoft Word Documents (.doc), Preservica converts them to PDF/A.  The Word document is saved in the system as the original, but the preservation copy is the PDF/A.

When a user clicks on a single file, s/he would see the metadata for that file: file name, unique ID, and accession metadata that is added by archives staff. Preservica also performs a number of “microservices” on each file when the file is ingested: it scans it for viruses, performs a fixity check, and creates a thumbnail to facilitate browsing. The system collects metadata about each file that enables staff to generate a report of all the file formats (including version numbers and frequency) in the archives.

What’s more, the system’s migration tool validates all file formats against the PRONOM registry, a file format registry developed by the British National Archives.

Kentucky’s success with Preservica suggests that the software might be adaptable to other non-gigantic entities, like state and local government digital archives.

And in closing…

It’s exciting to see organizations testing out cloud services in new and innovative ways. More cautious, less tech-forward organizations can learn from these early adopters’ experiences, whether generally good (Kentucky, North Carolina) or not-so-good (the nonprofit). Now that we have clear-cut reference models — The Principles for records management, ISO 16363 for digital archives — it’s great to see government and non-profit organizations working to figure out how use emerging technology to apply those models to their own programs. We have a lot to learn from these innovators!