A few weeks ago I was honored to attend the National Digital Stewardship Alliance meeting. The NDSA was planned by the Library of Congress as part of their NDIIP project.  There were more people there than I expected and it was a humbling experience to hear some of the brightest and most creative brains in Digital Preservation speak.

The high volume of information was overwhelming.  I spent more than six hours at the end of the conference compiling a PowerPoint of the important research highlights.   A small sample of this information is included below.

There were many wonderful presentations giving case studies on how institutions used their own creativity to try and enhance the longevity or migratability (new word?) of their digital files.  The amazing work often was done on a shoestring which though unfortunate, also forced a certain level of imagination and invention.

A few examples are:

Jack Brighton, of campus radio station WILL, gave a wonderful presentation on what a small station is doing to make their civil rights collection more accessible.

Kickstarter.com did a great presentation on how they are helping arts projects get funded and we hope that as they branch into community work that digital preservation might fit into that.

The UK Web Archiving project covered some of the complexities and true effort that it takes to try and tackle capturing the online history of its nation.   http://www.webarchive.org.uk/ukwa/

- As of December 2010 – 9 million sites with .uk, probably 1M more

- 10,027 websites archived

- Need skills in Linux, Java, Hadoop, and SOL

5 keys processes to web archiving

- Selection

- Harvesting

- Storage

- Preservation

- Access

http://www.webarchive.org.uk/ukwa/ngram/

So after taking in all this good information, what is it that I have left the conference with?

People just like us are doing some wonderful problem solving out there.  There is some potential being unlocked, but there is so much to do.

As I see it the Action Items are:

1)    Greater broadcasting of the successful case studies for migration and open solutions.

2)    Training classes in how to boil this down for each type of format/issue.  The NDSA Outreach group held a session called “Digital Preservation in a Box”.  This is the beginning of standardizing the tools that we need.

3)    Overarching education to information and production professionals, as well as, the general public about the dangers of digital fragility and the need for migration (at the least).

I have mentioned to my classes for years that future anthropologists, sociologists and historians will have little to sift through from the late 20^th century.

Some of it is being worked on by archivists now but much is gone.  Let’s keep making progress so that the future of our current history is not lost, like the way of silent films.

More informational tidbits from NDSA:

Other Great Projects

—       ThatCamp.org

—       http://www.scola.org/scola/sampledigitalarchive.aspx

NYPL Labs

—     http://search.creativecommons.org/?q=nypl+map+rectifier&sourceid=Mozilla-search

—     http://menus.nypl.org/

Archiving Facebook

Grad student designed Firefox add-on for individual archiving of Fb.

www.Bit.ly/archivefb

Preserving Virtual Worlds

—      www.ideals.illinois.edu/handle/2142/17097

Cool Tools

—  http://www.google.com/landing/historypin/

—  http://blogs.yu.edu/cpa/2011/02/23/open-source-video-platforms-kaltura-vs-entermedia/

Great Quotes

—    JackBrighton”(DAM) is more like an appliance than an Ecosystem.”

—    Michael Nelson “We need to raise the level of user expectations.”

—    Michael Nelson “In all good computer science functions you solve the problem through indirection.”

—    Wheatley and Frieze “The world does not change one person at a time.  It changes as networks of relationships form among people        who discover they share a common cause and vision of what’s possible.”

—    Tim O’Reilly(?) “Teach preservation as a mindset.  Bake this into the tools.”

New Phrases

—    Social Curation

—    Metadata Ecologists

There has been much written recently about digital fragility. Researchers and archivists have heard dueling longevity and futuristic projections. In trying to push this dire need without appearing like Chicken Little, I have embarked on serious primary research to expose the sheer volume of the problem. The in-depth article will be coming out in a professional journal within the year. Until then, I felt that I needed to speak up a bit about the need for our activism.

Between my teaching digital archiving courses and my work with clients, this issue has been prevalent in each work day. In fact, while re-processing an archives for a client, a case of 5 ¼” floppy discs were found. No one in-house knew exactly if these were of value, what was on them or even if they were created by the organization. When we offered to open them on a computer with a floppy drive, we were told just to throw them out. This is the fear that archivists are living with. Each time an archivist approaches this obsolete media, the questions come: How many others are out there? How many are being thrown out because it is easier? How many are left? How long do I save them? If I am able to find a player/drive/ etc. will I be open the software that the data is formatted in? Will it even be playable? Are we missing decades of human knowledge? How long will this continue? How can archivists slow down the moving train of media change? Can archivists increase re-formatting awareness? Is reformatting my only option? Where does emulation stand? Who do I call? Who do I write? How do I make a difference in this loss that flies in the face of everything my profession holds dear?

A colleague, Tom Featherstone, once told a class “Archivists get paid for throwing things out.” After the horrific silence, he explained that we cannot save everything. Archivists place value on what is received, appraisal occurs and the kernels of importance are retained.

In today’s Digital Age, few are seeing the kernels, the wheat, and perhaps even the farm!

A recent IDC/UC San Diego study estimated that the average American is taking in 34 gigabytes of information per day. As an archivist, let’s think about the volume of data that is being created and disseminated. If even half of one percent were historically significant, we still have a large preservation problem.

The loss of corporate, academic and personal data from the late 1980s to the current time is tragic for future generations of historians, technologists, anthropologists and sociologists. We are nearing twenty-five years with little implementation of preservation processes. This is not to say that Archivists have not offered plans. This is to say that they are not being followed. In pure sales terms, we have NOT “sold” the crisis to the people. This is not to imply that the issue is not real, it means that the dry facts were not enough to convince people of the crisis. More facts had to be gathered. Now a true implementation plan with typical business practices needs to be created.

Here are the options:

1) Do nothing and continue to sweep up after mass dumps of data. Process what passively comes to us. Complain a little (or a lot) and do the best that we can with the little that we receive

2) Be moderately proactive to educate the general public on the loss of human knowledge. Start education workshops at local archives, issue press releases individually and work at the grassroots level to educate your donors and users.

3) Be passionately proactive and begin a coordinated media campaign aimed at the public and the computer industry to work with archivists, historians, sociologists and anthropologists to stop the destruction of electronic records on all media. Work this campaign hand in hand into a reformatting program that is easy. Much like the environmentalists needed to educate consumers (e.g. “Reduce Reuse Recycle”). Catchy phrases work.

One of the biggest complaints leveled against our largest member associations is that they do not get involved in the issues that are most impactful for our day to day work. There is NOTHING bigger to archives than this, right now. Member associations are built on exactly that, their members. We can choose to have a voice.

How each professional decides to act on this data is an individual choice, but a large percentage of archivists and other professionals impacted by this severe and irreparable data loss would be a dominant force in the media, to donors and to the computer industry.

For twenty five years, archivists have been that little chick crying about disaster. It is time we grew up and became the rooster at the farm, crowing for the populace to wake up.

Illustrators are often the least known and most quickly forgotten members of the art world and as such, so many of their artistic creations are lost in our disposable society. Illustrators are the creative forces behind the images on everything from greeting cards to cereal boxes to advertising – including movie posters. All too often, the work of illustrators survives only because of the relationships between dealers who sell the ephemera and the collectors who drive the market. Scholarly information tends to be severely lacking, patchy, or extremely incomplete on movie poster ephemera. Often with collectibles, the best reference guides are written and compiled by collectors.

Ed and Susan Poole have been avid collectors of the movie industry’s “paper accessories” beginning with their first purchase in the mid-1970’s – a poster for one of Susan’s favorite movies: Gidget. Because of their decades long interest in the subject, the Pooles are regarded as experts in their field and have published several favorably reviewed books on movie posters. In addition, they maintain a website, Learn About Movie Posters (LAMP), where interested individuals can learn more about movie posters and buyers and sellers can connect to trade their wares. At the beginning of 2010, the Poole’s announced their commitment to developing an online archive of movie ephemera, Movie Poster Data Base. In early April, according to a posting to the Association of Moving Image Archivists listserv by Ed Poole, they already had developed a robust collection of information in their database including:

• 3,000+ silent studios worldwide
• 15,000+ NSS trailer numbers for identifying unknown trailers
• 25,000+ production codes for identifying unknown stills
• 30,000+ NSS poster and accessory numbers for identifying titles and reissues
• 81,000 poster images online in our archive and cross referenced

Upon visiting both of these sites, their commercial status is apparent due to advertising and promoting sellers, however, there seems to be a strong commitment to compiling accurate information and making it accessible. Seemingly, there is a plethora of information available, but it is not always easy to find. There are several search options on the site. The basic search is the simplest. There are advanced searches with an accompanying manual instructing visitor in their use. Here, one is able to search movies, posters, dealers, and items for sale. It is slightly confusing because all of these features seem to be accessible through the results of a basic search, too. Serious scholars and collectors are encouraged to sign up for specialized members only access, though it does not appear to deliver any more information.

Testing the site with a search for Star Wars movie posters is an excellent indicator of the breadth of information available. After selecting the original release from 1977 search results are returned for more than 40 posters and lobby cards. Selecting one of these items from the list delivers a paragraph about the history, the dimensions, the name of the artist, the number of printings and variances between each printing, and the NSS (National Screen Service) number. There is even cautionary information regarding counterfeits. And, of course, there are links to Poole-approved, reputable dealers selling this particular poster. It is disappointing that the artist’s name is hyperlinked, but not to biographical information. The link leads to an alphabetical list of all the artists on record with LAMP. Clicking on a hyperlinked name from the list delivers all of the posters and paper goods created by a particular artist.

Browsing the site, copyright issues do leap to mind. There are many featured items old enough to be in the public domain; however, our Star Wars example is definitely protected by copyright law and Lucasfilm could be a vigilant enforcer. The Poole’s have a statement of ownership on each entry reading:

This section is for reference use. Images found on this site are property of LAMP and are for reference purposes only with NO rights implied or given. See LAMP Disclaimer.

The LAMP disclaimer is a vague interpretation of Fair Use citing educational purposes, but it does warn patrons to investigate ownership rights before distributing images.

Fortunately, increasing the breadth and scope of the site is in the works. The Poole’s intend to include artist biographies, studio logos and printing companies into their documentation. As to be expected, this is an incredibly laborious task. This endeavor is an important task because it will preserve relatively unknown information on an esoteric, culturally intrinsic medium. The current Movie Poster Data Base is comprehensive and useful and, as it evolves, it will prove even more so. Scholars and collectors may soon be able to locate accurate and comprehensive information on one website, rather than sifting through the internet and searching for out of print books. If the Poole’s would only team up with a librarian/archivist to help streamline the user interface, improve the web graphics and design, and organize the information.

This is a big question for our AMPed Blog. The topic came up in a staff meeting talking about comments we have heard from local archives. When you talk about the costs for archival supplies, HVAC maintenance, staffing, reformatting, yearly examination for any degradation, rotating films, tapes, etc., the budgets can run into the tens of thousands of dollars, easily. Does this add up to one answer, which is pure migration?

The topic also came up roughly on the AMIA listserv early in 2010. Had we as archivists made a horrible error in judgment by focusing on cool storage instead of migration? Though I mentioned that in an earlier blog post, I have to say that the issue has remained on my mind ever since.

When I lecture, I tell my students that I take the responsibility of archiving very seriously. As few as 6% of the silent films ever created still exist. We are the last pass for not only artifact preservation but content preservation. This is akin to the push and pull of access copies and long-term preservation copies.

Can we afford to save everything? No. That has historically been accepted in the profession. Maybe there is a new question now, which is can we afford to save the originals?

Before you think I have jumped overboard to pure technology and left my archival skills behind, look at your budget. Where are your numbers headed? Where do you expect your budget money to be focused in 2011? 2012? Look at what grant money is out there. Is the money trail focused on preservation or digitization? Look again at your originals and think, how many of them need to be preserved as an artifact?

If I can find money to migrate more of those formats that do not have an artifactual value, isn’t that more ethical than keeping it in storage with bare access and a bleak future? As the AMIA listserv discussion mentioned, if we don’t migrate it now, there may be no way in the future to do so. Now or never has never been more real to me.

With budgets tight, can we continue to afford to optimally preserve the originals, migrate periodically and preserve the transitory /digital copies? That is a tall order and one that some archivists do not want to openly discuss.

I am not pushing for throwing originals out, as the original artifacts have historical value, but does a ¾” videotape have the artifactual value of a Victorian diary? Is content migration more important than spending a lot of time on a fragile and low quality original? Some examples of every media need to be saved as artifacts but does each one need the effort that other more important original formats deserve? For instance, if you have a script copy with original notes from a director or a master copy on film or a master ¾” videotape or several b-roll copies with no markings are they all equal? I would argue that those with a historic fingerprint need all the best archival tools for long-term preservation but some of the multimedia archive originals like 5 1/4” floppies and ¾” videotape might be great candidates for content migration and less appropriate for long-term storage.

The other thing that we struggle with is that many of the A/V originals just will not hold over time. The tapes will not make it to their 100th year like black and white film or photographic prints. Let’s face it, we all age. Even if you put me in cold storage, eventually my organs and joints will fail! So the best that you can do is a genetic clone one day, the worst is take an oral history in current technology and plan for the migration of my brain’s content. I am okay with that. Migration of oral histories allow for the “living history” for generations.

I can not make tape last four generations and I certainly can not foresee how to assure operational machines in twenty years, forty years and even ten for some. So the choice to make a stellar digital copy, is not really a choice but a necessity.

I still think that cold storage for the originals is the other side of the issue, but I wonder if our professional reality now, is that the originals are not going to be as big a priority for A/V archivists as migration is.

This is completely counter to what I was taught in graduate school and what I have practiced my entire professional career, but at my core I am a realist. We have to look at where the money comes from and when our equipment, copy media, and original formats will fail.

Can we focus on the content migration for certain formats and still be good practicing archivists? Is this giving up? Should we be fighting harder to change how funding comes to us? Should we work harder as a coalition with manufacturers? Do we even have the money en masse’ to make it financially sound to push for equipment/format stability? Do we need to be realists and move with the technology of the time?

Maybe a list is needed of the media that even if original, have a transitory nature and little intrinsic value. The list might be easier than we think as the two formats that I listed above were treated as transport medium from the beginning, whereas other formats such as film were meant to be the original and were treated better as far as description and care.

I struggle with this and fear the ensuing conversation but I also fear not having this conversation.

Interesting related sites on digital preservation:
Long-term preservation costs – some figures
Digital Preservation Cost Centers Digital Preservation Committee
Comparison of Methods & Costs of Digital Preservation
Keeping research data safe (Phase 1)
When will digital preservation come to an end?
Why Digital Preservation is Important for Everyone
Preservation Assessment and Planning
CARLI Preservation Working Group – Webliography

In a national economy in which double-digit unemployment figures travel in the opposite direction from the signs that things are said to be improving and a local economy that would jangle the nerves of the most fiercely optimistic, I am frequently riddled with doubt as to my decision to return to school for the academic qualifications to do something I truly enjoy. It was easy to buy into the notion of getting paid for doing a job for the sheer happiness it brings. I didn’t anticipate that timing is everything and having the know-how, energy and desire to take on an all new career was only half the battle.

Recently, while enjoying the company of my three granddaughters during their weeklong break from school, I was reminded why I decided to enter library school and select archival administration as my area of study. As usual, the eldest of the girls was shadowing me, eager to help with any little thing I asked her to do. I sent her to the spare room where she was to take a good long look and devise a plan to organize the boxes and boxes of stuff stacked among the cats’ toys, feeding dishes and litter boxes. After a few minutes she returned with her report. “There’s a lot of stuff in there, she said, but I found a box full of CDs.” I immediately wracked my brain in an attempt to remember if I had been remiss in the clutter-reducing organization of the CDs and DVDs I had painstakingly organized into albums.

Anticipating my next request, my granddaughter left the room and quickly returned holding an old milk crate filled with individually boxed audio cassettes. One glance at the plastic crate and I was instantly reminded of the reason I had upset the status quo by leaving behind a reasonably well-paying job and switching my academic focus to continue on in graduate school. “Those aren’t CDs, I said. “They are cassette tapes recorded by your great-grandfather, and they are in need of processing, along with the rest of his collection.” If at that moment my granddaughter didn’t quite understand what I meant, by the time she returned home at the end of the week, not only had she become somewhat acquainted with the great-grandfather who died shortly before she made it to 3 years old, she attained a basic understanding of the important role archivists play in the preservation of history and culture, and she began to comprehend why her Nana always gets a little misty whenever we go through the dozens of boxes, albums and other containers filled with photographs of people she will never actually meet.

To quote my esteemed mentor, “as archivists we wear multiple hats.” Our professional responsibilities require us to step away from the photographic and/or moving images in our personal collections and devote our time and energies to the clients and organizations from which we endeavor to earn a living. When we are presented with a collection, we delight in the discovery of the information contained therein. Our sense of history is sharpened (or dulled) when set upon a path to learn something new and/or review, organize and disseminate what we already knew. As we focus on these bread and butter tasks, the records of our personal histories all too often get short shrift, and containers filled with items in need of organization and care are left to languish and suffer the deterioration that results from the necessity of putting off until later the donning of the hat we wear as preservers of our family histories.

For most of his life, my father was a musician. He played music, wrote music, breathed ate and drank music – all the time. Although he earned a living as a civil servant, he never strayed far from his first love. He had been a music teacher and a choir director, an actor and a playwright, a lay minister, videographer, photographer and a broadcaster. He had also performed in prestigious productions and with some of the most legendary names in the business. But what my father really, really loved to do was sing, and boy could he sing! My earliest and best memories include the velvety sound of Dad’s rich baritone voice singing from one of the many American musicals he counted among his favorites. Contained among the scores of audio cassettes my granddaughter recovered from the spare room closet is a series of lecture concerts on the history of the American musical theater from 1866 to 1970, Dad’s recreation of his doctoral dissertation. Imagine my surprise when making this discovery. To say that the processing of my father’s collection will be both an arduous task and a labor of love seems inadequate upon the realization that Dad’s audiocassettes, photographs and sheet music very likely hold a cultural, historical, academic and possibly commercial value beyond family sentiment and the enormous pleasure I receive from hearing that rich, velvety baritone waft from the speakers of my stereo.

As I embark upon the journey represented by processing my father’s collection – a journey which will doubtless be filled with twists, turns and bumps in the road – I ask myself one question: Which hat do I wear now?

In teaching multimedia archive,s I think about the future of our content constantly. Like a new mother, I fret for its security, growth and health. What is THE answer for our degrading media, emulsions, for our software obsolescence and our equipment falling down around our ears?

Recent discussions on the AMIA listserv brought new energy to this discussion and I wanted to put my spin on this. The subject line was “What’s Not Cool About Cold?” and it solicited some serious discussion about whether we have made a horrible mistake for a generation of archivists and content.

Jim Lindner argues that the imminent demise of tape players is more important in an archivist’s preservation decision than our focus on the imminent degradation of the media itself. The latter being our big decision to place much of our media in cool or cold storage. The group discussion mentioned the fact that many of our players are no longer supported by their manufacturers and the simple math that the lack of machines and the existing wear on their parts will not even cover the playback of the volume of archival tapes awaiting…migration? This hits a deep reality. Have we lost hundreds of thousands of hours of archival motion under our watch? Maybe even millions or billions of hours?

With the current demand from patrons increasing and the desire to use motion outtakes, news, etc. for historic documentation this is a disappointing state. Motion is powerful, emotional and immediate. It puts patrons in the experience in a way that to the visual human being is dramatic.

This important discussion on the listserv was a strong exchange which took place over several days. Other important points included the argument that the research on degradation and proper handling has made the tape degradation less of an issue and now the issue is playback for migration and access.

Other points included:

1) The need to get these gems a proper venue to be seen (whether due to money or rights issues)

2) The natural evolution of gathering obsolete machines from production facilities and creating some centralized site for archival institutions to use.

3) How to custom make parts so that we can maintain the machines so that we can transfer the content.

4) Wait for a miracle technological innovation to happen in the next 20-30 years so that you can migrate. (Keep them cool until then).

We come back again and again to the complexities of managing archival multimedia. The next generation of archivists will have changed their paradigm and released themselves from the desire to preserve the artifact. I am not condoning refusing to preserve all originals but our focus as far as video tape and digital files will be on content migration.

Other formats still have certain inherent value and that is a different blog posting!

Future archivists will have a much better long-term understanding of what is an artifact and what is intellectual content that needs to be migrated. They will be more adept to the rapid pace of format change and will HAVE to adapt quickly.

Not that I am faulting us. We are the transitory archival generation, the one that bridges the 100 year film format and the thumb drive.

I agree that the great research done in the field has helped us to minimize that problem, but it does seem that we took the eye off the ball a bit on migration. We cite lack of funds, lack of understanding of the urgency by the non-archival world, the frustration with equipment manufacturers, etc. It sounds like we need a development guru to raise funds, awareness and help to join all the key players for a collaboration. Many mention that the creators often do not prioritize preservation as they should. That is true, but our role as professionals is to educate.

Hand-wringing is not allowed! The energy that we have spent on that could have been put forth to an international collaboration with a real potential for migration, managing equipment, and innovation. One manufacturer can not do this alone, everyone needs to get on board and when I say “everyone” I mean:

Archivists
Creators (producers, studios, channels, directors, talent, writers, etc.)
Media
Equipment and Media Manufacturers
Technologists
Professional Associations
Funders

I think all of the above agree that the loss is imminent. The question is can we use our collective economy of scale to work together? Or maybe it is will we?