A few weeks ago I was honored to attend the National Digital Stewardship Alliance meeting. The NDSA was planned by the Library of Congress as part of their NDIIP project.  There were more people there than I expected and it was a humbling experience to hear some of the brightest and most creative brains in Digital Preservation speak.

The high volume of information was overwhelming.  I spent more than six hours at the end of the conference compiling a PowerPoint of the important research highlights.   A small sample of this information is included below.

There were many wonderful presentations giving case studies on how institutions used their own creativity to try and enhance the longevity or migratability (new word?) of their digital files.  The amazing work often was done on a shoestring which though unfortunate, also forced a certain level of imagination and invention.

A few examples are:

Jack Brighton, of campus radio station WILL, gave a wonderful presentation on what a small station is doing to make their civil rights collection more accessible.

Kickstarter.com did a great presentation on how they are helping arts projects get funded and we hope that as they branch into community work that digital preservation might fit into that.

The UK Web Archiving project covered some of the complexities and true effort that it takes to try and tackle capturing the online history of its nation.   http://www.webarchive.org.uk/ukwa/

- As of December 2010 – 9 million sites with .uk, probably 1M more

- 10,027 websites archived

- Need skills in Linux, Java, Hadoop, and SOL

5 keys processes to web archiving

- Selection

- Harvesting

- Storage

- Preservation

- Access

http://www.webarchive.org.uk/ukwa/ngram/

So after taking in all this good information, what is it that I have left the conference with?

People just like us are doing some wonderful problem solving out there.  There is some potential being unlocked, but there is so much to do.

As I see it the Action Items are:

1)    Greater broadcasting of the successful case studies for migration and open solutions.

2)    Training classes in how to boil this down for each type of format/issue.  The NDSA Outreach group held a session called “Digital Preservation in a Box”.  This is the beginning of standardizing the tools that we need.

3)    Overarching education to information and production professionals, as well as, the general public about the dangers of digital fragility and the need for migration (at the least).

I have mentioned to my classes for years that future anthropologists, sociologists and historians will have little to sift through from the late 20^th century.

Some of it is being worked on by archivists now but much is gone.  Let’s keep making progress so that the future of our current history is not lost, like the way of silent films.

More informational tidbits from NDSA:

Other Great Projects

—       ThatCamp.org

—       http://www.scola.org/scola/sampledigitalarchive.aspx

NYPL Labs

—     http://search.creativecommons.org/?q=nypl+map+rectifier&sourceid=Mozilla-search

—     http://menus.nypl.org/

Archiving Facebook

Grad student designed Firefox add-on for individual archiving of Fb.

www.Bit.ly/archivefb

Preserving Virtual Worlds

—      www.ideals.illinois.edu/handle/2142/17097

Cool Tools

—  http://www.google.com/landing/historypin/

—  http://blogs.yu.edu/cpa/2011/02/23/open-source-video-platforms-kaltura-vs-entermedia/

Great Quotes

—    JackBrighton”(DAM) is more like an appliance than an Ecosystem.”

—    Michael Nelson “We need to raise the level of user expectations.”

—    Michael Nelson “In all good computer science functions you solve the problem through indirection.”

—    Wheatley and Frieze “The world does not change one person at a time.  It changes as networks of relationships form among people        who discover they share a common cause and vision of what’s possible.”

—    Tim O’Reilly(?) “Teach preservation as a mindset.  Bake this into the tools.”

New Phrases

—    Social Curation

—    Metadata Ecologists

A recent question posted on the AMP blog, “Is preservation cost-prohibitive?,” made me think about costs related to archives in general. As a former corporate archivist, I am painfully aware of budgets and bidding out work! Now that my shoe is on the other foot, and I am consulting in the field, the issue is even larger for me. When I was the “client” and was requesting bids for work, I (and my coworkers) were always concerned by the process. We knew many vendors were underbidding to get the work and that could pose a financial risk for them if they got the project. We also were forced to consider those bids because there was quite a bit of paperwork to do if the lowest bid was not selected. In the end, we could often work around it by choosing the firm with the most expertise in an area as long as their bid was not too much higher than the lowest one.

It’s easy for a client to forget about the hidden costs of operations related to projects. There are often random emails with questions, monthly or more frequent conference calls, technology testing or review, on-site meetings or visits, etc. All of these items take up staff time – and not just a little bit of it either – it really adds up. I think many clients might be shocked if they realized exactly how much time. Often a fair amount of this time can be billed back as project management time, but only if the client is willing or that category has been built into the project.

I recently took on some work that was priced out on a per-item basis. This method really made the most sense since there was a huge quantity of digital files to review. As any vendor would, I priced the project as low as I could while still hoping to cover any time required for emails, phone calls, meetings, technology issues, etc. Luckily for me, so far my pricing seems to be working for me and for them.

The project brought up another interesting financial concern though. The quantity of items is so high and the quality isn’t always great, that the client and I started to wonder if there was a better way to upload fewer items so as not to flood their database with useless information. I think I have found a workable solution, but the main issue with trying to cull out the bad items was the time required to do it and the cost associated with that. It’s a pure and simple appraisal issue along the lines of “More Product, Less Process,” although this time it’s “More Process, Less Product!” It’s been a unique budgetary issue for me and my client in that it could easily be cheaper for them to add thousands of images to a database rather than to review the images and pick the best ones – or more importantly, to delete the bad ones.

In the end, I employed some digital photo software that allows quick-ish review and refoldering of files so that I can at least remove the “worst of” images for the client. The system isn’t perfect, but in the end, I needed to present a plan to the client that would save them money. It didn’t make sense to have the project cost more money to give them fewer images even when fewer images was more desirable. Now the project is a mix of hourly review time and per item upload time (engineered never to exceed what the total upload time would have cost if no review was done), but the client is left with better quality images and a slightly cheaper overall cost.

Back to the question at hand: Is preservation cost-prohibitive? I guess it certainly can be – as can any archival procedure, but in the end, that’s what Archives are for – preserving and making available items of historical importance. Let’s just hope that all the holders of the purse strings continue to agree!

I spent the last week writing and editing a book chapter on process management for digitization. My head has been trying to process all that I contemplated while doing such an intensive session.

I think that a lot of process management comes down to skills that we often no longer practice. These oldies but goodies are classics that we need some reminding about. The most important is…LISTENING.

We Twitter, we Facebook and we Blog but we are not necessarily listening to each other. Engaging colleagues in the process of digitization or any other information management process makes a huge difference in creating efficiency. More brains are always better than one!

The next skill is one that many of us are skilled at but do not have time for: analysis.

There was a great quote that I read ages ago “If you want to live forever, write something worth reading.” That has stuck in my head and inspired me to continue to write personally. The reason that quote is so powerful is because of its qualification “worth reading.”

How much of the information explosion is worth reading?

One aspect of self-publishing that is great is that anyone can post any wonderful idea for all to see, but it is also the great disadvantage. Who has time to qualify all that is out there? Librarians would love to direct patrons to well-researched blogs, but who has the time to research their accuracy? We fall back to well-known publishers, many of whom have had their own authenticity problems.

We have sites where Librarians assess other sites and compile them so that we can be assured that we have access to high quality tools. In my latest nights, I wonder…how much checking can we really do to ascertain quality? In this world of self-promotion and personal status sites, are we really examining the information? Are we slowing down enough to get this?

Lastly, is the skill of planning.

Many times librarians and archivists (particularly of the solo version) spend so much time feeling overwhelmed that we only react instead of proactively changing our state. After reading hundreds of interviews that my students have done with leaders in the field I see one common thread between success and failure…planning. So that goes back to being proactive, taking a breath and listening to what it is your users and your collection needs.

Listening to the content and the users means that you have a better plan. Maybe it is a stolen hour a week to consider this and shut out the information noise, disseminate the good research and plan how to get where you need to be. Ignoring for just an hour the budget pressures, lack of staff skills, and bureaucratic realities allows you to envision where you can be and as many great business leaders espouse, visualizing your success plays a big part in getting to it.

So in the words of the great Frasier Crane, “I’m listening.”

It is quite common for those who work within the library and archival professions to get caught up in the never ending twists and turns of technology. This particularly happens to those who are new to the profession. For instance, I can remember quite fondly of my graduate work at Wayne State University where I was absolutely convinced that librarians and archivists must not only understand programming, but they must also strive to do as much of their own programming as possible. Naivety is a wonderful thing, especially when you awake from it.

After more than ten years of professional experience, I’ve learned a great deal about librarianship as well as technology; moreover, these lessons have open my eyes to the various ways to approach the challenges of organizing, managing and disseminating content. In addition, it goes without saying, during this time technology has quickly evolved and changed in new and unexpected ways. While I’m no longer such a staunch proponent of librarians and archivists as programmers, I still hold on to a strong notion that librarians and archivists must become intimate with the tools within which they use. But this is nothing new, for librarians and archivists have always had a strong relationship with the tools and materials with which we’ve used. Not surprisingly the change has come from the types of tools and ways in which we use them.

This perspective can best be illustrated in my changing view of librarianship and programming. Over the past 10 years we’ve witnessed an explosion of new tools which are able to leverage the ubiquitous nature of the web. We now have at our finger tips scripting languages, JAVA, XHTML, XML, along with improved tools for the creation of relational databases, MS Access, File Maker, etc. These tools all allow us different means of presenting, organizing and delivering materials. No longer are we restricted to the expertise of others. Now, today, librarians and archivists have the ability of not only learning, but mastering tools, which build on our professional training. In other words, our understanding of classification and its principles provide us with the intellectual skill set useful in the designing and development of web accessible collections.

Moreover, our training as librarians and archivists provides us with something else when we come to the table of problem solving. Our training and skill set allows us to not only understand the tool, but allows us the ability to think through the problem. Rather then relying on the programmer or the skills found in one set of tools, we think of how to achieve the goals. That is, if we have to migrate data from one system to the next then we contemplate how best to do it, while maintaining the data integrity and efficiency of our efforts. We understand that we’ll have to sacrifice one, usually the efficiency of our efforts, for another, the integrity of the data.

No, I was wrong as a budding librarian, we didn’t need to become programmers. Rather we needed to build on our intellectual skills as librarians and archivists and wait for the tools to come to us.