In writing, research, lecturing and just plain old working, I see over and over again the need for us not just to educate ourselves about technology, and project management but to remind ourselves that this is “not your Grandfather’s Archive” (with apologies to Oldsmobile aficionados).

In some ways the archives world that I was trained in was more of a warehouse atmosphere. Preservation and scholarship were the goals….and if you get a chance… do a little promotion.

That is a far cry from the reality that television documentaries and technology tools have created today. The archive is the star of more than a few television shows now and the archivist or librarian is the key to the new economy…the world of information.

As gate-keepers of the past, we took that job so seriously (maybe a little too seriously) that we often forgot to have fun with our collections and more importantly our users.

I am happy to report that fun is back in the archives! These technology tools have empowered archivists to be creative with promotion, crowd-sourcing and documentaries.

Many archives realize that the power of these new (often open source) tools have allowed us to creatively reach the masses and show the world how interesting their history is. For many years I have said that “History is not boring, history books are boring!”

The ability to show someone their great-grandmother screaming at a WWI protest march from halfway around the world, or transcribing the price of strawberries in NYC in 1915 (as in NYPL -What’s on the Menu Project) or developing a coalition of the great-grandchildren of a WWII regiment is not connecting people less to history, in fact it is connecting them more to history and to their global cousins with shared interests.

An archivist is supposed to be a good citizen of their community. It is what I and generations of archival professors have taught their students. You must be involved in your community. Today with a global presence potential in every archive, our global audience yearns for connection to our forefathers, our heritage, our hobbies and each other.

This is a really exciting time in the profession but it means some loss of control. Cutter numbers and authority files are going to take a hit in order to engage our communities to our collections. It does not mean that we will not continue to quality check our records but it means more time might be spent on those communication connections with our users. Certainly more time will be spent evaluating community tags than we will spend re-vamping the controlled vocabulary. Some of you might think it is better, some worse, but even in my deepest control freak heart of hearts, I am really enjoying the sense of community that these tools are creating.

Archives are a true part of this global community. Welcome neighbors!

How did one institution attract 50,000-plus volunteers to help with an archiving project?

The National Library of Finland is in the process of digitizing its archives so that they are fully searchable on the Internet. Scanning the centuries-old newspapers, journals, and documents isn’t so much the problem as is accurately transcribing the text. OCR (Optical Character Reading) software can only do so much. Standard fonts are easy enough for a computer to identify, but aging print in fancy scripts are more difficult. Obscure language, proper names, and decaying paper also interfere with OCR’s text recognition. In order for the materials to be accurately digitized, every document must then be double-checked by human eyes.

To help with the process, The National Library of Finland teamed up with Finnish technology company Microtask to come up with an innovative solution: make a game of it. Granted, it’s hard to imagine how anything like checking manuscript text against a computer’s digital interpretation could really be fun. But Microtask saw things differently—instead of pages of repetitive work, they broke down each individual word-check into what they (appropriately) call microtasks.

Taking each microtask as a tiny action to perform, two online flash games were created.

In Mole Bridge is a fast-paced typing challenge. An image of a digitized word from the original manuscript appears at the top of the screen. The player must type the word, as best he or she can read it, as quickly as possible. The typed word helps build a “bridge” that ensures the safety of some adorably cute, but evidently suicidal moles, who tirelessly try to cross the void. The quicker the words are entered, the more moles make it to safety. Words that are indecipherable—or too difficult to type (American keyboards lack the å, ö, and ä symbols common in Finnish) — can be skipped without penalty.

In Mole Hunt, cute critters hold up signs with individual manuscript words and the computer’s interpretation of each word below. Players identify whether or not the computer’s recognition of the word is accurate.

In terms of fun, the games are more on par with Mavis Beacon than Super Mario. But like many other online distractions, there’s something strangely addicting about them. Knowing that the games actually contribute to a worthwhile cause —preserving the cultural heritage of Finland — is evidently enough to keep people playing. Since the Mole Bridge and Mole Hunt were launched in February, they’ve logged over 50,000 players, hailing mostly from Finland, the UK, and the US. At last count, the grand total of work contributed to these games is over 3,500 hours, with 4.5 million microtasks achieved.

The real brilliance of these games is that not only do they get the work done, they get it done right. With thousands of people playing, gamers effectively cross-check one another’s work. The software developers at Microtask have even created a defense against the few people who might have malicious intent (apparently, even Finnish libraries occasionally encounter mean-spirited Internet trolls).

When the game starts, the computer shows words that have already been accurately identified. It can then determine if someone is making deliberate mistakes or is just too poor of a typist to benefit the project, and then disregards every further answer submitted by the player. Microtask reported that volunteers’ transcriptions were 99% accurate.

While the games delegate hours of work to the hands of others, they are in no way a substitute for trained professionals. No matter how many microtasks are achieved, the games only cover a fragment of the digitization process. Archivists, librarians, and other specialists are still necessary to develop and implement every step of digitization projects. Accurately digitizing a nation’s archives is an enormous task. For starters, it would require:

• Different classification schemes for different types of materials
• Categorization by subject matter
• Indexing every document
• Entering metadata on every file
• Secure storage space with prevention against digital decay
• A user-friendly interface that allows for effective browsing and searching

And that’s in addition to a plethora of other issues as well. How well does centuries-old newsprint fare in a scanner? What do you do with materials still under copyright? How do you anticipate the needs of both serious researchers and more casual users, such as genealogists?

Implementing gaming systems is not a quick solution to digitization projects, but many institutions could benefit from The Finland National Library‘s model. Introducing similar programs would not only help the workload behind large-scale digitization projects, it would (perhaps most importantly) establish a connection between the public and their libraries. It would raise awareness to libraries’ missions and likely cause more people to support such institutions. Preserving cultural heritage should not just be the work of archivists, and now, it doesn’t have to be.

Play Mole Bridge and Mole Hunt at http://www.digitalkoot.fi/en

Further reading:
Gamification time: What if everything were just a game?
The secrets of Digitalkoot: Lessons learned crowdsourcing data entry to 50,000 people (for free)
Digitalkoot e-programme breaks 25,000 participant mark

Please note that the images are used with permission from Microtask. Several attempts were made to receive written permission from the National Library of Finland, but there was no response. We assume that in the interest of education, this is fair use.

A few weeks ago I was honored to attend the National Digital Stewardship Alliance meeting. The NDSA was planned by the Library of Congress as part of their NDIIP project.  There were more people there than I expected and it was a humbling experience to hear some of the brightest and most creative brains in Digital Preservation speak.

The high volume of information was overwhelming.  I spent more than six hours at the end of the conference compiling a PowerPoint of the important research highlights.   A small sample of this information is included below.

There were many wonderful presentations giving case studies on how institutions used their own creativity to try and enhance the longevity or migratability (new word?) of their digital files.  The amazing work often was done on a shoestring which though unfortunate, also forced a certain level of imagination and invention.

A few examples are:

Jack Brighton, of campus radio station WILL, gave a wonderful presentation on what a small station is doing to make their civil rights collection more accessible.

Kickstarter.com did a great presentation on how they are helping arts projects get funded and we hope that as they branch into community work that digital preservation might fit into that.

The UK Web Archiving project covered some of the complexities and true effort that it takes to try and tackle capturing the online history of its nation.   http://www.webarchive.org.uk/ukwa/

- As of December 2010 – 9 million sites with .uk, probably 1M more

- 10,027 websites archived

- Need skills in Linux, Java, Hadoop, and SOL

5 keys processes to web archiving

- Selection

- Harvesting

- Storage

- Preservation

- Access

http://www.webarchive.org.uk/ukwa/ngram/

So after taking in all this good information, what is it that I have left the conference with?

People just like us are doing some wonderful problem solving out there.  There is some potential being unlocked, but there is so much to do.

As I see it the Action Items are:

1)    Greater broadcasting of the successful case studies for migration and open solutions.

2)    Training classes in how to boil this down for each type of format/issue.  The NDSA Outreach group held a session called “Digital Preservation in a Box”.  This is the beginning of standardizing the tools that we need.

3)    Overarching education to information and production professionals, as well as, the general public about the dangers of digital fragility and the need for migration (at the least).

I have mentioned to my classes for years that future anthropologists, sociologists and historians will have little to sift through from the late 20^th century.

Some of it is being worked on by archivists now but much is gone.  Let’s keep making progress so that the future of our current history is not lost, like the way of silent films.

More informational tidbits from NDSA:

Other Great Projects

—       ThatCamp.org

—       http://www.scola.org/scola/sampledigitalarchive.aspx

NYPL Labs

—     http://search.creativecommons.org/?q=nypl+map+rectifier&sourceid=Mozilla-search

—     http://menus.nypl.org/

Archiving Facebook

Grad student designed Firefox add-on for individual archiving of Fb.

www.Bit.ly/archivefb

Preserving Virtual Worlds

—      www.ideals.illinois.edu/handle/2142/17097

Cool Tools

—  http://www.google.com/landing/historypin/

—  http://blogs.yu.edu/cpa/2011/02/23/open-source-video-platforms-kaltura-vs-entermedia/

Great Quotes

—    JackBrighton”(DAM) is more like an appliance than an Ecosystem.”

—    Michael Nelson “We need to raise the level of user expectations.”

—    Michael Nelson “In all good computer science functions you solve the problem through indirection.”

—    Wheatley and Frieze “The world does not change one person at a time.  It changes as networks of relationships form among people        who discover they share a common cause and vision of what’s possible.”

—    Tim O’Reilly(?) “Teach preservation as a mindset.  Bake this into the tools.”

New Phrases

—    Social Curation

—    Metadata Ecologists

In years of teaching visual indexing and being called in to create metadata schemas, I have seen some crazy attempts at description.

Sometimes we have been involved from the beginning developing thesauri of specialized terms for a collection, more often we are called in to fix existing records.

As I roll up my sleeves to tackle either project, I often wonder why organizations do not know more about what they want.

I come down to the same answer that permeates our profession as a whole. The majority of people do not understand the work that goes into providing quality. In our current era of fast and cheap; people have lost the quality aspect almost completely. When they can not successfully execute an accurate search in their database, then they call us to fix it. I am absolutely happy to do so, but make no mistake, I wish for that collection to have done it right the first time; rather than to have called us after hundreds of hours of wasted work. Quality becomes a feature of importance often only after a failure rather than as a preventative measure.

As I tell my classes, let’s talk about why doing it right is rarely done:

1) Illusion – “Everyone is digitizing” is akin to what your mother taught you as a child. “If everyone jumped off a bridge would you do it too?” Many Asset Management Companies sell short the highest cost of digitization which is (dun, dun, dun) linking the metadata to the record. The metadata needs to mean something.

I once saw a vendor selling his “automatic indexing” system. I stopped to chat with him. His product, he told me, will negate needing a human to index. As this is one of our services, I thought that I had better pay attention. He proudly told me that the video clips that he was showing me worked off of closed captioning. I was glad to know that I was not out of business. If you have ever viewed closed captioning, it is a fantastic service to those hearing impaired but it is far from error free. Aside from the many spelling errors within this due to the pressure of typing the words as a show airs (for live shows), there is no intellectual analysis of what is being said and how it relates to the visual.

If an actor said about a child, “Her temperature is 105 degrees!” Assuming the spelling was correct, that is all that the search tool would allow for. A professional indexer could include “Fevers, Childhood Illnesses, Sickness, etc”. This extra analysis would allow a successful search. Most user’s would not find that video clip by looking for “temperature” and they might not know it was a child if that is what they wanted. They would have to pull up the clips and view them. If your collection is going to stay very small, maybe this kind of quality will not matter to you.

For some, I worry that when management, tax payers or a municipality sees bulky systems with little relevance in results, they will certainly shudder at writing more checks for the system or archive.

2) Internal Pressure – “Everything needs to be digitized”
We see this pressure to digitize everything without a clear plan for prioritization. A serious needs assessment is required to be done to understand what needs to be digitized, why and what needs to be researched and described.

I have often told my students that I would rather misfile a photo negative in a physical drawer than have misinformation on a digital record. I am more likely to find it again in the drawer than in a large database.

3) Money – “Scanners are cheap, how much could it cost?”
Money is tight and people are even more apt to cut corners now. It is always cheaper and more accurate to plan something out and do it right rather than to try and fix it afterwards.

Building a business case for the step by step process of tracking assets, designing metadata, the costs of hardware/software/maintenance, training, etc. is often looked at as daunting or impossible. It is not. You have to think like a cost accountant to spell out the savings and efficiency gained. There is also often a publicity component to having an organized and highly accessible collection. This is something that needs to be built in to the value.

4) Ignorance of Computational Linguistics/Human Computer Interfaces/ Usability Studies/Search Strategies/Term Linking/(More) – “Just throw some keywords on it.”

Many times I have seen upper management wave their hands in the air as if with a magic wand and say “Just get it done.” Unfortunately, I am too old to believe in the magic wand and hard work is the only way to create a successful search tool. When I say “hard work”, I actually mean really, really hard work. Research, focus groups, linguistic analysis, understanding search tool limitations, etc. all play a part in quality design.

Along these lines, there is an interesting project that was all over the news. IBM has a team from their labs that have designed a computer to compete on Jeopardy. PBS – Nova had a documentary on it and many of the things that the lead researcher, Dr. Ferrucci, mentioned in that documentary is relevant to our field. The primary comment that caught my interest was when he discussed the fact that a computer competing on Jeopardy can be fed thousands of background documents but they have to work very hard to understand the actual question being asked. So they have the answer, they just do not know the question.

The human brain “gets” the context of place and language. Computers have not yet mastered this. Those brilliant connections of slang, historical context, cultural cues, body language, etc. are a tremendous gift that humans have.

I have stated for years that I wished our culture valued the human brain as much as technology. Watson’s project is interesting and what it proved on Jeopardy is just a portion of what it will prove going forward.

Until then, we who aim to direct searchers into exactly the video clip, manuscript or image that they desire need to value our brains and find better ways to sell our skills.

My indexers know that “Picket Fences” have a certain lifestyle context. Automation or even off-shore indexers do not know that and we can do so much better than cutting corners on core concepts.

Let’s use the gifts that technology gives to us. The ability to link, create synonyms, cross-reference records, stream clips, etc. are all exciting tools and work best in conjunction with a well thought out plan designed by a human brain.

Good luck Dr. Ferrucci but I am not sure that it is Watson that is on trial but your brain.

A recent question posted on the AMP blog, “Is preservation cost-prohibitive?,” made me think about costs related to archives in general. As a former corporate archivist, I am painfully aware of budgets and bidding out work! Now that my shoe is on the other foot, and I am consulting in the field, the issue is even larger for me. When I was the “client” and was requesting bids for work, I (and my coworkers) were always concerned by the process. We knew many vendors were underbidding to get the work and that could pose a financial risk for them if they got the project. We also were forced to consider those bids because there was quite a bit of paperwork to do if the lowest bid was not selected. In the end, we could often work around it by choosing the firm with the most expertise in an area as long as their bid was not too much higher than the lowest one.

It’s easy for a client to forget about the hidden costs of operations related to projects. There are often random emails with questions, monthly or more frequent conference calls, technology testing or review, on-site meetings or visits, etc. All of these items take up staff time – and not just a little bit of it either – it really adds up. I think many clients might be shocked if they realized exactly how much time. Often a fair amount of this time can be billed back as project management time, but only if the client is willing or that category has been built into the project.

I recently took on some work that was priced out on a per-item basis. This method really made the most sense since there was a huge quantity of digital files to review. As any vendor would, I priced the project as low as I could while still hoping to cover any time required for emails, phone calls, meetings, technology issues, etc. Luckily for me, so far my pricing seems to be working for me and for them.

The project brought up another interesting financial concern though. The quantity of items is so high and the quality isn’t always great, that the client and I started to wonder if there was a better way to upload fewer items so as not to flood their database with useless information. I think I have found a workable solution, but the main issue with trying to cull out the bad items was the time required to do it and the cost associated with that. It’s a pure and simple appraisal issue along the lines of “More Product, Less Process,” although this time it’s “More Process, Less Product!” It’s been a unique budgetary issue for me and my client in that it could easily be cheaper for them to add thousands of images to a database rather than to review the images and pick the best ones – or more importantly, to delete the bad ones.

In the end, I employed some digital photo software that allows quick-ish review and refoldering of files so that I can at least remove the “worst of” images for the client. The system isn’t perfect, but in the end, I needed to present a plan to the client that would save them money. It didn’t make sense to have the project cost more money to give them fewer images even when fewer images was more desirable. Now the project is a mix of hourly review time and per item upload time (engineered never to exceed what the total upload time would have cost if no review was done), but the client is left with better quality images and a slightly cheaper overall cost.

Back to the question at hand: Is preservation cost-prohibitive? I guess it certainly can be – as can any archival procedure, but in the end, that’s what Archives are for – preserving and making available items of historical importance. Let’s just hope that all the holders of the purse strings continue to agree!

To say those from the South are proud of their heritage is putting it mildly. Every year dozens if not hundreds of groups, towns and culture centers celebrate some aspect of Southern life from Civil War reenactments to living history villages to a wide array of festivals honoring everything from fruit and food to music and specialty local events. ¹

To help commemorate the South’s (and also America’s) illustrious background and to give a voice to the Southern perspective, the University of North Carolina has put together a digital initiative of primary and secondary sources on the Internet. Since 2004, Documenting the American South has been the premier location for education and research materials on Southern life not just on the Internet, but also in the world. The fourteen thematic collections offer wide range of digital materials that includes books, diaries, posters, artifacts, letters, oral history interviews, and songs. This vibrant digital collection consists of numerous large print, photograph and rare materials collections made possible by Southern Historical Collection, the North Carolina Collection, UNC’s Rare Book Collectionand the Davis Library.

What makes Documenting the American South such a rich resource is not just because of the various large collections that have been consolidated into a single landing point for research and discovery rather it is also because the scope, design and ease of use of the site that makes it such a valuable resource. In addition, the site is consistantly updated with new information and materials, making it not only a valuable resource for things past, but also for things present and future.

Documenting the American South has a number of discovery and mapping tools to help viewers explore the collection. This includes a Highlights section, a monthly update that explores content in the collection, which is also available via RSS. In addition is the Collections section, a listing of the digital collections by theme, also available for researchers and educators to use for finding materials that range from first person narratives, slave narratives to Southern literature and more. Specific content can also be found by using the Subjects, Geographical, Authors and Titles tabs in the top navigation bar. And if you still can’t find what you’re looking for, there is also a search option to search the entire collection by keyword to find exactly what you need.

UNC has also put together Classroom Resources, which includes kits, lesson plans and additional materials to help bring Southern life vividly into the classroom.

This is a wonderful treasure trove of materials, that is not only rich for discovery but also a valuable archive of not just Southern life, but American life as well.

1. In addition, Southern Festivals is a fantastic resource for festivals happening in the South. This site is organized by state, so if you’re traveling south this year and looking for things to do, this would definitely be one site you want to bookmark.

In a national economy in which double-digit unemployment figures travel in the opposite direction from the signs that things are said to be improving and a local economy that would jangle the nerves of the most fiercely optimistic, I am frequently riddled with doubt as to my decision to return to school for the academic qualifications to do something I truly enjoy. It was easy to buy into the notion of getting paid for doing a job for the sheer happiness it brings. I didn’t anticipate that timing is everything and having the know-how, energy and desire to take on an all new career was only half the battle.

Recently, while enjoying the company of my three granddaughters during their weeklong break from school, I was reminded why I decided to enter library school and select archival administration as my area of study. As usual, the eldest of the girls was shadowing me, eager to help with any little thing I asked her to do. I sent her to the spare room where she was to take a good long look and devise a plan to organize the boxes and boxes of stuff stacked among the cats’ toys, feeding dishes and litter boxes. After a few minutes she returned with her report. “There’s a lot of stuff in there, she said, but I found a box full of CDs.” I immediately wracked my brain in an attempt to remember if I had been remiss in the clutter-reducing organization of the CDs and DVDs I had painstakingly organized into albums.

Anticipating my next request, my granddaughter left the room and quickly returned holding an old milk crate filled with individually boxed audio cassettes. One glance at the plastic crate and I was instantly reminded of the reason I had upset the status quo by leaving behind a reasonably well-paying job and switching my academic focus to continue on in graduate school. “Those aren’t CDs, I said. “They are cassette tapes recorded by your great-grandfather, and they are in need of processing, along with the rest of his collection.” If at that moment my granddaughter didn’t quite understand what I meant, by the time she returned home at the end of the week, not only had she become somewhat acquainted with the great-grandfather who died shortly before she made it to 3 years old, she attained a basic understanding of the important role archivists play in the preservation of history and culture, and she began to comprehend why her Nana always gets a little misty whenever we go through the dozens of boxes, albums and other containers filled with photographs of people she will never actually meet.

To quote my esteemed mentor, “as archivists we wear multiple hats.” Our professional responsibilities require us to step away from the photographic and/or moving images in our personal collections and devote our time and energies to the clients and organizations from which we endeavor to earn a living. When we are presented with a collection, we delight in the discovery of the information contained therein. Our sense of history is sharpened (or dulled) when set upon a path to learn something new and/or review, organize and disseminate what we already knew. As we focus on these bread and butter tasks, the records of our personal histories all too often get short shrift, and containers filled with items in need of organization and care are left to languish and suffer the deterioration that results from the necessity of putting off until later the donning of the hat we wear as preservers of our family histories.

For most of his life, my father was a musician. He played music, wrote music, breathed ate and drank music – all the time. Although he earned a living as a civil servant, he never strayed far from his first love. He had been a music teacher and a choir director, an actor and a playwright, a lay minister, videographer, photographer and a broadcaster. He had also performed in prestigious productions and with some of the most legendary names in the business. But what my father really, really loved to do was sing, and boy could he sing! My earliest and best memories include the velvety sound of Dad’s rich baritone voice singing from one of the many American musicals he counted among his favorites. Contained among the scores of audio cassettes my granddaughter recovered from the spare room closet is a series of lecture concerts on the history of the American musical theater from 1866 to 1970, Dad’s recreation of his doctoral dissertation. Imagine my surprise when making this discovery. To say that the processing of my father’s collection will be both an arduous task and a labor of love seems inadequate upon the realization that Dad’s audiocassettes, photographs and sheet music very likely hold a cultural, historical, academic and possibly commercial value beyond family sentiment and the enormous pleasure I receive from hearing that rich, velvety baritone waft from the speakers of my stereo.

As I embark upon the journey represented by processing my father’s collection – a journey which will doubtless be filled with twists, turns and bumps in the road – I ask myself one question: Which hat do I wear now?

Earlier this month, the Library of Congress announced that it would house every “tweet” ever posted on Twitter. Every 140-character-or-less blurb on the site is now part of the vast LoC archives. This got me thinking: what are the issues at hand in archiving social networking sites? And why is it important?

Recently, while cleaning out my apartment, I found a relic of primitive social networking—a printed-out Facebook message from 2005. Nostalgia instantly struck. Five years ago, Facebook was [thefacebook], with a much simpler interface. A toolbar on the left listed the humble features of the relatively new site: My Profile, My Groups, My Friends, My Away Messages. Clearly, Facebook was trying to emulate MySpace —which was then by far the preferred means of social networking.

The past five years of my Facebook existence flashed before my eyes. I remembered when the site was only accessible to students at selected universities, when users could upload only one photo, when “the wall” was merely a text box that anyone could edit. A Facebook before FarmVille, “like” buttons, and mini-feeds.

With hundreds of millions of users, there are many reasons to archive social networking sites. First, for historical documentation—millions of photos are uploaded to the site every week, capturing the trends of the present era. Additionally, no matter how worthless each status update might seem, taken as a whole they reflect our reactions to modern events. It would be worth keeping a log of Facebook updates from the night of the 2008 election, or as the news broke about the earthquake in Haiti, or even as the Tiger Woods scandal unfolded.

Second, following the history of social networking sites offers insight into the development and perceptions of Web 2.0. What was it that made Facebook become so much more popular than MySpace ? How has Facebook increased the average person’s awareness of internet safety and privacy? Studying the evolution of such sites, web developers can predict future trends in online technology.

Third, social networking sites are paramount to the way we communicate in the 21st century. How do we choose to represent ourselves online? Everyone has that one friend who reveals too much personal information (Matt Maclean: too much vodka + ice cream = the great pukefest of 2010). There’s the perpetual Facebook drama, including my favorite, the Passive Aggressive Status Update (Alexis Cooper thinks some people need to get over themselves!!!) And how has Facebook changed our interpersonal relationships? At some point in their cyber-existence, most people have undoubtedly become angry or hurt or jealous over something said on Facebook. Relationships that may end in real life passively continue online—the psychological implications of which are new to our generation.

But how should sites like Facebook be archived, and to what extent?

The Internet Archive has preserved 150 billion web pages, dating back to 1996. Using web crawlers to archive websites, the IA serves as a publicly-accessible digital library, allowing users to revisit older versions of their favorite sites. Documenting and preserving static websites such as aol.com is relatively easy because they are open to the public. Public profiles on MySpace and Twitter are also easily accessed. But how do web crawlers get inside websites, like Facebook that require login information to view its content?

It’s not easy. The Internet Archive has developed a software application called Archive-It, which allows organizations to make digital copies of their own websites. As more and more universities, corporations, and libraries create their own Facebook pages, they can now archive them. While this is great news for individual organizations, it doesn’t include the interactions of ordinary Facebook users—the cyber chatter that documents our day-to-day lives.

Recently, the White House announced plans to preserve its social networking content. Media capturing is done using software applications and daily screen shots, storing text, graphics, audio and video in the context in which they were originally presented. Once this project gets off the ground, it can serve as a model for other organizations.

But what about the interactions of everyday Americans—how will their voices be recorded? Perhaps the LoC or the Smithsonian could begin a digital initiative to capture people’s online identities. An open call for submissions, asking Facebook users to send in screenshots of their profiles, walls, photos, or feeds, would be a simple and effective start. Similar projects, including the September 11 Digital Archive and the Hurricane Digital Memory Bank, were created to obtain Americans’ reactions to history-making events.

It’s difficult to say if a comprehensive range of Facebook users would be willing to submit their personal information to an archive. However, thousands are already sending in their screenshots to blogs, including Failbook and Lamebook, which capture people’s regrettable and embarrassing Facebook moments.

No matter how trivial they may seem, social networking sites are a defining part of our generation—and that’s something worth preserving.

Further reading:

As I was working on a workshop about process planning for digitization, I came across this quote by Peter Drucker, ”Effective leadership is not about making speeches or being liked; leadership is defined by results not attributes.”

No matter how pleasant you are (or you think that you are) the bottom line is that the funding and reputation of your institution rests on success.

There is a reason that business principles exist. There is a reason that companies that fail to follow these principles also fail. Few managers of digitization projects have business backgrounds. The number one failure seems to be a lack of project management skills.

There is not enough in the way of professional literature on this topic. Most is focused on case studies that are presented in ways that are specific to that particular project.

We need to try to develop standards or at the least, guidelines that are portable to most projects. A smart manager follows four important rules:

1. Establish written procedures (aka process manuals)
2. Develop deliverables (aka due dates and accountability)
3. Track production (aka cost accounting)
4. Adjust/improve processes as needed (aka communication and report generation)

In the words of many great politicians, “Let me be perfectly clear, “developing tracking and implementing good project management is not more time consuming than NOT doing it. On the contrary, the risk in not doing it is too great. Why don’t we put the time in to do this deeper structural work? My answer is that most of us do not realize the impact of not doing it.

The number crunching, procedure writing, report generation and pie chart creation is not the most attractive aspect of digitization. Showing the world a digitized version of a little-documented historic event is what pulls us in to this field. Creating tracking forms is NOT why 99% of us enter the profession.

I can’t even count the number of institutions that I have seen go over budget and have to stop in the midst of a digitization project. The lack of ability to manage, project and track costs has stopped some people from even trying. So, I think this is a fair forum to introduce some of these concepts.

Look for a serialized version of “Business Principles for Digitization” to be coming to the blog over the next month.

As always if you have suggestions of what you might like to see, drop me a note!

In teaching multimedia archive,s I think about the future of our content constantly. Like a new mother, I fret for its security, growth and health. What is THE answer for our degrading media, emulsions, for our software obsolescence and our equipment falling down around our ears?

Recent discussions on the AMIA listserv brought new energy to this discussion and I wanted to put my spin on this. The subject line was “What’s Not Cool About Cold?” and it solicited some serious discussion about whether we have made a horrible mistake for a generation of archivists and content.

Jim Lindner argues that the imminent demise of tape players is more important in an archivist’s preservation decision than our focus on the imminent degradation of the media itself. The latter being our big decision to place much of our media in cool or cold storage. The group discussion mentioned the fact that many of our players are no longer supported by their manufacturers and the simple math that the lack of machines and the existing wear on their parts will not even cover the playback of the volume of archival tapes awaiting…migration? This hits a deep reality. Have we lost hundreds of thousands of hours of archival motion under our watch? Maybe even millions or billions of hours?

With the current demand from patrons increasing and the desire to use motion outtakes, news, etc. for historic documentation this is a disappointing state. Motion is powerful, emotional and immediate. It puts patrons in the experience in a way that to the visual human being is dramatic.

This important discussion on the listserv was a strong exchange which took place over several days. Other important points included the argument that the research on degradation and proper handling has made the tape degradation less of an issue and now the issue is playback for migration and access.

Other points included:

1) The need to get these gems a proper venue to be seen (whether due to money or rights issues)

2) The natural evolution of gathering obsolete machines from production facilities and creating some centralized site for archival institutions to use.

3) How to custom make parts so that we can maintain the machines so that we can transfer the content.

4) Wait for a miracle technological innovation to happen in the next 20-30 years so that you can migrate. (Keep them cool until then).

We come back again and again to the complexities of managing archival multimedia. The next generation of archivists will have changed their paradigm and released themselves from the desire to preserve the artifact. I am not condoning refusing to preserve all originals but our focus as far as video tape and digital files will be on content migration.

Other formats still have certain inherent value and that is a different blog posting!

Future archivists will have a much better long-term understanding of what is an artifact and what is intellectual content that needs to be migrated. They will be more adept to the rapid pace of format change and will HAVE to adapt quickly.

Not that I am faulting us. We are the transitory archival generation, the one that bridges the 100 year film format and the thumb drive.

I agree that the great research done in the field has helped us to minimize that problem, but it does seem that we took the eye off the ball a bit on migration. We cite lack of funds, lack of understanding of the urgency by the non-archival world, the frustration with equipment manufacturers, etc. It sounds like we need a development guru to raise funds, awareness and help to join all the key players for a collaboration. Many mention that the creators often do not prioritize preservation as they should. That is true, but our role as professionals is to educate.

Hand-wringing is not allowed! The energy that we have spent on that could have been put forth to an international collaboration with a real potential for migration, managing equipment, and innovation. One manufacturer can not do this alone, everyone needs to get on board and when I say “everyone” I mean:

Archivists
Creators (producers, studios, channels, directors, talent, writers, etc.)
Media
Equipment and Media Manufacturers
Technologists
Professional Associations
Funders

I think all of the above agree that the loss is imminent. The question is can we use our collective economy of scale to work together? Or maybe it is will we?