Tiered Storage and Archiving: Best Practices for Data Life Cycle Management and Long-term Digital Preservation: Dartmouth

Tiered Storage and Archiving: Best Practices for Data Life Cycle Management and Long-term Digital Preservation

Duration: 2 and a half hours, not including Q&A

Monday, March 09, 2009
8:30 am: Doors open / Registration Begins
9:00 am: Lecture begins: Digital Preservation and Data Life Cycle Management
11:30 pm: Q&A / Discussion

Some of the topics covered in the lecture :

  • In-band File System Virtualization
  • Out-of-band File System Virtualization
  • Archival File Systems
  • Clustered File Systems
  • File Stubbing and its associated challenges
  • MAID - Massive Array of Idle Drives
  • SSD - Solid State Disk
  • Data Replication Methods
  • Tiered Storage for Databases and Email
  • Eliminating Tape Media
  • Embracing Tape Media
  • Deduplication of Rich Media
  • Compensating for Silent Data Corruption

Across all industries digital content is accumulating:  office files, email, data warehouses, medical images, scientific data, rich media, etc.  Like a snowball rolling down a hill, we have to store and manage an ever increasing volume of digital content year after year.

The good news is that the cost of storage media gets less and less expensive over time.  Whereas a single terabyte of redundant storage might have cost $4 million in 1990, today you can pick up a mirrored pair of terabyte-sized hard drives for about 200 bucks.  The bad news is that the costs and complexity of managing storage gets greater and greater as capacities grow.  Take the backup system for example.  If you struggle to back up a 10TB storage environment, what is it going to be like when your capacity doubles to 20TB and then to 40TB and 80TB? In short, traditional technologies for managing storage (RAID groups, file systems, backup systems, tiered storage mechanisms, etc.) are not scaling up to keep pace with capacity growth.  To make matters worse, there are the physical and operational challenges with a growing storage environment.  One has to factor in power consumption, cooling, physical space in the data center, and the availability of personnel with the appropriate skills.

The picture is further complicated by the growing demand for higher performance storage systems.  While a subset of storage technologies address growing capacities, others address the need for greater and greater performance.   Performance comes at a price premium, so it is seldom possible to get bigger, faster, and cheaper.  The only way to get bigger, faster, and cheaper, is to manage the life cycle of data across multiple tiers of storage media.

The response to these challenges by the data storage industry has been generally disappointing.  Many of the leading vendors have simply adopted new acronyms for old technologies, hoping to expand the markets for their existing products!  Meanwhile, a myriad of new technologies have made it to market, but they lack a generally accepted set of best practices for integration.  Thus, the onus falls on the storage administrator or archivist to sort out these new technologies and attempt to integrate them into a comprehensive storage management strategy.

This seminar explores the problems and solutions associated with managing data storage as capacities grow and data ages.  It does not focus on specific products or specific vendors.  Rather it focuses on the problems that need to be solved and the underlying architectures of a variety of solutions.  Examples are drawn from multiple industries: scientific research, medical imaging, academic libraries, manufacturing, and general office computing.

 Major Themes:


  • Traditional data storage management technologies are not keeping pace with the growth of storage capacities. 
  • The bigger they are, the harder they fall.  How do you back up ever-growing storage repositories and ensure the integrity of data over long periods of time? 
  • Sometimes it makes sense to tier storage such that storage performance and storage capacity can scale independently of one another.  Sometimes it is less expensive and simpler to to settle into a single tier for all needs. 
  • Redundancy models need to change as capacities continue to grow.  Traditional redundancy models look great on paper, but can be very expensive and can even lead to an increased chance of data loss.

Intended Audience: System administrators running day-to-day operations as well as systems architects and IT managers. This tutorial is technical in nature, but it does not address command-line syntax or the operation of specific products or technologies. Rather, the focus is on general architectures and various approaches to scaling in both performance and capacity. Since storage networking technologies tend to be costly, there is some discussion of the relative cost of different technologies and of strategies for managing cost and achieving results on a limited budget.

About the Presenter: Our presenter is an industry-recognized expert on storage networking and data protection technologies. He has authored numerous papers and is a regular speaker at major industry events such as Storage Networking World, VMWorld, Interop, and the Usenix conferences. His no-nonsense, fast paced presentation style has won him many accolades. He is a regular lecturer at many of the nation’s leading colleges and universities. Inside the data storage industry, he is best known for having authored best practices for designing and optimizing enterprise backup systems and for his expertise in the marketplace for emerging storage networking technologies. He has served on the advisory boards of many of the most successful storage technology startups, and is well respected in the analyst community. Most recently, he was voted "Best Speaker" at Storage Networking World.

Questions: Please feel free to contact Megan at 781-250-3240.

Monday, March 09, 2009
8:30 am: Doors open / Registration Begins
9:00 am: Lecture begins: Digital Preservation and Data Life Cycle Management
11:30 pm: Q&A / Discussion

Thank you to our sponsors: USENIX and Dartmouth College

Campus Map


Name Sales End Price Fee
* FREE * For faculty, staff, students, and local IT end users Ended Free
Not Free - but still a great value - for all others (members of the press, salespeople, manufacturers' representatives, resellers, etc.) Ended $395 $9.88

On the Web

Tiered Storage and Archiving: Best Practices for Data Life ...

... Best Practices for Data Life Cycle Management and ... Tiered Storage and Archiving: Best Practices for Data Life Cycle Management and Long-term Digital ...


Tiered Storage and Archiving: Best Practices for Data Life ...

... Tiered Storage and Archiving: Best Practices for Data Life Cycle Management and Long-term ... Data Life Cycle Management and Long-term Digital ...


Tiered Storage & Archiving Lecture @ Virginia Tech - Sept ...

Tiered Storage & Archiving ... Best Practices for Data Life Cycle Management & Long ... options for digital preservation and data life cycle ...


Tiered Storage & Archiving Lecture - MCV Campus @ VCU ...

Tiered Storage & Archiving Lecture - MCV Campus @ VCU - Sept. 26, 2013 on Sep 26, 2013 in Richmond, VA at Virginia Commonwealth University ...


Tiered Storage & Archiving Lecture @ Virginia Tech - Sept ...

Find event times, reviews and photos on Local.com for Tiered Storage & Archiving Lecture @ Virginia Tech - Sept. 25, 2013, on September 25, 2013, in Blacksburg, VA ...


Active Archiving: Data Migration, Self-Healing Storage ...

Active ARCHIVING Data Migration, Self-Healing Storage, and Long-term Digital Preservation This ... archiving and data life cycle management. ...


Best Practices for Storage Management: Developing an ...

... through its life cycle, and at various points, storage, ... for long-term preservation. Create a digital ... "Best Practices for Data Management: ...


Usenix Definition. Crossword Dictionary. - Word …

... and Archiving: Best Practices for Data Life Cycle Management and Long-term Digital Preservation: Dartmouth ... Tiered Storage and Archiving: Best Practices …


Long-term archiving - Storage Technology Magazine

... officer or digital-preservation officer. Long-term archiving can ... second- or third-tier disk storage. ... data life cycle management (DLM) Load More ...


Storage Design for High Capacity and Long Term Storage

Capacity and Long Term Storage Lecturer: ... Best Practices for Life Cycle Management ... Data Storage Solutions for Digital Preservation


Data Integrity Means and Practices - Digital …

Data Integrity Means and Practices ... Long-term Retention, Green Storage Apps Tier 0 ... Future-proof Data Storage for data preservation


EDRM: Best practices when preparing for an e-discovery …

... need to know about EDRM; as well as some best practices for ... on Data storage compliance and archiving. All; ... data life cycle management ...


QStar Archive and Data Management Solutions

Long term data preservation for ... Network Migrator - Policy-based Tiered Storage Data Life-cycle Management ... 3-2-1 Archiving and Data Protection Best Practice ...


Implementing tiered storage from HP - ComputerworldUK.com

Implementing tiered storage ... and long-term data management in the ... service levels at different stages of the data life cycle. y Management software ...


Digital Preservation Policy - Dartmouth College

Life Cycle Management. Digital objects will be ... Best practice in digital preservation requires duplicating ... Long-term Storage – A conscious decision ...


Archival, Digital Preservation, and Records Management

IEEE Archive Storage Life Cycle ... Produce a “best practices for long-term digital information ... Data Management Forum (DMF) LTACSI (Long Term Archive ...


A Best Practice Guide to Archiving Persistent Data: …

A Best Practice Guide to Archiving Persistent Data: ... Why Tiered Storage is the Only Realistic Solution ... life cycle and


USENIX Education on the Road

Digital Preservation and Data Life Cycle ... as content archiving and/or life cycle management ... data storage industry, he is best known for having ...


Enigma Data Solutions, Ltd. Joins with Archivas to Offer ...

Jun 19, 2006 · ... a leading provider of data management ... a tiered storage A data ... accessed files to lower cost storage. When combined with a digital ...


Content Lifecycle Management - Best Practices for ...

Jul 30, 2009 · ... webinar discusses the possibility of a full content life cycle management ... Best Practices for Governance, Archiving, ... Long-term data …