ArcticBlue: Bridging the Gap Between Production Storage and Deep Archive
Striking a balance between performance and cost
As the demand for high-quality digital media continues to explode year over year, media companies are finding themselves acquiring, creating, distributing and storing massive amounts of content. With the volume and quality of the media growing, the sizes of files are getting larger and larger. The traditional ways people consume media are evolving, so both established and new media companies are clamoring to capture people’s attention through broadcasting, virtual reality (VR), augmented reality (AR), live streaming, social media and video streaming services on every conceivable platform, including smartphones, tablets, computers and televisions. Revolutionary and cost-effective storage solutions are required to keep up with the demand.
Media companies continue to face the prospect of increasing storage costs, bulging storage capacities and expanding infrastructures, as well as future expenditures for business development and staffing. Cloud options do exist and are gaining in popularity in some aspects of the digital supply chain, but are limited in their speed of delivery and do not provide a cost-effective means for media companies to store frequently accessed media. And, although the cost of spinning disk has come down in price, it is still expensive to retain content that is not accessed on a frequent basis or assets associated with a finished project on primary storage.
Gaining A Return on Investment
Any organization that utilizes media as part of their business can benefit from Spectra’s ArcticBlue® Disk Solution. ArcticBlue, simply said, as a mid-tier storage, bridges the gap between production storage and archival storage providing balance between cost of storage and expected performance for quick concurrent access to media. ArcticBlue delivers a solid return on investment for key media organizations in news. That’s because ArcticBlue offers all attributes required of a storage system, such as low cost, seamless scalability and fast response time required to store and retain breaking news, anniversaries, orbits and elections cycles, etc. for mid to long term. Creative and graphics groups benefit from ArcticBlue for storage of their seasonal campaign or tent-pole events. These assets and projects typically sit untouched for many months or for extended periods of time. As the event nears, these assets can be accessed easily and repurposed, then go dormant again.
Sports broadcasters and regional sports networks can also leverage ArcticBlue’s ability to quickly search and access prior season’s content, B-roll, promos, teasers, social media, new media, remote and studio productions for reference or to monetize. Video and post-production houses, with their video quality and file sizes growing exponentially (i.e., 4K, 8k and even 16K), need to offload their primary storage to store and park files natively - but also need files to be quickly accessible to users when needed for weeks prior to their permanent archive. Implementing a tiered storage infrastructure with ArcticBlue as a mid-tier storage, will help reduce the cost of doing business while still meeting the quick access needs of the users.
Tape storage technology clearly has cost and data security advantages, but for many organizations with highdemand environments, augmenting the workflow with disk storage to achieve quick and concurrent accessibility for media content is critical. However, the overall solution must balance cost and fast concurrent access and the total security of the corporate assets.
Leveraging Spectra’s BlackPearl Architecture
To truly understand the value of ArcticBlue, one must understand the power of BlackPearl. Spectra’s BlackPearl Converged Storage System was designed to provide a single portal to multiple tiers of storage – intelligently moving and retaining media seamlessly on disk, tape and/or cloud - in a way not previously possible. BlackPearl is an agnostic storage platform that can easily and seamlessly integrate with any MAM, PAM or DAM application. These integrations allow for a transparent, one-stop application that can ingest, manage, archive, search and retrieve all media assets seamlessly, regardless of what storage repository the media assets are currently being stored. Utilizing Advanced Bucket Management (ABM), BlackPearl’s intelligent data policies enable redundant copies of media on multiple storage mediums, including disk, tape, and/or public cloud, and can even automatically eject tape copies for offsite storage.
Utilizing the cloud as a storage repository in media workflows is an option organizations explore as part of their digital supply chain. As more businesses look to reduce operational costs, uploading media to the cloud seems to be an attractive offering to explore for media companies - though they should not be fooled. While there are some practical applications and workflows which utilize cloud services which help some organizations, the cost of bringing content back is extremely cost prohibitive compared to on-premise storage, which acts as a better repository for accessed content. Additionally, the latency of bringing media back from the cloud doesn’t meet the needs of many organizations who rely on media as an integral part of their business. Now, through utilizing an ArcticBlue and BlackPearl solution, media organizations can economically create a private cloud infrastructure.
Performance and Capacity: What to Expect
A unique feature of the ArcticBlue system is the use of large stripes of disks (20) or Very Wide Bands [VWB]. The spread of data over large number of drives delivers greater performance (nearly 1 GB/sec) while accessing data. This data distribution coupled with optional replication (discussed later in this whitepaper) delivers the best performance, protection and recovery. Other object storage systems use a 20+6 erasure coding which requires higher overhead, achieving only 20/26 of the capacity. While this method provides good data protection, it requires a greater number of drives and delivers lower useable capacity. Spectra’s implementation of ZFS software uses a triple parity to maximize data integrity. To counter the higher overhead (as is the case with erasure coded stripes), ArcticBlue uses a 20+3 band size, lending the system up to 83% usage (20/23) of raw capacity. Compared to other object storage systems, a common 20+6 erasure coding has 77% usage, while a 3 copy redundancy has only 33% usage.
Expansion: Incremental Scaling as Business Grows
Not only is ArcticBlue reliable (up to 7 years of disk life) and economical (as low as 10 cents a gigabyte), it also easily scales to accommodate growing content. In order to scale the ArcticBlue system to meet evolving storage requirements, users can easily increase capacity and performance by adding additional (VWB) bands and/or expansion chassis of ArcticBlue. Added bands are simply and automatically provisioned into available storage pool(s). Each ArcticBlue expansion chassis connects to the master node using a direct external SAS cable. BlackPearl supports up to 8 expansion chassis, scaling to 6.1PB in a single rack. Spectra Logic’s storage systems, including BlackPearl and ArcticBlue expansion nodes, are designed to easily add or swap drives and replace components with optional spare parts stored onsite as needed. Expansions of nodes require zero downtime with the optional install of SAS HBAs in BlackPearl. Each SAS HBA supports two ArcticBlue chassis (over 1.5PB of storage).
Cost Effective: The First Disk System of Its Kind
Each band of drives is intelligently spun down when the band has been idle for a period of time (an hour). This reduces power consumption, minimizing the wear factor on the drives, thereby extending the life of the drives. Each band can be powered down individually, and quickly spun back up as needed for ingest and retrieval. Spin down capability uses up to 75% less power and can extend the disk’s life span up to 7 years. The extended life span of ArcticBlue’s disk drives translates into sizable cost savings. Industry norms dictate that disk subsystems usually need to be replaced every 3-4 years. ArcticBlue extends the time between technology upgrades by almost double. ArcticBlue also eliminates costs associated with procuring, deploying and migrating content to a new disk system every 3-4 years – which is a labor intensive and expensive process. Finally, the largest cost benefit arises from the fact that ArcticBlue lowers the expansion burden placed on primary storage further easing the need to scale out these systems. Primary storage can moving forward be used only as needed for production purposes.
Each full ArcticBlue node (96 drives) uses 775 watts during read/write and only 140 watts once spun down. A full rack of ArcticBlue at 6.1PB can realize power savings up to 80%. The lower power consumption results in much less heat, decreasing the demand on required cooling, further lowering the total TCO.
Data Integrity: Keeping Content Safe
ArcticBlue systems preserve data integrity by providing multiple levels of integrity checks beyond that found in typical disk systems, resulting in a much better bit error rate. First level is advanced file system checksums that protect against undetected errors. ArcticBlue features an on-demand data integrity check that scans the drives for data corruption and corrects any errors found. The integrity check runs on an individual ArcticBlue pool. In addition to checksums, the ArcticBlue pool has triply parity protection with intelligent rebuilds using a hot spare hard drive. This level of protection allows for up-to three drive failures while providing continuous access to data. Statistically, a triple parity system will only lose data once in over 2 million years for the drive type and the 23 drive band size option used in ArcticBlue.
On top of the protection, at the disk drive and file system level is the second level of protection: Object Storage checksums. The Object Storage checksum provides a file-level or part-offile (chunk/blob) level checksum, where each file or chunk has its checksum calculated and stored with the object as metadata. An end-to-end checksum can also be used, where the host is required to provide a checksum for the file or blob before the data is sent to BlackPearl. Once sent and in the Cache, BlackPearl verifies the checksum, ensuring the integrity of data in transit to BlackPearl.
In addition to protecting the single copy of data kept on an ArcticBlue pool, another great way to provide more protection (while also increasing the availability of data) is to have an additional copy of data. Using BlackPearl data policies, multiple copies of data can be seamlessly made in any combination of the following: on a secondary ArcticBlue pool, on internal tape, on ejected tape, with a replicated BlackPearl and more. With ArcticBlue, organizations can handle data that continues to expand at dizzying rates while remaining confident in the data’s integrity.
- During reads and writes ArcticBlue performs continuous checks for bit rot and corrects them as needed
- As an object storage system, ArcticBlue provides the means to checksum and verify the integrity at the file or file-part level
- Optionally, multiple copies of assets can be created on separate ArcticBlue bands or other BlackPearl supported storage mediums - such as a remote BlackPearl, tape or even the public cloud (e.g. Amazon Glacier)
More and more users are realizing that their archives are not a place to warehouse media that will rarely or never be used again. Instead, organizations are recognizing that their archives are an invaluable, central and active part of their media workflows and media operations. As a result, the need for fast, reliable, scalable and affordable tiered storage solution isn’t a nice-to-have, but rather a must-have for an organization to be competitive in this fast-paced, ever-changing media landscape. End users now access archived media more often and faster than before, driving demand for immediate access to their archived assets. In addition, they need to restore and manipulate media very quickly. Standard tape archives are an effective and affordable storage repository for long-term infrequently accessed assets. These workflows may not meet the business requirements for a certain subset of assets that require instant access (millisecond to seconds) by users, hence augmenting such archives with ArcticBlue greatly enhances the response of the overall archival system. As previously mentioned the following organizations can greatly benefit from the performance, efficiencies and low cost attributes of ArcticBlue:
- News Organizations
- Creative Groups
- Sports Broadcasters and Regional Networks
- Post and Video Production Houses
ArcticBlue more than meets the needs of users who must frequently access archived media in a quick, efficient and affordable manner.
Deep Storage Experts
Spectra Logic develops deep storage solutions that solve the problem of long term storage for business and technology professionals dealing with exponential data growth.
Dedicated solely to storage innovation for nearly 40 years, Spectra Logic’s uncompromising product and customer focus is proven by the largest information users in multiple vertical markets globally.
Spectra enables affordable, multi?decade data storage and access by creating new methods of managing information in all forms of deep storage—including archive, backup, cold storage, cloud, and private cloud. For more information, please visit http://www.spectralogic.com.