Sponsors
Sponsor Products
Storage scalability
posted by Caleb Khor  on July 1, 2021, 4:59 p.m. (1 year, 10 months, 28 days ago)
3 Responses     0 Plus One's     0 Comments  

Does anyone reach a point where one has to scale storage and keep storage performance at the same time?

I have a file system on ZFS in FreeBSD for my studio to use, but I'm facing challenges into how to best scale this as well as keep the performance intact.  

Does anyone here have a similar setup?

Or rather, I'm interested to know how you scale up your storage while maintaining its performance. 

What do you use for your studio when storage can go up to petabytes and so on...

Thank you.


Thread Tags:
  zfs storage truenas vdev 

Response from Ty McMahan @ Feb. 7, 2022, 10:35 a.m.

Hi! I can recommend you take a look at www.truenas.com and get either the community edition to test (TrueNAS Core, formerly FreeNAS), or for production environments with support, TrueNAS Enterprise.

In terms of performance and scalability. the main point is getting the vdev layout RIGHT up front and making sure you understand your throughput needs (both network and disk). Tell us more about your use case, and I could give you better advice as it relates to your workload and bitrates, streams, etc.

 

For example, an 15-wide z2 vdev would perform horrible compared to a 5-wide z2 vdev. There is a lot that goes into properly planning and laying out storage. TrueNAS actually helps make all this a LOT easier.

 

-KX4MQ

 


0 Plus One's     0 Comments  
   

Response from Rob Giltrap @ July 5, 2021, 12:19 a.m.

Disclaimer: I work for Oracle

We have a lot of studios happily using Oracle ZFS Storage Appliances. They scale up well, providing multiple petabytes in a single system with plenty of performance and stability.

If you're growing at a rate that your storage is problematic then we have a good solution which is easier than having to maintain a DIY solution.

If you're at half a petabyte or more, then it certainly starts to make commercial sense.

Cheers, Rob.

 

 

 


0 Plus One's     0 Comments  
   

Response from Jeff Yana @ July 4, 2021, 11:58 p.m.

This site is mostly dormant by now so you might get more traction by posting on Slack. 

But regarding your question, are you looking for alternatives to ZFS, or are you looking to stay with it and improve performance? ZFS is pretty unique as filesystems go with many advanced features not found in many commercial products, such as COW, but its not a scale-out, cluster file system. I know there is some interest in the OpenZFS community re: adding scale-out features but no current road map.

If you are running a newer version of OpenZFS, then take a look at Allocation Classes, which will allow you offload your smallblock and metadata workloads from your slower data disks.


0 Plus One's     0 Comments