|
I have a file server performance issue thats been bugging me for a while and am hoping you all can help...
Nuke (9.0v5) is reading in DPX sequences at around 2-3fps when network traffic is low. When traffic is high or automatic backups are running, read times can drop to 2-3 seconds per frame. This becomes a huge time-sink when nearing the end of an episode and an artist is making small changes to a dozen shots per day.
My background: Character animator for the last 10 years but have always enjoyed tinkering with hardware and software and have become the de facto tech guy at my latest studio because of it. Im currently learning bash and python to help manage the render farm and automate mundane stuff. So please use small words and provide pictures when possible, Im pretty new to this stuff. :)
The setting: Small studio doing episodic television and commercials with about 15 artists but well ramp up to about 30 artists during heavy production. About 8-10 of the artists would be Nuke compositors. Were all Mac OS for the time being.
Now heres a whole lot of info Ive collected since Im not sure whats relevant and what isnt
Current file server configuration: (I did not build this, we contract with an outside company for hardware setup and support)
Cisco SG500X-48 Switch.
Server: Mac Pro 4,1 (2009) - Dual Quad-core Zeon E5520 @ 2.26GHz, 32GB RAM, OS 10.9.5
RAID (1+0 configuration) storage 44TB w/ 66TB backup via ChronoSync
RAID card = Areca ARC-1863 - http://www.areca.com.tw/products/1883.htm
Ethernet card = Small Tree P2E10G-2-XR 10GB running x8 width
Volumes are shared via AFP.
Current Test results:
Iperf3 - 940 Mb/s ethernet = theoretical max of 117.5 MB/s
Black magic Disk Speed test - Local Write (running locally on the server): avg. 1800 MB/s Local Read: avg. 1000 MB/s
Black magic Disk Speed test - Network Write (shared volume mounted on workstation): avg. 105 MB/s Network Read: avg. 105 MB/s
DPX sequences average about 8MB per frame giving a theoretical max of about 13fps to compositing machines.
A 1080p, 156 frame dpx sequence took 68 seconds to read in (just over 2 fps) and Activity Monitor reported that Data Received topped out around 14 MB/s. So about 13% of the theoretical max transfer.
Simply copying the same sequence to the workstation takes about 16 seconds.
Our current show was shot on Arri Alexas and a 2K (2048x1152) ProRes 4444 clip straight from the Alexa reads in at about 18-20fps. Activity Monitor reports that Data Received tops out around 30 MB/s about 28% of the theoretical max but nearly 10X faster than the dpx sequence from the Nuke artists perspective.
I get that theres a fair bit of overhead when transferring 100, 8MB files as opposed to one, 800MB file but does that seem right to you all? Am I delusional about what our little server should be able to do?
So this boils down to two main questions:
1. This feels like a RAID issue to me. What sort of settings could I look into on the RAID to minimize read times on image sequences or is this the best I can expect from our setup?
2. What advantages (if any) does working with DPX sequences offer when the raw footage is shot in ProRes 4444?
Thanks for reading through all of that. Any insight would be appreciated!
- JG
| 
24 spindles.