27 June 2018

CMS dark data volume after migrtaion to new SE at Tier1 RAL

CMS have recently moved all  full usage of disk only storage from CASTOR to CEPH at the RAL-LCG2 tier1. (They are still using CASTOR for tape services.) CMS had been using an approx 2PB of disk only CASTOR space on dedicated servers. What is of interest is that after CMS had moved their data and deleted the old data from CASTOR, there was still ~ 100TB of dark data left on the SE. WE have now been able to removed this data and have started the process of decommissioning the hardware. (Some hardware may be re-provisioned for other tasks, but most is creaking with old age.)

ATLAS are almost in a similar position. I wander how much dark data will be left for them.. ( Capacity is 3.65PB at the moement so my guess is ~150TB)

When all the dark data is removed , and the files removed form the namespace, we can also clean up the dark directory structure which is no longer needed.  I leave it to the read to guess how many dark directories CMS have left...

1 comment:

bgedavies said...

Addendum: ATLAS have now effectively moved off diskOnly CASTOR at RAL-LCG2 Tier1. 3.3PB sites had 106TB of dark data, so similar volume as CMS.