Here at the Tier1 at RAL-LCG2; we have been draining disk servers with a fury (achieving over 800MB/s on a 10G NIC machine.) Well we get that rate on some servers with large files; but machines with small files achieve a lower rate, but how many small files do we have and is there a VO dependency... So I decided to look at our three largest LCG VOs.
In tabula form; here is the analysis so far:
VO |
LHCb |
CMS |
ATLAS |
ATLAS |
ATLAS |
sub section |
All |
All |
All |
non-Log files |
Log files |
# Files |
16305 |
14717 |
396887 |
181799 |
215088 |
Size (TB) |
37.565 |
39.599 |
37.564 |
35.501 |
2.062 |
# Files > 10 GB |
1 |
24 |
75 |
75 |
0 |
# Files > 1GB |
8526 |
11902 |
9683 |
9657 |
26 |
# Files < 100MB |
4434 |
2330 |
3E+06 |
134137 |
3E+06 |
# Files < 10MB |
2200 |
569 |
265464 |
68792 |
196672 |
# Files < 1MB |
1429 |
294 |
85190 |
20587 |
64603 |
# Files < 100kB |
243 |
91 |
6693 |
2124 |
4569 |
# Files < 10kB |
6 |
13 |
635 |
156 |
479 |
Ave Filesize (GB) |
2.30 |
2.69 |
0.0946 |
0.195 |
0.00959 |
% space used by files >
1GB |
96.71 |
79.73 |
64.56 |
|
|
|
|
|
|
|
|
Now what I find interesting is how similar values LHCb and CMS are with each other, even though they are vastly different VOs. What worries me is that over 50% of ATLAS files are less than 10MB. Now just to find a tier2 to do a similar analysis to see if it just a T1 issue.....
No comments:
Post a Comment