Of course, I am also interested to see what happens when I copy a file into castor.The floowing is an example of a similar 275Gb file being copied across the WAN then is written to tape. As you can see the intial write phase (~75MB/s) is lower than the rate for the file to be written to tape (~260MB/s) Copy across the network for similar size file and then uploading into Castor:
The end of the log file for this transfer is shown here:
N.B This transfer was using four concurrent stream within the gsiftp transfer.
It is (I think) interesting to look at the theoretical rate limit for transfers for a single stream between the two hosts in this transfer using some predictions; (from website https://www.switch.ch/network/tools/tcp_throughput/) :
So we may need to work on this...
The higher level monitoring worried me for this file until I realised the display options greatly effect the perceived rate. In this (in what might be an atypical) example; by solely changing the bin size I was able to change the perceived rate from 75MB/s to 460MB/s as seen in the two pictures below:
Now my site has the advantage (hindrance) of two storage systems using different hardware configurations and separate implementation of a gsiftp server; so I decided to see what rate I could get between the two.... And I managed over 100MB/s. This 100MB/s was for a 5 section poll time in the transfer as seen here:
What I also find interesting is that there seems to be a systematic difference between the average rate for transfers depending on transfer direction. (70-80MB/s one direction 90-95MB/s in reverse.) whether this difference is worth investigating is a question I will leave to the reader to decide. Also of interest may be to see what the effect of changing data transfer protocol has; but that it for another day...