RAL Castor issues also affecting LHCb

The Oracle problems that heve been an issue with the CASTOR ATLAS system at RAL also appear to be affecting LHCb according to an email from Bonny this morning. Problems started at 2008-08-20 10:56.

dpm-drain

Just a heads up for any DPM users - We might have discovered an issue with the draining files from a pool. If for any reason dpm thinks there's no allocatable space (not the same thing as Free space - its assuming space reservations are going to be filled) then it may move the data out of that pool and into another (such as generalPool).

Even worse it may remove those files out of a spacetoken.

I believe the problem you've run into is that dpm-drain is moving the files out of their space and putting the new copy in no particular space. The ATLAS pool would be preferentially chosen for ATLAS files but the free (non space reservation) portion has been exhausted, so it falls backs to the general pool.

I shall update when I have more information. (needless to say we stopped draining uki-scotgrid-glasgow pools this morning)

See Bug #40273

GridKa school of computing 2008

I have been asked to present a talk titles "The Importance of Data Storage" at this years GridKa school of computing. In particular, I have been asked to discuss issues like:

* computing and data storage: differences in the challenges.
* several examples of communities needing large amounts of data on the grid (if possible, communities having different access models).
* usage of different storage technologies (disk, tape, solid state,...).
* different tools for storing huge amounts of data.

If anyone has any pertinent ideas then I'd like to know about them, particularly when it comes to non-HEP users of Grid storage.

Cheers,
Greig

GridPP DPM toolkit 1.3.0 released

I meant to post about this last week, as I have released a new version of my DPM admin toolkit. This release includes a new tool called gridpp_dpm_dpns_du. This reports on the usage of directories in the DPNS namespace. This tool is something that CMS people at Bristol were asking for such that users could manage their own space.

$ gridpp_dpm_dpns_du -h
usage: gridpp_dpm_dpns_du /dpm/path/to/directory

options:
-h, --help show this help message and exit
-s, --si Use powers of 1000, not 1024.
-xEXCLUDE, --exclude=EXCLUDE
Directory to ignore.

One thing that I have noticed is that the tool sometimes claims that a directory contains (say) 914354654K. However, when you dpns-ls -l the directory it does not contain any files or sub-dirs. Deeper investigation using dpns-ls --delete shows that DPM still has some remnant of an old file replica which was previously in that directory, but has now been marked as "D" for deleted. If there are no problems, these files can be removed with rfrm. Let me know if you find any other bugs.

Note that the rpm depends on DPM-interfaces > 1.6.11-4 due to the inclusion of the dpm-listspaces tool (which needs the latest API). You can find the latest rpm here.