Musings on data confidentiality

Recently I was asked whether STFC should store classified data, such as Secret data (being a gov't facility, all our data is already Official).

If you look at a "normal" data centre like those run by the big cloud providers, they are typically set up to ensure data confidentiality. They have special personnel who are authorised to enter the data centre, and they have all sorts of physical security measures. If they store Secret data they will need clearance.

We have security measures, but we also take visitors round our data centre and if they are monitored all the time it is more for their own health and safety than because we don't trust them. They can take pictures if they like. Of course we would very much like them to not press any buttons but that's also why there's someone with them.We have students who come and work with us also in the data centre, and leave feeling they have made real contributions.

The three basic data security goals are confidentiality, integrity, and availability, and all three are of course important. A "conventional" data centre would probably prioritise confidentiality first, then integrity, and finally availability: it is better that data is temporarily inaccessible than leaked. RAL's data centre on the other hand is different: for us integrity is top - we spend a lot of time checksumming files at rest and in flight, and comparing lists of files with other lists, data volumes with data volumes. Availability is also highly important as science data is collected, transmitted, and processed around the clock. And then in a sense confidentiality is last: for example, hardly anything is encrypted in flight because it would just slow transfers down. Of course we still need to protect scientists' data because "there is definitely a Nobel prize in there!" but our data is not national security, nor even personal/medical data. Yes, of course we protect the science data, but there something to be said for openness too - making open data available, and showing the public some of the good stuff we do. And it would be quite costly to protect against a "highly capable threat," money which is better spent making things go faster. Leave other data centres to guard the national secrets.

ARGUS user suspension with DPM

Many grid services that need to authenticate their users do so with LCAS/LCMAPS plugins, making integration with a site central authentication server such as ARGUS relatively straightforward. With the ARGUS client LCAS/LCMAPS plugins configured, all authentication decisions are referred to the central service at the time they're made. When the site ARGUS is configured to use the EGI/NGI emergency user suspension policies, any centrally suspended user DN will be automatically blocked from accessing the site's services.

However, DPM does it's own authentication and maintains its own list of banned DNs, so rather than referring each decision to the site ARGUS, we need a specific tool to update DPM's view based on the site ARGUS server. Just to complicate matters further, DPM's packages live in the Fedora EPEL repository, which means that they cannot depend on the ARGUS client libraries, which do not.

The solution is the very small 'dpm-argus' package which is available from the EMI3 repositories for both SL5 and SL6; a package dependency bug has prevented its installation in the past, but this has been fixed as of EMI3 Update 19. It should be installed on the DPM head node (if installing manually rather than with yum, you'll also need the argus-pep-api-c package from EMI) and contains two files, the 'dpns-arguspoll' binary, and its manual page.

Running the tool is simple - it needs a 'resource string' to identify itself to the ARGUS server (for normal purposes it doesn't actually matter what it is) and the URL for the site ARGUS:

dpns-arguspoll my_resource_id https://argus.example.org:8154/authz

when run, it will iterate over the DNs known to the DPM, check each one against the ARGUS server, and update the DPM banning state accordingly. All that remains is to run it periodically. At Oxford we have an '/etc/cron.hourly/dpm-argus' script that simply looks like this:

#!/bin/sh
# Sync DPM's internal user banning states from argus

export DPNS_HOST=t2se01.physics.ox.ac.uk
dpns-arguspoll dpm_argleflargle https://t2argus04.physics.ox.ac.uk:8154/authz 2>/dev/null

And that's it. If you want to be able to see the current list of DNs that your DPM server considers to be banned, then you can query the head node database directly:

echo "SELECT username from Cns_userinfo WHERE banned = 1;" | mysql -u dpminfo -p cns_db

At the moment that should show you my test DN, and probably nothing else.

DPM vulnerability

Another security hole in the DPM gridftp server has been found and subsequently patched.

All details of the update can be found here.

The security advisory issued by the GSVG can be found here.

All DPM sites should use YAIM (or your method of choice) to upgrade to the latest version (DPM-gridftp-server-1.6.5-6) ASAP. Depending on how regularly you have been updating, there may also be new rpms available for other components of the DPM (all of these are on 1.6.5-5).

DPM gridftp security

Apologies for not posting for a while, it's been a busy few weeks. First thing that should be mentioned is the gaping security hole that existed in the DPM gridftp server. Users using the uberftp (or some other suitable) client could log into the server and change permissions on anyones files, move files to different areas of the DPM namespace or even move files outside of the namespace altogether. Thanks to Kostas and Olivier at Imperial for spotting this. Unfortunately, it took a couple of weeks, 3 patch releases and a lot of testing within GridPP before we finally plugged the hole.

Initially only patched version of the 1.6.5 server was produced. I asked for the fix to be back-ported to 1.5.10 as there were a few sites still running this version, unable to upgrade to the latest release (due to the upgrade problems) due to ongoing experiment tests and wanting to be as secure as they could be. This was done, so thanks to the DPM team.

All sites should upgrade to the latest version of DPM and ensure that they are running patch -4 of the gridftp server.

Storage security service challenge

WLCG are asking each ROC to run a security service challenge their sites. Someone (probably Alessandra) will submit a job to each site which will attempt to write a file to the local SE, read it back, copy it to a remote SE, delete the file... Once complete, the submitter will issue a GGUS ticket against the site, asking them to provide information on which operations were performed on the file. You can see an example of what is expected here:

https://gus.fzk.de/pages/ticket_details.php?ticket=22012

The aim of this testing is to determine if SEs record sufficient information for tracing user operations and also to check that site admins are able to gather that information. I am currently putting together some scripts that will perform the querying and parsing of DPM/dCache databases and log files in order to gather the information.

In addition to going through the SE files, it is likely that sites will have to parse the PBS logs on the CE to determine the UI that was originally used for the job submission.