25 February 2014

Big data picture

Not as in ((big data) picture) but (big (data picture)), if that makes sense.

I find myself in Edinburgh - it's been far too long since I last was here, I am embarrassed to say.

We are looking at data movement for EUDAT and PRACE, and by a natural extension (being at EPCC), GridPP and DiRAC. The main common data mover is GridFTP: useful because we can (more or less) all move data with GridFTP, it gets great performance, we know how to tune and monitor it, and it supports third party copying. We also need to see how to bridge in GlobusOnline, with the new delegated credentials. In fact both Contrail and NCSA developed OAuth-delegated certificates (and while the original CILogon work was OAuth1, the new stuff is OAuth2.)

One use case is data sharing (the link is to a nice little video Adam Carter from EPCC showed in the introduction). You might argue that users are not jumping up and down screaming for interdisciplinary collaborations, yet if they were possible they might happen! When data policies require data be made available, as a researcher producing data you really have no choice: your data must be shareable with other communities.

No comments: