ARCS Data Fabric - Easy to use virtual storage for researchers integrating with ARCS services.
Overview
The ARCS Data Fabric is an easy to use way for researchers or groups of researchers (virtual organisations or VOs) to store their data. It has features useful for data collaboration and eResearch.
Here a user refers to an individual researcher or a VO of researchers.
- Users will be provided with their own virtual permanent storage on request.
- There will be a quota applied to the storage.
- A user does not necessarily need to know the physical location of their data but they will be able to find out. This is what is meant by "virtual storage".
- A user (or VO) may negotiate or pay for more space than the default amount. ARCS will facilitate this. The extra space will effectively be an increase in the user's quota without them needing to know where it is located.
- A user (or VO) may also negotiate with the storage provider about the lifetime of their permanent storage. ARCS will facilitate this also. If there is no such negotiation the storage may be reclaimed if it appears the user is no longer using it.
- There will be an easy to use desktop graphical interface to this storage as well as command line tools and a web interface.
- Where necessary users may work with ARCS to replicate or move their data to a certain physical location if data locality is important to their needs.
- It will be possible to cache replicas, temporarily, of user's data in locations close to compute resources to improve the performance of compute workflows.
- The data fabric will integrate well with other ARCS services such as compute job submission.
- There will be close cooperation between ARCS and ANDS (Australian National Data Service) so that the ARCS data fabric will integrate well with services provided by ANDS.
- Examples demonstrating how the data fabric can be used for several common use cases will be available on the ARCS web site.
Status
At the moment a prototype of the Data Fabric is undergoing testing and analysis. Early adopters will be invited to use it from July 2008.
Examples using existing ARCS data services
Things you might want to do with existing ARCS data services and examples on how you might do them. Examples for the proposed data fabric will be added when the service is available to users.
Implementation of the Data Fabric
A description of how the ARCS Data Fabric is being developed and implemented.