Data Examples
Data Examples
This page is under development
| I want to be able to stage my data in and out of a compute job on the grid. | If you intend to submit compute jobs using the common grid technology Globus you will probably want to make your input data available on a grid file server called a GridFTP server. Similarly your job's output can be sent to a GridFTP server as well. | See here for a description of GridFTP, tools you can use to transfer data using GridFTP and a list of GridFTP servers. |
| I have an instrument writing data to a hard disk. I want to move that data regularly to a data store. | What you need is something that will reliably discover new data from the instrument and transfer it to the data store. | A solution for this problem was developed for the Skymapper project using the Lightweight Data Replicator software. Other options are being investigated by ARCS including the use of the Globus Data Replication Service - which is developed to be a generalisation of LDR |
| I want to replicate a data set to one or more other places. | There are several ways to do this but it is actually a general solution of the question above. | At the moment we suggest you use LDR. If this doesn't suit or you have any other questions please contact help@arcs.org.au |
| I want to store some data with its metadata so that I can search on the metadata. | Once again there are several ways to do this. | We suggest you use SRB (Storage Resource Broker). It not only handles your metadata but can hide away from you the details of where your data is stored. |