Compute Cloud User Guide
Accessing the ARCS Compute Cloud
Access the ARCS Compute Cloud Using Your Web Browser
The ARCS Compute Cloud offers researchers access to high performance computing (HPC) resources through an easy-to-use Web interface.
Accessing the ARCS Compute Cloud
All services offered by ARCS are centrally accessible through the front page of the ARCS website. To access any of the services, navigate to http://www.arcs.org.au and click on the “Services Login” button.
Follow the instructions on the Services Login screen to either authenticate using your Australian Access Federation (AAF) identity provided by your institution or to apply for an AAF identity provided by ARCS.
Access the ARCS Compute Cloud Using Your Web Browser
After successfully authenticating to ARCS, you will see a menu of buttons that will give you access to all ARCS services available or allow you to request a service to be set up for you. For more information on any service, click the green “i” next to each service. To access the ARCS Compute Cloud, click the “Compute Cloud” button. Please note that if you are accessing ARCS services for the first time, it may take up to five minutes for your account on the ARCS Compute Cloud to be set up. You will be notified by email once your account has been successfully activated.
The ARCS Compute Cloud will open in a new window.
When you log in to the Compute Cloud, you will be presented with a row of options to manage your jobs:
-
My Job List – List of jobs you have run or are currently running.
-
New Job – Job submission front end with a range of pre-defined job templates.
-
Data Fabric – Portal for accessing files in the ARCS Data Fabric, Australia’s national file system for sharing data across institutional and regional boundaries.
-
Resources – A list of current HPC resources to which you can submit jobs.
-
Advanced Tools – A list of other tools available for assisting in job submission (e.g., MyProxy login).
You will also see a table called “My Job List” that as the name suggests lists your compute jobs.
Job List
This table displays a detailed list of your jobs submitted to the ARCS Compute Cloud. Each entry lists the job name, type, application used, creation time, actual start time, finish time, queue submitted to and the current status.
Navigation options for your jobs lists are available at the bottom of the display pane. Of particular importance is the refresh button, which is used to update the status of your jobs.
Action Buttons
Above the job list table, you will see a series of action buttons. You can select one or more jobs by clicking the check box to the left of the job name and perform the following actions on those jobs:
- Resume - This action will resubmit the selected jobs if you previously killed them.
- Kill Jobs - This will terminate all selected job submissions. The selected entries will still remain in your job list.
- Delete Jobs - This will remove the selected jobs from your job list.
If you select a job from the job list table by clicking on the box to the left of a row, you will see a series of tabs appear below the table with detailed information about the selected job.
Job Properties
This tab displays a range of detailed parameters relating to the currently highlighted job entry. This information includes details such as resources allocated, additional arguments, and site and queue information. There is also a link to the Data Fabric folder that contains the output files for this job.
Log Messages
This tab displays a detailed log of the job submission and execution for the currently highlighted job entry. This information can be useful in determining what went wrong with a failed job submission.
Stage-in List
This tab displays a list of all input files that were staged in for the currently highlighted job entry.
Candidate Queues
This tab displays a range of details for the potential sites and queues to which the currently highlighted job may have been submitted. This information also includes any preferred sites for the given application as well as the selected site at which the job will run.
Each queue possesses a ranking that determines how suitable that queue is for the given job. These rankings are calculated dynamically based on metrics such as free jobs slots, free memory, application requirements, etc.
Files
This tab displays a list of all files generated by the job. Here you can preview or download the output files that have been generated. If you select a job from the job list table and then select the Files tab, you will see a list of output files relating to that job. There are two links to the right of each file – a Preview and a Download link. If you click Preview, the file will be displayed in a separate window. Currently, text files and image files are supported in Preview mode. If you click Download, you are prompted to either save the file locally or open it with an application (e.g., a PDF viewer if the file is in pdf format).
To submit a job to the Compute Cloud, click “New Job” on the top menu.
Selecting an Application
On the left side of the screen you will see a list of applications currently available on the Compute Cloud. Click on the application you would like to run and the job submission parameters will change as required. If no template exists for your job, you can select the generic template that allows you to simply enter a command line string as you would from a general Unix terminal.
File System Browser
On the right hand side of the screen you will see a drop-down listbox that allows you to browse your Data Fabric home directory or your Compute Cloud output directory, which also resides on the Data Fabric. You can use either of these areas as the location for input files for your compute jobs. If you select one of these locations, a file and folder list will appear below the listbox. You can navigate folders by clicking on the small arrow to the left of a folder.
Job Templates
Although each job template is different since it is designed for a specific application, there are some common elements across each template that are important.
Walltime
This is the expected time that your job will take to run. Input the amount of time appropriate for your given job, or accept the default.,
Job Name
This is the name of your job. This must be unique to any job currently in your job list. You can enter a specific name for a job or accept the default.
Advanced Options
Clicking this button displays information regarding the current VO (virtual organisation) being used (the default is ACC) and the available queues to run your job. Only those sites that match your job requirements are shown.
By default, the Compute Cloud will select the best site for you based on several metrics. You can, however, elect to run your job at a specific site in the displayed list. Simply select your desired site and submit your job.
Example
The following demonstration is a basic submission of a POV-Ray rendering job to the ARCS Compute Cloud.
While the job submission itself is rather trivial, it demonstrates the most common features that you will use when submitting your own job to the ARCS Compute Cloud.
Login to the Compute Cloud
- Go to http://www.arcs.org.au and click the “Services Login” button.
- Follow the instructions on the Services Login screen to authenticate using your institution's AAF Identity Provider. If your institution's Identity Provider is not listed, you can register here for an ARCS AAF identity.
Fill in Job Parameters and Template Information
- Once logged in, select “New Job” from the top menu. Select the POV-Ray template from the left-hand side menu.
- Fill in the parameters as follows:
- Image Width - Set to 1000.
- Image Height - Set to 1000.
- Walltime - Set to 10 minutes.
- Jobname - You can leave this at the default value or change it to something else (e.g., example_povray_job_001).
Load the Input File
There are two sources that you can load input files from. The first is your own PC from which you are submitting the job. The second is via the ARCS Data Fabric.
For this example we will load the input file from the Data Fabric. In the File System Browser drop-down listbox on the right-hand side of the screen, select Data Fabric Home. Click on the small arrow to the left of Cloud_Input_Examples. Select the file named POV-Ray_example.pov and then click the “Retrieve selected file from the tree” button to the right of the Input File field.
If you had an input file on your local computer, you could instead select the Browse button next to the Input File field and navigate to the folder on your computer where the input file resides, then upload it.
Advanced Options and Job Submission
- Click “Submit” and your job will be submitted to the most appropriate site for processing. You can track your job progress under the “My Job List” pane.
- OPTIONAL: You can click the “Advanced Options” button to view a range of sites to which you can elect to submit your job. Select any site you wish. Only those sites that have POV-Ray available will be visible.
Retrieve/Review the Output
- Click the “My Job List” button. You should now see the job you just submitted listed below. If the status of the job is listed as anything but “finished,” simply click the refresh button to update the display. The job should take no longer then 10 minutes to complete.
- Once the status is listed as “finished,” click the job entry in the list below and you should see a series of tabs below the job list table. Select the Files tab.
- You should now see several files. The one you are interested in is called POV-Ray_example.png. Select Preview to preview this file. If all went well you should see the following image as the end result of your job.
Data Fabric Integration
You can access the Data Fabric directly by selecting “Data Fabric” from the top menu. You will see the same view of the Data Fabric as you would if you had launched it directly from the ARCS website. You can now perform all the familiar Data Fabric functions such as moving files to other folders, sharing files with others and using the QuickShare feature.
Input filenames that contain spaces cause problems during the “stage in” process.
Internet Explorer 9 causes problems with the display of the Grisu interface. If possible, avoid using IE 9 in this context; IE 8 appears to be OK.