RESOURCES-Cross Campus Grid (XCG)

Cross Campus Grid (XCG)

If you have the need for faster computation or the need to securely and easily share data with colleagues, we invite you to use the cross campus grid. The objectives of the cross campus grid are to:

  • provide computational resources for science and engineering research
  • provide a secure data sharing platform for members of the University community and their colleagues at other institutions, and
  • serve as a test-bed for emerging Grid standards

The Cross Campus Grid is implemented using Genesis II, which was developed here at the University of Virginia. Genesis II is the first integrated implementation of the standards and profiles coming out of the Open Grid Forum (OGF) Open Grid Service Architecture (OGSA) Working Group. The OGF is the standards organization for Grids. Genesis II is a complete set of Grid services for users and applications which not only follows our maxim “by default the user should not have to think”, but is also a from-scratch implementation of the standards and profiles, not a wrapping of existing artifacts. Genesis II is open source under the Apache license.

What can I do with XCG?

There are basically two things you can do with the Grounds Grid: you can use it to access Grid compute resources or to export and access data

Need Speed?

Suppose you have an application that you need to run for your research, and that you need to execute the application many times; for example many executions with different parameters or input files, or simply a large number of times to establish a statistical property. In this case, you have a high-throughput problem. If each execution takes many minutes or hours, and you have hundreds or thousands of jobs, this could take days or weeks to run on your desktop. The Grounds Grid compute queues can be used to solve this sort of problem. Descriptions of the jobs to be executed are submitted to the Grounds Grid, and the jobs are distributed throughout the Grid and executed concurrently. Data management and movement in/out of your local compute environment is managed by the Grounds Grid. The result is you get your results tens to hundreds of times faster.

Similarly, if you already have an MPI application you can use the Grounds Grid to select a cluster for your job and run it.

Need to Share Data?

Suppose you are working on a project with a colleague in another department or at another institution, and having direct access to each others files would accelerate your research. For example, you have a colleague with an instrument that writes its data directly onto a hard disk in their lab, and you would like to be able to read those files as if they were local to your machine. You may perhaps post-process those results and store them locally, and your colleagues may in turn want to be able to read, modify, or view the results. This is a data sharing problem.

To solve this sort of problem using the Grounds Grid, the person who wants to share their data “exports” the data into the Grounds Grid, and the person(s) who want to access that data mount the Grid into their local Linux or Windows file system and then access the data as if it were in a local file system. Data access is fully secure using the latest Web Services security standards.

What does it mean to “share” or “export” data? The basic idea is really simple. Suppose you have a directory (also called a “folder” in Windows) on your hard drive that you want to share with a friend or colleague. You can use the Genesis II export Grid Shell command. Invoked without any command-line options, the export command starts up a simple GUI (which requires X-windows in Linux environments), as shown below. The GUI allows you to specify a local directory path that you want to export, and the directory path where you want to place it in the Grounds Grid name space. The process is similar to mapping/mounting a drive, except that you are mounting a portion of your local file system into the Grid namespace.

Once exported, the directory and its contents can be manipulated both via local users and via Grid users. Updates made by local users are visible by remote users, and vice versa. Access by remote users is governed by access-control lists that you establish. The identity(ies) of the user who performed the export are given initial, exclusive Grid access to the exported resources. After performing an export, you will typically modify the access-control policies for the newly-exported resources to specify which users and groups can read, write, and execute (i.e., make subdirectories) these exported resources.

 

Return to Computing