One of the aims of GRIDportal is indeed to allow the user to submit jobs to grids other than the local one (in fact NorduGrid itself should decide to which grid the job is assigned transparently), so the use of runtime environments is a key issue and should be exploited by GRIDportal.
Jobs submitted through GRIDportal are submitted to the NorduGrid network. In the most basic scenario, all jobs are executed on the local grid (if there is one). Otherwise, GRIDportal can also allow a user to submit jobs to any other grid within Nordugrid. But a job is a set of instructions meant for a specific application, so in order to keep track of which grid site has which applications, NorduGrid governs this dilemma by the use of runtime environments.
A runtime environment is quite simply a shell script written for a specific application (and its version), which sets up the necessary environmental variables (such as the path to the executable, libraries etc), allowing the execution of jobs for that particular application. When a user submits a job to NorduGrid, the job definition may include a string in the form:
(runTimeEnvironment=APPS/MATH/MATLAB-7.0.0-1.0)This defines a runtime environment for Matlab version 7.0, whereas the value 1.0 indicates this is version 1.0 of the runtime environment. This job will be accepted by a grid advertising this exact runtime environment and rejected by those which do not.
More background on runtime environments:
+ Runtime Environment Registry
While some applications are well suited for a runtime environment definition, BLAST is not. The rationale behind a runtime environment is to pinpoint which application is requested and which version of the application is needed. But that still does not define a BLAST deployment adequately. In order to use BLAST one needs to install not only the application but also one or more databases to search in. A BLAST job requesting a database which is not found on the grid where the job is executed will fail. Meanwhile, there is no logic in the current format of runtime environments to declare which databases are present on the grid.
The current format of runtime environments does not enable an unambiguous definition of a BLAST runtime environment. This in fact means that BLAST jobs can only be executed on grids which are "manually" synchronized between grid administrators (that is, an agreement reached about which databases are installed).
Unfortunately, mpiBLAST offers no solution to this problem, in fact it adds to it. All of the above holds true for mpiBLAST, but in addition, mpiBLAST divides each database into a number of segments (with mpiformatdb), which are then distributed onto the nodes which are chosen to execute the job. This "segmentation" is considered to be static (ie. it is only done upon database installation, not on every job run), because it takes quite a long time. This effectively means that every mpiblast job should be ran with the same number of nodes as there are segments.
There is a loose correlation between the number of nodes selected in the job definition (the xrsl file),
(count=20)and the arguments passed to mpirun, set with the count parameter,
mpirun -np 20 mpiblast -p blastp -d nr -i infile -o outfile(to run on 20 nodes). This declaration does not directly assign a number of nodes, it simply states that the job is executed in 20 parallell sets, whether that translates into one node for each set or something completely different depends on the underlying job scheduler.
The local cluster has 3 databases installed, each of them is divided into segments depending on its size:
It is fully possible to write logic to handle this scenario and create job definitions thereafter. But there is no way to pass this state onto the runtime environment, thus local jobs cannot be run on remote grids.
Adding to the problems of BLAST, mpiBLAST introduces problems of its own. The decisions on segmentation could be streamlined but it remains a decision of the grid administrator of how to best utilize the given resources, it should not be overridden from an outside body. In effect, this leaves several issues unresolved and renders an mpiBLAST runtime environment definition impossible.