FBSNG Application Programmer’s Interface

Reference Manual

Version 1.1

Jim Fromm, Krzysztof Genser, Tanya Levshina, Igor Mandrichenko

Farms and Clustered Systems Group

Fermi National Accelerator Laboratory

Table of Contents

1 Introduction. 5

2 FBSNG API Overview. 5

3 Using FBSNG API 8

4 Common Features of API Classes 9

4.1 Returning Status 9

4.2 Generated Exceptions 9

5 FBSClient Class 11

5.1 FBSClient Methods 11

Constructor FBSClient 11

submitJob. 11

getJobList 11

getJob. 12

killJob. 12

getSection. 13

getSectionOutput 13

holdSection. 14

releaseSection. 14

incSectPrio. 15

killSection. 15

getProcess 16

createQueue. 17

getQueueList 17

getQueue. 17

holdQueue. 18

releaseQueue. 18

lockQueue. 19

unlockQueue. 19

createGlobalResource. 20

setGlobalResource. 20

getGlobalRsrcList 21

createLocalResource. 21

getLocalRsrcList 22

getGblRsrcQuota. 22

getGblRsrcUsage. 23

getGlobalPoolList 23

getLclRsrcQuota. 24

getLclRsrcUsage. 24

createRsrcPool 25

setRsrcPool 26

getLocalPoolList 26

getResourcePool 27

createNodeClass 27

getNodeClassList 28

getNodeClass 28

getNode. 29

holdNode. 29

releaseNode. 30

getProcessTypeList 30

createProcessType. 30

getProcessType. 31

setTimeOut 31

6 FBSJobDesc Class 33

6.1 Methods 33

Constructor 33

getSection. 33

addSection. 34

hasSection. 34

sections 35

validateDependencies 35

__repr__. 35

7 FBSSectionDesc Class 37

7.1 Data Members 37

7.2 Methods 37

Constructor 37

clone. 40

__repr__. 40

8 FBSJobInfo Class 42

8.1 Data Members 42

8.2 Methods 42

sections 42

getSection. 42

kill 43

refresh. 43

9 FBSSectionInfo Class 45

9.1 Data Members 45

9.2 Methods 46

getProcess 46

isHeld. 46

hold. 47

release. 47

incPrio. 48

kill 48

refresh. 49

10 FBSProcessInfo Class 50

10.1 Data Members 50

10.2 Methods 50

refresh. 50

11 FBSSubProcessInfo Class 52

11.1 Data Members 52

12 FBSNodeClassInfo Class 52

12.1 Data Members 52

12.2 Methods 52

refresh. 52

setRsrcCap. 53

setLocalDisks 54

addNode. 54

removeNode. 55

13 FBSNodeInfo Class 56

13.1 Data Members 56

13.2 Methods 56

refresh. 56

14 FBSQueueInfo Class 58

14.1 Data Members 58

14.2 Methods 58

hold. 58

release. 59

lock 59

unlock 60

update. 60

refresh. 61

15 FBSProcTypeInfo Class 63

15.1 Data Members 63

15.2 Methods 63

setSectRsrcDefs 63

setProcRsrcDefs 64

setRsrcQuota. 64

setMaxPrioInc. 65

refresh. 65

Appendix A: JDF Format 67

Appendix B: Hold Time Representation. 70

Appendix C: More API Examples 71

Appendix D: Glossary of Terms 77

1 Introduction

This document is a part of set of the FBSNG documentation. It describes functionality of Python version of FBSNG Application Programmer’s Interface (API). Description of FBSNG features is left beyond the scope of this document. Other parts of documentation should be consulted for more information on specific FBSNG features:

• FBSNG User’s Guide describes FBSNG command line interface as well as basic features;
• FBSNG Resources Concepts document describes the concepts of FBSNG resource management;
• FBSNG Scheduler document is about scheduling algorithms and related configuration issues;
• FBSNG Installation and Administration Guide describes installation and support procedures.

2 FBSNG API Overview

The FBSNG API provides access to FBSNG run-time and configuration information. It is organized as a set of classes divided into the following major groups:

• API front-end FBSClient class. This class performs all communication transactions between API client and FBSNG components. In order to use FBSNG API, user application must create an object of this class. An FBSClient object provides methods for:
• Job submission, monitoring and control
• Farm resource utilization monitoring
• Queue monitoring and control
• Monitoring and control of farm nodes
• Obtaining farm configuration information
• Dynamic modification of farm configuration

FBSClient object creates objects of “informational” API classes such as FBSJobInfo, FBSSectionInfo, FBSProcessInfo, etc.

• Job construction and submission:
• FBSJobDesc – representation of an FBSNG job to be submitted. Each FBSJobDesc object is a container of one or more FBSSectionDesc objects.
• FBSSectionDesc – representation of individual FBSNG job section as a part of a job to be submitted to FBSNG.

In order to submit a job using API, user must

1. Create an FBSClient object
2. Create FBSJobDesc object and populate it with one or more FBSSectionDesc objects either using existing Job Description File (see Appendix A) or creating FBSSectionDesc objects and adding them to the FBSJobDesc object.
3. Use FBSClient object to send FBSJobDesc object for submission.

For more information, refer to FBSJobDesc and FBSSectionDesc sections of this document.

• Run-time batch job and farm status monitoring. The following “informational” classes provide run-time information about batch jobs, farm resource utilization and FBSNG configuration.
• FBSJobInfo – job monitoring and control;
• FBSSectionInfo – monitoring and control of individual job section;
• FBSProcessInfo and FBSSubProcessInfo – monitoring of individual batch processes and their subprocesses;
• FBSQueueInfo – FBSNG queue monitoring, control and configuration;
• FBSNodeClassInfo – farm node class configuration and monitoring;
• FBSNodeInfo – individual farm node status monitoring and control;
• FBSProcTypeInfo – configuration and run-time information about a process type.

Objects of all informational classes are created by FBSClient object or objects of other FBSNG API classes. API client application should not create objects of these classes directly.

Currently, only Python binding of FBSNG API is available.

Relationships between different API classes are shown on Fig. 1.

Figure 1. FBSNG API Classes

3 Using FBSNG API

In order to use FBSNG API, client application must import all or only necessary names from FBS_API module using one of the following methods:

from FBS_API import * # import all names

from FBS_API import FBSClient, FBSJobDesc # import some names

import FBS_API # import only the module

If first method is used, the following names will be imported: FBSClient, FBSJobDesc, FBSSectionDesc and FBSError. If third method is used, class names will have to be explicitly qualified with module name “FBS_API”. All examples of Python code in this document are written assuming first or second method was used.

Actual location of FBS_API.py module must be included into Python library search path defined by PYTHONPATH. FBSNG comes with necessary user environment set-up scripts that assign correct value to this and other environment variables. UPS users can use FBSNG API after issuing “setup fbsng” command. See FBSNG User’s Guide for more details on setting up user environment.

4 Common Features of API Classes

4.1 Returning Status

Many API methods return completion status information as a 2-tuple (status, reason). By convention, status 1 indicates successful completion, and status 0 indicates failure. On success, the reason field is either ‘OK’ or some warning message. On failure, the reason field of the tuple contains a specific explanation of what happened.

For example:

# Cancel or kill a section

from FBS_API import FBSClient

fc=FBSClient()

si = fc.getSection(‘123.MAIN’)

sts, reason = si.kill()

if sts:

if reason == ‘OK’:

print ‘Done’

else:

print ‘Warning: ‘, reason

else:

print ‘Failed: ‘, reason

4.2 Generated Exceptions

Almost all methods of API classes described below generate standard Python KeyError