FBSNG Application Programmer’s Interface

Reference Manual

Version 1.2

Jim Fromm, Krzysztof Genser, Tanya Levshina, Igor Mandrichenko

Farms and Clustered Systems Group

Fermi National Accelerator Laboratory

Table of Contents

1 Introduction. 5

2 FBSNG API Overview. 5

3 Using FBSNG API 8

4 Common Features of API Classes 9

4.1 Returning Status 9

4.2 Generated Exceptions 9

5 FBSClient Class 11

5.1 FBSClient Methods 11

Constructor FBSClient 11

submitJob. 11

getJobList 11

getJob. 12

killJob. 12

getSection. 13

getSectionOutput 13

holdSection. 14

releaseSection. 14

incSectPrio. 15

killSection. 15

getProcess 16

createQueue. 17

getQueueList 17

getQueue. 17

holdQueue. 18

releaseQueue. 18

lockQueue. 19

unlockQueue. 19

removeQueue. 20

createGlobalResource. 20

setGlobalResource. 21

getGlobalRsrcList 21

getGblRsrcQuota. 22

getGblRsrcUsage. 22

removeGlobalResource. 23

createLocalResource. 23

getLocalRsrcList 24

getLclRsrcQuota. 24

getLclRsrcUsage. 25

removeLocalResource. 26

createRsrcPool 26

setRsrcPool 27

getLocalPoolList 27

getGlobalPoolList 28

getResourcePool 28

removeResourcePool 29

createNodeClass 29

getNodeClassList 30

getNodeClass 30

removeNodeClass 31

getNode. 31

holdNode. 32

releaseNode. 32

getProcessTypeList 33

createProcessType. 33

getProcessType. 34

removeProcessType. 34

setTimeOut 35

6 FBSJobDesc Class 36

6.1 Methods 36

Constructor 36

getSection. 36

addSection. 37

hasSection. 37

sections 38

validateDependencies 38

__repr__. 38

7 FBSSectionDesc Class 40

7.1 Data Members 40

7.2 Methods 41

Constructor 41

clone. 43

__repr__. 43

8 FBSJobInfo Class 45

8.1 Data Members 45

8.2 Methods 45

sections 45

getSection. 45

kill 46

refresh. 46

9 FBSSectionInfo Class 48

9.1 Data Members 48

9.2 Methods 49

getProcess 49

isHeld. 49

hold. 50

release. 50

incPrio. 51

kill 51

refresh. 52

10 FBSProcessInfo Class 53

10.1 Data Members 53

10.2 Methods 54

refresh. 54

11 FBSSubProcessInfo Class 55

11.1 Data Members 55

12 FBSNodeClassInfo Class 55

12.1 Data Members 55

12.2 Methods 55

refresh. 55

setRsrcCap. 56

setLocalDisks 57

addNode. 57

removeNode. 58

13 FBSNodeInfo Class 59

13.1 Data Members 59

13.2 Methods 59

refresh. 59

14 FBSQueueInfo Class 61

14.1 Data Members 61

14.2 Methods 61

hold. 61

release. 62

lock 62

unlock 63

update. 63

refresh. 64

15 FBSProcTypeInfo Class 66

15.1 Data Members 66

15.2 Methods 66

setSectRsrcDefs 66

setProcRsrcDefs 67

setRsrcQuota. 67

setMaxPrioInc. 68

refresh. 68

Appendix A: JDF Format 70

Appendix B: Hold Time Representation. 73

Appendix C: More API Examples 74

Appendix D: Glossary of Terms 80

1 Introduction

This document is a part of set of the FBSNG documentation. It describes functionality of Python version of FBSNG Application Programmer’s Interface (API). Description of FBSNG features is left beyond the scope of this document. Other parts of documentation should be consulted for more information on specific FBSNG features:

• FBSNG User’s Guide describes FBSNG command line interface as well as basic features;
• FBSNG Resources Concepts document describes the concepts of FBSNG resource management;
• FBSNG Scheduler document is about scheduling algorithms and related configuration issues;
• FBSNG Installation and Administration Guide describes installation and support procedures.

2 FBSNG API Overview

The FBSNG API provides access to FBSNG run-time and configuration information. It is organized as a set of classes divided into the following major groups:

• API front-end FBSClient class. This class performs all communication transactions between API client and FBSNG components. In order to use FBSNG API, user application must create an object of this class. An FBSClient object provides methods for:
• Job submission, monitoring and control
• Farm resource utilization monitoring
• Queue monitoring and control
• Monitoring and control of farm nodes
• Obtaining farm configuration information
• Dynamic modification of farm configuration

FBSClient object creates objects of “informational” API classes such as FBSJobInfo, FBSSectionInfo, FBSProcessInfo, etc.

• Job construction and submission:
• FBSJobDesc – representation of an FBSNG job to be submitted. Each FBSJobDesc object is a container of one or more FBSSectionDesc objects.
• FBSSectionDesc – representation of individual FBSNG job section as a part of a job to be submitted to FBSNG.

In order to submit a job using API, user must

1. Create an FBSClient object
2. Create FBSJobDesc object and populate it with one or more FBSSectionDesc objects either using existing Job Description File (see Appendix A) or creating FBSSectionDesc objects and adding them to the FBSJobDesc object.
3. Use FBSClient object to send FBSJobDesc object for submission.

For more information, refer to FBSJobDesc and FBSSectionDesc sections of this document.

• Run-time batch job and farm status monitoring. The following “informational” classes provide run-time information about batch jobs, farm resource utilization and FBSNG configuration.
• FBSJobInfo – job monitoring and control;
• FBSSectionInfo – monitoring and control of individual job section;
• FBSProcessInfo and FBSSubProcessInfo – monitoring of individual batch processes and their subprocesses;
• FBSQueueInfo – FBSNG queue monitoring, control and configuration;
• FBSNodeClassInfo – farm node class configuration and monitoring;
• FBSNodeInfo – individual farm node status monitoring and control;
• FBSProcTypeInfo – configuration and run-time information about a process type.

Objects of all informational classes are created by FBSClient object or objects of other FBSNG API classes. API client application should not create objects of these classes directly.

Currently, only Python binding of FBSNG API is available.

Relationships between different API classes are shown on Fig. 1.

Figure 1. FBSNG API Classes

3 Using FBSNG API

In order to use FBSNG API, client application must import all or only necessary names from FBS_API module using one of the following methods:

from FBS_API import * # import all names

from FBS_API import FBSClient, FBSJobDesc # import some names

import FBS_API # import only the module

If first method is used, the following names will be imported: FBSClient, FBSJobDesc, FBSSectionDesc and FBSError. If third method is used, class names will have to be explicitly qualified with module name “FBS_API”. All examples of Python code in this document are written assuming first or second method was used.

Actual location of FBS_API.py module must be included into Python library search path defined by PYTHONPATH. FBSNG comes with necessary user environment set-up scripts that assign correct value to this and other environment variables. UPS users can use FBSNG API after issuing “setup fbsng” command. See FBSNG User’s Guide for more details on setting up user environment.

4 Common Features of API Classes

4.1 Returning Status

Many API methods return completion status information as a 2-tuple (status, reason). By convention, status 1 indicates successful completion, and status 0 indicates failure. On success, the reason field is either ‘OK’ or some warning message. On failure, the reason field of the tuple contains a specific explanation of what happened.