The lsb.resources file contains configuration information for resource allocation limits, exports, resource usage limits, and guarantee policies. This file is optional.
The lsb.resources file is stored in the directory LSB_CONFDIR/cluster_name/configdir, where LSB_CONFDIR is defined in lsf.conf.
After making any changes to lsb.resources, run badmin reconfig to reconfigure mbatchd.
For limits to be enforced, jobs must specify rusage resource requirements (bsub -R or RES_REQ in lsb.queues).
The blimits command displays view current usage of resource allocation limits configured in Limit sections in lsb.resources:
Each set of limits is defined in a Limit section enclosed by Begin Limit and End Limit.
Vertical tabular
Horizontal
The file can contain sections in both formats. In either format, you must configure a limit for at least one consumer and one resource. The Limit section cannot be empty.
Use the vertical format for simple configuration conditions involving only a few consumers and resource limits.
SLOTS or SLOTS_PER_PROCESSOR
MEM (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
SWP (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
TMP (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
JOBS
RESOURCE
USERS or PER_USER
QUEUES or PER_QUEUE
HOSTS or PER_HOST
PROJECTS or PER_PROJECT
LIC_PROJECTS or PER_LIC_PROJECT
Multiple entries must be enclosed in parentheses. For RESOURCE limits, RESOURCE names must be enclosed in parentheses.
Use the horizontal format to give a name for your limits and to configure more complicated combinations of consumers and resource limits.
The first line of the Limit section gives the name of the limit configuration.
Job slots and per-processor job slots
Memory (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
Swap space (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
Tmp space (MB or unit set in LSF_UNIT_FOR_LIMITS in lsf.conf)
Running and suspended (RUN, SSUSP, USUSP) jobs
Other shared resources
Users and user groups
Hosts and host groups
Queues
Projects
Jobs from user1 and user3 are limited to 2 job slots on hostA
Jobs from user2 on queue normal are limited to 20 MB of memory or the unit set in LSF_UNIT_FOR_LIMITS in lsf.conf.
The short queue can have at most 200 running and suspended jobs
Begin Limit
NAME USERS QUEUES HOSTS SLOTS MEM SWP TMP JOBS
limit1 (user1 user3) - hostA 2 - - - -
- user2 normal - - 20 - - -
- - short - - - - - 200
End Limit
Jobs that do not match these limits; that is, all users except user1 and user3 running jobs on hostA and all users except user2 submitting jobs to queue normal, have no limits.
Begin Limit
# ugroup1 except user1 uses queue1 and queue2 with 2 job slots
# on each host in hgroup1
NAME = limit1
# Resources
SLOTS_PER_PROCESSOR = 2
#Consumers
QUEUES = queue1 queue2
USERS = ugroup1 ~user1
PER_HOST = hgroup1
End Limit
The Limit section of lsb.resources does not support the keywords or format used in lsb.users, lsb.hosts, and lsb.queues. However, your existing job slot limit configuration in these files will continue to apply.
Job slot limits are the only type of limit you can configure in lsb.users, lsb.hosts, and lsb.queues. You cannot configure limits for user groups, host groups and projects in lsb.users, lsb.hosts, and lsb.queues. You should not configure any new resource allocation limits in lsb.users, lsb.hosts, and lsb.queues. Use lsb.resources to configure all new resource allocation limits, including job slot limits. Limits on running and suspended jobs can only be set in lsb.resources.
Existing limits in lsb.users, lsb.hosts, and lsb.queues with the same scope as a new limit in lsb.resources, but with a different value are ignored. The value of the new limit in lsb.resources is used. Similar limits with different scope enforce the most restrictive limit.
HOSTS
JOBS
MEM
NAME
PER_HOST
PER_PROJECT
PER_QUEUE
PER_USER
PROJECTS
QUEUES
RESOURCE
SLOTS
SLOTS_PER_PROCESSOR
SWP
TMP
USERS
HOSTS=all [~]host_name ... | all [~]host_group ...
HOSTS
( [-] | all [~]host_name ... | all [~]host_group ... )
A space-separated list of hosts, host groups defined in lsb.hosts on which limits are enforced. Limits are enforced on all hosts or host groups listed.
If a group contains a subgroup, the limit also applies to each member in the subgroup recursively.
To specify a per-host limit, use the PER_HOST keyword. Do not configure HOSTS and PER_HOST limits in the same Limit section.
If you specify MEM, TMP, or SWP as a percentage, you must specify PER_HOST and list the hosts that the limit is to be enforced on. You cannot specify HOSTS.
In horizontal format, use only one HOSTS line per Limit section.
Use the keyword all to configure limits that apply to all hosts in a cluster.
Use the not operator (~) to exclude hosts from the all specification in the limit. This is useful if you have a large cluster but only want to exclude a few hosts from the limit definition.
In vertical tabular format, multiple host names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
all (limits are enforced on all hosts in the cluster).
HOSTS=Group1 ~hostA hostB hostC
Enforces limits on hostB, hostC, and all hosts in Group1 except for hostA.
HOSTS=all ~group2 ~hostA
Enforces limits on all hosts in the cluster, except for hostA and the hosts in group2.
HOSTS SWP (all ~hostK ~hostM) 10
Enforces a 10 MB (or the unit set in LSF_UNIT_FOR_LIMITS in lsf.conf) swap limit on all hosts in the cluster, except for hostK and hostM
JOBS=integer
JOBS
- | integer
Maximum number of running or suspended (RUN, SSUSP, USUSP) jobs available to resource consumers. Specify a positive integer greater than or equal 0. Job limits can be defined in both vertical and horizontal limit formats.
With MultiCluster resource lease model, this limit applies only to local hosts being used by the local cluster. The job limit for hosts exported to a remote cluster is determined by the host export policy, not by this parameter. The job limit for borrowed hosts is determined by the host export policy of the remote cluster.
If SLOTS are configured in the Limit section, the most restrictive limit is applied.
If HOSTS are configured in the Limit section, JOBS is the number of running and suspended jobs on a host. If preemptive scheduling is used, the suspended jobs are not counted against the job limit.
Use this parameter to prevent a host from being overloaded with too many jobs, and to maximize the throughput of a machine.
If only QUEUES are configured in the Limit section, JOBS is the maximum number of jobs that can run in the listed queues.
If only USERS are configured in the Limit section, JOBS is the maximum number of jobs that the users or user groups can run.
If only HOSTS are configured in the Limit section, JOBS is the maximum number of jobs that can run on the listed hosts.
If only PROJECTS are configured in the Limit section, JOBS is the maximum number of jobs that can run under the listed projects.
Use QUEUES or PER_QUEUE, USERS or PER_USER, HOSTS or PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit jobs available to resource consumers.
In horizontal format, use only one JOBS line per Limit section.
In vertical format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
No limit
JOBS=20
MEM=integer[%]
MEM
- | integer[%]
Maximum amount of memory available to resource consumers. Specify a value in MB or the unit set in LSF_UNIT_FOR_LIMITS in lsf.conf as a positive integer greater than or equal 0.
Without PER_HOST, or
With HOSTS
In horizontal format, use only one MEM line per Limit section.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
If only QUEUES are configured in the Limit section, MEM must be an integer value. MEM is the maximum amount of memory available to the listed queues.
If only USERS are configured in the Limit section, MEM must be an integer value. MEM is the maximum amount of memory that the users or user groups can use.
If only HOSTS are configured in the Limit section, MEM must be an integer value. It cannot be a percentage. MEM is the maximum amount of memory available to the listed hosts.
If only PROJECTS are configured in the Limit section, MEM must be an integer value. MEM is the maximum amount of memory available to the listed projects.
Use QUEUES or PER_QUEUE, USERS or PER_USER, HOSTS or PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit memory available to resource consumers.
No limit
MEM=20
NAME=limit_name
NAME
- | limit_name
Name of the Limit section
Specify any ASCII string 40 characters or less. You can use letters, digits, underscores (_) or dashes (-). You cannot use blank spaces.
If duplicate limit names are defined, the Limit section is ignored. If value of NAME is not defined in vertical format, or defined as (-), blimtis displays NONAMEnnn.
None. In horizontal format, you must provide a name for the Limit section. NAME is optional in the vertical format.
NAME=short_limits
PER_HOST=all [~]host_name ... | all [~]host_group ...
PER_HOST
( [-] | all [~]host_name ... | all [~]host_group ... )
A space-separated list of host or host groups defined in lsb.hosts on which limits are enforced. Limits are enforced on each host or individually to each host of the host group listed. If a group contains a subgroup, the limit also applies to each member in the subgroup recursively.
Do not configure PER_HOST and HOSTS limits in the same Limit section.
In horizontal format, use only one PER_HOST line per Limit section.
If you specify MEM, TMP, or SWP as a percentage, you must specify PER_HOST and list the hosts that the limit is to be enforced on. You cannot specify HOSTS.
Use the keyword all to configure limits that apply to each host in a cluster. If host groups are configured, the limit applies to each member of the host group, not the group as a whole.
Use the not operator (~) to exclude hosts or host groups from the all specification in the limit. This is useful if you have a large cluster but only want to exclude a few hosts from the limit definition.
In vertical tabular format, multiple host names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
None. If no limit is specified for PER_HOST or HOST, no limit is enforced on any host or host group.
PER_HOST=hostA hgroup1 ~hostC
PER_PROJECT=all [~]project_name ...
PER_PROJECT
( [-] | all [~]project_name ... )
A space-separated list of project names on which limits are enforced. Limits are enforced on each project listed.
Do not configure PER_PROJECT and PROJECTS limits in the same Limit section.
In horizontal format, use only one PER_PROJECT line per Limit section.
Use the keyword all to configure limits that apply to each project in a cluster.
Use the not operator (~) to exclude projects from the all specification in the limit.
In vertical tabular format, multiple project names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
None. If no limit is specified for PER_PROJECT or PROJECTS, no limit is enforced on any project.
PER_PROJECT=proj1 proj2
PER_QUEUE=all [~]queue_name ..
PER_QUEUE
( [-] | all [~]queue_name ... )
A space-separated list of queue names on which limits are enforced. Limits are enforced on jobs submitted to each queue listed.
Do not configure PER_QUEUE and QUEUES limits in the same Limit section.
In horizontal format, use only one PER_QUEUE line per Limit section.
Use the keyword all to configure limits that apply to each queue in a cluster.
Use the not operator (~) to exclude queues from the all specification in the limit. This is useful if you have a large number of queues but only want to exclude a few queues from the limit definition.
In vertical tabular format, multiple queue names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
None. If no limit is specified for PER_QUEUE or QUEUES, no limit is enforced on any queue.
PER_QUEUE=priority night
PER_USER=all [~]user_name ... | all [~]user_group ...
PER_USER
( [-] | all [~]user_name ... | all [~]user_group ... )
A space-separated list of user names or user groups on which limits are enforced. Limits are enforced on each user or individually to each user in the user group listed. If a user group contains a subgroup, the limit also applies to each member in the subgroup recursively.
User names must be valid login names. User group names can be LSF user groups or UNIX and Windows user groups. Note that for LSF and UNIX user groups, the groups must be specified in a UserGroup section in lsb.users first.
Do not configure PER_USER and USERS limits in the same Limit section.
In horizontal format, use only one PER_USER line per Limit section.
Use the keyword all to configure limits that apply to each user in a cluster. If user groups are configured, the limit applies to each member of the user group, not the group as a whole.
Use the not operator (~) to exclude users or user groups from the all specification in the limit. This is useful if you have a large number of users but only want to exclude a few users from the limit definition.
In vertical tabular format, multiple user names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
None. If no limit is specified for PER_USER or USERS, no limit is enforced on any user or user group.
PER_USER=user1 user2 ugroup1 ~user3
PROJECTS=all [~]project_name ...
PROJECTS
( [-] | all [~]project_name ... )
A space-separated list of project names on which limits are enforced. Limits are enforced on all projects listed.
To specify a per-project limit, use the PER_PROJECT keyword. Do not configure PROJECTS and PER_PROJECT limits in the same Limit section.
In horizontal format, use only one PROJECTS line per Limit section.
Use the keyword all to configure limits that apply to all projects in a cluster.
Use the not operator (~) to exclude projects from the all specification in the limit. This is useful if you have a large number of projects but only want to exclude a few projects from the limit definition.
In vertical tabular format, multiple project names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
all (limits are enforced on all projects in the cluster)
PROJECTS=projA projB
QUEUES=all [~]queue_name ...
QUEUES
( [-] | all [~]queue_name ... )
A space-separated list of queue names on which limits are enforced. Limits are enforced on all queues listed.
The list must contain valid queue names defined in lsb.queues.
To specify a per-queue limit, use the PER_QUEUE keyword. Do not configure QUEUES and PER_QUEUE limits in the same Limit section.
In horizontal format, use only one QUEUES line per Limit section.
Use the keyword all to configure limits that apply to all queues in a cluster.
Use the not operator (~) to exclude queues from the all specification in the limit. This is useful if you have a large number of queues but only want to exclude a few queues from the limit definition.
In vertical tabular format, multiple queue names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
all (limits are enforced on all queues in the cluster)
QUEUES=normal night
RESOURCE=[shared_resource,integer] [[shared_resource,integer] ...]
RESOURCE
( [[shared_resource,integer] [[shared_resource,integer] ...] )
Maximum amount of any user-defined shared resource available to consumers.
In horizontal format, use only one RESOURCE line per Limit section.
In vertical tabular format, resource names must be enclosed in parentheses.
In vertical tabular format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
None
RESOURCE=[stat_shared,4]
Begin Limit
RESOURCE PER_HOST
([stat_shared,4]) (all ~hostA)
([dyn_rsrc,1] [stat_rsrc,2]) (hostA)
End Limit
SLOTS=integer
SLOTS
- | integer
Maximum number of job slots available to resource consumers. Specify a positive integer greater than or equal 0.
With MultiCluster resource lease model, this limit applies only to local hosts being used by the local cluster. The job slot limit for hosts exported to a remote cluster is determined by the host export policy, not by this parameter. The job slot limit for borrowed hosts is determined by the host export policy of the remote cluster.
If JOBS are configured in the Limit section, the most restrictive limit is applied.
If HOSTS are configured in the Limit section, SLOTS is the number of running and suspended jobs on a host. If preemptive scheduling is used, the suspended jobs are not counted as using a job slot.
To fully use the CPU resource on multiprocessor hosts, make the number of job slots equal to or greater than the number of processors.
Use this parameter to prevent a host from being overloaded with too many jobs, and to maximize the throughput of a machine.
Use “!” to make the number of job slots equal to the number of CPUs on a host.
If the number of CPUs in a host changes dynamically, mbatchd adjusts the maximum number of job slots per host accordingly. Allow the mbatchd up to 10 minutes to get the number of CPUs for a host. During this period the value of SLOTS is 1.
If only QUEUES are configured in the Limit section, SLOTS is the maximum number of job slots available to the listed queues.
If only USERS are configured in the Limit section, SLOTS is the maximum number of job slots that the users or user groups can use.
If only HOSTS are configured in the Limit section, SLOTS is the maximum number of job slots that are available to the listed hosts.
If only PROJECTS are configured in the Limit section, SLOTS is the maximum number of job slots that are available to the listed projects.
Use QUEUES or PER_QUEUE, USERS or PER_USER, HOSTS or PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit job slots per processor available to resource consumers.
In horizontal format, use only one SLOTS line per Limit section.
In vertical format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
No limit
SLOTS=20
SLOTS_PER_PROCESSOR=number
SLOTS_PER_PROCESSOR
- | number
Per processor job slot limit, based on the number of processors on each host affected by the limit.
Maximum number of job slots that each resource consumer can use per processor. This job slot limit is configured per processor so that multiprocessor hosts will automatically run more jobs.
Without PER_HOST, or
With HOSTS
In vertical format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
To fully use the CPU resource on multiprocessor hosts, make the number of job slots equal to or greater than the number of processors.
Use this parameter to prevent a host from being overloaded with too many jobs, and to maximize the throughput of a machine.
This number can be a fraction such as 0.5, so that it can also serve as a per-CPU limit on multiprocessor machines. This number is rounded up to the nearest integer equal to or greater than the total job slot limits for a host. For example, if SLOTS_PER_PREOCESSOR is 0.5, on a 4-CPU multiprocessor host, users can only use up to 2 job slots at any time. On a single-processor machine, users can use 1 job slot.
Use “!” to make the number of job slots equal to the number of CPUs on a host.
If the number of CPUs in a host changes dynamically, mbatchd adjusts the maximum number of job slots per host accordingly. Allow the mbatchd up to 10 minutes to get the number of CPUs for a host. During this period the number of CPUs is 1.
If only QUEUES and PER_HOST are configured in the Limit section, SLOTS_PER_PROCESSOR is the maximum amount of job slots per processor available to the listed queues for any hosts, users or projects.
If only USERS and PER_HOST are configured in the Limit section, SLOTS_PER_PROCESSOR is the maximum amount of job slots per processor that the users or user groups can use on any hosts, queues, license projects, or projects.
If only PER_HOST is configured in the Limit section, SLOTS_PER_PROCESSOR is the maximum amount of job slots per processor available to the listed hosts for any users, queues or projects.
If only PROJECTS and PER_HOST are configured in the Limit section, SLOTS_PER_PROCESSOR is the maximum amount of job slots per processor available to the listed projects for any users, queues or hosts.
Use QUEUES or PER_QUEUE, USERS or PER_USER, PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit job slots per processor available to resource consumers.
No limit
SLOTS_PER_PROCESSOR=2
SWP=integer[%]
SWP
- | integer[%]
Maximum amount of swap space available to resource consumers. Specify a value in MB or the unit set in LSF_UNIT_FOR_LIMITS in lsf.conf as a positive integer greater than or equal 0.
Without PER_HOST, or
With HOSTS
In horizontal format, use only one SWP line per Limit section.
In vertical format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
If only USERS are configured in the Limit section, SWP must be an integer value. SWP is the maximum amount of swap space that the users or user groups can use on any hosts, queues or projects.
If only HOSTS are configured in the Limit section, SWP must be an integer value. SWP is the maximum amount of swap space available to the listed hosts for any users, queues or projects.
If only PROJECTS are configured in the Limit section, SWP must be an integer value. SWP is the maximum amount of swap space available to the listed projects for any users, queues or hosts.
If only LIC_PROJECTS are configured in the Limit section, SWP must be an integer value. SWP is the maximum amount of swap space available to the listed projects for any users, queues, projects, or hosts.
Use QUEUES or PER_QUEUE, USERS or PER_USER, HOSTS or PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit swap space available to resource consumers.
No limit
SWP=60
TMP=integer[%]
TMP
- | integer[%]
Maximum amount of tmp space available to resource consumers. Specify a value in MB or the unit set in LSF_UNIT_FOR_LIMITS in lsf.conf as a positive integer greater than or equal 0.
Without PER_HOST, or
With HOSTS
In horizontal format, use only one TMP line per Limit section.
In vertical format, use empty parentheses () or a dash (-) to indicate the default value (no limit). Fields cannot be left blank.
If only QUEUES are configured in the Limit section, TMP must be an integer value. TMP is the maximum amount of tmp space available to the listed queues for any hosts, users projects.
If only USERS are configured in the Limit section, TMP must be an integer value. TMP is the maximum amount of tmp space that the users or user groups can use on any hosts, queues or projects.
If only HOSTS are configured in the Limit section, TMP must be an integer value. TMP is the maximum amount of tmp space available to the listed hosts for any users, queues or projects.
If only PROJECTS are configured in the Limit section, TMP must be an integer value. TMP is the maximum amount of tmp space available to the listed projects for any users, queues or hosts.
If only LIC_PROJECTS are configured in the Limit section, TMP must be an integer value. TMP is the maximum amount of tmp space available to the listed projects for any users, queues, projects, or hosts.
Use QUEUES or PER_QUEUE, USERS or PER_USER, HOSTS or PER_HOST, LIC_PROJECTS or PER_LIC_PROJECT, and PROJECTS or PER_PROJECT in combination to further limit tmp space available to resource consumers.
No limit
TMP=20%
USERS=all [~]user_name ... | all [~]user_group ...
USERS
( [-] | all [~]user_name ... | all [~]user_group ... )
A space-separated list of user names or user groups on which limits are enforced. Limits are enforced on all users or groups listed. Limits apply to a group as a whole.
If a group contains a subgroup, the limit also applies to each member in the subgroup recursively.
User names must be valid login names. User group names can be LSF user groups or UNIX and Windows user groups. UNIX user groups must be configured in lsb.user.
To specify a per-user limit, use the PER_USER keyword. Do not configure USERS and PER_USER limits in the same Limit section.
In horizontal format, use only one USERS line per Limit section.
Use the keyword all to configure limits that apply to all users or user groups in a cluster.
Use the not operator (~) to exclude users or user groups from the all specification in the limit. This is useful if you have a large number of users but only want to exclude a few users or groups from the limit definition.
In vertical format, multiple user names must be enclosed in parentheses.
In vertical format, use empty parentheses () or a dash (-) to indicate an empty field. Fields cannot be left blank.
all (limits are enforced on all users in the cluster)
USERS=user1 user2
Defines a guarantee policy. A guarantee is a commitment to ensure availability of a number of resources to a service class, where a service class is a container for jobs. Each guarantee pool can provide guarantees to multiple service classes, and each service class can have guarantees in multiple pools.
To use guaranteed resources, configure service classes with GOALS=[GUARANTEE] in the lsb.serviceclasses file.
Each resource pool is defined in a GuaranteedResourcePool section and enclosed by Begin GuaranteedResourcePool and End GuaranteedResourcePool.
You must configure a NAME, TYPE and DISTRIBUTION for each GuaranteedResourcePool section.
The order of GuaranteedResourcePool sections is important, as the sections are evaluated in the order configured. Each host can only be in one pool of host-based resources (slots, hosts, or package, each of which can have its own GuaranteedResourcePool section); ensure all GuaranteedResourcePool sections (except the last one) define the HOSTS parameter, so they do not contain the default of all hosts.
When LSF starts up, it goes through the hosts and assigns each host to a pool that will accept the host based on the pool's RES_SELECT and HOSTS parameters. If multiple pools will accept the host, the host will be assigned to the first pool according to the configuration order of the pools.
Begin GuaranteedResourcePool
NAME = linuxGuarantee
TYPE = slots
HOSTS = linux_group
DISTRIBUTION = [sc1, 25] [sc2, 30]
LOAN_POLICIES=QUEUES[all] DURATION[15]
DESCRIPTION = This is the resource pool for the hostgroup linux_group, with\
and 30 slots guaranteed to sc2. Resources are loaned to 25 slots guaranteed\
to sc1 jobs from any queue with runtimes of up to 15 minutes
End GuaranteedResourcePool
Begin GuaranteedResourcePool
NAME = x86Guarantee
TYPE = slots
HOSTS = linux_x86
DISTRIBUTION = [sc1, 25]
LOAN_POLICIES=QUEUES[short_jobs] DURATION[15]
DESCRIPTION = This is the resource pool for the hostgroup\
linux_x86 using the queue solaris, with 25 slots guaranteed\
to sc1. Resources are loaned to jobs for up to 15 minutes.
End GuaranteedResoucePool
Begin GuaranteedResourcePool
NAME = resource2pool
TYPE = resource[f2]
DISTRIBUTION = [sc1, 25%] [sc2, 25%]
LOAN_POLICIES=QUEUES[all] DURATION[10]
DESCRIPTION = This is the resource pool for all f2 resources managed by IBM\
Platform License Scheduler, with 25% guaranteed to each of sc1 and sc2. \
Resources are loaned to jobs from any queue with runtimes of up to 10 minutes.
End GuaranteedResourcePool
NAME
TYPE
HOSTS
RES_SELECT
DISTRIBUTION
LOAN_POLICIES
DESCRIPTION
NAME=name
The name of the guarantee policy.
None. You must provide a name for the guarantee.
TYPE = slots | hosts | resource[shared_resource] | package[slots=[slots_per_package][:mem=mem_per_package]]
Defines the type of resources to be guaranteed in this guarantee policy. These can either be slots, whole hosts, packages composed of an amount of slots and memory bundled on a single host, or licenses managed by License Scheduler.
Specify resource[license] to guarantee licenses (which must be managed by License Scheduler) to service class guarantee jobs.
Specifies the combination of memory and slots that defines the packages that will be treated as resources reserved by service class guarantee jobs. For example,
TYPE=package[slots=1:mem=1000]
Each unit guaranteed is for one slot and 1000 MB of memory.
LSF_UNIT_FOR_LIMITS in lsf.conf determines the units of memory in the package definition. The default value of LSF_UNIT_FOR_LIMITS is MB, therefore the guarantee is for 1000 MB of memory.
A package need not have both slots and memory. Setting TYPE=package[slots=1] is the equivalent of slots. In order to provide guarantees for parallel jobs that require multiple CPUs on a single host where memory is not an important resource, you can use packages with multiple slots and not specify mem.
Each host can belong to at most one slot/host/package guarantee pool.
None. You must specify the type of guarantee.
HOSTS=all | allremote | all@cluster_name ... | [~]host_name | [~]host_group
A space-separated list of hosts or host groups defined in lsb.hosts, on which the guarantee is enforced.
Use the keyword all to include all hosts in a cluster. Use the not operator (~) to exclude hosts from the all specification in the guarantee.
Use host groups for greater flexibility, since host groups have additional configuration options.
Ensure all GuaranteedResourcePool sections (except the last one) define the HOSTS or RES_SELECT parameter, so they do not contain the default of all hosts.
all
RES_SELECT=res_req
Resource requirement string with which all hosts defined by the HOSTS parameter are further filtered. For example, RES_SELECT=type==LINUX86
Only static host attributes can be used in RES_SELECT. Do not use consumable resources or dynamic resources.
None. RES_SELECT is optional.
DISTRIBUTION=([service_class_name, amount[%]]...)
Assigns the amount of resources in the pool to the specified service classes, where amount can be an absolute number or a percentage of the resources in the pool. The outer brackets are optional.
When configured as a percentage, the total can exceed 100% but each assigned percentage cannot exceed 100%. For example:
DISTRIBUTION=[sc1,50%] [sc2,50%] [sc3,50%] is an acceptable configuration even though the total percentages assigned add up to 150%.
DISTRIBUTION=[sc1,120%] is not an acceptable configuration, since the percentage for sc1 is greater than 100%.
Each service class must be configured in lsb.serviceclasses, with GOALS=[GUARANTEE].
When configured as a percentage and there are remaining resources to distribute (because the calculated number of slots is rounded down), LSF distributes the remaining resources using round-robin distribution, starting with the first configured service class. Therefore, the service classes that you define first will receive additional resources regardless of the configured percentage. For example, there are 93 slots in a pool and you configure the following guarantee distribution:
DISTRIBUTION=[sc1,30%] [sc2,10%] [sc3,30%]
The number of slots assigned to guarantee policy are: floor((30% + 10% + 30%)*(93 slots)) = 65 slots
The slots are distributed to the service classes as follows:
As a result of rounding down, the total number of distributed slots is 27+9+27=63 slots, which means there are two remaining slots to distribute. Using round-robin distribution, LSF distributes one slot each to sc1 and sc2 because these service classes are defined first. Therefore, the final slot distribution to the service classes are as follows:
If you configure sc3 before sc2 (DISTRIBUTION=[sc1,30%] [sc3,30%] [sc2,10%]), LSF distributes the two remaining slots to sc1 and sc3. Therefore, the slots are distributed as follows:
None. You must provide a distribution for the resource pool.
LOAN_POLICIES=QUEUES[queue_name ...|all] [CLOSE_ON_DEMAND] [DURATION[minutes][RETAIN[amount[%]]]
By default, LSF will reserve sufficient resources in each guarantee pool to honor the configured guarantees. To increase utilization, use LOAN_POLICIES to allow jobs without guarantees to use these reserved resources when not needed by jobs with guarantees. When resources are loaned out, jobs with guarantees may have to wait for jobs to finish before they are able to dispatch in the pool.
QUEUES[all | queue_name ...] loans only to jobs from the specified queue or queues. You must specify which queues are permitted to borrow resources reserved for guarantees.
When CLOSE_ON_DEMAND is specified, LSF stops loading out from a pool whenever there is pending demand from jobs with guarantees in the pool.
DURATION[minutes] only allows jobs to borrow the resources if the job run limit (or estimated run time) is no larger than minutes. Loans limited by job duration make the guaranteed resources available within the time specified by minutes. Jobs running longer than the estimated run time will run to completion regardless of the actual run time.
RETAIN[amount[%]] enables LSF to reserve some amount of the resources in the pool exclusively for jobs with guarantees in the pool when there are guarantees that are not met.
None. LOAN_POLICIES is optional.
DESCRIPTION=description
A description of the guarantee policy.
None. DESCRIPTION is optional.
Defines an export policy for a host or a group of related hosts. Defines how much of each host’s resources are exported, and how the resources are distributed among the consumers.
Each export policy is defined in a separate HostExport section, so it is normal to have multiple HostExport sections in lsb.resources.
Use empty parentheses ( ) or a dash (-) to specify the default value for an entry. Fields cannot be left blank.
Begin HostExport PER_HOST= hostA hostB SLOTS= 4 DISTRIBUTION= [cluster1, 1]
[cluster2, 3] MEM= 100 SWP= 100 End HostExport
PER_HOST
RES_SELECT
NHOSTS
DISTRIBUTION
MEM
SLOTS
SWAP
TYPE
PER_HOST=host_name...
Required when exporting special hosts.
Determines which hosts to export. Specify one or more LSF hosts by name. Separate names by space.
RES_SELECT=res_req
Required when exporting workstations.
Determines which hosts to export. Specify the selection part of the resource requirement string (without quotes or parentheses), and LSF will automatically select hosts that meet the specified criteria. For this parameter, if you do not specify the required host type, the default is type==any.
When LSF_STRICT_RESREQ=Y is configured in lsf.conf, resource requirement strings in select sections must conform to a more strict syntax. The strict resource requirement syntax only applies to the select section. It does not apply to the other resource requirement sections (order, rusage, same, span, or cu). When LSF_STRICT_RESREQ=Y in lsf.conf, LSF rejects resource requirement strings where an rusage section contains a non-consumable resource.
The criteria is only evaluated once, when a host is exported.
NHOSTS=integer
Required when exporting workstations.
Maximum number of hosts to export. If there are not this many hosts meeting the selection criteria, LSF exports as many as it can.
DISTRIBUTION=([cluster_name, number_shares]...)
Required. Specifies how the exported resources are distributed among consumer clusters.
The syntax for the distribution list is a series of share assignments. The syntax of each share assignment is the cluster name, a comma, and the number of shares, all enclosed in square brackets, as shown. Use a space to separate multiple share assignments. Enclose the full distribution list in a set of round brackets.
cluster_name
Specify the name of a remote cluster that will be allowed to use the exported resources. If you specify a local cluster, the assignment is ignored.
number_shares
Specify a positive integer representing the number of shares of exported resources assigned to the cluster.
The number of shares assigned to a cluster is only meaningful when you compare it to the number assigned to other clusters, or to the total number. The total number of shares is just the sum of all the shares assigned in each share assignment.
MEM=megabytes
Used when exporting special hosts. Specify the amount of memory to export on each host, in MB or in units set in LSF_UNIT_FOR_LIMITS in lsf.conf.
- (provider and consumer clusters compete for available memory)
SLOTS=integer
Required when exporting special hosts. Specify the number of job slots to export on each host.
To avoid overloading a partially exported host, you can reduce the number of job slots in the configuration of the local cluster.
SWAP=megabytes
Used when exporting special hosts. Specify the amount of swap space to export on each host, in MB or in units set in LSF_UNIT_FOR_LIMITS in lsf.conf.
- (provider and consumer clusters compete for available swap space)
TYPE=shared
Changes the lease type from exclusive to shared.
If you export special hosts with a shared lease (using PER_HOST), you cannot specify multiple consumer clusters in the distribution policy.
Undefined (the lease type is exclusive; exported resources are never available to the provider cluster)
Optional. Requires HostExport section. Defines an export policy for a shared resource. Defines how much of the shared resource is exported, and the distribution among the consumers.
The shared resource must be available on hosts defined in the HostExport sections.
All parameters are required.
Begin SharedResourceExport
NAME= AppRes
NINSTANCES= 10
DISTRIBUTION= ([C1, 30] [C2, 70])
End SharedResourceExport
NAME
NINSTANCES
DISTRIBUTION
NAME=shared_resource_name
Shared resource to export. This resource must be available on the hosts that are exported to the specified clusters; you cannot export resources without hosts.
NINSTANCES=integer
Maximum quantity of shared resource to export. If the total number available is less than the requested amount, LSF exports all that are available.
DISTRIBUTION=([cluster_name, number_shares]...)
Specifies how the exported resources are distributed among consumer clusters.
The syntax for the distribution list is a series of share assignments. The syntax of each share assignment is the cluster name, a comma, and the number of shares, all enclosed in square brackets, as shown. Use a space to separate multiple share assignments. Enclose the full distribution list in a set of round brackets.
cluster_name
Specify the name of a cluster allowed to use the exported resources.
number_shares
Specify a positive integer representing the number of shares of exported resources assigned to the cluster.
The number of shares assigned to a cluster is only meaningful when you compare it to the number assigned to other clusters, or to the total number. The total number of shares is the sum of all the shares assigned in each share assignment.
By default, only LSF administrators or root can add or delete advance reservations.
Users or user groups that can create reservations
Hosts that can be used for the reservation
Time window when reservations can be created
Each advance reservation policy is defined in a separate ResourceReservation section, so it is normal to have multiple ResourceReservation sections in lsb.resources.
Begin ResourceReservation
NAME = dayPolicy
USERS = user1 user2 # optional
HOSTS = hostA hostB # optional
TIME_WINDOW = 8:00-18:00 # weekly recurring reservation
End ResourceReservation
% user1@hostB> brsvadd -m "hostA" -n 1 -u "user2" -t "5:9:0-5:11:0"
Reservation "user2#2" is created
Users can only delete reservations they created themselves. In the example, only user user1 can delete the reservation; user2 cannot. Administrators can delete any reservations created by users.
HOSTS
NAME
TIME_WINDOW
USERS
HOSTS=[~]host_name | [~]host_group | all | allremote | all@cluster_name ...
A space-separated list of hosts, host groups defined in lsb.hosts on which administrators or users specified in the USERS parameter can create advance reservations.
The hosts can be local to the cluster or hosts leased from remote clusters.
If a group contains a subgroup, the reservation configuration applies to each member in the subgroup recursively.
Use the keyword all to configure reservation policies that apply to all local hosts in a cluster not explicitly excluded. This is useful if you have a large cluster but you want to use the not operator (~) to exclude a few hosts from the list of hosts where reservations can be created.
You cannot specify host groups or host partitions that contain the allremote keyword.
Use all@cluster_name to specify the group of all hosts borrowed from one remote cluster. You cannot specify a host group or partition that includes remote resources.
With MultiCluster resource leasing model, the not operator (~) can be used to exclude local hosts or host groups. You cannot use the not operator (~) with remote hosts.
HOSTS=hgroup1 ~hostA hostB hostC
Advance reservations can be created on hostB, hostC, and all hosts in hgroup1 except for hostA.
HOSTS=all ~group2 ~hostA
Advance reservations can be created on all hosts in the cluster, except for hostA and the hosts in group2.
all allremote (users can create reservations on all server hosts in the local cluster, and all leased hosts in a remote cluster).
NAME=text
Required. Name of the ResourceReservation section
Specify any ASCII string 40 characters or less. You can use letters, digits, underscores (_) or dashes (-). You cannot use blank spaces.
NAME=reservation1
None. You must provide a name for the ResourceReservation section.
TIME_WINDOW=time_window ...
Optional. Time window for users to create advance reservations. The time for reservations that users create must fall within this time window.
time_window = begin_time-end_time
[day:]hour[:minute]
day of the week: 0-6 (0 is Sunday)
hour: 0-23
minute: 0-59
hour-hour
hour:minute-hour:minute
day:hour:minute-day:hour:minute
The default value for minute is 0 (on the hour); the default value for day is every day of the week.
You must specify at least the hour. Day of the week and minute are optional. Both the start time and end time values must use the same syntax. If you do not specify a minute, LSF assumes the first minute of the hour (:00). If you do not specify a day, LSF assumes every day of the week. If you do specify the day, you must also specify the minute.
timeWindow(8:00-14:00 18:00-22:00)
timeWindow(8:00-14:00 11:00-15:00)
is not valid.
TIME_WINDOW=8:00-14:00
Users can create advance reservations with begin time (brsvadd -b), end time (brsvadd -e), or time window (brsvadd -t) on any day between 8:00 a.m. and 2:00 p.m.
Undefined (any time)
USERS=[~]user_name | [~]user_group ... | all
A space-separated list of user names or user groups who are allowed to create advance reservations. Administrators, root, and all users or groups listed can create reservations.
If a group contains a subgroup, the reservation policy applies to each member in the subgroup recursively.
User names must be valid login names. User group names can be LSF user groups or UNIX and Windows user groups.
Use the keyword all to configure reservation policies that apply to all users or user groups in a cluster. This is useful if you have a large number of users but you want to exclude a few users or groups from the reservation policy.
Use the not operator (~) to exclude users or user groups from the list of users who can create reservations.
The not operator does not exclude LSF administrators from the policy.
USERS=user1 user2
all (all users in the cluster can create reservations)
To enable greater flexibility for reserving numeric resources that are reserved by jobs, configure the ReservationUsage section in lsb.resources to reserve resources as PER_JOB, PER_SLOT, or PER_HOST. For example:
Begin ReservationUsage
RESOURCE METHOD RESERVE
resourceX PER_JOB Y
resourceY PER_HOST N
resourceZ PER_SLOT N
End ReservationUsage
RESOURCE
METHOD
RESERVE
The name of the resource to be reserved. User-defined numeric resources can be reserved, but only if they are shared (they are not specific to one host).
mem
tmp
swp
Any custom resource can also be reserved if it is shared (defined in the Resource section of lsf.shared) or host based (listed in the Host section of the lsf.cluster file in the resource column).
PER_JOB
PER_HOST
PER_SLOT
The cluster-wide RESOURCE_RESERVE_PER_SLOT parameter in lsb.params is obsolete.
RESOURCE_RESERVE_PER_SLOT parameter still controls resources not configured in lsb.resources. Resources not reserved in lsb.resources are reserved per job.
PER_HOST reservation means that for the parallel job, LSF reserves one instance of a for each host. For example, some application licenses are charged only once no matter how many applications are running provided those applications are running on the same host under the same user.
Use no method ("-") when setting mem, swp, or tmp as RESERVE=Y.
Reserves the resource for pending jobs that are waiting for another resource to become available.
For example, job A requires resources X, Y, and Z to run, but resource Z is a high demand or scarce resource. This job pends until Z is available. In the meantime, other jobs requiring only X and Y resources run. If X and Y are set as reservable resources (the RESERVE parameter is set to "Y"), as soon as Z resource is available, job A runs. If they are not, job A may never be able to run because all resources are never available at the same time.
Only the following built-in resources can be defined as reservable:
mem
swp
tmp
Use no method ("-") when setting mem, swp, or tmp as RESERVE=Y.
When submitting a job, the queue must have RESOURCE_RESERVE defined.
Backfill of the reservable resources is also supported when you submit a job with reservable resources to a queue with BACKFILL defined.
Valid values are Y and N. If not specified, resources are not reserved.
Per-resource configuration defines resource usage for individual resources, but it does not change any existing resource limit behavior (PER_JOB, PER_SLOT).
In a MultiCluster environment, you should configure resource usage in the scheduling cluster (submission cluster in lease model or receiving cluster in job forward model).
Variable configuration is used to automatically change LSF configuration based on time windows. You define automatic configuration changes in lsb.resources by using if-else constructs and time expressions. After you change the files, reconfigure the cluster with the badmin reconfig command.
The expressions are evaluated by LSF every 10 minutes based on mbatchd start time. When an expression evaluates true, LSF dynamically changes the configuration based on the associated configuration statements. Reconfiguration is done in real time without restarting mbatchd, providing continuous system availability.
# limit usage of hosts for group and time
# based configuration
# - 10 jobs can run from normal queue
# - any number can run from short queue between 18:30
# and 19:30
# all other hours you are limited to 100 slots in the
# short queue
# - each other queue can run 30 jobs
Begin Limit
PER_QUEUE HOSTS SLOTS # Example
normal Resource1 10
# if time(18:30-19:30)
short Resource1 -
#else
short Resource1 100
#endif
(all ~normal ~short) Resource1 30
End Limit
This section enables and defines a power management policy.
Begin PowerPolicy
NAME = policy_night
HOSTS = hostGroup1 host3
TIME_WINDOW = 23:00-8:00
MIN_IDLE_TIME = 1800
CYCLE_TIME = 60
End PowerPolicy
NAME
HOSTS
TIME_WINDOW
MIN_IDLE_TIME
CYCLE_TIME
NAME=string
Required. Unique name for the power management policy.
Specify any ASCII string 60 characters or less. You can use letters, digits, underscores (_), dashes (-), or periods (.). You cannot use blank spaces.
NAME=policy_night1
None. You must provide a name to define a PowerPolicy.
HOSTS=host_list
host_list is a space-separated list of host names, host groups, host partitions, or compute units.
Required. Specified hosts should not overlap between power policies.
HOSTS=hostGroup1 host3
If a host is not defined, the default value is all other hosts that are not included in power policy. (Does not contains the master and master candidate)
TIME_WINDOW=time_window ...
Required. Time window is the time period to which the power policy applies.
time_window = begin_time-end_time
[day:]hour[:minute]
day of the week: 0-6 (0 is Sunday)
hour: 0-23
minute: 0-59
hour-hour
hour:minute-hour:minute
day:hour:minute-day:hour:minute
The default value for minute is 0 (on the hour); the default value for day is every day of the week.
You must specify at least the hour. Day of the week and minute are optional. Both the start time and end time values must use the same syntax. If you do not specify a minute, LSF assumes the first minute of the hour (:00). If you do not specify a day, LSF assumes every day of the week. If you do specify the day, you must also specify the minute.
timeWindow(8:00-14:00 18:00-22:00)
timeWindow(8:00-14:00 11:00-15:00)
is not valid.
TIME_WINDOW=8:00-14:00
Not defined (any time)
MIN_IDLE_TIME=minutes
This parameter takes effect if TIME_WINDOW is configured and is valid. It defines the host idle time before power operations are issued for defined hosts.
MIN_IDLE_TIME=60
0
CYCLE_TIME=minutes
This parameter takes effect if TIME_WINDOW is configured and is valid. It defines the minimum time (in minutes) between changes in power states for defined hosts.
CYCLE_TIME=15
0