public abstract class AbstractCSQueue extends Object implements CSQueue
| Modifier and Type | Field and Description |
|---|---|
protected ActivitiesManager |
activitiesManager |
protected org.apache.hadoop.yarn.security.YarnAuthorizationProvider |
authorizer |
protected CapacitySchedulerContext |
csContext |
protected org.apache.hadoop.yarn.security.PrivilegedEntity |
queueEntity |
protected ReentrantReadWriteLock.ReadLock |
readLock |
protected ReentrantReadWriteLock.WriteLock |
writeLock |
| Constructor and Description |
|---|
AbstractCSQueue(CapacitySchedulerContext cs,
String queueName,
CSQueue parent,
CSQueue old) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
accept(org.apache.hadoop.yarn.api.records.Resource cluster,
ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request) |
boolean |
accessibleToPartition(String nodePartition) |
void |
activeQueue()
Active the queue.
|
protected void |
appFinished() |
CSAssignment |
assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource,
FiCaSchedulerNode node,
ResourceLimits resourceLimits,
SchedulingMode schedulingMode) |
void |
decPendingResource(String nodeLabel,
org.apache.hadoop.yarn.api.records.Resource resourceToDec)
When an outstanding resource is fulfilled or canceled, calling this will
decrease pending resource in a queue.
|
void |
decReservedResource(String partition,
org.apache.hadoop.yarn.api.records.Resource reservedRes)
Decrement Reserved Capacity
|
void |
decUsedResource(String nodeLabel,
org.apache.hadoop.yarn.api.records.Resource resourceToDec,
SchedulerApplicationAttempt application)
When partition of node updated, we will update queue's resource usage if it
has container(s) running on that.
|
float |
getAbsoluteCapacity()
Get capacity of the parent of the queue as a function of the
cumulative capacity in the cluster.
|
float |
getAbsoluteMaximumCapacity()
Get maximum-capacity of the queue as a funciton of the cumulative capacity
of the cluster.
|
float |
getAbsoluteUsedCapacity()
Get the current absolute used capacity of the queue
relative to the entire cluster.
|
Set<String> |
getAccessibleNodeLabels()
Get labels can be accessed of this queue
labels={*}, means this queue can access any label
labels={ }, means this queue cannot access any label except node without label
labels={a, b, c} means this queue can access a or b or c
|
Map<org.apache.hadoop.yarn.security.AccessType,org.apache.hadoop.security.authorize.AccessControlList> |
getACLs() |
float |
getCapacity()
Get the configured capacity of the queue.
|
long |
getDefaultApplicationLifetime()
Get the default lifetime in seconds of an application which is submitted to
this queue.
|
org.apache.hadoop.yarn.api.records.Priority |
getDefaultApplicationPriority()
Get the Default Application Priority for this queue
|
boolean |
getDefaultAppLifetimeWasSpecifiedInConfig()
Get the indicator of whether or not the default application lifetime was
set by a config property or was calculated by the capacity scheduler.
|
String |
getDefaultNodeLabelExpression()
Get default label expression of this queue.
|
boolean |
getIntraQueuePreemptionDisabled()
Check whether intra-queue preemption is disabled for this queue
|
boolean |
getIntraQueuePreemptionDisabledInHierarchy()
Determines whether or not the intra-queue preemption disabled switch is set
at any level in this queue's hierarchy.
|
Iterator<RMContainer> |
getKillableContainers(String partition) |
org.apache.hadoop.yarn.api.records.Resource |
getMaximumAllocation() |
long |
getMaximumApplicationLifetime()
Get the maximum lifetime in seconds of an application which is submitted to
this queue.
|
float |
getMaximumCapacity()
Get the configured maximum-capacity of the queue.
|
CSQueueMetrics |
getMetrics()
Get the queue metrics
|
org.apache.hadoop.yarn.api.records.Resource |
getMinimumAllocation() |
Set<String> |
getNodeLabelsForQueue()
Get valid Node Labels for this queue
|
int |
getNumContainers() |
CSQueue |
getParent()
Get the parent
Queue. |
boolean |
getPreemptionDisabled()
Check whether disable_preemption property is set for this queue
|
org.apache.hadoop.yarn.api.records.Priority |
getPriority()
Get priority of queue
|
org.apache.hadoop.yarn.security.PrivilegedEntity |
getPrivilegedEntity() |
QueueCapacities |
getQueueCapacities()
Get QueueCapacities of this queue
|
Map<String,org.apache.hadoop.yarn.api.records.QueueConfigurations> |
getQueueConfigurations() |
protected org.apache.hadoop.yarn.api.records.QueueInfo |
getQueueInfo() |
String |
getQueueName()
Get the queue name.
|
String |
getQueuePath()
Get the full name of the queue, including the heirarchy.
|
ResourceUsage |
getQueueResourceUsage()
Get ResourceUsage of this queue
|
org.apache.hadoop.yarn.api.records.QueueStatistics |
getQueueStatistics() |
ReentrantReadWriteLock.ReadLock |
getReadLock()
Get readLock associated with the Queue.
|
boolean |
getReservationContinueLooking() |
org.apache.hadoop.yarn.api.records.QueueState |
getState()
Get the current run-state of the queue
|
org.apache.hadoop.yarn.api.records.Resource |
getTotalKillableResource(String partition) |
float |
getUsedCapacity()
Get the current used capacity of nodes without label(s) of the queue
and it's children (if any).
|
org.apache.hadoop.yarn.api.records.Resource |
getUsedResources()
Get the currently utilized resources which allocated at nodes without any
labels in the cluster by the queue and children (if any).
|
Map<String,Float> |
getUserWeights()
Get a map of usernames and weights
|
boolean |
hasAccess(org.apache.hadoop.yarn.api.records.QueueACL acl,
org.apache.hadoop.security.UserGroupInformation user)
Check if the
user has permission to perform the operation |
boolean |
hasChildQueues() |
void |
incPendingResource(String nodeLabel,
org.apache.hadoop.yarn.api.records.Resource resourceToInc)
When new outstanding resource is asked, calling this will increase pending
resource in a queue.
|
void |
incReservedResource(String partition,
org.apache.hadoop.yarn.api.records.Resource reservedRes)
Increment Reserved Capacity
|
void |
incUsedResource(String nodeLabel,
org.apache.hadoop.yarn.api.records.Resource resourceToInc,
SchedulerApplicationAttempt application)
When partition of node updated, we will update queue's resource usage if it
has container(s) running on that.
|
protected void |
releaseResource(org.apache.hadoop.yarn.api.records.Resource clusterResource,
org.apache.hadoop.yarn.api.records.Resource resource,
String nodePartition) |
void |
setParent(CSQueue newParentQueue)
Set the parent
Queue. |
protected void |
setupConfigurableCapacities() |
void |
updateQueueState(org.apache.hadoop.yarn.api.records.QueueState queueState)
Update the queue state.
|
void |
validateSubmitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
String userName,
String queue)
Validate submitApplication api so that moveApplication do a pre-check.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitapply, assignContainers, attachContainer, collectSchedulerApplications, completedContainer, detachContainer, finishApplication, finishApplicationAttempt, getAbstractUsersManager, getChildQueues, getNumApplications, reinitialize, submitApplication, submitApplicationAttempt, updateClusterResourcestopQueuegetQueueInfo, getQueueUserAclInfo, recoverContainerprotected final org.apache.hadoop.yarn.security.PrivilegedEntity queueEntity
protected CapacitySchedulerContext csContext
protected org.apache.hadoop.yarn.security.YarnAuthorizationProvider authorizer
protected ActivitiesManager activitiesManager
protected ReentrantReadWriteLock.ReadLock readLock
protected ReentrantReadWriteLock.WriteLock writeLock
public AbstractCSQueue(CapacitySchedulerContext cs, String queueName, CSQueue parent, CSQueue old) throws IOException
IOExceptionprotected void setupConfigurableCapacities()
public String getQueuePath()
CSQueuegetQueuePath in interface CSQueuepublic float getCapacity()
CSQueuegetCapacity in interface CSQueuepublic float getAbsoluteCapacity()
CSQueuegetAbsoluteCapacity in interface CSQueuepublic float getAbsoluteMaximumCapacity()
CSQueuegetAbsoluteMaximumCapacity in interface CSQueuepublic float getAbsoluteUsedCapacity()
CSQueuegetAbsoluteUsedCapacity in interface CSQueuepublic float getMaximumCapacity()
CSQueuegetMaximumCapacity in interface CSQueuepublic float getUsedCapacity()
CSQueuegetUsedCapacity in interface CSQueuepublic org.apache.hadoop.yarn.api.records.Resource getUsedResources()
CSQueuegetUsedResources in interface CSQueuepublic int getNumContainers()
public org.apache.hadoop.yarn.api.records.QueueState getState()
CSQueuegetState in interface CSQueuegetState in interface SchedulerQueue<CSQueue>public CSQueueMetrics getMetrics()
QueuegetMetrics in interface Queuepublic String getQueueName()
CSQueuegetQueueName in interface CSQueuegetQueueName in interface Queuepublic org.apache.hadoop.yarn.security.PrivilegedEntity getPrivilegedEntity()
getPrivilegedEntity in interface CSQueuepublic CSQueue getParent()
CSQueueQueue.getParent in interface CSQueuegetParent in interface SchedulerQueue<CSQueue>public void setParent(CSQueue newParentQueue)
CSQueueQueue.public Set<String> getAccessibleNodeLabels()
QueuegetAccessibleNodeLabels in interface Queuepublic boolean hasAccess(org.apache.hadoop.yarn.api.records.QueueACL acl,
org.apache.hadoop.security.UserGroupInformation user)
CSQueueuser has permission to perform the operationpublic String getDefaultNodeLabelExpression()
QueuegetDefaultNodeLabelExpression in interface Queueprotected org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo()
public org.apache.hadoop.yarn.api.records.QueueStatistics getQueueStatistics()
public Map<String,org.apache.hadoop.yarn.api.records.QueueConfigurations> getQueueConfigurations()
@InterfaceAudience.Private public org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()
@InterfaceAudience.Private public org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()
protected void releaseResource(org.apache.hadoop.yarn.api.records.Resource clusterResource,
org.apache.hadoop.yarn.api.records.Resource resource,
String nodePartition)
@InterfaceAudience.Private public boolean getReservationContinueLooking()
@InterfaceAudience.Private public Map<org.apache.hadoop.yarn.security.AccessType,org.apache.hadoop.security.authorize.AccessControlList> getACLs()
@InterfaceAudience.Private public boolean getPreemptionDisabled()
CSQueuegetPreemptionDisabled in interface CSQueue@InterfaceAudience.Private public boolean getIntraQueuePreemptionDisabled()
CSQueuegetIntraQueuePreemptionDisabled in interface CSQueue@InterfaceAudience.Private public boolean getIntraQueuePreemptionDisabledInHierarchy()
CSQueuegetIntraQueuePreemptionDisabledInHierarchy in interface CSQueue@InterfaceAudience.Private public QueueCapacities getQueueCapacities()
CSQueuegetQueueCapacities in interface CSQueue@InterfaceAudience.Private public ResourceUsage getQueueResourceUsage()
CSQueuegetQueueResourceUsage in interface CSQueuepublic ReentrantReadWriteLock.ReadLock getReadLock()
CSQueuegetReadLock in interface CSQueuepublic boolean hasChildQueues()
public void incReservedResource(String partition, org.apache.hadoop.yarn.api.records.Resource reservedRes)
QueueincReservedResource in interface Queuepartition - asked by applicationreservedRes - reserved resource askedpublic void decReservedResource(String partition, org.apache.hadoop.yarn.api.records.Resource reservedRes)
QueuedecReservedResource in interface Queuepartition - asked by applicationreservedRes - reserved resource askedpublic void incPendingResource(String nodeLabel, org.apache.hadoop.yarn.api.records.Resource resourceToInc)
QueueincPendingResource in interface QueuenodeLabel - asked by applicationresourceToInc - new resource askedpublic void decPendingResource(String nodeLabel, org.apache.hadoop.yarn.api.records.Resource resourceToDec)
CSQueuedecPendingResource in interface CSQueuedecPendingResource in interface QueuenodeLabel - asked by applicationresourceToDec - new resource askedpublic void incUsedResource(String nodeLabel, org.apache.hadoop.yarn.api.records.Resource resourceToInc, SchedulerApplicationAttempt application)
CSQueueincUsedResource in interface CSQueuepublic void decUsedResource(String nodeLabel, org.apache.hadoop.yarn.api.records.Resource resourceToDec, SchedulerApplicationAttempt application)
CSQueuedecUsedResource in interface CSQueuepublic boolean accessibleToPartition(String nodePartition)
public org.apache.hadoop.yarn.api.records.Priority getDefaultApplicationPriority()
QueuegetDefaultApplicationPriority in interface Queuepublic Set<String> getNodeLabelsForQueue()
CSQueuegetNodeLabelsForQueue in interface CSQueuepublic org.apache.hadoop.yarn.api.records.Resource getTotalKillableResource(String partition)
public Iterator<RMContainer> getKillableContainers(String partition)
public CSAssignment assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerNode node, ResourceLimits resourceLimits, SchedulingMode schedulingMode)
assignContainers in interface CSQueuepublic boolean accept(org.apache.hadoop.yarn.api.records.Resource cluster,
ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request)
public void validateSubmitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
String userName,
String queue)
throws org.apache.hadoop.security.AccessControlException
CSQueuevalidateSubmitApplication in interface CSQueueapplicationId - Application IDuserName - User Namequeue - Queue Nameorg.apache.hadoop.security.AccessControlException - if any acl violation is there.public void updateQueueState(org.apache.hadoop.yarn.api.records.QueueState queueState)
SchedulerQueueupdateQueueState in interface SchedulerQueue<CSQueue>queueState - the queue statepublic void activeQueue()
throws org.apache.hadoop.yarn.exceptions.YarnException
SchedulerQueueactiveQueue in interface SchedulerQueue<CSQueue>org.apache.hadoop.yarn.exceptions.YarnException - if the queue can not be activated.protected void appFinished()
public org.apache.hadoop.yarn.api.records.Priority getPriority()
CSQueuegetPriority in interface CSQueuepublic Map<String,Float> getUserWeights()
CSQueuegetUserWeights in interface CSQueuepublic long getMaximumApplicationLifetime()
CSQueuegetMaximumApplicationLifetime in interface CSQueuepublic long getDefaultApplicationLifetime()
CSQueuegetDefaultApplicationLifetime in interface CSQueuepublic boolean getDefaultAppLifetimeWasSpecifiedInConfig()
CSQueuegetDefaultAppLifetimeWasSpecifiedInConfig in interface CSQueueCopyright © 2022 Apache Software Foundation. All rights reserved.