public class ZoneawareEnsemblePlacementPolicyImpl extends TopologyAwareEnsemblePlacementPolicy
| Modifier and Type | Class and Description |
|---|---|
static class |
ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation
Zone and UpgradeDomain pair of a node.
|
TopologyAwareEnsemblePlacementPolicy.DefaultResolver, TopologyAwareEnsemblePlacementPolicy.DNSResolverDecorator, TopologyAwareEnsemblePlacementPolicy.EnsembleForReplacementWithNoConstraints, TopologyAwareEnsemblePlacementPolicy.RRTopologyAwareCoverageEnsemble, TopologyAwareEnsemblePlacementPolicy.TruePredicateITopologyAwareEnsemblePlacementPolicy.Ensemble<T extends org.apache.bookkeeper.net.Node>, ITopologyAwareEnsemblePlacementPolicy.Predicate<T extends org.apache.bookkeeper.net.Node>EnsemblePlacementPolicy.PlacementPolicyAdherence, EnsemblePlacementPolicy.PlacementResult<T>| Modifier and Type | Field and Description |
|---|---|
protected java.util.concurrent.ConcurrentMap<org.apache.bookkeeper.net.BookieId,ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation> |
address2NodePlacement |
protected int |
desiredNumZonesPerWriteQuorum |
protected boolean |
enforceStrictZoneawarePlacement |
protected org.apache.bookkeeper.stats.Counter |
failedToResolveNetworkLocationCounter |
(package private) static org.slf4j.Logger |
LOG |
protected int |
maxWeightMultiple |
protected int |
minNumZonesPerWriteQuorum |
protected org.apache.bookkeeper.net.BookieNode |
myNode |
protected java.lang.String |
myZone |
protected org.apache.bookkeeper.stats.Gauge<java.lang.Integer> |
numWritableBookiesInDefaultFaultDomain |
protected boolean |
reorderReadsRandom |
protected int |
reorderThresholdPendingRequests |
protected com.google.common.cache.Cache<org.apache.bookkeeper.net.BookieId,java.lang.Long> |
slowBookies |
protected int |
stabilizePeriodSeconds |
protected org.apache.bookkeeper.stats.StatsLogger |
statsLogger |
protected io.netty.util.HashedWheelTimer |
timer |
static java.lang.String |
UNKNOWN_ZONE |
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation |
unresolvedNodeLocation |
bookieAddressResolver, bookieInfoMap, bookiesJoinedCounter, bookiesLeftCounter, dnsResolver, historyBookies, isWeighted, knownBookies, readOnlyBookies, REPP_DNS_RESOLVER_CLASS, rwLock, topology, weightedSelection| Constructor and Description |
|---|
ZoneawareEnsemblePlacementPolicyImpl() |
| Modifier and Type | Method and Description |
|---|---|
protected java.util.Set<org.apache.bookkeeper.net.BookieId> |
addDefaultFaultDomainBookies(java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies) |
boolean |
areAckedBookiesAdheringToPlacementPolicy(java.util.Set<org.apache.bookkeeper.net.BookieId> ackedBookies,
int writeQuorumSize,
int ackQuorumSize)
Returns true if the bookies that have acknowledged a write adhere to the minimum fault domains as defined in the
placement policy in use.
|
java.lang.String |
getDefaultFaultDomain() |
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation |
getZoneAwareNodeLocation(org.apache.bookkeeper.net.BookieId addr) |
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation |
getZoneAwareNodeLocation(org.apache.bookkeeper.net.BookieNode node) |
EnsemblePlacementPolicy |
initialize(ClientConfiguration conf,
java.util.Optional<org.apache.bookkeeper.net.DNSToSwitchMapping> optionalDnsResolver,
io.netty.util.HashedWheelTimer timer,
FeatureProvider featureProvider,
org.apache.bookkeeper.stats.StatsLogger statsLogger,
org.apache.bookkeeper.proto.BookieAddressResolver bookieAddressResolver)
Initialize the policy.
|
EnsemblePlacementPolicy.PlacementPolicyAdherence |
isEnsembleAdheringToPlacementPolicy(java.util.List<org.apache.bookkeeper.net.BookieId> ensembleList,
int writeQuorumSize,
int ackQuorumSize)
returns AdherenceLevel if the Ensemble is strictly/softly/fails adhering
to placement policy, like in the case of
RackawareEnsemblePlacementPolicy, bookies in the writeset are from
'minNumRacksPerWriteQuorum' number of racks.
|
EnsemblePlacementPolicy.PlacementResult<java.util.List<org.apache.bookkeeper.net.BookieId>> |
newEnsemble(int ensembleSize,
int writeQuorumSize,
int ackQuorumSize,
java.util.Map<java.lang.String,byte[]> customMetadata,
java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies)
Choose numBookies bookies for ensemble.
|
EnsemblePlacementPolicy.PlacementResult<java.util.List<org.apache.bookkeeper.net.BookieId>> |
newEnsemble(int ensembleSize,
int writeQuorumSize,
int ackQuorumSize,
java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> parentEnsemble,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> parentPredicate)
Create an ensemble with parent ensemble.
|
void |
registerSlowBookie(org.apache.bookkeeper.net.BookieId bookieSocketAddress,
long entryId)
Register a bookie as slow so that it is tried after available and read-only bookies.
|
DistributionSchedule.WriteSet |
reorderReadLACSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble,
BookiesHealthInfo bookiesHealthInfo,
DistributionSchedule.WriteSet writeSet)
Reorder the read last add confirmed sequence of a given write quorum writeSet.
|
DistributionSchedule.WriteSet |
reorderReadSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble,
BookiesHealthInfo bookiesHealthInfo,
DistributionSchedule.WriteSet writeSet)
Reorder the read sequence of a given write quorum writeSet.
|
EnsemblePlacementPolicy.PlacementResult<org.apache.bookkeeper.net.BookieId> |
replaceBookie(int ensembleSize,
int writeQuorumSize,
int ackQuorumSize,
java.util.Map<java.lang.String,byte[]> customMetadata,
java.util.List<org.apache.bookkeeper.net.BookieId> currentEnsemble,
org.apache.bookkeeper.net.BookieId bookieToReplace,
java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies)
Choose a new bookie to replace bookieToReplace.
|
org.apache.bookkeeper.net.BookieNode |
selectFromNetworkLocation(java.util.Set<java.lang.String> excludeRacks,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
Select a node from cluster excluding excludeBookies and bookie nodes of
excludeRacks.
|
org.apache.bookkeeper.net.BookieNode |
selectFromNetworkLocation(java.lang.String networkLoc,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
Select a node from a given network location.
|
org.apache.bookkeeper.net.BookieNode |
selectFromNetworkLocation(java.lang.String networkLoc,
java.util.Set<java.lang.String> excludeRacks,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
Select a node from networkLoc rack excluding excludeBookies.
|
void |
uninitalize()
Uninitialize the policy.
|
ZoneawareEnsemblePlacementPolicyImpl |
withDefaultFaultDomain(java.lang.String defaultFaultDomain) |
convertBookiesToNodes, convertBookieToNode, createBookieNode, createDummyLocalBookieNode, differBetweenBookies, getNetworkLocations, handleBookiesThatJoined, handleBookiesThatLeft, onBookieRackChange, onClusterChanged, resolveNetworkLocation, shuffleWithMask, updateBookieInfoclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetStickyReadBookieIndex, replaceToAdherePlacementPolicystatic final org.slf4j.Logger LOG
public static final java.lang.String UNKNOWN_ZONE
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation unresolvedNodeLocation
protected org.apache.bookkeeper.stats.StatsLogger statsLogger
protected com.google.common.cache.Cache<org.apache.bookkeeper.net.BookieId,java.lang.Long> slowBookies
protected org.apache.bookkeeper.net.BookieNode myNode
protected java.lang.String myZone
protected boolean reorderReadsRandom
protected int stabilizePeriodSeconds
protected int reorderThresholdPendingRequests
protected int maxWeightMultiple
protected int minNumZonesPerWriteQuorum
protected int desiredNumZonesPerWriteQuorum
protected boolean enforceStrictZoneawarePlacement
protected io.netty.util.HashedWheelTimer timer
protected final java.util.concurrent.ConcurrentMap<org.apache.bookkeeper.net.BookieId,ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation> address2NodePlacement
@StatsDoc(name="FAILED_TO_RESOLVE_NETWORK_LOCATION_TOTAL",
help="Counter for number of times DNSResolverDecorator failed to resolve Network Location")
protected org.apache.bookkeeper.stats.Counter failedToResolveNetworkLocationCounter
@StatsDoc(name="NUM_WRITABLE_BOOKIES_IN_DEFAULT_FAULTDOMAIN",
help="Gauge for the number of writable Bookies in default fault domain")
protected org.apache.bookkeeper.stats.Gauge<java.lang.Integer> numWritableBookiesInDefaultFaultDomain
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation getZoneAwareNodeLocation(org.apache.bookkeeper.net.BookieId addr)
protected ZoneawareEnsemblePlacementPolicyImpl.ZoneAwareNodeLocation getZoneAwareNodeLocation(org.apache.bookkeeper.net.BookieNode node)
public EnsemblePlacementPolicy initialize(ClientConfiguration conf, java.util.Optional<org.apache.bookkeeper.net.DNSToSwitchMapping> optionalDnsResolver, io.netty.util.HashedWheelTimer timer, FeatureProvider featureProvider, org.apache.bookkeeper.stats.StatsLogger statsLogger, org.apache.bookkeeper.proto.BookieAddressResolver bookieAddressResolver)
EnsemblePlacementPolicyconf - client configurationoptionalDnsResolver - dns resolvertimer - timerfeatureProvider - feature providerstatsLogger - stats loggerpublic ZoneawareEnsemblePlacementPolicyImpl withDefaultFaultDomain(java.lang.String defaultFaultDomain)
public java.lang.String getDefaultFaultDomain()
public EnsemblePlacementPolicy.PlacementResult<java.util.List<org.apache.bookkeeper.net.BookieId>> newEnsemble(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> parentEnsemble, ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> parentPredicate) throws BKException.BKNotEnoughBookiesException
ITopologyAwareEnsemblePlacementPolicyensembleSize - ensemble sizewriteQuorumSize - write quorum sizeackQuorumSize - ack quorum sizeexcludeBookies - exclude bookiesparentEnsemble - parent ensembleBKException.BKNotEnoughBookiesExceptionpublic org.apache.bookkeeper.net.BookieNode selectFromNetworkLocation(java.lang.String networkLoc,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
throws BKException.BKNotEnoughBookiesException
ITopologyAwareEnsemblePlacementPolicynetworkLoc - network locationexcludeBookies - exclude bookies setpredicate - predicate to applyensemble - ensemblefallbackToRandom - fallbackToRandomBKException.BKNotEnoughBookiesExceptionpublic org.apache.bookkeeper.net.BookieNode selectFromNetworkLocation(java.util.Set<java.lang.String> excludeRacks,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
throws BKException.BKNotEnoughBookiesException
ITopologyAwareEnsemblePlacementPolicyBKException.BKNotEnoughBookiesExceptionpublic org.apache.bookkeeper.net.BookieNode selectFromNetworkLocation(java.lang.String networkLoc,
java.util.Set<java.lang.String> excludeRacks,
java.util.Set<org.apache.bookkeeper.net.Node> excludeBookies,
ITopologyAwareEnsemblePlacementPolicy.Predicate<org.apache.bookkeeper.net.BookieNode> predicate,
ITopologyAwareEnsemblePlacementPolicy.Ensemble<org.apache.bookkeeper.net.BookieNode> ensemble,
boolean fallbackToRandom)
throws BKException.BKNotEnoughBookiesException
ITopologyAwareEnsemblePlacementPolicyBKException.BKNotEnoughBookiesExceptionpublic void uninitalize()
EnsemblePlacementPolicypublic EnsemblePlacementPolicy.PlacementResult<java.util.List<org.apache.bookkeeper.net.BookieId>> newEnsemble(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Map<java.lang.String,byte[]> customMetadata, java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies) throws BKException.BKNotEnoughBookiesException
EnsemblePlacementPolicyBKException.BKNotEnoughBookiesException is thrown.
The implementation should respect to the replace settings. The size of the returned bookie list
should be equal to the provide ensembleSize.
customMetadata is the same user defined data that user provides
when BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[], Map).
If 'enforceMinNumRacksPerWriteQuorum' config is enabled then the bookies belonging to default faultzone (rack) will be excluded while selecting bookies.
ensembleSize - Ensemble SizewriteQuorumSize - Write Quorum SizeackQuorumSize - the value of ackQuorumSize (added since 4.5)customMetadata - the value of customMetadata. it is the same user defined metadata that user
provides in BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[])excludeBookies - Bookies that should not be considered as targets.BKException.BKNotEnoughBookiesException - if not enough bookies available.public EnsemblePlacementPolicy.PlacementResult<org.apache.bookkeeper.net.BookieId> replaceBookie(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Map<java.lang.String,byte[]> customMetadata, java.util.List<org.apache.bookkeeper.net.BookieId> currentEnsemble, org.apache.bookkeeper.net.BookieId bookieToReplace, java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies) throws BKException.BKNotEnoughBookiesException
EnsemblePlacementPolicyBKException.BKNotEnoughBookiesException is thrown.
If 'enforceMinNumRacksPerWriteQuorum' config is enabled then the bookies belonging to default faultzone (rack) will be excluded while selecting bookies.
ensembleSize - the value of ensembleSizewriteQuorumSize - the value of writeQuorumSizeackQuorumSize - the value of ackQuorumSize (added since 4.5)customMetadata - the value of customMetadata. it is the same user defined metadata that user
provides in BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[])currentEnsemble - the value of currentEnsemblebookieToReplace - bookie to replaceexcludeBookies - bookies that should not be considered as candidate.BKException.BKNotEnoughBookiesExceptionprotected java.util.Set<org.apache.bookkeeper.net.BookieId> addDefaultFaultDomainBookies(java.util.Set<org.apache.bookkeeper.net.BookieId> excludeBookies)
public void registerSlowBookie(org.apache.bookkeeper.net.BookieId bookieSocketAddress,
long entryId)
EnsemblePlacementPolicybookieSocketAddress - Address of bookie hostentryId - Entry ID that caused a speculative timeout on the bookie.public DistributionSchedule.WriteSet reorderReadSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble, BookiesHealthInfo bookiesHealthInfo, DistributionSchedule.WriteSet writeSet)
EnsemblePlacementPolicyreorderReadSequence in interface EnsemblePlacementPolicyreorderReadSequence in class TopologyAwareEnsemblePlacementPolicyensemble - Ensemble to read entries.bookiesHealthInfo - Health info for bookieswriteSet - Write quorum to read entries. This will be modified, rather than
allocating a new WriteSet.public DistributionSchedule.WriteSet reorderReadLACSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble, BookiesHealthInfo bookiesHealthInfo, DistributionSchedule.WriteSet writeSet)
EnsemblePlacementPolicyreorderReadLACSequence in interface EnsemblePlacementPolicyreorderReadLACSequence in class TopologyAwareEnsemblePlacementPolicyensemble - Ensemble to read entries.bookiesHealthInfo - Health info for bookieswriteSet - Write quorum to read entries. This will be modified, rather than
allocating a new WriteSet.public EnsemblePlacementPolicy.PlacementPolicyAdherence isEnsembleAdheringToPlacementPolicy(java.util.List<org.apache.bookkeeper.net.BookieId> ensembleList, int writeQuorumSize, int ackQuorumSize)
EnsemblePlacementPolicyensembleList - list of BookieId of bookies in the ensemblewriteQuorumSize - writeQuorumSize of the ensembleackQuorumSize - ackQuorumSize of the ensemblepublic boolean areAckedBookiesAdheringToPlacementPolicy(java.util.Set<org.apache.bookkeeper.net.BookieId> ackedBookies,
int writeQuorumSize,
int ackQuorumSize)
EnsemblePlacementPolicyackedBookies - list of BookieId of bookies that have acknowledged a write.writeQuorumSize - writeQuorumSize of the ensembleackQuorumSize - ackQuorumSize of the ensembleCopyright © 2011–2024 The Apache Software Foundation. All rights reserved.