org.apache.bookkeeper.client

Class TopologyAwareEnsemblePlacementPolicy

java.lang.Object

org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy

All Implemented Interfaces:

EnsemblePlacementPolicy, ITopologyAwareEnsemblePlacementPolicy<org.apache.bookkeeper.net.BookieNode>

Direct Known Subclasses:

RackawareEnsemblePlacementPolicyImpl, ZoneawareEnsemblePlacementPolicyImpl

abstract class TopologyAwareEnsemblePlacementPolicy
extends java.lang.Object
implements ITopologyAwareEnsemblePlacementPolicy<org.apache.bookkeeper.net.BookieNode>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	TopologyAwareEnsemblePlacementPolicy.DefaultResolver
(package private) static class	TopologyAwareEnsemblePlacementPolicy.DNSResolverDecorator Decorator for any existing dsn resolver.
protected static class	TopologyAwareEnsemblePlacementPolicy.EnsembleForReplacementWithNoConstraints
protected static class	TopologyAwareEnsemblePlacementPolicy.RRTopologyAwareCoverageEnsemble A predicate checking the rack coverage for write quorum in RoundRobinDistributionSchedule, which ensures that a write quorum should be covered by at least two racks.
protected static class	TopologyAwareEnsemblePlacementPolicy.TruePredicate

Nested classes/interfaces inherited from interface org.apache.bookkeeper.client.ITopologyAwareEnsemblePlacementPolicy

ITopologyAwareEnsemblePlacementPolicy.Ensemble<T extends org.apache.bookkeeper.net.Node>, ITopologyAwareEnsemblePlacementPolicy.Predicate<T extends org.apache.bookkeeper.net.Node>

Nested classes/interfaces inherited from interface org.apache.bookkeeper.client.EnsemblePlacementPolicy

EnsemblePlacementPolicy.PlacementPolicyAdherence, EnsemblePlacementPolicy.PlacementResult<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.bookkeeper.proto.BookieAddressResolver	bookieAddressResolver
protected java.util.Map<org.apache.bookkeeper.net.BookieNode,WeightedRandomSelection.WeightedObject>	bookieInfoMap
protected org.apache.bookkeeper.stats.OpStatsLogger	bookiesJoinedCounter
protected org.apache.bookkeeper.stats.OpStatsLogger	bookiesLeftCounter
protected org.apache.bookkeeper.net.DNSToSwitchMapping	dnsResolver
protected java.util.Map<org.apache.bookkeeper.net.BookieId,org.apache.bookkeeper.net.BookieNode>	historyBookies
(package private) boolean	isWeighted
protected java.util.Map<org.apache.bookkeeper.net.BookieId,org.apache.bookkeeper.net.BookieNode>	knownBookies
(package private) static org.slf4j.Logger	LOG
protected com.google.common.collect.ImmutableSet<org.apache.bookkeeper.net.BookieId>	readOnlyBookies
static java.lang.String	REPP_DNS_RESOLVER_CLASS
protected java.util.concurrent.locks.ReentrantReadWriteLock	rwLock
protected org.apache.bookkeeper.net.NetworkTopology	topology
protected WeightedRandomSelection<org.apache.bookkeeper.net.BookieNode>	weightedSelection

Constructor Summary

Constructors

Constructor and Description
TopologyAwareEnsemblePlacementPolicy()

Method Summary

Modifier and Type	Method and Description
protected java.util.Set<org.apache.bookkeeper.net.Node>	convertBookiesToNodes(java.util.Collection<org.apache.bookkeeper.net.BookieId> bookies)
protected org.apache.bookkeeper.net.BookieNode	convertBookieToNode(org.apache.bookkeeper.net.BookieId addr)
protected org.apache.bookkeeper.net.BookieNode	createBookieNode(org.apache.bookkeeper.net.BookieId addr)
protected org.apache.bookkeeper.net.BookieNode	createDummyLocalBookieNode(java.lang.String hostname)
static int	differBetweenBookies(java.util.List<org.apache.bookkeeper.net.BookieId> bookiesA, java.util.List<org.apache.bookkeeper.net.BookieId> bookiesB)
(package private) static java.util.Set<java.lang.String>	getNetworkLocations(java.util.Set<org.apache.bookkeeper.net.Node> bookieNodes)
void	handleBookiesThatJoined(java.util.Set<org.apache.bookkeeper.net.BookieId> joinedBookies) Handle bookies that joined.
void	handleBookiesThatLeft(java.util.Set<org.apache.bookkeeper.net.BookieId> leftBookies) Handle bookies that left.
void	onBookieRackChange(java.util.List<org.apache.bookkeeper.net.BookieId> bookieAddressList) Handle rack change for the bookies.
java.util.Set<org.apache.bookkeeper.net.BookieId>	onClusterChanged(java.util.Set<org.apache.bookkeeper.net.BookieId> writableBookies, java.util.Set<org.apache.bookkeeper.net.BookieId> readOnlyBookies) A consistent view of the cluster (what bookies are available as writable, what bookies are available as readonly) is updated when any changes happen in the cluster.
DistributionSchedule.WriteSet	reorderReadLACSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble, BookiesHealthInfo bookiesHealthInfo, DistributionSchedule.WriteSet writeSet) Reorder the read last add confirmed sequence of a given write quorum writeSet.
DistributionSchedule.WriteSet	reorderReadSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble, BookiesHealthInfo bookiesHealthInfo, DistributionSchedule.WriteSet writeSet) Reorder the read sequence of a given write quorum writeSet.
protected java.lang.String	resolveNetworkLocation(org.apache.bookkeeper.net.BookieId addr)
(package private) static void	shuffleWithMask(DistributionSchedule.WriteSet writeSet, int mask, int bits) Shuffle all the entries of an array that matches a mask.
void	updateBookieInfo(java.util.Map<org.apache.bookkeeper.net.BookieId,BookieInfoReader.BookieInfo> bookieInfoMap) Send the bookie info details.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.bookkeeper.client.ITopologyAwareEnsemblePlacementPolicy

newEnsemble, selectFromNetworkLocation, selectFromNetworkLocation, selectFromNetworkLocation

Methods inherited from interface org.apache.bookkeeper.client.EnsemblePlacementPolicy

areAckedBookiesAdheringToPlacementPolicy, getStickyReadBookieIndex, initialize, isEnsembleAdheringToPlacementPolicy, newEnsemble, registerSlowBookie, replaceBookie, replaceToAdherePlacementPolicy, uninitalize

Field Detail

LOG

static final org.slf4j.Logger LOG

REPP_DNS_RESOLVER_CLASS

public static final java.lang.String REPP_DNS_RESOLVER_CLASS

See Also:

Constant Field Values

knownBookies

protected final java.util.Map<org.apache.bookkeeper.net.BookieId,org.apache.bookkeeper.net.BookieNode> knownBookies

historyBookies

protected final java.util.Map<org.apache.bookkeeper.net.BookieId,org.apache.bookkeeper.net.BookieNode> historyBookies

rwLock

protected final java.util.concurrent.locks.ReentrantReadWriteLock rwLock

bookieInfoMap

protected java.util.Map<org.apache.bookkeeper.net.BookieNode,WeightedRandomSelection.WeightedObject> bookieInfoMap

readOnlyBookies

protected com.google.common.collect.ImmutableSet<org.apache.bookkeeper.net.BookieId> readOnlyBookies

isWeighted

boolean isWeighted

weightedSelection

protected WeightedRandomSelection<org.apache.bookkeeper.net.BookieNode> weightedSelection

topology

protected org.apache.bookkeeper.net.NetworkTopology topology

dnsResolver

protected org.apache.bookkeeper.net.DNSToSwitchMapping dnsResolver

bookieAddressResolver

protected org.apache.bookkeeper.proto.BookieAddressResolver bookieAddressResolver

bookiesJoinedCounter

@StatsDoc(name="BOOKIES_JOINED",
          help="The distribution of number of bookies joined the cluster on each network topology change")
protected org.apache.bookkeeper.stats.OpStatsLogger bookiesJoinedCounter

bookiesLeftCounter

@StatsDoc(name="BOOKIES_LEFT",
          help="The distribution of number of bookies left the cluster on each network topology change")
protected org.apache.bookkeeper.stats.OpStatsLogger bookiesLeftCounter

Constructor Detail

TopologyAwareEnsemblePlacementPolicy

TopologyAwareEnsemblePlacementPolicy()

Method Detail

getNetworkLocations

static java.util.Set<java.lang.String> getNetworkLocations(java.util.Set<org.apache.bookkeeper.net.Node> bookieNodes)

shuffleWithMask

static void shuffleWithMask(DistributionSchedule.WriteSet writeSet,
                            int mask,
                            int bits)

Shuffle all the entries of an array that matches a mask. It assumes all entries with the same mask are contiguous in the array.

reorderReadSequence

public DistributionSchedule.WriteSet reorderReadSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble,
                                                         BookiesHealthInfo bookiesHealthInfo,
                                                         DistributionSchedule.WriteSet writeSet)

Description copied from interface: EnsemblePlacementPolicy

Reorder the read sequence of a given write quorum writeSet.

Specified by:

reorderReadSequence in interface EnsemblePlacementPolicy

Parameters:

ensemble - Ensemble to read entries.

bookiesHealthInfo - Health info for bookies

writeSet - Write quorum to read entries. This will be modified, rather than allocating a new WriteSet.

Returns:

The read sequence. This will be the same object as the passed in writeSet.

reorderReadLACSequence

public DistributionSchedule.WriteSet reorderReadLACSequence(java.util.List<org.apache.bookkeeper.net.BookieId> ensemble,
                                                            BookiesHealthInfo bookiesHealthInfo,
                                                            DistributionSchedule.WriteSet writeSet)

Description copied from interface: EnsemblePlacementPolicy

Reorder the read last add confirmed sequence of a given write quorum writeSet.

Specified by:

reorderReadLACSequence in interface EnsemblePlacementPolicy

Parameters:

ensemble - Ensemble to read entries.

bookiesHealthInfo - Health info for bookies

writeSet - Write quorum to read entries. This will be modified, rather than allocating a new WriteSet.

Returns:

The read sequence. This will be the same object as the passed in writeSet.

onClusterChanged

public java.util.Set<org.apache.bookkeeper.net.BookieId> onClusterChanged(java.util.Set<org.apache.bookkeeper.net.BookieId> writableBookies,
                                                                          java.util.Set<org.apache.bookkeeper.net.BookieId> readOnlyBookies)

Description copied from interface: EnsemblePlacementPolicy

A consistent view of the cluster (what bookies are available as writable, what bookies are available as readonly) is updated when any changes happen in the cluster.

The implementation should take actions when the cluster view is changed. So subsequent EnsemblePlacementPolicy.newEnsemble(int, int, int, Map, Set) and EnsemblePlacementPolicy.replaceBookie(int, int, int, java.util.Map, java.util.List, BookieId, java.util.Set) can choose proper bookies.

Specified by:

onClusterChanged in interface EnsemblePlacementPolicy

Parameters:

writableBookies - All the bookies in the cluster available for write/read.

readOnlyBookies - All the bookies in the cluster available for readonly.

Returns:

the dead bookies during this cluster change.

handleBookiesThatLeft

public void handleBookiesThatLeft(java.util.Set<org.apache.bookkeeper.net.BookieId> leftBookies)

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Handle bookies that left.

Specified by:

handleBookiesThatLeft in interface ITopologyAwareEnsemblePlacementPolicy<org.apache.bookkeeper.net.BookieNode>

Parameters:

leftBookies - bookies that left

handleBookiesThatJoined

public void handleBookiesThatJoined(java.util.Set<org.apache.bookkeeper.net.BookieId> joinedBookies)

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Handle bookies that joined.

Specified by:

handleBookiesThatJoined in interface ITopologyAwareEnsemblePlacementPolicy<org.apache.bookkeeper.net.BookieNode>

Parameters:

joinedBookies - bookies that joined.

onBookieRackChange

public void onBookieRackChange(java.util.List<org.apache.bookkeeper.net.BookieId> bookieAddressList)

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Handle rack change for the bookies.

Specified by:

onBookieRackChange in interface ITopologyAwareEnsemblePlacementPolicy<org.apache.bookkeeper.net.BookieNode>

differBetweenBookies

public static int differBetweenBookies(java.util.List<org.apache.bookkeeper.net.BookieId> bookiesA,
                                       java.util.List<org.apache.bookkeeper.net.BookieId> bookiesB)

updateBookieInfo

public void updateBookieInfo(java.util.Map<org.apache.bookkeeper.net.BookieId,BookieInfoReader.BookieInfo> bookieInfoMap)

Description copied from interface: EnsemblePlacementPolicy

Send the bookie info details.

Specified by:

updateBookieInfo in interface EnsemblePlacementPolicy

Parameters:

bookieInfoMap - A map that has the bookie to BookieInfo

createBookieNode

protected org.apache.bookkeeper.net.BookieNode createBookieNode(org.apache.bookkeeper.net.BookieId addr)

createDummyLocalBookieNode

protected org.apache.bookkeeper.net.BookieNode createDummyLocalBookieNode(java.lang.String hostname)

resolveNetworkLocation

protected java.lang.String resolveNetworkLocation(org.apache.bookkeeper.net.BookieId addr)

convertBookiesToNodes

protected java.util.Set<org.apache.bookkeeper.net.Node> convertBookiesToNodes(java.util.Collection<org.apache.bookkeeper.net.BookieId> bookies)

convertBookieToNode

protected org.apache.bookkeeper.net.BookieNode convertBookieToNode(org.apache.bookkeeper.net.BookieId addr)