Package io.axoniq.framework.springcloud.shared

Class SpringCloudMemberDiscovery

java.lang.Object
io.axoniq.framework.springcloud.shared.SpringCloudMemberDiscovery
All Implemented Interfaces:
DescribableComponent
public class SpringCloudMemberDiscovery extends Object implements DescribableComponent
Discovers the members of the cluster through Spring Cloud Discovery, and resolves the member a command or query is routed to from what it discovered.

This is what both connectors route with and publish their capabilities through. Commands and queries are routed differently: a command is hashed onto a ConsistentHash ring by its routing key, so that commands for one entity keep landing on one member, while a query carries no routing key and goes to whichever member advertises its name, rotating over them to spread the load. The administration behind that, which members there are and what each of them handles, is kept in a MemberRegistry this discovery owns.

Who is in the cluster and what each member handles are learned separately, because they change for different reasons and are signalled differently:

  • Membership comes from the discovery implementation, which signals a change with a HeartbeatEvent. Eureka publishes one on every registry fetch, Spring Cloud Kubernetes whenever the set of endpoints changes. On each, updateMemberships() reads the instances discovery reports, asks the ones it has not heard from for their capabilities, and drops the ones that are gone.
  • Capabilities change when a member subscribes or unsubscribes a handler, which no discovery implementation signals. Each instance's answer is therefore kept for a refresh interval, after which refreshCapabilities() asks that instance again, without involving discovery.
After either, the ring is derived anew from the members as now known, so a member that has gone away simply does not reappear. Both run one at a time, and startMembershipRound() runs one on the Executor this discovery is given, so a heartbeat never waits on the requests it causes.

Members are identified by the node id each one reports about itself, not by what discovery reports about them. An application that discovery reports more than once, for example under two service ids, is one member, reached at the first of those instances in discovery's own order. This application recognizes its own instance the same way: it is the one answering with this application's node id. No Spring Cloud Registration is needed for that, so the connector works the same with discovery implementations that register the application, such as Eureka, and with those that leave registration to the platform, such as Spring Cloud Kubernetes.

Registered by the Spring Boot autoconfiguration, which starts a membership round on every HeartbeatEvent and InstanceRegisteredEvent, and schedules the capability refresh. Both connectors share one, so that each publishes what it handles without erasing what the other published.

Since:
5.4.0
Author:
Allard Buijze

  • Field Details

    • DEFAULT_CAPABILITIES_REFRESH_INTERVAL

      public static final Duration DEFAULT_CAPABILITIES_REFRESH_INTERVAL
      The default for how long an instance's capabilities are taken as current before it is asked again: 30 seconds, Eureka's default registry fetch interval, at which capabilities used to be refreshed along with membership.

  • Constructor Details

    • SpringCloudMemberDiscovery

      public SpringCloudMemberDiscovery(org.springframework.cloud.client.discovery.DiscoveryClient discoveryClient, CapabilityDiscoveryMode discoveryMode)
      Constructs a SpringCloudMemberDiscovery discovering members through the given discoveryClient.

      Capabilities are refreshed after DEFAULT_CAPABILITIES_REFRESH_INTERVAL, and rounds run on virtual threads.

      Parameters:
      discoveryClient - the client reporting the service instances making up the cluster
      discoveryMode - the mode used to learn which member each discovered instance is, and what it handles

    • SpringCloudMemberDiscovery

      public SpringCloudMemberDiscovery(org.springframework.cloud.client.discovery.DiscoveryClient discoveryClient, CapabilityDiscoveryMode discoveryMode, Predicate<org.springframework.cloud.client.ServiceInstance> serviceInstanceFilter, @Nullable String contextRootMetadataPropertyName, Duration capabilitiesRefreshInterval, Executor executor)
      Constructs a SpringCloudMemberDiscovery discovering members through the given discoveryClient.
      Parameters:
      discoveryClient - the client reporting the service instances making up the cluster
      discoveryMode - the mode used to learn which member each discovered instance is, and what it handles
      serviceInstanceFilter - decides which discovered instances are considered at all. Instances rejected here are never asked for their capabilities, which is cheaper than relying on the ignore list when whole services can be excluded up front.
      contextRootMetadataPropertyName - the metadata key holding an instance's context root, to be appended to its URI, or null when instances are served from the root
      capabilitiesRefreshInterval - how long an instance's capabilities are taken as current before it is asked again. A zero or negative interval asks every instance on every round, and leaves scheduleCapabilityRefresh(ScheduledExecutorService) nothing to schedule
      executor - runs the rounds startMembershipRound() or the capability refresh starts, and the capability requests within them. A round waits for its requests, so the executor must run more than one task at a time

  • Method Details

    • findCommandDestination

      public Optional<Member> findCommandDestination(String routingKey, QualifiedName commandName)
      Resolves the member a command of the given commandName carrying the given routingKey should be handled by.
      Parameters:
      routingKey - the routing key of the command to resolve a member for
      commandName - the QualifiedName of the command to resolve a member for
      Returns:
      the member that should handle the command, or Optional.empty() when no known member handles commands of the given commandName

    • findQueryDestination

      public Optional<Member> findQueryDestination(QualifiedName queryName)
      Resolves the member a query of the given queryName should be handled by.

      Unlike a command, a query carries no routing key and asks nothing of where it is handled, so any member advertising the name will do. Successive queries of the same name therefore rotate over the members advertising it, spreading the load rather than sending every query of a name to the same member. Members are rotated over in name order, so that the rotation is over a stable sequence even as the ring changes.

      This member is one of the candidates like any other. Preferring the local handler is a policy of the DistributedQueryBus, which applies it before the query reaches a connector at all.

      Parameters:
      queryName - the QualifiedName of the query to resolve a member for
      Returns:
      the member that should handle the query, or Optional.empty() when no known member handles queries of the given queryName

    • findAllQueryDestinations

      public List<Member> findAllQueryDestinations(QualifiedName queryName)
      Returns every member advertising the given queryName, in a stable order.

      Where findQueryDestination(QualifiedName) picks one member to answer a query, a subscription query needs them all: an update is emitted on whichever member's state changed, and a member only matches a subscription it holds a registration for. A subscriber that reached only one member would miss every update emitted on the others.

      Parameters:
      queryName - the QualifiedName of the query to resolve members for
      Returns:
      every member advertising the given queryName, empty when none does

    • onMembershipChanged

      public Registration onMembershipChanged(Consumer<ConsistentHash> listener)
      Registers a listener to be notified after every change to the ring, with the ring that replaced the previous one.

      A subscription query relies on this: it has to know when a member that advertises its name appears, because it cannot have received the updates that member emitted before it was subscribed to. Notified outside the lock the ring is replaced under, so a listener may read this discovery freely. It runs on whichever thread made the change, which for a discovery round is the thread running that round.

      Parameters:
      listener - notified with the new ring after every membership change
      Returns:
      a registration cancelling the notification

    • publishLocalCommands

      public void publishLocalCommands(int loadFactor, Set<QualifiedName> commands)
      Publishes the given commands as the commands this application handles, asking for the given loadFactor worth of the command load.

      Reflected in this application's own ring right away, so that a command dispatched right after its handler subscribed can already be routed here, and visible to other members on their next round.

      Parameters:
      loadFactor - the share of the command load this application asks for
      commands - the names of the commands this application handles

    • publishLocalQueries

      public void publishLocalQueries(Set<QualifiedName> queries)
      Publishes the given queries as the queries this application handles.

      Reflected in this application's own ring right away, and visible to other members on their next round.

      Parameters:
      queries - the names of the queries this application handles

    • localMember

      public Member localMember()
      Returns the Member representing this application, identified by this application's node id.
      Returns:
      the member representing this application

    • members

      public Set<Member> members()
      Returns every member currently known, this application included when it is part of the cluster, whatever they handle.
      Returns:
      every member currently known, in no particular order

    • capabilitiesOf

      public Optional<MemberCapabilities> capabilitiesOf(Member member)
      Returns what the given member handles, as currently known.
      Parameters:
      member - the member to look up the capabilities of
      Returns:
      the capabilities of the given member, or Optional.empty() when it is not a known member

    • startMembershipRound

      public void startMembershipRound()
      Starts a membership round on this discovery's executor, as discovery signalled that the instances it reports may have changed.

      The thread signalling the change does not wait on the capability requests the round makes.

    • updateMemberships

      public void updateMemberships()
      Brings the ring in line with the instances discovery currently reports.

      Instances discovery no longer reports are dropped. Instances that are new, that have not answered before, or whose answer is older than the capabilities refresh interval are asked for their capabilities; the others keep what they answered before.

      Exposed so that a deployment whose discovery implementation publishes no heartbeats, or a test, can drive membership itself. Runs on the calling thread, after any round in progress has finished.

    • refreshCapabilities

      public void refreshCapabilities()
      Asks the instances whose capabilities are older than the capabilities refresh interval, or that have not answered before, for their capabilities again.

      Does not consult discovery: which instances there are is up to updateMemberships(). This is what keeps the ring current with discovery implementations that only signal membership changes, such as Spring Cloud Kubernetes, while a member subscribes or unsubscribes handlers. Runs on the calling thread, after any round in progress has finished.

    • scheduleCapabilityRefresh

      public Registration scheduleCapabilityRefresh(ScheduledExecutorService scheduler)
      Schedules refreshCapabilities() on the given scheduler, twice per capabilities refresh interval, so that no instance's capabilities are older than one and a half intervals.

      The scheduler only decides when a refresh is due. The refresh itself runs on this discovery's executor, and is skipped while the previous one has not finished, so the scheduler may be shared with other work that must not be held up. Does nothing when the capabilities refresh interval is zero or negative.

      Parameters:
      scheduler - the scheduler deciding when a refresh is due
      Returns:
      a registration cancelling the schedule

    • markUnreachable

      public void markUnreachable(Member member)
      Takes the given member off the ring, as a message could not be delivered to it.

      What the member answered is forgotten as well, so that the next round does not put it back before it has answered again. It is asked again on that round, so this only keeps messages away from a member while it is unreachable. It does not weaken any consistency guarantee: routing a command to one member is a matter of locality, while consistency is enforced where the events are appended.

      Parameters:
      member - the member that could not be reached

    • describeTo

      public void describeTo(ComponentDescriptor descriptor)
      Description copied from interface: DescribableComponent
      Describe the properties of this DescribableComponent with the given descriptor.

      Components should call the appropriate describeProperty methods on the descriptor to register their properties. The descriptor is responsible for determining how these properties are formatted and structured in the final output.

      Best Practices: As a general rule, all relevant fields of a DescribableComponent implementation should be described in this method. However, developers have discretion to include only the fields that make sense in the context. Not every field may be meaningful for description purposes, especially internal implementation details. Furthermore, components might want to expose different information based on their current state. The final decision on what properties to include lies with the person implementing the describeTo method, who should focus on providing information that is useful for understanding the component's configuration and state.

      Example implementation: 

       public void describeTo(ComponentDescriptor descriptor) {
     descriptor.describeProperty("name", this.name);
     descriptor.describeProperty("enabled", this.enabled);
     descriptor.describeProperty("configuration", this.configuration); // A nested component
     descriptor.describeProperty("handlers", this.eventHandlers);      // A collection
 }
      Specified by:
      describeTo in interface DescribableComponent
      Parameters:
      descriptor - The component descriptor to describe this DescribableComponentn its properties in.