Skip to content

feature: zone aware routing - Option 1#3117

Open
szuecs wants to merge 17 commits intomasterfrom
feature/zone-aware-routing
Open

feature: zone aware routing - Option 1#3117
szuecs wants to merge 17 commits intomasterfrom
feature/zone-aware-routing

Conversation

@szuecs
Copy link
Copy Markdown
Member

@szuecs szuecs commented Jun 18, 2024

  • feature: zone aware eskip.LBEndpoints
  • feature: routesrv add zone aware API endpoint /routes/:zone

@szuecs szuecs added enhancement architectural all changes in the hot path, big changes in the control plane, control flow changes in filters labels Jun 18, 2024
@szuecs szuecs force-pushed the feature/zone-aware-routing branch 4 times, most recently from b15a1b3 to 2f8af02 Compare June 19, 2024 13:02
@szuecs szuecs changed the title feature: zone aware routing feature: zone aware routing - Option 1 Jun 24, 2024
@szuecs szuecs mentioned this pull request Dec 11, 2025
@szuecs szuecs force-pushed the feature/zone-aware-routing branch from 2f8af02 to e6350b4 Compare February 27, 2026 08:19
@greeshma1196 greeshma1196 force-pushed the feature/zone-aware-routing branch from 1dc1dc5 to 6e33de7 Compare March 13, 2026 12:59
@greeshma1196
Copy link
Copy Markdown
Collaborator

greeshma1196 commented Mar 20, 2026

Test run on 19/03:
I created an application

zkubectl -n zone-aware-routing get all
NAME                          READY   STATUS    RESTARTS   AGE
pod/app1-v1-cdd55fdd9-4rk87   1/1     Running   0          3h1m
pod/app1-v1-cdd55fdd9-8qjbb   1/1     Running   0          20m
pod/app1-v1-cdd55fdd9-9ndls   1/1     Running   0          3h27m
pod/app1-v1-cdd55fdd9-hwxcl   1/1     Running   0          3h1m
pod/app1-v1-cdd55fdd9-jdrnx   1/1     Running   0          3h27m
pod/app1-v1-cdd55fdd9-mj86h   1/1     Running   0          3h27m
pod/app1-v1-cdd55fdd9-vv454   1/1     Running   0          80s
pod/app1-v1-cdd55fdd9-x9g24   1/1     Running   0          94s
pod/app1-v1-cdd55fdd9-xvnq7   1/1     Running   0          13m

Endpointslices for the above application:

kubectl -n zone-aware-routing get endpointslices app1-v1-cwjdl -oyaml | yq
addressType: IPv6
apiVersion: discovery.k8s.io/v1
endpoints:
  - addresses:
      - 2a05:d024:90:1104:565e::8
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-mj86h
      namespace: zone-aware-routing
      uid: 69ae5fc1-b155-4ced-a2c6-c095965b859d
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1104:565e::d
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-jdrnx
      namespace: zone-aware-routing
      uid: eab3475a-2537-4a51-93ab-c373596d96a8
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1104:565e::b
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-9ndls
      namespace: zone-aware-routing
      uid: 689323c4-d8b0-4c24-917e-f0fcfa666f95
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1103:b425::a
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-3-212.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-4rk87
      namespace: zone-aware-routing
      uid: ec6d33a8-5f86-45e3-b949-895bc0d59cd4
    zone: eu-central-1a
  - addresses:
      - 2a05:d024:90:1103:b425::7
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-3-212.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-hwxcl
      namespace: zone-aware-routing
      uid: a924265d-d2db-4fa3-838d-757128f39b45
    zone: eu-central-1a
  - addresses:
      - 2a05:d024:90:1103:4b45::8
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-3-106.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-xvnq7
      namespace: zone-aware-routing
      uid: f247f345-e105-4d8b-a292-cda577323c2f
    zone: eu-central-1a
  - addresses:
      - 2a05:d024:90:1105:c367::4
    conditions:
      ready: false
      serving: true
      terminating: true
    nodeName: ip-172-31-20-21.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-x9g24
      namespace: zone-aware-routing
      uid: f9e7a920-b8b4-4088-ab47-a0392b235fff
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:2a8::4
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-69.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-vv454
      namespace: zone-aware-routing
      uid: fc10ab35-e0f5-45df-945d-2ee2492eef1e
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:2a8::7
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-69.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-xrhjw
      namespace: zone-aware-routing
      uid: 100b0a9d-cbfd-4759-be8e-1cc99b2d8b3f
    zone: eu-central-1c
kind: EndpointSlice
metadata:
  creationTimestamp: "2026-03-19T12:35:42Z"
  generateName: app1-v1-
  generation: 87
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: app1-v1
    stack-version: v1
    stackset: app1
  name: app1-v1-cwjdl
  namespace: zone-aware-routing
  ownerReferences:
    - apiVersion: v1
      blockOwnerDeletion: true
      controller: true
      kind: Service
      name: app1-v1
      uid: b94959a9-4d9c-4254-a1ea-9290a809b159
  resourceVersion: "73922010"
  uid: 2c00ea63-e807-4f4d-b76e-c13d235fe9b9
ports:
  - name: port-0-0
    port: 9090
    protocol: TCP

Port-forwarded a routesrv pod, this pod's zone is set to eu-central-1c

Response

curl localhost:9990/routes/eu-central-1c
kube_rg__zone_aware_routing__app1_v1_25rzp__all__0_0: Path("/test") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?|app1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Weight(3) && Method("GET") && Header("X-Forwarded-Proto", "https") -> disableAccessLog(2, 3, 404, 429) -> fifo(2000, 20, "1s") -> oauthTokeninfoAnyKV("realm", "/services") -> unverifiedAuditLog("sub") -> oauthTokeninfoAllScope("uid") -> flowId("reuse") -> forwardToken("X-TokenInfo-Forward", "uid", "scope", "realm") -> oauthTokeninfoValidate("{optOutAnnotations: [iam.zalando.org/public], optOutHosts: [\"^.*[.]ingress[.]cluster[.]local\"], unauthorizedResponse: \"Authentication required, see https://cloud.docs.zalando.net/howtos/authenticate-requests/#default-authentication\\n\"}") -> stateBagToTag("auth-user", "client.uid") -> <powerOfRandomNChoices, "http://[2a05:d024:90:1105:2a8::4]:9090", "http://[2a05:d024:90:1105:2a8::7]:9090", "http://[2a05:d024:90:1105:fb28::7]:9090">;

This returns the correct set of Endpoints. But when I got the same set of endpoints when I queried for eu-central-1a and eu-central-1b. This is not the expected behavior. Ideally it should fallback and return all endpoints.

Updated JTBD:
When I deploy skipper with routesrv in a Kubernetes cluster with pods distributed across multiple availability zones / I want to have zone-aware routing that automatically adapts based on endpoint availability / So that requests prefer same-zone backends for low latency when sufficient endpoints exist, but gracefully fall back to cross-zone routing when a zone is under-provisioned

Updated Tasks:

  • Remove zone filtering from dataclient - return all endpoints with zone metadata
  • Ensure route construction always creates LBEndpoints with zone labels
  • Implement threshold-based zone filtering in routesrv when serving routes
  • Run tests to verify fallback behavior

@greeshma1196
Copy link
Copy Markdown
Collaborator

greeshma1196 commented Mar 20, 2026

Test run on 20/3:

The application has the following:
Zone A - 0 endpoints
Zone B - 3 endpoints
Zone C - 6 endpoints

kubectl -n zone-aware-routing get endpointslices app1-v1-cwjdl -oyaml | yq
addressType: IPv6
apiVersion: discovery.k8s.io/v1
endpoints:
  - addresses:
      - 2a05:d024:90:1104:565e::10
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-lt4bp
      namespace: zone-aware-routing
      uid: 5217d86f-bdd9-43c6-9ed5-b0b9656a5d39
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1104:565e::d
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-p6wvq
      namespace: zone-aware-routing
      uid: 81ae26e3-fd13-46ff-a182-5d31271a9fdf
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1104:565e::b
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-8-220.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-kbk76
      namespace: zone-aware-routing
      uid: f6cc804c-c466-431a-9cac-69610f358e8d
    zone: eu-central-1b
  - addresses:
      - 2a05:d024:90:1105:43ef::c
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-143.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-v7s9p
      namespace: zone-aware-routing
      uid: 37483208-2726-4b46-bc70-24381e2f561c
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:43ef::b
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-143.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-jz99r
      namespace: zone-aware-routing
      uid: ec278009-4007-4210-ac2f-0c2d07c6d7a2
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:43ef::12
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-143.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-ft7lj
      namespace: zone-aware-routing
      uid: eba2cced-17cb-4e5d-bf01-dd197f45ee23
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:43ef::8
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-143.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-c9xb7
      namespace: zone-aware-routing
      uid: 31b139fa-c61b-4842-8358-ce62ec826d11
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:43ef::14
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-18-143.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-2pqjj
      namespace: zone-aware-routing
      uid: bce6bb49-5363-48d0-88d1-35030b4caf12
    zone: eu-central-1c
  - addresses:
      - 2a05:d024:90:1105:bdf5::6
    conditions:
      ready: true
      serving: true
      terminating: false
    nodeName: ip-172-31-23-22.eu-central-1.compute.internal
    targetRef:
      kind: Pod
      name: app1-v1-cdd55fdd9-4t9pv
      namespace: zone-aware-routing
      uid: c6f4b7f0-30c3-4ef4-a544-c6957ea933bb
    zone: eu-central-1c
kind: EndpointSlice
metadata:
  annotations:
    endpoints.kubernetes.io/last-change-trigger-time: "2026-03-20T14:19:07Z"
  creationTimestamp: "2026-03-19T12:35:42Z"
  generateName: app1-v1-
  generation: 201
  labels:
    endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io
    kubernetes.io/service-name: app1-v1
    stack-version: v1
    stackset: app1
  name: app1-v1-cwjdl
  namespace: zone-aware-routing
  ownerReferences:
    - apiVersion: v1
      blockOwnerDeletion: true
      controller: true
      kind: Service
      name: app1-v1
      uid: b94959a9-4d9c-4254-a1ea-9290a809b159
  resourceVersion: "74745418"
  uid: 2c00ea63-e807-4f4d-b76e-c13d235fe9b9
ports:
  - name: port-0-0
    port: 9090
    protocol: TCP

Results:
/routes/eu-central-1a - returns all 9 endpoints as expected

curl localhost:9990/routes/eu-central-1a 2>/dev/null | grep "app1_v1_25rzp__all__0_0"
kube_rg__zone_aware_routing__app1_v1_25rzp__all__0_0: Path("/test") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?|app1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Weight(3) && Method("GET") && Header("X-Forwarded-Proto", "https") -> disableAccessLog(2, 3, 404, 429) -> fifo(2000, 20, "1s") -> oauthTokeninfoAnyKV("realm", "/services") -> unverifiedAuditLog("sub") -> oauthTokeninfoAllScope("uid") -> flowId("reuse") -> forwardToken("X-TokenInfo-Forward", "uid", "scope", "realm") -> oauthTokeninfoValidate("{optOutAnnotations: [iam.zalando.org/public], optOutHosts: [\"^.*[.]ingress[.]cluster[.]local\"], unauthorizedResponse: \"Authentication required, see https://cloud.docs.zalando.net/howtos/authenticate-requests/#default-authentication\\n\"}") -> stateBagToTag("auth-user", "client.uid") -> <powerOfRandomNChoices, "http://[2a05:d024:90:1104:565e::10]:9090", "http://[2a05:d024:90:1104:565e::b]:9090", "http://[2a05:d024:90:1104:565e::d]:9090", "http://[2a05:d024:90:1105:43ef::12]:9090", "http://[2a05:d024:90:1105:43ef::14]:9090", "http://[2a05:d024:90:1105:43ef::8]:9090", "http://[2a05:d024:90:1105:43ef::b]:9090", "http://[2a05:d024:90:1105:43ef::c]:9090", "http://[2a05:d024:90:1105:bdf5::6]:9090">;

/routes/eu-central-1b - returns 3 endpoints as expected

curl localhost:9990/routes/eu-central-1b 2>/dev/null | grep "app1_v1_25rzp__all__0_0"
kube_rg__zone_aware_routing__app1_v1_25rzp__all__0_0: Path("/test") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?|app1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Weight(3) && Method("GET") && Header("X-Forwarded-Proto", "https") -> disableAccessLog(2, 3, 404, 429) -> fifo(2000, 20, "1s") -> oauthTokeninfoAnyKV("realm", "/services") -> unverifiedAuditLog("sub") -> oauthTokeninfoAllScope("uid") -> flowId("reuse") -> forwardToken("X-TokenInfo-Forward", "uid", "scope", "realm") -> oauthTokeninfoValidate("{optOutAnnotations: [iam.zalando.org/public], optOutHosts: [\"^.*[.]ingress[.]cluster[.]local\"], unauthorizedResponse: \"Authentication required, see https://cloud.docs.zalando.net/howtos/authenticate-requests/#default-authentication\\n\"}") -> stateBagToTag("auth-user", "client.uid") -> <powerOfRandomNChoices, "http://[2a05:d024:90:1104:565e::10]:9090", "http://[2a05:d024:90:1104:565e::b]:9090", "http://[2a05:d024:90:1104:565e::d]:9090">;

/routes/eu-central-1c - returns 6 endpoints as expected

curl localhost:9990/routes/eu-central-1c 2>/dev/null | grep "app1_v1_25rzp__all__0_0"
kube_rg__zone_aware_routing__app1_v1_25rzp__all__0_0: Path("/test") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?|app1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Host("^(app1-v1[.]pg9-test[.]zalan[.]do[.]?(:[0-9]+)?)$") && Weight(3) && Method("GET") && Header("X-Forwarded-Proto", "https") -> disableAccessLog(2, 3, 404, 429) -> fifo(2000, 20, "1s") -> oauthTokeninfoAnyKV("realm", "/services") -> unverifiedAuditLog("sub") -> oauthTokeninfoAllScope("uid") -> flowId("reuse") -> forwardToken("X-TokenInfo-Forward", "uid", "scope", "realm") -> oauthTokeninfoValidate("{optOutAnnotations: [iam.zalando.org/public], optOutHosts: [\"^.*[.]ingress[.]cluster[.]local\"], unauthorizedResponse: \"Authentication required, see https://cloud.docs.zalando.net/howtos/authenticate-requests/#default-authentication\\n\"}") -> stateBagToTag("auth-user", "client.uid") -> <powerOfRandomNChoices, "http://[2a05:d024:90:1105:43ef::12]:9090", "http://[2a05:d024:90:1105:43ef::14]:9090", "http://[2a05:d024:90:1105:43ef::8]:9090", "http://[2a05:d024:90:1105:43ef::b]:9090", "http://[2a05:d024:90:1105:43ef::c]:9090", "http://[2a05:d024:90:1105:bdf5::6]:9090">;

Context

Currently dataclient does not do filtering based on zone. There should be two code paths:

  • if routesrv is configured then dataclient needs to return all endpointslices
  • if skipper directly calls the dataclient it needs to filter based on zone

JTBD

When I configure skipper to use dataclient directly (without routesrv) in a multi-zone Kubernetes cluster / I want the dataclient to filter endpoints by my configured zone with threshold-based fallback / So that my skipper instance gets zone-aware routes without needing routesrv as an intermediary

When I configure routesrv to use dataclient to serve zone-specific routes / I want the dataclient to return all endpoints with zone metadata unfiltered / So that routesrv can perform dynamic per-request zone filtering based on the requested zone

Tasks

  • Modify GetEndpointSlicesByService and GetEndpointSlicesByTarget to check zone parameter for filtering mode
  • When zone is empty (routesrv mode), return all endpoints with zone metadata
  • When zone is set (direct skipper mode), filter by zone with threshold fallback
  • Verify dataclient and routesrv modes work correctly

@greeshma1196 greeshma1196 force-pushed the feature/zone-aware-routing branch from 8f58563 to 751dc5e Compare March 20, 2026 14:45
@greeshma1196 greeshma1196 marked this pull request as ready for review March 20, 2026 14:46
}

eps = state.GetEndpointsByService(ic.zone, ns, svcName, protocol, servicePort)
if state.enableEndpointSlices {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubernetes deprecated endpoints in favor of endpointslices and we used this switch only to enable it when it was more recent.
I think we can get rid of this.
Let's do the cleanup in a different PR.

Copy link
Copy Markdown
Member Author

@szuecs szuecs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some small changes should be enough

@szuecs szuecs force-pushed the feature/zone-aware-routing branch from 751dc5e to e7d1884 Compare March 23, 2026 11:11
@szuecs
Copy link
Copy Markdown
Member Author

szuecs commented Mar 26, 2026

👍

szuecs and others added 11 commits March 26, 2026 19:46
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
- update formatAndSet to filter based on zone
- update ServeHTTP based on zone
- add handler for `/routes/{zone}`
- add testcase - only 1 has been added; currently it fails (potentially we are losing zone information when LoadAll() is called)

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
- update filtering logic
- todo: add more test cases

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
…ts if the condition endpoints >= minEndpoints is met

- update logic in ServeHTTP - return all zone unaware endpoints if the above condition is not met
- add one more test case - currently failing need to check

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
…d zoneUnaware endpoints

- fix route construction in ingress
- update formatAndSet() to only store shallow copies of the route
- add test cases

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
…quality check works

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
…one metadata

- Ensure route construction always creates LBEndpoints with zone labels
- Implement threshold-based zone filtering in routesrv when serving routes
- Update expectations of testing zone-aware-routing in dataclient

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
@szuecs szuecs force-pushed the feature/zone-aware-routing branch from d935e61 to 3a72be5 Compare March 26, 2026 18:48
@szuecs
Copy link
Copy Markdown
Member Author

szuecs commented Mar 26, 2026

rebased on master

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
Copy link
Copy Markdown
Member Author

szuecs commented Mar 26, 2026

👍

Copy link
Copy Markdown
Member

@MustafaSaber MustafaSaber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did one run, but I think I need to have another deeper run after discussion.

}

func eqStrings(left, right []string) bool {
func eqLBEndpoints(left, right []*LBEndpoint) bool {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should be just another function? This is more of a general one while we change to more specific one, or we can implement string for LBEndpoint and just use it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot have LBEndpoint as string because we need zone information, unless I misunderstood this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an internal function and if it is not in use anymore we can drop it.

return result
}

func LBEndpointString(eps []*LBEndpoint) []string {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we sort, why not just implement Sting and use eqString?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String() on what and how to you suggest to use it?
We have a String() in LBEndpoint and use it in LBEndpointString(), so what is the suggestion?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant is something like

type LBEndpoints []*LBEndpoint

func (eps LBEndpoints) String() string {
	if len(sl) == 0 {
		return ""
	}
	return strings.Join(sl, ", ")
}


func (ep LBEndpoint) String() string {
	return fmt.Sprintf("%s/%s", ep.Zone, ep.Address)
}

So we can just run eqStrings(lc.LBEndpoints.String(), rc.LBEndpoints.String()) but if eqStrings is internal and not used anywhere else then it's fine to delete.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eqStrings is not used anywhere.


type LBEndpoint struct {
Address string
Zone string
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't seem to use this anywhere in this package

Copy link
Copy Markdown
Member Author

@szuecs szuecs Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see routesrv, the problem is that dataclients.LoadAll() returns []*eskip.Routes and to pass around zone information by endpoint we need to have this struct for routesrv.

Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
@greeshma1196
Copy link
Copy Markdown
Collaborator

Somewhere in the merge process this PR got messed up, I do not see changes from this PR here :/
#3927


for i := range left {
if left[i] != right[i] {
if left[i].Address != right[i].Address || left[i].Zone != right[i].Zone {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated question.
is that situation possible - left[i].Address == right[i].Address && left[i].Zone != right[i].Zone?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think this is possible, because each pod gets assigned a unique IP within a cluster and a pod exists in a zone. Probably this could come if there is stale data, i.e. a pod dies, new one comes and there is some time with stale data. But I need to double check on this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any time there is only one unique IP per pod, stale data would not change it, because then you would also have one unique IP per pod, maybe the pod does not exist, but the IP is unique at the old point of time.

greeshma1196 and others added 3 commits March 27, 2026 18:07
check zone parameter for filtering mode
- [x] When zone is empty (routesrv mode), return all endpoints with zone
metadata
- [x] When zone is set (direct skipper mode), filter by zone with
threshold fallback
- [x] Verify dataclient and routesrv modes work correctly

---------

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Co-authored-by: Sandor Szücs <sandor.szuecs@zalando.de>
Signed-off-by: greeshma1196 <greeshma.mathew@gmail.com>
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
Copy link
Copy Markdown
Member Author

szuecs commented Mar 28, 2026

I added a couple of missing tests that are only somewhat related to routesrv. These basically test that the change does not change valkey endpoints (these tests were completely missing before).

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@ponimas
Copy link
Copy Markdown
Member

ponimas commented Mar 31, 2026

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

architectural all changes in the hot path, big changes in the control plane, control flow changes in filters enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants