What would you like to be added:
From the ai gateway example, we'll create the gateway resources manually, this is just because envoy ai gateway is not that mature right now, for example, the health probe support: envoyproxy/ai-gateway#587.
Once the issue is fixed, we should pipeline this inside the controller. Also, we should consider how to set the configurations, like the weights.
Why is this needed:
Easy to use with llmaz.
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.
What would you like to be added:
From the ai gateway example, we'll create the gateway resources manually, this is just because envoy ai gateway is not that mature right now, for example, the health probe support: envoyproxy/ai-gateway#587.
Once the issue is fixed, we should pipeline this inside the controller. Also, we should consider how to set the configurations, like the weights.
Why is this needed:
Easy to use with llmaz.
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.