Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/docs/agent/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,20 @@

由于 python VeADK 的 Agent 基于 [LiteLLM]() 实现,因此您可以使用 LiteLLM 支持的所有模型提供商。您可以查看 [LiteLLM 支持的模型提供商列表](https://docs.litellm.ai/docs/providers)来设置 `model_provider` 参数。

基于 LiteLLM 的 [Fallbacks](https://docs.litellm.ai/docs/completion/reliable_completions) 容错机制,VeADK 实现了智能的模型切换能力。当目标模型经多次重试仍调用失败时,系统将自动切换至备选模型继续执行任务:

=== "Python"

```python title="agent.py" linenums="1" hl_lines="4"
from veadk import Agent

agent = Agent(
model_name=["doubao-seed-1-8-251215", "doubao-seed-1-6-251015", "doubao-seed-1.6-250615"]
)
```

在上述配置中,VeADK 默认会首先尝试 doubao-seed-1-8-251215,如果失败再去尝试后续模型,直至成功。

golang 暂不支持 LiteLLM,因此 VeADK 的 Agent 基于 OpenAI 实现,因此您可以使用所有支持OpenAI协议的模型。

#### 设置模型客户端
Expand Down
27 changes: 24 additions & 3 deletions veadk/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ class Agent(LlmAgent):
name (str): The name of the agent.
description (str): A description of the agent, useful in A2A scenarios.
instruction (Union[str, InstructionProvider]): The instruction or instruction provider.
model_name (str): Name of the model used by the agent.
model_name (Union[str, List[str]]): Name of the model used by the agent.
model_provider (str): Provider of the model (e.g., openai).
model_api_base (str): The base URL of the model API.
model_api_key (str): The API key for accessing the model.
Expand All @@ -93,7 +93,9 @@ class Agent(LlmAgent):
description: str = DEFAULT_DESCRIPTION
instruction: Union[str, InstructionProvider] = DEFAULT_INSTRUCTION

model_name: str = Field(default_factory=lambda: settings.model.name)
model_name: Union[str, list[str]] = Field(
default_factory=lambda: settings.model.name
)
model_provider: str = Field(default_factory=lambda: settings.model.provider)
model_api_base: str = Field(default_factory=lambda: settings.model.api_base)
model_api_key: str = Field(default_factory=lambda: settings.model.api_key)
Expand Down Expand Up @@ -183,10 +185,29 @@ def model_post_init(self, __context: Any) -> None:
min_tokens=0,
)
else:
fallbacks = None
if isinstance(self.model_name, list):
if self.model_name:
model_name = self.model_name[0]
fallbacks = [
f"{self.model_provider}/{m}" for m in self.model_name[1:]
]
logger.info(
f"Using primary model: {model_name}, with fallbacks: {self.model_name[1:]}"
)
else:
model_name = settings.model.name
logger.warning(
f"Empty model_name list provided, using default model from settings: {model_name}"
)
else:
model_name = self.model_name

self.model = LiteLlm(
model=f"{self.model_provider}/{self.model_name}",
model=f"{self.model_provider}/{model_name}",
api_key=self.model_api_key,
api_base=self.model_api_base,
fallbacks=fallbacks,
**self.model_extra_config,
)
logger.debug(
Expand Down