AIAgent State

The AIAgentState is used to define an AI Agent state within a serverless workflow. This state encapsulates the configuration for an AI agent, detailing its name, type, language model, messages, tools, outcomes, and other essential properties. This specification provides a comprehensive overview of the AIAgentState and its related objects.

AIAgentState

The AIAgentState is a specialized state in a serverless workflow designed to integrate AI agents. The state defines the agent's configuration, including the AI model, system and user messages, tools for the agent to use, possible outcomes, and other properties.

Parameter	Description	Type	Required
agentName	The name of the agent.	string	yes
aiModel	The name of AI Language Model. Default value is 'gpt-4o'.	string	no
llmConfig	The configuration for the language model.	object	no
systemMessage	The system message used for constructing LLM prompt. Defaults to "You are a helpful AI Assistant."	string	yes
userMessage	The user message.	string	yes
maxToolExecutions	The maximum number of tool executions. Default is 10.	integer	no
memory	The memory of the agent. If not specify, the workflow process instance scope is used.	object	no
output	JSON schema for agent data output. See AgentDataOutput.	object	yes
tools	Define list of tools. Each tool is described by the ToolForAI schema.	array	no
agentOutcomes	Define list of agent outcomes. Each outcome is described by the OnAgentOutcome schema.	array	yes
dataFilter	Filter to apply to the state data.	string	no
multiAgentCollaboration	Multi-agent collaboration enables AI agent to delegate tasks to other collaborator agents. When enabled, the agent acts as a supervisor, coordinating responses from its collaborator agents.	object	no

LLMConfig

The LLMConfig defines the configuration for the AI Language Model to be used by the AI Agent.

Property	Type	Description	Required
provider	string	The name of the provider.	yes
apiKey	string	The API key to access the AI Language Model.	yes
overrideParams	object	The parameters to override for the provider. (Optional)	no
multiLLMsConfig	object	The strategy to use when multiple AI Language Models are used.	no

LLMParams

The LLMParams defines the parameters for the LLM.

Property	Type	Description	Required
model	string	The model to use for generating responses.	no
temperature	number	The sampling temperature.	no
top_p	number	The top-p sampling parameter.	no
n	integer	The number of completions to generate.	no
logprobs	integer	The number of log probabilities to return.	no
echo	boolean	Whether to echo back the prompt.	no
stop	array	Sequences where the model should stop generating further tokens.	no
max_tokens	integer	The maximum number of tokens to generate.	no
presence_penalty	number	The presence penalty parameter.	no
frequency_penalty	number	The frequency penalty parameter.	no
logit_bias	object	Logit bias configuration.	no

MultiLLMsConfig

The MultiLLMsConfig defines the configuration for multiple AI Language Models and the strategy to use when multiple AI Language Models are used.

Property	Type	Description	Required
strategy	object	The strategy to use when multiple AI Language Models are used.	yes
targets	array	The list of AI Language Models to use. See LLMProviderConfig	yes

MultiLLMStrategy

The MultiLLMStrategy defines the strategy to use when multiple AI Language Models are used.

Property	Type	Description	Required
mode	string	The mode for handling the request. It can be 'single', 'fallback', or 'loadbalance'.	yes
onStatusCodes	array	The status codes to trigger the strategy.	yes

The mode for handling the request. It can be 'single', 'fallback', or 'loadbalance'.

• single: This mode uses a single, specified Language Model API to handle requests.

• fallback: With numerous Language Model APIs available, each with unique strengths, seamlessly switching between them based on performance or availability is ideal. The fallback mode allows you to specify a prioritized list of providers/models. If the primary LLM fails, The AI Agent automatically switches to the next one, ensuring the execution robustness and reliability.

• loadbalance: Load Balance mode efficiently distributes network traffic across multiple LLMs. This ensures high availability and optimal performance of AI Agent, preventing any single LLM from becoming a performance bottleneck.

We can combine multiple handling strategies, such as fallback and load balance, to maximize both reliability and performance. This approach ensures efficient traffic distribution while providing fallback options in case of model failures.

Examples:

• single

{
  "apiKey": "sk-123456",
  "overrideParams": {
    "model": "claude-1"
  }
}

• fallback

{
  "strategy": {
      "mode": "fallback",
  },
  "targets": [
    {
      "apiKey": "sk-123456"
    },
    {
      "apiKey": "sk-789101",
      "overrideParams": {
          "model": "claude-1"
      }
    }
  ]
}

{
  "strategy": {
    "mode": "fallback",
    "onStatusCodes": [ 429 ]
  },
  "targets": [
    {
      "apiKey": "sk-123456"
    },
    {
      "apiKey": "sk-789101"
    }
  ]
}

• loadbalance

{
  "strategy": {
      "mode": "loadbalance",
  },
  "targets": [
    {
      "apiKey": "sk-123456",
      "weight": 0.75
    },
    {
      "apiKey": "sk-789101",
      "weight": 0.25
    }
  ]
}

• Combination:

{
  "strategy": {
    "mode": "fallback",
    "onStatusCodes": [
      429
    ]
  },
  "targets": [
    {
      "apiKey": "sk-main-123456"
    },
    {
      "strategy": {
        "mode": "loadbalance"
      },
      "targets": [
        {
          "apiKey": "sk-sub-456789",
          "weight": 0.75
        },
        {
          "apiKey": "sk-sub-987654",
          "weight": 0.25
        }
      ]
    }
  ]
}

LLMProviderConfig

The LLMProviderConfig defines the configuration for an AI provider.

Property	Type	Description	Required
provider	string	The name of the LLM provider.	yes
apiKey	string	The API key to use for the provider.	yes
weight	number	The weight of the provider, used for load balancing. (Optional)	no
overrideParams	object	The parameters to override for the provider. See LLMParams. (Optional)	no
strategy	object	The strategy to use for the target. (Optional)	no
targets	array	The list of LLM provider configurations for the target. (Optional)	no

ChatMemory

The ChatMemory object specifies the schema for the AI Agent chat memory configuration.

Field	Description	Type	Required
memoryId	The memory id to store the agent history.	string	yes
memoryType	Type of memory (message_window or token_window). Default is message_window.	string	no
maxMessages	The maximum number of messages to retain. If there isn't enough space for a new message, the oldest one is evicted. Used for message_window memory type.	integer	yes (if memoryType is message_window)
maxTokens	The maximum number of tokens to retain. Chat memory will retain as many of the most recent messages as can fit into maxTokens. Messages are indivisible. If an old message doesn't fit, it is evicted completely. Used for token_window memory type.	integer	yes (if memoryType is token_window)

AgentDataOutput

The AgentDataOutput object specifies the schema for the data output produced by the AI agent.

Field	Description	Type	Required
schema	URI of the JSON Schema used to describe the agent data output	string	yes

OnAgentOutcome

The OnAgentOutcome object defines the actions to be performed based on the outcome of the AI agent. The actions are executed if the outcome matches the specified conditions.

Field	Description	Type	Required
condition	Expression, if defined, must evaluate to true for this outcome to be matched. If false, the outcome is disregarded.	string	no
finish	If true, the agent will finish after this action is executed. If false, the agent will continue to process.	boolean	no
dataFilter	Event data filter described by the EventDataFilter schema.	object	no
transition	The outgoing transition when the outcome is selected, if not defined the default transition will be used.	object	no

ToolExecution

The ToolExecution object specifies how actions are to be performed by the tools used by the AI agent.

Field	Description	Type	Required
actionMode	Specifies how actions are to be performed (in sequence or parallel). Default is sequential.	string	no
actions	Actions to be performed, each described by the Action schema.	array	yes

ToolForAI

The ToolForAI object describes the tools available for use by the AI agent, including their parameters, output, and execution details.

Field	Description	Type	Required
name	Tool name	string	yes
description	Tool description	string	yes
parameters	JSON schema for tool parameters described by ToolParams schema.	object	yes
output	JSON schema for tool output described by ToolOutput schema.	object	no
execution	Tool execution definition described by ToolExecution schema.	object	no
metadata	Metadata	object	no

ToolOutput

The ToolOutput object specifies the schema for the output produced by the tools used by the AI agent.

Field	Description	Type	Required
schema	URI of the JSON Schema used to describe the tool output	string	yes

ToolParams

The ToolParams object defines the parameters required by the tools used by the AI agent.

Field	Description	Type	Required
schema	URI of the JSON Schema used to describe the parameters	string	yes

MultiAgentCollaboration

The MultiAgentCollaboration object defines the configuration of agents collaboration, enables AI agent to delegate tasks to other collaborator agents. When enabled, the agent acts as a supervisor, coordinating responses from its collaborator agents.

Field	Description	Type	Required
enabled	Turn on multi-agent collaboration to appoint this agent as a supervisor agent. A supervisor agent can designate one or more collaborator agents, delegate responses, and can also be used as an collaborator in another multi-agent collaboration.	boolean	no
collaborators	Define list of collaborator agents. See the CollaboratorAgent.	array	no

CollaboratorAgent

The CollaboratorAgent object defines the configuration of a collaborator agent which is designated to assist the supervior agent in a multi-agent collaboration. It can be used to delegate tasks and provide specialized responses.

Field	Description	Type	Required
name	The name of the collaborator agent.	string	yes
collaboratorInstruction	Describes the task that the collaborator agent performs within multi-agent collaboration. We recommend providing clear, specific scenarios. You can also define the style and tone. Refer to the agent using its collaborator name.	string	yes
agentRef	The reference to the agent state that this collaborator represents	string	yes

Example:

YAML

states: - name: AgentSelector agentName: AgentSelector type: aiagent aiModel: llama3-70b-8192 systemMessage: |- You are an assistant for selecting which tool is the most useful to use. \n Based on the tool's description, you have to return the name of the selected tool. userMessage: '${ "User: " + .request.question }' output: |- { "type": "object", "properties": { "selectedAgent": { "type": "string", "description": "The exact name of agent to be selected." } }, "required": [ "selectedAgent" ] } tools: - name: RAG_REACT description: |- RAG_REACT[input]: This tool is great for answering questions about searching products information, pricing, description, give advise and others about querying data. parameters: |- { "type": "object", "properties": { "input": { "type": "string", "description": "The search query" } }, "required": ["input"] } output: |- { "type": "object", "properties": { "answer": { "type": "string", "description": "The answer to the user question" }, "images": { "type": "array", "description": "The list of images that related to the answer to be displayed to the user", "items": { "type": "string", "format": "uri" } } }, "required": ["answer"] } - name: TRANSACTION_PROCESSING description: |- TRANSACTION_PROCESSING[request]: This tool is great for handle ordering, buying, booking products, payments, and others related to transactions request. parameters: |- { "type": "object", "properties": { "request": { "type": "string", "description": "The user request" } }, "required": ["request"] } output: |- { "type": "object", "properties": { "answer": { "type": "string", "description": "The answer." } }, "required": [ "answer" ] } agentOutcomes: - condition: '${ $agentOutcome.returnValues.selectedAgent == "RAG_REACT" }' finish: true transition: RagAgent - condition: '${ $agentOutcome.returnValues.selectedAgent == "TRANSACTION_PROCESSING" }' finish: true transition: ProductRetriver - condition: '${ true }' finish: true transition: RagAgent - name: RagAgent agentName: RagAgent type: aiagent aiModel: gpt-4o systemMessage: |- You are an assistant for question-answering tasks. Correct question to ensure the correctness of spelling and clarity of meaning in Vietnamese before answering. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Question: {question} Context: {context} Answer: userMessage: '${ .request.question }' output: |- { "type": "object", "properties": { "answer": { "type": "string", "description": "The answer to the user question" }, "images": { "type": "array", "description": "The list of images that related to the answer to be displayed to the user", "items": { "type": "object", "properties": { "url": { "type": "string", "format": "uri", "description": "The URL of the image" } } } } }, "required": ["answer"] } tools: - name: FIND_RELEVANT_DOCUMENTS description: |- FIND_RELEVANT_DOCUMENTS[question]: This tool is great for searching relevant documents, articles in the knowledge base related to the user question parameters: |- { "type": "object", "properties": { "input": { "type": "string", "description": "The search query" } }, "required": ["input"] } output: |- { "type": "object", "properties": { "documents": { "type": "array", "items": { "type": "string", "format": "uri" } } }, "required": ["documents"] } execution: actionMode: sequential actions: - name: FindRelevantDocuments functionRef: refName: FuncFindRelevantDocuments arguments: question: '${ .request.question }' actor: '${ .request.userContext }' actionDataFilter: results: '${ { "documents": .data } }' toStateData: ${ .request } agentOutcomes: - condition: '${ true }' finish: true transition: InformRagResult

---

This document provides a detailed view of the AIAgentState state and its related objects, including comprehensive schema definitions, required fields, and descriptions for each attribute within the AIAgentState and associated schemas. This specification ensures clarity and completeness for integrating AI agents within serverless workflows.