FastAPI#

The system consists of two main components:

Mesop Client App#

The client App serves as the frontend interface for the system. It includes:

• MesopUI: A friendly web interface for users to interact with the workflows. It facilitates the communication with the user and the FastAPIProvider.
• FastAPIProvider: A component that facilitates communication between the Mesop client and the FastAPIAdapter.

This app handles all client interactions and presents the results back to the user.

FastAPI App#

The FastAPI App forms the backend of our system and consists of:

• AutoGen Workflows: These define the core logic and behavior of our application, utilizing agents to perform various tasks and achieve specific goals.
• The FastAPIAdapter: This component communicates with AutoGen, and implements routes and websocket for FastAPI.
• FastAPI: Provides the infrastructure for building and exposing AutoGen workflows via REST API.

The system consists of two main components:

Custom Client App#

This application serves as the frontend interface for the system. It includes:

• Custom Client: A client application built in a different language, (e.g., HTML/JavaScript, Go, Java, etc.) facilitates communication between the user and the FastAPIAdapter.

This application handles all interactions with the FastAPIAdapter and presents the results back to the user.

FastAPI App#

The FastAPI App forms the backend of our system and consists of:

• AutoGen Workflows: These define the core logic and behavior of our application, utilizing agents to perform various tasks and achieve specific goals.
• The FastAPIAdapter: This component communicates with AutoGen, and implements routes and websocket for FastAPI.
• FastAPI: Provides the infrastructure for building and exposing AutoGen workflows via REST API.

Building Custom Client Applications#

To write a custom application that interacts with FastAPIAdapter, it's essential to first understand the available server routes it provides and their purposes. This knowledge forms the foundation of the client-server interaction model.

Available API Endpoints#

FastAPIAdapter provides three primary endpoints for building client applications:

Now that we understand the available routes, let's visualize how these components interact in a typical client-server communication flow.

System Interaction Flow#

The following sequence diagram illustrates the step-by-step process of how a custom client application interacts with the FastAPIAdapter:

sequenceDiagram
participant Client as Custom Client Application
participant FastAPI as FastAPIAdapter

Note over Client,FastAPI: 1. Setup & Discovery Phase
Client->>FastAPI: GET /fastagency/discovery
FastAPI-->>Client: Available Workflows

Client->>Client: Display workflow options to user

Note over Client,FastAPI: 2. Workflow Initiation
Client->>FastAPI: POST /fastagency/initiate_workflow
FastAPI-->>Client: Workflow Configuration

Note over Client,FastAPI: 3. WebSocket Connection
Client->>FastAPI: Initiate a WebSocket Connection (/fastagency/ws)
FastAPI-->>Client: WebSocket Connection Established

Note over Client,FastAPI: 4. Real-time Communication
Client->>FastAPI: Send Initial WebSocket Message
FastAPI-->>Client: Acknowledge Connection

activate FastAPI
activate Client
Note right of FastAPI: Message Processing Loop
FastAPI->>Client: Send Workflow Message
Client->>FastAPI: Send Response If Required
FastAPI->>Client: Send Next Workflow Message
deactivate Client
deactivate FastAPI

To better understand this diagram, let's break down the key steps involved in the client-server interaction:

Understanding the Flow#

The interaction between client and server follows these key steps:

• Discovery: Client fetches available workflows from the server.
• Selection: User selects a workflow to execute.
• Initiation: Client requests to start the chosen workflow.
• Connection: WebSocket connection established for real-time communication. These includes:
  • Server sending workflow message to the client.
  • Client sends optional response to server if previous server message requires user input.
  • Server processes and sends the next workflow message.

Message Types#

Before diving into the implementation, we need to learn a bit about the message types that FastAPIAdapter provides. Understanding these will help us handle messages in our custom client and display them properly to the users.

FastAgency tags each message sent from the server to the client over WebSockets with a type attribute. This helps the client differentiate between different types of messages and handle them accordingly. Let’s break them down into two categories:

Messages for Display:

• text_message: A basic text message from the server, intended for display to the user. It doesn’t require any action from the user, serving purely as information or status updates.
• workflow_started: This message indicates the start of a new workflow. The message includes the workflow’s name and a description along with other details.
• workflow_completed: This message signals that the current workflow has been successfully completed. The client can use this to notify the user or transition to the next step in the application.
• suggested_function_call: Indicates that the LLM has suggested a function call.
• function_call_execution: Indicates that the LLM has executed the suggested function call.
• error: Indicates that an error occurred during the workflow. The client can handle this by displaying an error message or prompting the user to retry the action.

Messages That Require User Response:

• text_input: This message prompts the client to gather input from the user. It could be a question or request for data. The client should provide a way for the user to respond (e.g., a text input or text area) and then send the response back to the server.
• multiple_choice: This message requires the user to choose from a predefined set of options provided by the LLM. The client should present these options (e.g., checkboxes or radio buttons) and submit the user’s selection back to the server.

A full list of message types and their detailed usage will be available soon in the FastAgency Adapter’s OpenAPI specification—stay tuned!

Implementation Guide#

In the following sections, we'll walk through the process of creating a custom client application that implements the flow we've just described. We'll build a simple web-based client that demonstrates how to interact with FastAPIAdapter effectively.

Our implementation will cover these key aspects:

• Fetching and displaying available workflows.
• Handling workflow initiation.
• Managing WebSockets connection.
• Processing real-time messages.

Let's begin by looking at the code structure and then break down each component.

Before getting started, ensure that FastAgency is installed with support for the AutoGen runtime, along with the mesop, fastapi, and server submodules by running the following command:

pip install "fastagency[autogen,mesop,fastapi,server]"

This command installs FastAgency with support for both the mesop and console interfaces for AutoGen workflows, but with FastAPI serving input requests and running workflows.

Before getting started, ensure that FastAgency is installed with support for the AutoGen runtime, along with the fastapi, and server submodules by running the following command:

pip install "fastagency[autogen,fastapi,server]"

This command installs FastAgency, but with FastAPI serving input requests and running workflows.

In this example, we'll create a simple learning chatbot where a student agent asks questions and a teacher agent responds, simulating a learning environment. We'll use MesopUI for the web interface and the FastAPIAdapter to expose the workflow as a REST API.

In this example, we'll create a simple learning chatbot where a student agent asks questions and a teacher agent responds, simulating a learning environment. We'll create a custom client using HTML and JavaScript for the web interface and the FastAPIAdapter to expose the workflow as a REST API.

Next, import the required modules from the FastAgency. Import the FastAPIAdapter class to expose the workflows as a REST API.

from typing import Any

from fastagency.adapters.fastapi import FastAPIAdapter
from fastapi import FastAPI

Next, import the required modules from the FastAgency and AutoGen. These imports provide the essential building blocks for creating agents, workflows, and integrating with the custom client. Additionally, import the FastAPIAdapter and HTMLResponse class to expose the workflows as a REST API.

import os
from typing import Any

from autogen.agentchat import ConversableAgent
from fastapi import FastAPI
from fastapi.responses import HTMLResponse

from fastagency import UI
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.runtimes.autogen import AutoGenWorkflows

4. Adapter Chaining#

Finally, create an additional specification file for an application using MesopUI to connect to the FastAPIAdapter.

from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.app import FastAgency
from fastagency.ui.mesop import MesopUI

fastapi_url = "http://localhost:8008"

provider = FastAPIAdapter.create_provider(
    fastapi_url=fastapi_url,
)
ui = MesopUI()


app = FastAgency(
    provider=provider,
    ui=ui,

4. Serving the Custom HTML Client#

Finally, use the HTML Response from FastAPI to serve the custom client code.

html = """
<!DOCTYPE html>
<html>
   <head>
      <title>FastAgency Chat App</title>
   </head>
   <body>
      <h1>FastAgency Chat App</h1>
      <div id="workflows"></div>
      <ul id="messages"></ul>
      <script>
         const API_URL = 'http://127.0.0.1:8008/fastagency';
         const WS_URL = 'ws://127.0.0.1:8008/fastagency/ws'; // nosemgrep
         let socket;

         async function fetchWorkflows() {
             const response = await fetch(`${API_URL}/discovery`);
             const workflows = await response.json();
             const container = document.getElementById('workflows');
             workflows.forEach(workflow => {
                 const button = document.createElement('button');
                 button.textContent = workflow.description;
                 button.onclick = () => startWorkflow(workflow.name);
                 container.appendChild(button);
             });
         }

         async function startWorkflow(name) {
             const payload = {
                 workflow_name: name,
                 workflow_uuid: generateUUID(),
                 user_id: null, // Set to null for single-user applications; otherwise, provide the appropriate user ID
                 params: {}
             };
             const response = await fetch(`${API_URL}/initiate_workflow`, {
                 method: 'POST',
                 headers: { 'Content-Type': 'application/json' },
                 body: JSON.stringify(payload)
             });
             const workflowJson = await response.json();
             connectWebSocket(workflowJson);
         }

         function connectWebSocket(workflowJson) {
             socket = new WebSocket(WS_URL);
             socket.onopen = () => {
                 const initMessage = {
                     name: workflowJson.name,
                     workflow_uuid: workflowJson.workflow_uuid,
                     user_id: workflowJson.user_id,
                     params: {}
                 };
                 socket.send(JSON.stringify(initMessage));
             };
             socket.onmessage = (event) => handleMessage(JSON.parse(event.data));
         }

         function handleMessage(message) {
             const messagesList = document.getElementById('messages');
             const li = document.createElement('li');
             if (message.type === 'text_input') {
                 const response = prompt(message.content.prompt);
                 socket.send(response);
                 li.textContent = `${message.sender} -> ${message.recipient}: ${message.content.prompt}`;
             } else {
                 li.textContent = `${message.sender} -> ${message.recipient}: ${message.content?.body || message?.type || JSON.stringify(message)}`;
             }
             messagesList.appendChild(li);
         }

         fetchWorkflows();

         // Helper function for generating UUID
         function generateUUID() {
            return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
                if (c === 'x') {
                return (Math.random() * 16 | 0).toString(16);
                } else {
                return (Math.random() * 16 | 0 & 0x3 | 0x8).toString(16);
                }
            });
         }
      </script>
   </body>
</html>
"""

@app.get("/")
async def get() -> HTMLResponse:
    return HTMLResponse(html)

In this setup, we need to run two commands in separate terminal windows:

uvicorn main_1_fastapi:app --host 0.0.0.0 --port 8008 --reload

gunicorn main_2_mesop:app -b 0.0.0.0:8888 --reload

pip install uvicorn
uvicorn main_1_fastapi:app --host 0.0.0.0 --port 8008 --reload

pip install gunicorn
gunicorn main_2_mesop:app -b 0.0.0.0:8888 --reload

pip install uvicorn
uvicorn main_1_fastapi:app --host 0.0.0.0 --port 8008 --reload

pip install waitress
waitress-serve --listen=0.0.0.0:8888 main_2_mesop:app

Once everything is set up, you can run your FastAgency application using the following command.

• Start FastAPI application using uvicorn:

uvicorn main_fastapi_custom_client:app --port 8008 --reload