Quick start#
You can try out FastAgency to build and deploy your multi-agent application in less than one hour by following this guide. You will learn how to write you workflow, test and debug it and finally deploy it on cloud. The deployed application will have chat interface.
You can use three different setups, depending on how scalable solution you need:
-
Mesop: This setup uses Mesop to build a web application for interacting with our workflow. It supports a single-worker deployments only, limiting its scalability. However, it is the fastest way to build and test your application.
-
FastAPI + Mesop: This is fairly scalable setup using FastAPI to execute your workflows and Mesop for interactive web application. FastAPI supports execution with multiple workers, with each workflow being executed in the context of a WebSocket connection. Mesop is still limited to a single worker, although there is much less load on it due to workflows being executed in the FastAPI workers.
-
NATS + FastAPI + Mesop: This is the most scalable setup using a distributed message broker NATS.io MQ. Workflows are being executed with multiple workers that attach to the MQ waiting for initiate workflow messages. Such workers can be running on different machines or even different data centers/cloud providers. Message queues are highly scalable, but more difficult to integrate with end-clients. In order to make such integrations easier, we will connect our NATS-based message queue with the FastAPI application.
Project setup#
We strongly recommend using Cookiecutter for setting up a FastAgency project. It creates the project folder structure, default workflow, automatically installs all the necessary requirements, and creates a devcontainer that can be used with Visual Studio Code for development.
You could also use virtual environment managers such as venv, and a Python package manager, such as pip.
-
Install Cookiecutter with the following command:
-
Run the
cookiecuttercommand: -
Depending on the type of the project, choose the appropriate option in step 3:
[1/5] project_name (My FastAgency App): [2/5] project_slug (my_fastagency_app): [3/5] Select app_type 1 - fastapi+mesop 2 - mesop 3 - nats+fastapi+mesop Choose from [1/2/3] (1): 2 [4/5] Select python_version 1 - 3.12 2 - 3.11 3 - 3.10 Choose from [1/2/3] (1): [5/5] Select authentication 1 - basic 2 - google 3 - none Choose from [1/2/3] (1):This command installs FastAgency with support for both the Console and Mesop interfaces for AutoGen workflows.
[1/5] project_name (My FastAgency App): [2/5] project_slug (my_fastagency_app): [3/5] Select app_type 1 - fastapi+mesop 2 - mesop 3 - nats+fastapi+mesop Choose from [1/2/3] (1): 1 [4/5] Select python_version 1 - 3.12 2 - 3.11 3 - 3.10 Choose from [1/2/3] (1): [5/5] Select authentication 1 - basic 2 - google 3 - none Choose from [1/2/3] (1):This command installs FastAgency with support for both the Console and Mesop interfaces for AutoGen workflows, with FastAPI handling input requests and workflow execution.
[1/5] project_name (My FastAgency App): [2/5] project_slug (my_fastagency_app): [3/5] Select app_type 1 - fastapi+mesop 2 - mesop 3 - nats+fastapi+mesop Choose from [1/2/3] (1): 3 [4/5] Select python_version 1 - 3.12 2 - 3.11 3 - 3.10 Choose from [1/2/3] (1): [5/5] Select authentication 1 - basic 2 - google 3 - none Choose from [1/2/3] (1):This command installs FastAgency with support for both the Console and Mesop interfaces for AutoGen workflows, with FastAPI serving input and independent workers communicating over the NATS.io protocol workflows. This is the most scable setup, recommended for large production workloads.
-
Executing the
cookiecuttercommand will create the following file structure:my_fastagency_app ├── deployment │ └── firebase │ ├── allowed_users.yaml │ └── firebase_config.yaml ├── docker │ ├── content │ │ ├── nginx.conf.template │ │ └── run_fastagency.sh │ └── Dockerfile ├── my_fastagency_app │ ├── deployment │ │ ├── __init__.py │ │ └── main.py │ ├── local │ │ ├── __init__.py │ │ ├── main_console.py │ │ └── main_mesop.py │ ├── __init__.py │ └── workflow.py ├── scripts │ ├── build_docker.sh │ ├── deploy_to_fly_io.sh │ ├── lint-pre-commit.sh │ ├── lint.sh │ ├── run_docker.sh │ ├── run_mesop_locally.sh │ ├── static-analysis.sh │ └── static-pre-commit.sh ├── tests │ ├── __init__.py │ ├── conftest.py │ └── test_workflow.py ├── README.md ├── fly.toml └── pyproject.tomlmy_fastagency_app ├── deployment │ └── firebase │ ├── allowed_users.yaml │ └── firebase_config.yaml ├── docker │ ├── content │ │ ├── nginx.conf.template │ │ └── run_fastagency.sh │ └── Dockerfile ├── my_fastagency_app │ ├── deployment │ │ ├── __init__.py │ │ ├── main_1_fastapi.py │ │ └── main_2_mesop.py │ ├── local │ │ ├── __init__.py │ │ ├── main_console.py │ │ └── main_mesop.py │ ├── __init__.py │ └── workflow.py ├── scripts │ ├── build_docker.sh │ ├── deploy_to_fly_io.sh │ ├── lint-pre-commit.sh │ ├── lint.sh │ ├── run_docker.sh │ ├── run_mesop_locally.sh │ ├── static-analysis.sh │ └── static-pre-commit.sh ├── tests │ ├── __init__.py │ ├── conftest.py │ └── test_workflow.py ├── README.md ├── fly.toml └── pyproject.tomlmy_fastagency_app ├── deployment │ └── firebase │ ├── allowed_users.yaml │ └── firebase_config.yaml ├── docker │ ├── content │ │ ├── nginx.conf.template │ │ └── run_fastagency.sh │ └── Dockerfile ├── my_fastagency_app │ ├── deployment │ │ ├── __init__.py │ │ ├── main_1_nats.py │ │ ├── main_2_fastapi.py │ │ └── main_3_mesop.py │ ├── local │ │ ├── __init__.py │ │ ├── main_console.py │ │ └── main_mesop.py │ ├── __init__.py │ └── workflow.py ├── scripts │ ├── build_docker.sh │ ├── deploy_to_fly_io.sh │ ├── lint-pre-commit.sh │ ├── lint.sh │ ├── run_docker.sh │ ├── run_mesop_locally.sh │ ├── static-analysis.sh │ └── static-pre-commit.sh ├── tests │ ├── __init__.py │ ├── conftest.py │ └── test_workflow.py ├── README.md ├── fly.toml └── pyproject.toml -
To run LLM-based applications, you need an API key for the LLM used. The most commonly used LLM is OpenAI. To use it, create an OpenAI API Key and set it as an environment variable in the terminal using the following command:
If you want to use a different LLM provider, follow this guide.
Alternatively, you can skip this step and set the LLM API key as an environment variable later in the devcontainer's terminal. If you open the project in Visual Studio Code using GUI, you will need to manually set the environment variable in the devcontainer's terminal.
-
Open the generated project in Visual Studio Code with the following command:
-
Once the project is opened, you will get the following option to reopen it in a devcontainer:
-
After reopening the project in devcontainer, you can verify that the setup is correct by running the provided tests with the following command:
You should get the following output if everything is correctly setup.
=================================== test session starts =================================== platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0 rootdir: /workspaces/my_fastagency_app configfile: pyproject.toml plugins: asyncio-0.24.0, anyio-4.6.2.post1 asyncio: mode=Mode.STRICT, default_loop_scope=None collected 1 item tests/test_workflow.py . [100%] ==================================== 1 passed in 1.02s ====================================Running the test could take up to 30 seconds, depending on latency and throughput of OpenAI (or other LLM providers).
Info
If you used a different project_slug than the default my_fastagency_app this will be reflected in the project module naming. Keep this in mind when running the commands further in this guide (in Run Application), you will need to replace my_fastagency_app with your project_slug name.
Workflow Development#
Define the Workflow#
You need to define the workflow that your application will use. This is where you specify how the agents interact and what they do. Here's a simple example of a workflow definition as it is generated by the cookie cutter under my_fastagency_app/workflow.py:
import os
from typing import Any
from autogen.agentchat import ConversableAgent
from fastagency import UI
from fastagency.runtimes.autogen import AutoGenWorkflows
llm_config = {
"config_list": [
{
"model": "gpt-4o-mini",
"api_key": os.getenv("OPENAI_API_KEY"),
}
],
"temperature": 0.8,
}
wf = AutoGenWorkflows()
@wf.register(name="simple_learning", description="Student and teacher learning chat") # type: ignore[misc]
def simple_workflow(ui: UI, params: dict[str, Any]) -> str:
initial_message = ui.text_input(
sender="Workflow",
recipient="User",
prompt="I can help you learn about mathematics. What subject you would like to explore?",
)
student_agent = ConversableAgent(
name="Student_Agent",
system_message="You are a student willing to learn.",
llm_config=llm_config,
)
teacher_agent = ConversableAgent(
name="Teacher_Agent",
system_message="You are a math teacher.",
llm_config=llm_config,
)
chat_result = student_agent.initiate_chat(
teacher_agent,
message=initial_message,
summary_method="reflection_with_llm",
max_turns=3,
)
return str(chat_result.summary)
This code snippet sets up a simple learning chat between a student and a teacher. It defines the agents and how they should interact and specify how the conversation should be summarized.
Run and Debug the Workflow#
To ensure that the workflow we have defined is working properly, we can test it locally using MesopUI. The code below can be found under my_fastagency_app/local/main_mesop.py and imports the defined workflow and sets up MesopUI:
from fastagency import FastAgency
from fastagency.ui.mesop import MesopUI
from ..workflow import wf
app = FastAgency(
provider=wf,
ui=MesopUI(),
title="My FastAgency App",
)
# start the fastagency app with the following command
# gunicorn my_fastagency_app.local.main_mesop:app
Run MesopUI locally with the following command:
Open the MesopUI URL http://localhost:8000 in your browser. You can now use the graphical user interface to start, run, test and debug the autogen workflow manually.
Run Workflow Tests#
We can also use pytest to test the autogen workflow automatically, instead of manually testing it using MesopUI by using the generated pytest test found under tests/test_workflow.py.
from uuid import uuid4
import pytest
from fastagency.ui.console import ConsoleUI
from my_fastagency_app.workflow import wf
from tests.conftest import InputMock
def test_workflow(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr("builtins.input", InputMock([""] * 5))
result = wf.run(
name="simple_learning",
ui=ConsoleUI().create_workflow_ui(workflow_uuid=uuid4().hex),
)
assert result is not None
Run the test with the following command:
Running the test could take up to 30 seconds, depending on latency and throughput of OpenAI (or other LLM providers).
Deployment files#
Depending on the interface you choose when setting up the project withcookiecutter, appropriate deployment files will be generated under my_fastagency_app/deployment folder. We'll quickly explain what they contain and how to use them.
Registering workflow with a UI or a provider#
The workflow definition will be registered either directly with an UI object or a network provider that will propagate messages back and forth over a network protocol.
In the case of a simple Mesop application, will register the workflow directly with MesopUI object and run it in the same process.
from fastagency import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
from ..workflow import wf
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=wf,
ui=ui,
title="My FastAgency App",
)
# start the fastagency app with the following command
# gunicorn my_fastagency_app.deployment.main:app
In the case of FastAPI application, we will create an FastAPIAdapter and then include a router to the FastAPI application. The adapter will have all REST and Websocket routes for communicating with a client.
from typing import Any
from fastagency.adapters.fastapi import FastAPIAdapter
from fastapi import FastAPI
from ..workflow import wf
adapter = FastAPIAdapter(provider=wf)
app = FastAPI()
app.include_router(adapter.router)
# this is optional, but we would like to see the list of available workflows
@app.get("/")
def list_workflows() -> dict[str, Any]:
return {"Workflows": {name: wf.get_description(name) for name in wf.names}}
# start the adapter with the following command
# uvicorn my_fastagency_app.deployment.main_1_fastapi:app --reload
In the case of NATS.io application, we will create an NatsAdapter and then add it to a FastAPI application using the lifespan parameter. The adapter will have all REST and Websocket routes for communicating with a client.
import os
from typing import Any
from fastagency.adapters.nats import NatsAdapter
from fastapi import FastAPI
from ..workflow import wf
nats_url = os.environ.get("NATS_URL", "nats://localhost:4222")
user: str = os.environ.get("FASTAGENCY_NATS_USER", "fastagency")
password: str = os.environ.get("FASTAGENCY_NATS_PASSWORD", "fastagency_nats_password")
adapter = NatsAdapter(provider=wf, nats_url=nats_url, user=user, password=password)
app = FastAPI(lifespan=adapter.lifespan)
# this is optional, but we would like to see the list of available workflows
@app.get("/")
def list_workflows() -> dict[str, Any]:
return {"Workflows": {name: wf.get_description(name) for name in wf.names}}
# start the adapter with the following command
# uvicorn my_fastagency_app.deployment.main_1_nats:app --reload
NatsAdapter requires a running NATS server. The easiest way to start the NATS server is by using Docker. FastAgency uses the JetStream feature of NATS and also utilizes authentication. websocket {
# listen: localhost:9222
port: 9222
no_tls: true
compress: true
}
jetstream {}
accounts {
AUTH {
jetstream: enabled
users: [
{ user: fastagency, password: $FASTAGENCY_NATS_PASSWORD }
]
}
APP {
jetstream: enabled
}
SYS {}
}
authorization {
auth_callout {
issuer: $NATS_PUB_NKEY
auth_users: [ fastagency ]
account: AUTH
}
}
system_account: SYS
In the above NATS configuration, we define a user called fastagency, and its password is read from the environment variable FASTAGENCY_NATS_PASSWORD. We also enable JetStream in NATS and configure NATS to serve via the appropriate ports.
Adapter Chaining#
In case we used network adapters in the step above, we can chain additional network adapters or UI objects to it.
Not applicable for this setup as there are no adapters used.
There is an additional specification file for an application using MesopUI to connect to the FastAPIAdapter
main_2_mesop.py
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.app import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
fastapi_url = "http://localhost:8008"
provider = FastAPIAdapter.create_provider(
fastapi_url=fastapi_url,
)
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=provider,
ui=ui,
title="My FastAgency App",
)
# start the provider with the following command
# gunicorn my_fastagency_app.deployment.main_2_mesop:app -b 0.0.0.0:8888 --reload
Above, we created NATS.io provider that will start brokers waiting to consume initiate workflow messages from the message broker. Now, we create FastAPI service interacting with NATS.io provider:
main_2_fastapi.py
from os import environ
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.adapters.nats import NatsAdapter
from fastapi import FastAPI
nats_url = environ.get("NATS_URL", "nats://localhost:4222")
nats_user: str = "fastagency"
nats_password: str = environ.get("FASTAGENCY_NATS_PASSWORD", "fastagency_nats_password")
provider = NatsAdapter.create_provider(
nats_url=nats_url, user=nats_user, password=nats_password
)
adapter = FastAPIAdapter(
provider=provider,
)
app = FastAPI()
app.include_router(adapter.router)
# this is optional, but we would like to see the list of available workflows
@app.get("/")
def read_root() -> dict[str, dict[str, str]]:
return {
"Workflows": {name: provider.get_description(name) for name in provider.names}
}
# start the provider with the following command
# uvicorn my_fastagency_app.deployment.main_2_fastapi:app --host 0.0.0.0 --port 8008 --reload
Finally, we create Mesop app communicating with the FastAPI application:
main_3_mesop.py
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.app import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
fastapi_url = "http://localhost:8008"
provider = FastAPIAdapter.create_provider(
fastapi_url=fastapi_url,
)
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=provider,
ui=ui,
title="My FastAgency App",
)
# start the provider with the following command
# gunicorn my_fastagency_app.deployment.main_3_mesop:app -b 0.0.0.0:8888 --reload
Authentication#
FastAgency provides three types of authentication mechanisms: Basic Authentication, Firebase Authentication and no authentication (more will be added soon). The default authentication mechanism is Basic Authentication. You can choose the type of authentication while setting up the project with Cookiecutter.
To use Basic Authentication, configure the desired usernames and their bcrypt hashed passwords in the BasicAuth class and apply the authentication object to the MesopUI object.
main.py
from fastagency import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
from ..workflow import wf
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=wf,
ui=ui,
title="My FastAgency App",
)
# start the fastagency app with the following command
# gunicorn my_fastagency_app.deployment.main:app
main_2_mesop.py
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.app import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
fastapi_url = "http://localhost:8008"
provider = FastAPIAdapter.create_provider(
fastapi_url=fastapi_url,
)
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=provider,
ui=ui,
title="My FastAgency App",
)
# start the provider with the following command
# gunicorn my_fastagency_app.deployment.main_2_mesop:app -b 0.0.0.0:8888 --reload
main_3_mesop.py
from fastagency.adapters.fastapi import FastAPIAdapter
from fastagency.app import FastAgency
from fastagency.ui.mesop import MesopUI
from fastagency.ui.mesop.auth.basic_auth import BasicAuth
fastapi_url = "http://localhost:8008"
provider = FastAPIAdapter.create_provider(
fastapi_url=fastapi_url,
)
auth = BasicAuth(
# TODO: Replace `allowed_users` with the desired usernames and their
# bcrypt-hashed passwords. One way to generate bcrypt-hashed passwords
# is by using online tools such as https://bcrypt.online
# Default password for all users is `password`
allowed_users={
"admin": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
"user@example.com": "$2y$10$ZgcGQlsvMoMRmmW4Y.nUVuVHc.vOJsOA7iXAPXWPFy9DX2S7oeTDa", # nosemgrep: generic.secrets.security.detected-bcrypt-hash.detected-bcrypt-hash
},
)
ui = MesopUI(auth=auth)
app = FastAgency(
provider=provider,
ui=ui,
title="My FastAgency App",
)
# start the provider with the following command
# gunicorn my_fastagency_app.deployment.main_3_mesop:app -b 0.0.0.0:8888 --reload
Run Application Locally#
Once everything is set up, you can run your FastAgency application using the following command:
The preferred way to run the Mesop application is using a Python WSGI HTTP server like Gunicorn on Linux and Mac or Waitress on Windows.
- In the root of your project run:
In this setup, we need to run two commands in separate terminal windows:
- Start FastAPI application using uvicorn, in the root of your project run:
Terminal 1
- Start Mesop web interface using gunicorn, in the root of your project run:
The NATS docker container is started automatically by Cookiecutter for this setup. In this setup, we need to run three commands in separate terminal windows:
- Start FastAPI application that provides a conversational workflow, in the root of your project run::
- Start FastAPI application integrated with a NATS messaging system, in the root of your project run::
Terminal 2
- Start Mesop web interface using gunicorn, in the root of your project run::
Output#
The outputs will vary based on the interface, here is the output of the last terminal starting UI:
[2024-10-10 13:19:18 +0530] [23635] [INFO] Starting gunicorn 23.0.0
[2024-10-10 13:19:18 +0530] [23635] [INFO] Listening at: http://127.0.0.1:8000 (23635)
[2024-10-10 13:19:18 +0530] [23635] [INFO] Using worker: sync
[2024-10-10 13:19:18 +0530] [23645] [INFO] Booting worker with pid: 23645
Deployment#
Building the Docker Image#
If you created the project using Cookiecutter, then building the Docker image is as simple as running the provided script, as shown below:
Running the above command will build the Docker image. If the build is successful, you will see output similar to the following:
Output
Building fastagency docker image
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.41kB done
#1 DONE 0.0s
#2 [internal] load metadata for docker.io/library/python:3.12
#2 DONE 1.6s
#3 [internal] load .dockerignore
#3 transferring context: 34B done
#3 DONE 0.0s
#4 [internal] load build context
#4 transferring context: 14.06kB done
#4 DONE 0.0s
#5 [1/8] FROM docker.io/library/python:3.12@sha256:fccc38d7080ff9883ee85a65a340384d04eb1c148a7222439b3dc5d4f0f72025
...
...
...
#12 DONE 0.2s
#13 exporting to image
#13 exporting layers
#13 exporting layers 0.9s done
#13 writing image sha256:d5b5432294fa293e3f8d5a2128c2ff012faa640fb552c43ce5faf240bce8bc0f done
#13 naming to docker.io/library/deploy_fastagency done
#13 DONE 0.9s
Successfully built fastagency docker image
Running the Docker Image#
Similarly, running the Docker container is as simple as running the provided script, as shown below:
Running the above command will start the Docker container in the foreground with the following output:
Output
Number of workers: 1
Nginx config:
upstream mesop_backend {
# Enable sticky sessions with IP hash
ip_hash;
server 127.0.0.1:8889;
}
server {
listen 8888;
server_name localhost;
# Security headers
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options "nosniff";
add_header X-XSS-Protection "1; mode=block";
location / {
proxy_pass http://mesop_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_buffering off;
# WSGI support
proxy_set_header X-Forwarded-Host $server_name;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
Starting gunicorn on port 8889
2024/11/12 07:52:47 [warn] 10#10: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1
[2024-11-12 07:52:47 +0000] [12] [INFO] Starting gunicorn 23.0.0
[2024-11-12 07:52:47 +0000] [12] [INFO] Listening at: http://0.0.0.0:8889 (12)
[2024-11-12 07:52:47 +0000] [12] [INFO] Using worker: sync
[2024-11-12 07:52:47 +0000] [29] [INFO] Booting worker with pid: 29
flaml.automl is not available. Please install flaml[automl] to enable AutoML functionalities.
2024-11-12 07:52:49,054 [INFO] Patching static file serving in Mesop
2024-11-12 07:52:49,055 [INFO] Initializing MesopUI: <fastagency.ui.mesop.mesop.MesopUI object at 0x74b4128986b0>
2024-11-12 07:52:49,059 [INFO] Initialized MesopUI: <fastagency.ui.mesop.mesop.MesopUI object at 0x74b4128986b0>
2024-11-12 07:52:49,059 [INFO] Initializing MesopUI: <fastagency.ui.mesop.mesop.MesopUI object at 0x74b403fffb30>
2024-11-12 07:52:49,059 [INFO] Initialized MesopUI: <fastagency.ui.mesop.mesop.MesopUI object at 0x74b403fffb30>
2024-11-12 07:52:49,059 [INFO] Initializing FastAgency <FastAgency title=Write FastAgency Docs> with workflows: <fastagency.adapters.fastapi.base.FastAPIProvider object at 0x74b416f21010> and UI: <fastagency.ui.mesop.mesop.MesopUI object at 0x74b403fffb30>
2024-11-12 07:52:49,059 [INFO] Initialized FastAgency: <FastAgency title=Write FastAgency Docs>
2024-11-12 07:52:49,068 [INFO] Importing autogen.base.py
INFO: Started server process [11]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8008 (Press CTRL+C to quit)
Deploying to Fly.io#
If you created the project using Cookiecutter, there is a built-in script to deploy your workflow to Fly.io. Run it as shown below:
Running the above command will prompt you to log in to your Fly.io account (if not already logged in) by opening a fly.io URL in your browser. The login prompt will look like this:
Logging into fly.io
failed opening browser. Copy the url (https://fly.io/app/auth/cli/78366a6d347a377a6e346465776167726f6b693537666a333674346978626d37) into a browser and continue
Opening https://fly.io/app/auth/cli/78366a6d347a377a6e346465776167726f6b693537666a333674346978626d37 ...
Waiting for session...
After logging in, the script will deploy your application to Fly.io without any further input. The output will look like this:
Output
successfully logged in
Deploying to fly.io
An existing fly.toml file was found for app write-fastagency-docs-cool-dew-7193
Using build strategies '[the "docker/Dockerfile" dockerfile]'. Remove [build] from fly.toml to force a rescan
Creating app in /workspaces/write_fastagency_docs
We're about to launch your app on Fly.io. Here's what you're getting:
Organization: Organization Name (fly launch defaults to the personal org)
Name: write-fastagency-docs-delicate-waterfall-7272 (generated)
Region: Amsterdam, Netherlands (from your fly.toml)
App Machines: shared-cpu-1x, 1GB RAM (from your fly.toml)
Postgres: <none> (not requested)
Redis: <none> (not requested)
Tigris: <none> (not requested)
Created app 'write-fastagency-docs-delicate-waterfall-7272' in organization 'personal'
Admin URL: https://fly.io/apps/write-fastagency-docs-delicate-waterfall-7272
Hostname: write-fastagency-docs-delicate-waterfall-7272.fly.dev
Wrote config file fly.toml
Validating /workspaces/write_fastagency_docs/fly.toml
✓ Configuration is valid
==> Building image
==> Building image with Depot
--> build: ()
[+] Building 15.2s (13/13) FINISHED
...
...
...
--> Build Summary: ()
--> Building image done
image: registry.fly.io/write-fastagency-docs-delicate-waterfall-7272:deployment-01JCFQP31QFJWA9HVVYPZYH1QN
image size: 498 MB
Watch your deployment at https://fly.io/apps/write-fastagency-docs-delicate-waterfall-7272/monitoring
Provisioning ips for write-fastagency-docs-delicate-waterfall-7272
Dedicated ipv6: 2a09:8280:1::4f:f553:0
Shared ipv4: 66.241.124.140
Add a dedicated ipv4 with: fly ips allocate-v4
This deployment will:
* create 2 "app" machines
No machines in group app, launching a new machine
Creating a second machine to increase service availability
Finished launching new machines
-------
NOTE: The machines for [app] have services with 'auto_stop_machines = "stop"' that will be stopped when idling
-------
Checking DNS configuration for write-fastagency-docs-delicate-waterfall-7272.fly.dev
Visit your newly deployed app at https://write-fastagency-docs-delicate-waterfall-7272.fly.dev/
Setting secrets
Updating existing machines in 'write-fastagency-docs-delicate-waterfall-7272' with rolling strategy
-------
✔ [1/2] Machine 48e2764ce93e58 [app] update succeeded
✔ [2/2] Machine e825942c739518 [app] update succeeded
-------
Checking DNS configuration for write-fastagency-docs-delicate-waterfall-7272.fly.dev