Custom Agent - Process with LLM using Rest API

Overview and Purpose

Custom Agents in ai12z empower your chatbot with the ability to retrieve and process data from external APIs—both REST and GraphQL—allowing the Reasoning Engine LLM to generate richer, context-driven responses. By connecting to these endpoints, the LLM is no longer constrained to static or preloaded data, but instead can dynamically fetch up-to-the-minute information, transform it as needed, and seamlessly incorporate it into the conversation.

Why choose a REST or GraphQL API data source?

• REST APIs:
  Ideal for established services or legacy systems, RESTful endpoints offer a wide array of structured responses that can be easily fetched and parsed. Whether it’s a weather service, a financial data feed, or a product inventory database, REST APIs are a tried-and-true method of tapping into external data.
• GraphQL APIs:
  Perfect for scenarios where you need precise, tailored data fetching, GraphQL allows you to query exactly what you need in a single request. This reduces payload size and minimizes the number of roundtrips, making it simpler and more efficient to pull complex, interrelated data.

Once the data is retrieved from these external sources, you can opt to pass it directly back to the LLM for processing. The LLM can then interpret, summarize, and integrate the fetched information into its response—enriching user interactions with current, accurate, and contextually relevant details.

Example Use Cases

Below are eight example scenarios where a Custom Agent might request data from an API, enabling the LLM to produce intelligent answers or drive automated workflows:

1. Weather Forecast Retrieval (REST API)

   • Request: Use an OpenWeatherMap REST endpoint to fetch current conditions for a user-specified city and country. Note: ai12z provides weather as an out of the box agent.
   • Use Case: The LLM can then provide a live weather forecast to a user planning a trip or deciding what to wear.

2. Product Inventory Check (GraphQL API)

   • Request: Query a Shopify GraphQL endpoint to retrieve updated inventory counts and product details.
   • Use Case: The LLM can inform a user whether a particular product is in stock and suggest alternatives if it’s not.

3. Flight Status Lookup (REST API)

   • Request: Call an airline’s REST endpoint to check the status of a flight by its flight number.
   • Use Case: The LLM can inform a traveler if their flight is on time, delayed, or canceled, and propose next steps.

4. Restaurant Recommendations (GraphQL API)

   • Request: Query Yelp’s GraphQL API for a user’s preferred cuisine and location.
   • Use Case: The LLM can provide personalized dining recommendations, menus, and even reservation options.

5. Stock Market Data (REST API)

   • Request: Pull real-time stock quotes and company fundamentals from a financial data REST service.
   • Use Case: The LLM can answer queries about a stock’s current price, historical trends, and suggest portfolio adjustments.

6. Appointment Scheduling (REST API)

   • Request: Retrieve available appointment slots from a CRM or booking system using a RESTful endpoint.
   • Use Case: The LLM can guide a user through scheduling a service appointment, automatically suggesting times and booking it on the user’s behalf.

7. Content Personalization (GraphQL API)

   • Request: Query a CMS’s GraphQL endpoint to fetch user-specific articles, news feeds, or training modules.
   • Use Case: The LLM can deliver customized reading lists or educational resources tailored to the user’s profile and preferences.

8. Logistics and Delivery Tracking (REST API)

   • Request: Access a shipping company’s REST endpoint with a user’s tracking number to get real-time delivery updates.
   • Use Case: The LLM can notify the user of their package’s current location and estimated arrival time, and suggest alternative options if delays occur.

---

By integrating Custom Agents with your REST or GraphQL data sources, the LLM becomes a powerful, action-oriented assistant capable of retrieving information on demand, reasoning over it, and producing responses that are both timely and highly relevant.

Example Custom Weather Agent

See the Edit Parameter dialog below:

Screen Elements

1. Fields:

   • Method:

     • Description: The HTTP method used for the agent’s request (e.g., GET, POST).
     • Input: Dropdown menu.

   • URL:

     • Description: The endpoint URL that the agent will call.
     • Input: Text field.

   • Headers:

     • Description: Any headers required for the API call.
     • Input: Expandable section to add key-value pairs.

2. Parameters:

   • Description: The LLM parameters, that the LLM passes to the Agent.
   • Table Columns:
     • Parameter: The name of the parameter.
     • Type: The data type of the parameter (e.g., string).
     • Parameter (mapping): How the parameter is used in the query (e.g., as a query parameter).
     • Action: Options to add, edit, or delete parameters.

3. JSONata Request:

   • Description: JSONata expression for modifying the request payload. Only used on a method POST. The LLM provides a flat list of parameter, what an API like HubSpot, requires is a more complex object. JSONata can transform the LLM data to what the end point API format needs.
   • Input: Text area.

4. JSONata Response:

   • Description: JSONata expression for transforming the response data. Typically used with Rest API responses, since you often get too much data back, with JSONata you can transform the return JSON to be what only you need.
   • Input: Text area.

5. Action Buttons:

   • Test:
     • Function: Allows you to test the APIs calling the endpoint with the current configuration, to validate the response from the endpoint.
   • Cancel:
     • Function: Discards changes and returns to the previous screen.
   • Save:
     • Function: Saves the changes made to the agent. Save as Publish or Draft. If draft the LLM will not see this Agent.

How to Use

1. Setting the Request Method and URL:

   • Select the appropriate HTTP method from the Method dropdown. Typically GET or POST
   • Enter the API endpoint URL in the URL field.

2. Adding Headers:

   • Click on the "+" icon next to Headers to add a new header.
   • Enter the header name and value.
     Example1 key: accept and value: application/json Example2 key: Authorization and value: Bearer eyJhbGciOiJIUzI1NiJ9....

3. LLM Parameter:

   • Under Parameters, click "Add Parameter" to include a new parameter.
   • Fill in the Parameter name, description, Type, and how it should be used, for example is it a query parmeter i.e. exists after the ?

4. Using JSONata:

   • JSONata Request: Input the JSONata expression for the request transformation. Note Generate JSONata button will only appear if you have body Param Type fiels, that you are doing a POST
   • JSONata Response: Input the JSONata expression for the response transformation.

5. Testing the Agent:

   • Click the "Test" button to test the agent with the current settings.

6. Saving Changes:

   • Click the "Save" button, Select Save as Draft or Save and Publish to save the changes.
   • If you wish to discard the changes, click the "Cancel" button.

Demo

JSON sent to the LLM

{
  "base": "stations",
  "clouds": {
    "all": 75
  },
  "cod": 200,
  "coord": {
    "lat": 42.3584,
    "lon": -71.0598
  },
  "dt": 1733680191,
  "id": 4930956,
  "main": {
    "feels_like": 41.86,
    "grnd_level": 1000,
    "humidity": 49,
    "pressure": 1004,
    "sea_level": 1004,
    "temp": 48.43,
    "temp_max": 50.29,
    "temp_min": 46.09
  },
  "name": "Boston",
  "sys": {
    "country": "US",
    "id": 2013408,
    "sunrise": 1733659279,
    "sunset": 1733692303,
    "type": 2
  },
  "timezone": -18000,
  "visibility": 10000,
  "weather": [
    {
      "description": "broken clouds",
      "icon": "04d",
      "id": 803,
      "main": "Clouds"
    }
  ],
  "wind": {
    "deg": 240,
    "gust": 26.46,
    "speed": 18.41
  }
}