Skip to main content

Creating and Uploading a Custom Sitemap using JSON

Overview​

Use a custom sitemap when you need to batch upload URLs and their associated content. While the "Add Website" feature supports filtering and is typically more efficient, the custom sitemap method is beneficial for specific bulk upload needs.

Sitemap CSV Format​

This CSV format lets you upload web URLs, functioning like a sitemap. Once uploaded, the system ingests all the web content linked to these URLs.

Example CSV:

{
  "type": "webLinks",
  "mode": "auto",
  "urls": [
    {
      "url": "https://docs.ai12z.net/docs/ai-settings/overview-prompt"
    },
    {
      "url": "https://docs.ai12z.net/docs/ai-settings/ai-cards"
    },
    {
      "url": "https://dev-cdn.ai12z.net/65afeadc9660870171632cf1/65c4f27e31d75cb01a26fafe/Trust%20within%20the%20ai12z%20copilot%20platform.pdf"
    }
  ]
}

Parameters​

  • type: Must be set to webLinks.
  • crawlDelay (Optional): Sets the delay between crawls. Default is 2 seconds, which is also the minimum value.
  • mode (Optional): Options are auto, sync,async or advanceCrawl. Default is auto. For most cases (99%) you should be set to auto.