📄️ Overview
Overview
📄️ Add URL
Overview
📄️ Website ingestion
ai12z enables you to ingest entire websites for your AI copilot. This process extracts and indexes web content so your assistant can answer questions based on up-to-date site data.
📄️ Website and URL ingestion transform
Table of Contents
📄️ Add File
Your browser does not support the video tag.
📄️ Whitelisting the ai12z Crawler
Some websites run bot-protection services (Cloudflare, Akamai, Imperva, AWS WAF, and similar) that block or challenge automated crawlers by default. If a site you're trying to ingest is protected this way, the ai12z crawler can be silently blocked, and the ingested content will be empty, garbled, or a challenge/CAPTCHA page instead of the real content.
📄️ PDF
Overview
📄️ Custom Sitemap JSON
Overview
📄️ Custom Sitemap CSV
Overview
📄️ Bulk Content Upload JSON
Uploading bulk content through JSON is an efficient method for users who wish to import a large amount of content that is not already hosted on a website. This process is streamlined, allowing for the quick and seamless transfer of multiple content items in a single operation.
📄️ Bulk Content Upload CSV
Uploading bulk content through CSV is an efficient method for users who wish to import a large number of content items into the ai12z platform at once.
📄️ Local Crawler
The ai12z Local Crawler is a command-line tool you run on your own computer. It crawls a website with a real (headless) Chrome browser, extracts page text, JSON-LD, and PDF content, and packages everything into a JSON file that you upload to ai12z with Add File.
📄️ YouTube Videos
Your agent can answer questions from a YouTube video and play the video right in its response. To set this up, add the video's description and transcript to the Knowledge Base as an article, with the video's URL as the link back. Then add an instruction to the Answer AI system prompt that turns YouTube citation links into an embedded player.
📄️ Wordpress Website Custom Types
WordPress post types, including both default and popular custom types. However, there are a couple of points to note for accuracy: