What you’ll learn

  • How to add website content to your Knowledge Base
  • How to configure website crawling settings
  • Best practices for managing website-based Knowledge Bases

Adding Website Content to Your Knowledge Base

Overview

Adding a website as a knowledge base enables your AI assistant to access and utilize information from your organization’s website. This feature allows the assistant to provide accurate, up-to-date information directly from your web content.

Configuration Process

Website URL

Enter the complete website URL where your organization’s information is hosted. The URL should begin with ‘https://’ or ‘http://’ and point to the main domain you want to include in the knowledge base. For example: “https://example.com

Maximum Crawl Depth

The Maximum Crawl Depth setting determines how many levels deep the system will navigate through your website’s structure. A depth of 3 means the system will follow links from the main page through three levels of navigation. This helps control the scope of information gathered while ensuring relevant content is included.

Maximum Number of Pages

Set the maximum number of pages the system should process from your website. This number helps manage the amount of content added to your knowledge base. For instance, setting this to 10 means the system will process up to 10 pages from your website, prioritizing the most relevant content based on your configuration.

Best Practices

URL Selection

Choose the most relevant section of your website that contains information your AI assistant needs. Consider selecting specific sections rather than the entire website if only certain information is needed.

Depth Configuration

Select an appropriate crawl depth based on your website’s structure. Too shallow might miss important information, while too deep might include unnecessary content.

Page Limitation

Set the maximum number of pages based on your website’s size and the specific information needed. Consider starting with a smaller number and adjusting based on performance.

Best Practices

Regular Updates

Regularly update your website content to ensure your AI assistant provides the most accurate information.

Comprehensive Coverage

Use a combination of website and document knowledge bases for comprehensive coverage.