Skip to main content

Store web page text in Retool Vectors

Learn how to store web page text in a Retool-managed vector database for use with AI models.

You can use Retool Vectors to store and generate AI-related vector data for text from publicly accessible web pages. Retool then crawls the specified URLs and fetches the text content. Retool can then provide AI models with related information when performing AI actions.

Create URL Vector​

You can create a URL Vector and add URLs for publicly accessible web pages manually or query the Retool Vectors resource in apps or workflows. You only need to specify a base URL, such as https://docs.retool.com—Retool crawls the URL and fetches text for any pages at this location.

  1. Navigate to the Resources settings and click Retool Vectors.
  2. Click Add Vector.
  3. Click Add URLs.
  4. Specify the web pages in the Links to crawl field.

Retool crawls the specified URLs and automatically fetches the text content and creates embeddings for each.

Refresh a URL vector​

If the content of crawled web pages has changed, use the Refresh URL Vector action to crawl the URLs again. Retool updates existing embeddings with any new content and calculates new vector data automatically.

  1. Set Vector action to Refresh URL Vector.
  2. Specify the vector to refresh in the Vector field.

List all URLs in a vector​

Use the List URLs action to retrieve a list of all crawled URLs within the specified vector.

  1. Set Vector action to List URLs.
  2. Specify the vector to refresh in the Vector field.

Get URL chunks​

Combine with the List URLs action

Use the List URLs action to get a list of URLs within a vector in a separate query. You can then reference a specific URL using {{ }} notation, such as {{ listUrls.data[2] }}.

Use the Get URL chunks action to retrieve the chunked text for a specified URL in a vector.

  1. Set Vector action to Get URL chunks.
  2. Specify the vector to use in the Vector field.
  3. Specify the URL for which to get chunks in the url name field.