Disable search engine indexing

You can prevent search engines from indexing pages, folders, your entire site, or just your webflow.io subdomain. This is uesful for things like 404 pages, and other not useful content.

How to prevent webflow.io subdomain from being indexed:

  1. Open your Project Settings
  2. Go to the SEO tab
  3. Under the Indexing section
  4. Set Disable Subdomain Indexing to "Yes"
  5. Click Save Changes
  6. Publish your site

How to prevent your entire site, a page, or folder of pages from being indexed:

  1. Open your Project Settings
  2. Go to the SEO tab
  3. Under the Indexing section
  4. Add robots.txt rules:
    Entire site:
    User-agent: * 
    Disallow: /

    Individual pages:
    User-agent: *
    Disallow: /page-name

    Entire folder of pages:
    User-agent: *
    Disallow: /folder-name/

  5. Click Save Changes
  6. Publish your site

How /robots.txt files work

The User-agent: * means this section applies to all robots. The Disallow:  tells the robot to not visit the site, page, or folder.