A robots.txt file tells search engines what files and directories you don't want accessed or crawled on your site. Every Jive-powered community has this file, and it's stored in the root directory. Example: https://domainname.com/robots.txt.
If set up properly, a robots.txt file can be a useful tool. Here are a few tips to help you get started:
If you're not sure which files to block, you can work with your company's developer team or your Jive support team.
Noindex is the preferred method to prevent pages from being displayed in organic search results. While far better than robots.txt, this meta tag cannot be applied to community pages at this time (Jive 7 or lower). As an alternative, leverage your robots.txt file.