4 Replies Latest reply on Jul 13, 2017 7:03 AM by mnevill

    Gating Content (Google Crawling)


      Hey All,


      Just to confirm, if we were to put Forums discussions behind our SSO (changing permissions to view for only the "Registered Users" permission group), Google would no longer crawl these discussions, correct? Additionally, for anything that may appear in search results previously, that would disappear from Google's system after some time, I'd imagine....




        • Re: Gating Content (Google Crawling)

          Hey Ryan - Google is the #1 self-help channel according to several polls I've heard of and taking your community results out will likely prevent many people from finding your community and/or finding an answer they are seeking.  Additionally you create a login barrier that users have to jump through.  Let's say someone thinks useful content is in your community.  Now they either have to go through the registration process or find/remember their username and password before they can begin to self-help, which can be very frustrating. Unless you have truly sensitive content I would recommend not doing this.  I'm just curious why you would want to do this on an external community. I would recommend, you can only apply these restricted permissions to certain parts of your community for very good reasons and keep the majority open.  Also I wanted to mention, you can use the Robots.txt file to suggest search engines not crawl certain parts of the community.  Again that comes with the trade off of people not finding potentially useful content from Google, but sometimes it makes sense for things like a Community Help/Feedback section vs sections for products.

          • Re: Gating Content (Google Crawling)

            Correct x 2.  Google essentially honors the sitemap that you submit (http://yourcommunityURL.com/sitemap-index.jspa).  This sitemap is created automatically by Jive and its submission is controlled within the Admin console under System->Settings->Sitemap XML Settings.


            We do something similar with our detailed developer discussions and technical documentation/software. Essentially, users need to register to access both.


            We have worked with our platform consultant to develop what we call a teaser widget, and this teaser exposes 200 characters or so from the discussion to the public (i.e. Google and other spiders) once invoked (actions menu).  This gives us more control over the quality of threads that get exposed and helps SEO trremendously.

              • Re: Gating Content (Google Crawling)

                I see pros and cons to teaser content and widgets, but in general I've gotten a lot more bad feedback than good.  It sounds like they may be working well for you though.  As a consumer they frustrate me, because I may get a potential hit from Google only to discover I have to put in more effort (register or lookup login info) to read more and see if something is really useful and I'm a bit agitated I already invested a couple mins just getting to that point.  When confronted with that situation, I just go back to Google and find the next best potential hit, which very well may not be the company's official site.  If the content is really sensitive and a teaser is the only option I think it is better than nothing, but you run the risk of something coming out within the title and first 200 characters though many details could be missing.  I think it's important to have the philosophical discussion of what is really sensitive and why:Making the case for open content .  In my experience exposing as much as possible reduces customer pain points, helps to optimize self-help, improves call deflection, and increases customer satisfaction.  You have to get a little more creative to entice people to register, but it is worth that balance to make things quicker and easier for the majority.  Just my 2 cents, I know all of this depends on a lot of factors like use case, industry, etc.

                1 person found this helpful