Feeds: Frequently Asked Questions

Version 8

    Answers to questions on consuming and creating feeds.

    What's a simple way to pull data from the community?

    One easy way to syndicate content from the community to another site without having to do anything on the server is to use JSON. In fact, every single one of the RSS/Atom feeds can be rendered in JSON format by simply appending /json to the feed URL.  So in the case of syndicating Jive Software Community, you could simply have someone responsible for creating the skin add something like this to the HTML:

    <div id="jive-content"/>
    <script type="text/javascript" src="http://jivesoftware.com/community/community/feeds/allcontent/json"/>
     <script type="text/javascript">
    
         var ct = document.getElementById('jive-content');
         var content = '';
         var entries = Community.feed;
         for (var k in entries) {
             content += '<strong><a href="' + entries[k] + '">' + entries[k] +
             '</a></strong><br/>'; content += entries[k] + '<br/>';
         }
         content += '... <a href="' + Community.feed + '">(view all)</a><br/><br/>';
         ct.innerHTML = content;
    
     </script>
    

    This won't work by default with most communities because we lock down access to JSON for security reasons (mostly for internal deployments), but it's a lot easier than aggregating RSS feeds on the server.

     

    People who want to do this will need to set a system property secure-json-requests=false and they'll probably want to be using a public instance.

    What is a Struts feed result type?

    Struts actions can generate feeds, among other result types. When you define your Struts action, you need to specify "feed" as your result type. For example:

    <action name="examplefeed" class="com.example.actions.feeds.MyFeedAction">
         <result name="unauthorized" type="basicauth" ></result>
         <result name="success" type="feed" ></result>
    </action>
    

    How is authentication handled for feeds?

    By default HTTP Basic Authentication is turned on for feeds. It's important to note that your feed reader won't necessarily use any browser sessions that may be active. See security documentation for more details.

     

    If you want to avoid getting the Basic Auth prompt in your browser, you can use the URL syntax to embed the username and password into the URL. Here's an example:

    http://username:password@yoursite.com/community/feeds/allcontent
    

    What types you should generate?

    If you're creating a plugin for general use, you should support both RSS and JSON, but if it's for private use it's up to you which feed type is optimal. JSON is very handy for embedding data into UI elements or for making AJAX calls. RSS is best for content syndication.

    Can I disable feeds? Can I change the default feed format? What are the available feed formats? Can I set Basic Authentication on feeds?

    You can disable feeds, change the default feed format (it's RSS 2.0 out of the box) to Atom 1.0 or RSS 1.0, or require HTTP Basic Authentication for every feed in the system, you can change those settings in the admin console: click on system, then settings, then feeds.

    How do I create a feed in a plugin?

    The first thing you'll want to do is create a class that extends RSSActionSupport, implements the FeedAction interface and is annotated with the @AlwaysAllowAnonymous marker. You'll end up with a skeleton that looks something like this:

    package com.example.actions.feeds; @AlwaysAllowAnonymous
    public class MyFeedAction extends RSSActionSupport implements FeedAction {
        public Object getFeed() {
             
        }
    }
    

     

    Since the system supports multiple types of feeds, the getFeed() method returns an object whose type is simply Object. The @AlwaysAllowAnonymous annotation lets the request for this feed pass through the security interceptors so that the feed can handle security using HTTP Basic Auth, if necessary.  Extending RSSActionSupport gives you some helper methods that'll make the creation of the feed easier and implementing FeedAction enables your class to play nicely with the FeedResult class which itself takes care of some of the inane details associated with creating feeds: stuff like HTTP client caching, and feed serialization. You'll need to create an instance variable feed and override the:

    public String execute()

    method of the RSSActionSupport parent class, which will result in the body of your class looking like this:

    private Object feed;
    
    public Object getFeed() {
       return feed;
    }
    
    public String execute() {
       String result = super.execute();
       if (!result.equalsIgnoreCase(SUCCESS)) {
         return result;
       }
       ...
    }
    

    The call to RSSActionSupport.execute() handles two things for you: a) if feeds have been globally disabled, the parent execute() method will return the status string "disabled" and b) if feeds have been configured to require HTTP Basic Authentication, the parent execute() method will return "unauthorized". Assuming a Struts mapping that looks like this:

    <action name="examplefeed" class="com.example.actions.feeds.MyFeedAction">     <result name="disabled" type="dispatcher">/rsserror.jsp</result>
        <result name="error" type="dispatcher">/rsserror.jsp</result>
        <result name="unauthorized" type="basicauth" ></result>
        <result name="success" type="feed" ></result>
    </action>
    

    the user would be shown the RSS error page if feeds are disabled or if an error is encountered while creating the feed and would be presented with an HTTP Basic Authorization challenge if they didn't send Basic Authorization credentials in the request.

     

    Finally, you're ready to implement the guts of your feed. We use a feed library called ROME for creating RSS and Atom feeds so you don't need to know all the nitty gritty details about the RSS 2.0 specification, you just need to be able to get your list of widgets, tasks, pictures or whatever you're creating a feed of and then turn the list into something that ROME understands:

    List<Widget> widgets = getWidgets(); SyndFeed xmlFeed = new SyndFeedImpl();
     xmlFeed.setTitle("My Widgets");
     xmlFeed.setLanguage(JiveGlobals.getLocale().getLanguage());
     xmlFeed.setPublishedDate(publishDate);
     xmlFeed.setLink("http://example.com/widgets/");
     xmlFeed.setDescription("List of widgets");
     List<SyndEntry> entries = new ArrayList<SyndEntry>();
     SyndEntry entry;
     SyndContent description;
     for (Widget widget : widgets) {
         entry = new SyndEntryImpl();
         entry.setTitle(widget.getSubject());
         entry.setLink(getDefaultBaseURL() + "/widgets?widgetID=" + widget.getID());
         entry.setPublishedDate(widget.getCreationDate());
         entry.setUpdatedDate(widget.getModificationDate());
         description = new SyndContentImpl();
         description.setType("html");
         if (isFull()) {
             description.setValue(widget.getBody());
         }
         else {
             description.setValue(StringUtils.chopAtWord(
                 StringUtils.stripTags(widget.getBody(), true),
                 255));
         }
         entry.setDescription(description);
         entry.setAuthors(getAuthors(widget));
         entries.add(entry);
     }
     xmlFeed.setEntries(entries);
     feed = xmlFeed;
    

    A couple things to note in this code: the publish date variable is important. because feed aggregators use that date to determine whether or not your feed has actually been updated since the last time it tried to retrieve the feed. When we're creating feeds, we typically try to get the first item in the list of things that we're creating a feed of and then use the creation date of that item. The link 'http://example.com/widgets' is the link that represents the page where the user can view the non RSS representation of this feed. So for example, the feed for a blog is retrieved at this URL:

     

    http://example.com/community/blogs/aaron/feeds/posts

     

    and the link for the feed is this URL:

     

    http://example.com/community/blogs/aaron

     

    The idea behind the link for each individual entry is almost exactly the same idea so I won't go into explaining it here. One of the helper methods RSSActionSupport gives you is a full or not full instance variable. You can use the isFull() method to determine whether or not to return the full description of the item or just a teaser. If you or the user chooses a teaser, you can use the StringUtils class to strip out any HTML tags and then chop the description at N number of words. Finally, you can use the getAuthors(JiveContentObject) method to populate the list of author or authors associated with each entry, given that the item implements JiveContentObject. So that's it for creating a feed. Notice that you don't need to implement anything specific to RSS 2.0, RSS 1.0 or Atom 1.0. The conversion between the different feed formats is handled behind the scenes by the FeedResult class.

     

    There is one more thing you'll need to do if you want to support the JSON (JavaScript Object Notation) format.

     

    Right after you get that list of widgets:

    List<Widget> widgets = getWidgets();

    you'll need to add a check to see if the current request is for a JSON feed:

    if (feedType.equalsIgnoreCase(FeedAction.JSON))
    

    and then if so, you'll create a JSONObject to represent the same feed as a JSON feed:

    Map<String, Object> widgetFeed = new HashMap<String, Object>();
     widgetFeed.put("title", "My Widgets");
     widgetFeed.put("link", "http://example.com/widgets/");
     widgetFeed.put("description", "List of widgets");
     widgetFeed.put("publishDate", publishDate);
     Map<String, Object> entry;
     List<Map<String, Object>> entries = new ArrayList<Map<String, Object>>();
     for (Widget widget : widgets) {
         entry = new HashMap<String, Object>();
         entry.put("title", widgetFeed.getSubject());
         entry.put("link", getDefaultBaseURL() + "/widgets?widgetID=" + widget.getID());
         entry.put("date", widget.getCreationDate());
         entry.put("body", StringUtils.chopAtWord(
             StringUtils.stripTags(widget.getBody(), true), 255));
         entry.put("author", widget.getUser() == null ? "guest" : widget.getUser().getUsername());
         entries.add(entry);
     }
     JSONArray jsonEntries = JSONArray.fromCollection(entries);
     JSONObject jsonFeed = JSONObject.fromMap(widgetFeed);
     jsonFeed.put("entries", jsonEntries);
     feed = jsonFeed;
    

    The code is basically the same as before except this time you're using JSONObject and JSONArray. The benefit now is that you can syndicate your feed out to users and feed aggregators using RSS or Atom AND you can syndicate your feed out to browsers using JSON.

     

    How do you consume feeds? How do you fetch feeds?

    ROME has a subproject called ROME Fetcher. We've extended ROME Fetcher a bit, adding in some functionality around proxy servers and SSL, in a class called JiveHttpClientFeedFetcher so if you're writing a plugin that needs to consume an external RSS feed, say a list of bugs from JIRA or a list of commits from a Subversion server, you can use the JiveHttpClientFeedFetcher. If you're trying to consume a feed in a Java environment outside the application, you can use HttpClientFeedFetcher, which is part of the ROME Feed Fetcher library. Not coincidentally, we use this exact model here on Jivespace to get recent threads and blog posts into other parts of the site. Here's a snippet of the code we use on Jivespace:

    FeedFetcherCache feedInfoCache = HashMapFeedInfoCache.getInstance();
     FeedFetcher feedFetcher = new HttpClientFeedFetcher(feedInfoCache);
     SyndFeed feed = feedFetcher.retrieveFeed(new URL("http://jivesoftware.com/community/community/feeds/threads"));
     if (feed != null && feed.getEntries().size() > 0) {
         List entries = feed.getEntries();
         for (int i = 0; i < 3; i++) {
             SyndEntry entry = (SyndEntry) entries.get(i);
             out.write("<li>" + entry.getAuthor() + " wrote " + entry.getTitle() + '<br />');
             out.write(StringUtils.chop(StringUtils.stripTags(entry.getDescription().getValue()), 60) + "</li>");
         }
         out.write("... <a href='" + feed.getLink() + "'>(view all)</a>");
     }
    

    Pretty straightforward. The FeedFetcherCache keeps an in-memory cache of every feed the system has requested and FeedFetcher uses that cache to check to see if the feed has changed since the last time it was requested. If it hasn't, then it doesn't take the time to download the new feed but simply uses the version from it's cache.

     

    The code above will work fine as long as you've got access to Java or JSP on your website. If you've got PHP or some other non-Java based website or a static HTML website and you don't want to take the time to request and parse RSS on the server-side, you'll definitely want to take a look at JSON because it doesn't have to run on your server, it's run in the browser. So instead of writing Java code to consume the feed and then write the output to the response stream, you'll add a couple of lines of JavaScript to your page:

    <div id="threads"></div>
     <script type="text/javascript" src="http://jivesoftware.com/community/community/feeds/threads/json"></script>
     <script type="text/javascript">
         var ct = document.getElementById('threads');
         var content = '';
         var entries = Community.feed;
         for (var k in entries) {
             content += '<li>' + entries[k] + ' wrote ' + entries[k] + '<br />';
             // stripTags() and truncate(...) both come from the excellent prototype JavaScript library
             content += entries[k].stripTags().truncate(255) + '</li>';
         }
         content += '... <a href="' + Community.feed + '">(view all)</a>';
         ct.innerHTML = content;
     </script>
    

    which run in the browser. The first line defines a div element with an ID that you then populate with content from the JSON feed, you can actually see the source of the JSON feed if you view the URL in the example in your browser:

     

    http://jivesoftware.com/community/community/feeds/threads/json

     

    The JavaScript code then loops over every element in the feed, creates a list element for each item, adds the entry title and author to the list and then adds the resulting content to the inner HTML of the div defined on the first line.