Jive Support recognizes that the performance of your Jive implementation is the most important factor of your site's user experience. We're committed to working with our customers to optimize performance wherever possible, and we keep a constant line of communication with our Engineering team to ensure that performance problems in the platform and code are reported and fixed as quickly and effectively as possible.
Performance problems are often very challenging to understand. There are many layers of the user's interaction with the application in which performance issues can arise, and a multitude of factors can contribute to an end user experience which is summarized in one simple phrase: "it's too slow". Additionally, each instance is different, and the way our application is used "in the wild" is evolving with each release and each new customer site. The aim of this document is to shed some light on the process that Jive Support uses to gather the information that is needed to diagnose and resolve performance issues.
The Four W's of Performance Problems
Inevitably, the Jive Support Engineer assigned to your case will want to know some or all of the answers to the following questions:
- Who reported the problem? Have multiple users reported the problems?
- Can you reproduce the reported behavior?
- If only specific users have reported the problem, who are the users? Do they have anything in common? Are they administrators, external users, LDAP users, moderators?
- What is the problematic behavior? Requests hanging? HTTP errors (i.e. 403/404/500/503 etc.)? Long page loads? If so, how long?
- What is the size of the instance? How many users/spaces/groups/permission groups?
- What URL was the user accessing? What action was s/he taking at the time of the reported performance problem? Does this URL/action consistently reproduce the performance problem?
- What are the typical usage patterns of the site? For example, an internal corporate site is typically used between 9 AM and 5 PM local time, with the heaviest traffic around 11 AM and 3 PM. Or, an external site may receive a traffic spike one hour after the work day ends in a particular time zone. Knowing this information goes a long way toward understanding whether or not your problem is caused by high user load.
- What node are you on?
- What plugins do you have installed on your Jive instance?
- If performance issues are detected during a load test, what are the details of the load test? How many nodes in the cluster? What are the baseline metrics?
- When does the performance problem occur? Is it sporadic or does it occur consistently at a particular time? Provide as much information as you can about the exact time that the problem occurs. There's no such thing as "random" behavior in software. Every tiny data point is a clue that can help us track down a larger problem.
- If there is a window of time in which it occurs, when is that window and how long does it last?
- When does it perform well? Often, this information is as important as understanding when it performs poorly, as it provides us some baseline information to which we can compare the periods of poor performance.
- Where is the user physically located?
- Is the user accessing the site from home over a VPN? In the office? Which browser is s/he using? Are web request routed through a local proxy? Is there an Intrusion Detection System (IDS) on the machine or local network?
Tools of the Trade
Jive Engineers utilize a variety of homegrown and third party tools to gather information about the application. Some of these can be found here: Providing Troubleshooting Information to Support for Performance Issues. Here's a quick summary of tools you can use to gather information to address The Four W's:
|Charles||6.0 and earlier: appsnap Command / 7.0+ jive snap Command||top|
|Fiddler||sbs.out and sbs.log||ksar : a sar grapher|
|HTTP access logs|
|Live HTTP Headers||sbs-gc.log|
What to expect from Jive Support
Expect a lot of questions. Expect requests for data. Expect to be asked to reproduce in a test environment. Expect to be asked to upgrade to the latest patch release if you're running into known bugs.
Don't necessarily expect a quick fix.
Some issues we encounter are caused by known bugs or can be directly attributed to application settings that can easily be tweaked. Most aren't. Many reported problems fall outside the scope of our application (i.e. browser problems, local network configurations, geographic distance from the server, etc.). If we're doing our jobs properly, we're going to investigate all possible angles, and it can be a time-consuming process. Working through performance problems is often a collaborative process between Jive and the customer, rather than a "give us what we need and we'll take care of you" situation.
If we're able to identify an issue with the application, we'll follow up with you on the next steps. These are the most common scenarios:
Known bug: We'll request that you upgrade to a current patch release in which this bug is fixed.
New bug: Performance issues are often among the highest priority bug fixes. However, they aren't always quick or easy fixes. We'll work with you to find the best way to deliver the changes you need to fix your immediate problems, as well as suggesting a path to get you on a version in which the bug will be fixed.
Configurations: Small configuration tweaks such as the number of results returned in an RSS feed or the search filters you're using for your LDAP integration can often have huge performance implications. Very often there are no hard rules for how these settings should be configured. Expect to work with the Jive Support Engineer to implement one or more suggested configuration tweaks in a process to fully resolve the issue.
Test, Measure, Repeat: A good engineer knows that performing consistent, repeatable tests and gathering hard data are the best methods for understanding a problem. Repeating a process may make you feel like you're spinning your wheels but it's a necessary part of the engineering process.