Providing Troubleshooting Information to Support

    In most Support cases, you will likely be asked to provide one or more types of diagnostic reports: application logs, thread dumps, heap dumps, TCP dumps, HTTP logs, LDIFs, Tomcat config files, etc. Each of these logs provides valuable information to Support which greatly facilitates the path to resolution. This document will explain what each type of log is used for and how you can gather the information that is needed to effectively troubleshoot your issues.

     

     

    TCP Dump

    When to use it:

    TCP dumps are useful for tracking all network traffic to and from a particular client or server. This is very useful for determining if your SBS and LDAP servers are communicating properly, and for taking a deeper look into the content of the LDAP requests and responses. Understanding the traffic between two nodes helps isolate the problematic behavior to the particular node which is causing it.


    Tools for the Job:

     

    Other tips:

    • In Wireshark, you can use a capture filter like:
    host <IP address>
    
    

    using the IP address of the LDAP server to capture all traffic between the SBS server and the LDAP server. This will give you a snapshot of the request and response traffic and give you some clues about whether the problem is on the SBS side or the LDAP side.

     

     

    HTTP Monitoring

    When to use it:

    Very useful for detecting problems with web resource loading (i.e. CSS, JavaScript, JSON calls, etc.). These tools will help you understand the fine-grained details of performance and functional problems at an HTTP level.

     

    Tools for the Job:

     

    Web Page Inspection

    When to use it:

    These tools are indispensable for the modern web developer. If your pages don't display the way you'd like them to, it is very easy to inspect a particular HTML element and determine what styling is applied to it. In addition, these tools allow you to change the page on the fly so you can test out your fixes in real time. This also can be very helpful for planning your customizations.

     

    Tools for the Job:

     

    Heap Dumps

    When to use it:

    Heap dumps are useful when attempting to identify the cause of OutOfMemoryErrors and/or poor performance. Heap dumps can be quite laborious to capture, transfer, and analyze, so Support will not usually ask for them unless there is a real need.

     

    How to create a Heap Dump using HeapDumper

     

    The program is intended to run on the server that you want to get a heap dump on. You must supply the absolute path to a directory to write the heap dump into using -dir=/some/path. For example, /tmp/. You can optionally supply a java process id using -pid=xxxx. This is useful for any java process including the eae or cache server. If you don't supply one, it will attempt to get the Tomcat process id automatically by default.

     

    The output file name is in the format heapdump_pid-<pid>_date-<date>.hprof, where date is month, day, year - hour, minute, second - time zone (all in the server's timezone). To run the program do the following:

    • copy the attached jar to the file system and chown it to jive
    • run it as the jive user: java -jar heapdumper.jar -pid=12345 -dir=/some/place
      • -dir= is required and -pid= is optional, but if not included the tomcat pid will be used

     

    For example:

    1. /usr/local/jive/java/bin/java -jar heapdump.jar -dir=/mnt/jive_temp 
    2. successfully wrote heap dump to /mnt/jive_temp/heapdump_pid-21552_date-020413-143603-MST.hprof in 30 seconds 

     

    Other tools for the Job:

    • jmap (older tool also used for capture)
    • MAT (for analysis)

     

    Thread Dumps

    When to use it:

    Thread dumps are a "snapshot" of all active Java processing threads in the Java Virtual Machine. Thread dumps are often the only tool which can help an engineer identify certain concurrency problems such as deadlocks. They also can be quite useful in determining the nature of the problem of a "hung" application.

     

    Tools for the Job:


     

    sar

    When to use it:

    sar is a Linux tool which gathers critical system data and stores it in files in /var/log/sa/ (may vary by distribution). This tool offers a convenient way to view a snapshot of critical system metrics over a given time period. Visualize sar data with ksar or another tool to view a profile of your system's performance and identify trouble areas in your hardware/software configuration.

     

    Tools for the Job:

    • ksar - point it at your server and run sar -A or sar -A -f /var/log/sa/saXX to graphically display your sar data. Love those pretty graphs!

    Other Tools