Mchr3k - InMemProfiler

Home

InMemProfiler is a tool for tracking the allocation and lifetime of Java objects.

Download (see below for how to use these files):

Latest release made on 27/01/2012. See Release Notes for details.

The Problem

Java makes it really easy to write code which allocates a lot of short lived memory (e.g. String concatenation) or to use memory structures which are larger than you might expect (e.g. Vectors).

The Solution

This tool aims to answer the following questions.

  • How many instances of a class are being allocated?
  • What are the most common stack traces of these allocations?
  • How long do these instances live?
  • What class is responsible for their allocation?

Here is some example output from the tool.

Live objects:
  56:1 - [Ljava.lang.Object;
  24:1 - java.util.ArrayList

Live objects summary:
  80:2

Collected objects:
  0(s) - 5(s) :
    440:6 - [Ljava.lang.Object;
    24:1 - java.util.ArrayList
    8:1 - other.test.TestObject
  5(s) - 15(s) :
    80:10 - other.test.TestObject
    56:1 - [Ljava.lang.Object;
    24:1 - java.util.ArrayList
  15(s) - 25(s) :
    160:20 - other.test.TestObject
    112:1 - [Ljava.lang.Object;
    24:1 - java.util.ArrayList
  25(s) - 35(s) :
    240:30 - other.test.TestObject
    168:1 - [Ljava.lang.Object;
    24:1 - java.util.ArrayList
  35(s) - 45(s) :
  45(s) - 55(s) :
  55(s) - inf(s) :

  Buckets summary:
    472:8 - 0(s) - 5(s)
    160:12 - 5(s) - 15(s)
    296:22 - 15(s) - 25(s)
    432:32 - 25(s) - 35(s)
    0:0 - 35(s) - 45(s)
    0:0 - 45(s) - 55(s)
    0:0 - 55(s) - inf(s)

This output is generated from the following test program:

package test;

import java.util.ArrayList;
import java.util.List;

import other.test.TestObject;

public class Test
{
  public static void main(String[] args) throws Exception
  {
    // Allocate 30 objects
    List round1 = allocateObjects(30);
    Thread.sleep(10 * 1000);

    // Allocate 20 objects
    List round2 = allocateObjects(20);
    Thread.sleep(10 * 1000);

    // Allocate 10 objects
    List round3 = allocateObjects(10);
    Thread.sleep(10 * 1000);

    // Allocate 1 object, discarding 30 which are 30 seconds old
    round1 = allocateObjects(1);
    System.out.println(round1);

    // Discard 1 object which is 0 seconds old
    round1 = null;
    System.out.println(round2);

    // Discard 20 objects which are 20 seconds old
    round2 = null;
    System.out.println(round3);

    // Discard 30 objects which are 30 seconds old
    round3 = null;
    Thread.sleep(6 * 1000);
    System.out.println("===");
  }

  private static List allocateObjects(int len)
  {
    List list = new ArrayList();
    for (int ii = 0; ii < len; ii++)
    {
      list.add(new TestObject());
    }
    return list;
  }
}

Note: The extra ArrayList instance alive at the end of this code is created by a JVM system class. This will become clearer below.

This output is made possible by adding the following arguments to the JVM used to run this program.

-Xbootclasspath/a:./inmemprofiler.jar -agentpath:./inmemprofiler.dll=#bucket-5,15,25,35,45,55#gc-1#include-other,[Ljava.lang.Object,java.util.ArrayList#trackcollection

To find out more about the source of these objects you can use the #trace option to generate the following output:

Live objects:
  56:1 - [Ljava.lang.Object;

    Allocation Sites:
      56:1
        java.util.ArrayList.(ArrayList.java)
        java.util.ArrayList.(ArrayList.java)
        java.io.FilePermissionCollection.(FilePermission.java)
        java.io.FilePermission.newPermissionCollection(FilePermission.java)
        java.security.Permissions.getPermissionCollection(Permissions.java)
        java.security.Permissions.add(Permissions.java)
        java.net.URLClassLoader.getPermissions(URLClassLoader.java)
        sun.misc.Launcher$AppClassLoader.getPermissions(Launcher.java)
        java.security.SecureClassLoader.getProtectionDomain(SecureClassLoader.java)
        java.security.SecureClassLoader.defineClass(SecureClassLoader.java)
        java.net.URLClassLoader.defineClass(URLClassLoader.java)
        java.net.URLClassLoader.access$000(URLClassLoader.java)
        java.net.URLClassLoader$1.run(URLClassLoader.java)
        java.security.AccessController.doPrivileged(AccessController.java)
        java.net.URLClassLoader.findClass(URLClassLoader.java)
        java.lang.ClassLoader.loadClass(ClassLoader.java)
        sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java)
        java.lang.ClassLoader.loadClass(ClassLoader.java)

    Allocation Classes:
      java.security.Permissions
        56:1 - All methods
          56:1 - add
          56:1 - getPermissionCollection
      java.security.SecureClassLoader
        56:1 - All methods
          56:1 - getProtectionDomain
          56:1 - defineClass
      sun.misc.Launcher$AppClassLoader
        56:1 - All methods
          56:1 - loadClass
          56:1 - getPermissions
      java.io.FilePermission
        56:1 - All methods
          56:1 - newPermissionCollection
      java.lang.ClassLoader
        56:1 - All methods
          56:1 - loadClass
      java.net.URLClassLoader
        56:1 - All methods
          56:1 - access$000
          56:1 - findClass
          56:1 - defineClass
          56:1 - getPermissions
      java.util.ArrayList
        56:1 - All methods
          56:1 - 
          0:0 - add
          0:0 - ensureCapacity
      java.security.AccessController
        56:1 - All methods
          56:1 - doPrivileged
      java.net.URLClassLoader$1
        56:1 - All methods
          56:1 - run
      java.io.FilePermissionCollection
        56:1 - All methods
          56:1 - 

  24:1 - java.util.ArrayList

    Allocation Sites:
      24:1
        java.util.ArrayList.(ArrayList.java)
        java.io.FilePermissionCollection.(FilePermission.java)
        java.io.FilePermission.newPermissionCollection(FilePermission.java)
        java.security.Permissions.getPermissionCollection(Permissions.java)
        java.security.Permissions.add(Permissions.java)
        java.net.URLClassLoader.getPermissions(URLClassLoader.java)
        sun.misc.Launcher$AppClassLoader.getPermissions(Launcher.java)
        java.security.SecureClassLoader.getProtectionDomain(SecureClassLoader.java)
        java.security.SecureClassLoader.defineClass(SecureClassLoader.java)
        java.net.URLClassLoader.defineClass(URLClassLoader.java)
        java.net.URLClassLoader.access$000(URLClassLoader.java)
        java.net.URLClassLoader$1.run(URLClassLoader.java)
        java.security.AccessController.doPrivileged(AccessController.java)
        java.net.URLClassLoader.findClass(URLClassLoader.java)
        java.lang.ClassLoader.loadClass(ClassLoader.java)
        sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java)
        java.lang.ClassLoader.loadClass(ClassLoader.java)

    Allocation Classes:
      java.security.Permissions
        24:1 - All methods
          24:1 - add
          24:1 - getPermissionCollection
      java.security.SecureClassLoader
        24:1 - All methods
          24:1 - getProtectionDomain
          24:1 - defineClass
      sun.misc.Launcher$AppClassLoader
        24:1 - All methods
          24:1 - loadClass
          24:1 - getPermissions
      java.io.FilePermission
        24:1 - All methods
          24:1 - newPermissionCollection
      java.lang.ClassLoader
        24:1 - All methods
          24:1 - loadClass
      java.net.URLClassLoader
        24:1 - All methods
          24:1 - access$000
          24:1 - findClass
          24:1 - defineClass
          24:1 - getPermissions
      java.util.ArrayList
        24:1 - All methods
          24:1 - 
      java.security.AccessController
        24:1 - All methods
          24:1 - doPrivileged
      java.net.URLClassLoader$1
        24:1 - All methods
          24:1 - run
      java.io.FilePermissionCollection
        24:1 - All methods
          24:1 - 


Live objects summary:
  80:2

Collected objects:
  0(s) - 60(s) :
    776:9 - [Ljava.lang.Object;

      Allocation Sites:
        552:5
          java.util.Arrays.copyOf(Arrays.java)
          java.util.Arrays.copyOf(Arrays.java)
          java.util.ArrayList.ensureCapacity(ArrayList.java)
          java.util.ArrayList.add(ArrayList.java)
          test.Test.allocateObjects(Test.java)
          test.Test.main(Test.java)
        224:4
          java.util.ArrayList.(ArrayList.java)
          java.util.ArrayList.(ArrayList.java)
          test.Test.allocateObjects(Test.java)
          test.Test.main(Test.java)

      Allocation Classes:
        java.util.ArrayList
          776:9 - All methods
            552:5 - add
            552:5 - ensureCapacity
            224:4 - 
        test.Test
          776:9 - All methods
            776:9 - allocateObjects
            776:9 - main
        java.util.Arrays
          552:5 - All methods
            552:5 - copyOf

    488:61 - other.test.TestObject

      Allocation Sites:
        488:61
          test.Test.allocateObjects(Test.java)
          test.Test.main(Test.java)

      Allocation Classes:
        test.Test
          488:61 - All methods
            488:61 - allocateObjects
            488:61 - main

    96:4 - java.util.ArrayList

      Allocation Sites:
        96:4
          java.util.ArrayList.(ArrayList.java)
          test.Test.allocateObjects(Test.java)
          test.Test.main(Test.java)

      Allocation Classes:
        java.util.ArrayList
          96:4 - All methods
            96:4 - 
        test.Test
          96:4 - All methods
            96:4 - allocateObjects
            96:4 - main

  60(s) - inf(s) :

  Buckets summary:
    1360:74 - 0(s) - 60(s)
    0:0 - 60(s) - inf(s)
`

This was generated with the following set of options:

-Xbootclasspath/a:./inmemprofiler.jar -agentpath:./inmemprofiler.dll=#bucket-60#gc-1#include-other,[Ljava.lang.Object,java.util.ArrayList#trackcollection#trace

This output clearly shows that the extra ArrayList instance and corresponding Object[] comes from a call to URLClassLoader.getPermissions during classloading.

Usage

To use this tool you should download the following two files (select the appropriate dll or so file) into your application�s working directory:

  • InMemProfile jar
  • InMemProfile dll
  • InMemProfile so

You then need to add the following arguments to your java command.

Windows

-Xbootclasspath/a:./inmemprofiler.jar -agentpath:./inmemprofiler.dll

Linux

-Xbootclasspath/a:./inmemprofiler.jar -agentpath:./libinmemprofiler.so

You can specify arguments to the agent in string form e.g.

-Xbootclasspath/a:./inmemprofiler.jar -agentpath:./inmemprofiler.dll=#bucket-5,15,25,35,45,55#include-other,java.util.ArrayList#gc-1

The full list of options is described on the Profiler Options page.