The gProfiler Agent

How does the gProfiler agent work?

The gProfiler agent is the engine that orchestrates, collects and stitches together data from sampling profilers. This data is then processed and presented in human-readable format (as a flame graph or in table view).

The gProfiler interface is a unified, system-wide display of what the CPU is spending resources on, and is based on the profiling data file created in the backend. Continuous, live samples of the CPU are shown as stack traces of all captured processes in native programs (including Golang), Java and Python runtimes, and kernel routines.

This cumulative view can be explored as a flame graph or in table format, and is derived from the following supplementary profilers:

• Python eBPF (Granulate’s proprietary profiler.)

• async-profiler by Andrei Pangin. See our fork.

• py-spy by Ben Frederickson. See our fork.

• bcc (for PyPerf) by the IO Visor project. See our fork.

• phpspy by Adam Saponara. See our fork.

• rbspy by the rbspy project. See our fork.

Data Collection

gProfiler does not collect personal information, and only looks at data that is relevant for its purpose. In terms of information regarding a provider’s cluster in the cloud, gProfiler reads each instance’s region, type, lifecycle, agent version, memory capacity and usage, and CPU count and current utilization.

Information collected by gProfiler which is used to create runtime-specific stack traces includes functions in Java and Python, for example, unlike perf, which produces native stacks of the JVM / CPython interpreter. Runtime stacks are then merged into the data collected by perf, and are substituted for the native stacks it has collected, in order to produce the most accurate system-wide view possible.

Theory of Operation

The comprehensive understanding of resource utilization enabled by gProfiler is in part due to its invocation of perf in system wide mode, which collects profiling data for all running processes. Alongside perf, and mentioned previously, gProfiler also invokes runtime-specific profilers for processes based on these environments:

• Java SE 7+: Based on the HotSpot JVM, including the Oracle JDK and other builds of OpenJDK like AdoptOpenJDK and Azul Zulu. Uses async-profiler.

• CPython Interpreter v2.7 and 3.5-3.10: Uses eBPF profiling (based on PyPerf) and requires Linux 4.14 or higher. See Python profiling options for more info. Note that if eBPF is not available then py-spy is used.

• PHP (Zend Engine), v7.0-8.0: Granulate's fork of the phpspy project.

• Ruby v1.9.1 - 3.0.1: Granulate's fork of the rbspy profiler.

• NodeJS v10+: Used for functioning -perf-prof. Uses perf inject --jit and NodeJS's ability to generate jitdump files. See NodeJS profiling options.