Quickstart: SQL-based analysis and trace-based metrics

This quickstart explains how to use trace_processor as well as its Python API to programmatically query the trace contents through SQL and compute trace-based metrics.

Trace Processor

TraceProcessor is a multi-format trace importing and query engine based on SQLite. It comes both as a C++ library and as a standalone executable: trace_processor_shell (or just trace_processor).

Setup

# Download prebuilts (Linux and Mac only)
curl -LO https://get.perfetto.dev/trace_processor
chmod +x ./trace_processor

# Start the interactive shell
./trace_processor trace.perfetto-trace

# Start a local trace processor instance to replace wasm module in the UI
./trace_processor trace.perfetto-trace --httpd

NOTE: In HTTP mode the trace will be loaded into the trace_processor and the UI will connect and issue queries over TCP. This can allow arbitrary sized traces to be loaded since there are no memory constraints, unlike the WASM module. In addition, this can improve performance in the UI as it issues SQL queries.

See Trace Processor docs for the full TraceProcessor guide.

Sample queries

For more exhaustive examples see the SQL section of the various Data sources docs.

Slices

Slices are stackable events which have name and span some duration of time.

> SELECT ts, dur, name FROM slice
ts                   dur                  name
-------------------- -------------------- ---------------------------
     261187017446933               358594 eglSwapBuffersWithDamageKHR
     261187017518340                  357 onMessageReceived
     261187020825163                 9948 queueBuffer
     261187021345235                  642 bufferLoad
     261187121345235                  153 query
     ...

Counters

Counters are events with a value which changes over time.

> SELECT ts, value FROM counter
ts                   value
-------------------- --------------------
     261187012149954          1454.000000
     261187012399172          4232.000000
     261187012447402         14304.000000
     261187012535839         15490.000000
     261187012590890         17490.000000
     261187012590890         16590.000000
...

Scheduler slices

Scheduler slices indicate which thread was scheduled on which CPU at which time.

> SELECT ts, dur, cpu, utid FROM sched
ts                   dur                  cpu                  utid
-------------------- -------------------- -------------------- --------------------
     261187012170489               267188                    0                  390
     261187012170995               247153                    1                  767
     261187012418183                12812                    2                 2790
     261187012421099               220000                    6                  683
     261187012430995                72396                    7                 2791
...

Trace-based metrics

Trace Processor offers also a higher-level query interface that allows to run pre-baked queries, herein called "metrics". Metrics are generally curated by domain experts, often the same people who add the instrumentation points in the first place, and output structured JSON/Protobuf/text. Metrics allow to get a summarized view of the trace without having to type any SQL or having to load the trace in the UI.

The metrics` schema files live in the /protos/perfetto/metrics directory. The corresponding SQL queries live in /src/trace_processor/metrics.

Run a single metric

Let's run the android_cpu metric. This metrics computes the total CPU time and the total cycles (CPU frequency * time spent running at that frequency) for each process in the trace, breaking it down by CPU (core) number.

./trace_processor --run-metrics android_cpu trace.perfetto-trace

android_cpu {
  process_info {
    name: "/system/bin/init"
    threads {
      name: "init"
      core {
        id: 1
        metrics {
          mcycles: 1
          runtime_ns: 570365
          min_freq_khz: 1900800
          max_freq_khz: 1900800
          avg_freq_khz: 1902017
        }
      }
      core {
        id: 3
        metrics {
          mcycles: 0
          runtime_ns: 366406
          min_freq_khz: 1900800
          max_freq_khz: 1900800
          avg_freq_khz: 1902908
        }
      }
      ...
    }
    ...
  }
  process_info {
    name: "/system/bin/logd"
    threads {
      name: "logd.writer"
      core {
        id: 0
        metrics {
          mcycles: 8
          runtime_ns: 33842357
          min_freq_khz: 595200
          max_freq_khz: 1900800
          avg_freq_khz: 1891825
        }
      }
      core {
        id: 1
        metrics {
          mcycles: 9
          runtime_ns: 36019300
          min_freq_khz: 1171200
          max_freq_khz: 1900800
          avg_freq_khz: 1887969
        }
      }
      ...
    }
    ...
  }
  ...
}

Running multiple metrics

Multiple metrics can be flagged using comma separators to the --run-metrics flag. This will output a text proto with the combined result of running both metrics.

$ ./trace_processor --run-metrics android_mem,android_cpu trace.perfetto-trace

android_mem {
  process_metrics {
    process_name: ".dataservices"
    total_counters {
      anon_rss {
        min: 19451904
        max: 19890176
        avg: 19837548.157829277
      }
      file_rss {
        min: 25804800
        max: 25829376
        avg: 25827909.957489081
      }
      swap {
        min: 9289728
        max: 9728000
        avg: 9342355.8421707246
      }
      anon_and_swap {
        min: 29179904
        max: 29179904
        avg: 29179904
      }
    }
    ...
  }
  ...
}
android_cpu {
  process_info {
    name: "/system/bin/init"
    threads {
      name: "init"
      core {
        id: 1
        metrics {
          mcycles: 1
          runtime_ns: 570365
          min_freq_khz: 1900800
          max_freq_khz: 1900800
          avg_freq_khz: 1902017
        }
      }
      ...
    }
    ...
  }
  ...
}

JSON and binary output

The trace processor also supports binary protobuf and JSON as alternative output formats. This is useful when the intended reader is an offline tool.

Both single and multiple metrics are supported as with proto text output.

./trace_processor --run-metrics android_mem --metrics-output=binary trace.perfetto-trace
<binary protobuf output>

./trace_processor --run-metrics android_mem,android_cpu --metrics-output=json trace.perfetto-trace
{
  "android_mem": {
    "process_metrics": [
      {
        "process_name": ".dataservices",
        "total_counters": {
          "anon_rss": {
            "min": 19451904.000000,
            "max": 19890176.000000,
            "avg": 19837548.157829
          },
          "file_rss": {
            "min": 25804800.000000,
            "max": 25829376.000000,
            "avg": 25827909.957489
          },
          "swap": {
            "min": 9289728.000000,
            "max": 9728000.000000,
            "avg": 9342355.842171
          },
          "anon_and_swap": {
            "min": 29179904.000000,
            "max": 29179904.000000,
            "avg": 29179904.000000
          }
        },
        ...
      },
      ...
    ]
  }
  "android_cpu": {
    "process_info": [
      {
        "name": "\/system\/bin\/init",
        "threads": [
          {
            "name": "init",
            "core": [
              {
                "id": 1,
                "metrics": {
                  "mcycles": 1,
                  "runtime_ns": 570365,
                  "min_freq_khz": 1900800,
                  "max_freq_khz": 1900800,
                  "avg_freq_khz": 1902017
                }
              },
              ...
            ]
            ...
          }
          ...
        ]
        ...
      },
      ...
    ]
    ...
  }
}

Python API

The API can be run without requiring the trace_processor binary to be downloaded or installed.

Setup

$ pip install perfetto

NOTE: The API is only compatible with Python3.

Example functions

See the Python API section of Trace Processor (SQL) to get more details on all available functions.

Query

from perfetto.trace_processor import TraceProcessor
tp = TraceProcessor(trace='trace.perfetto-trace')

qr_it = tp.query('SELECT name FROM slice')
for row in qr_it:
  print(row.name)

Output

eglSwapBuffersWithDamageKHR
onMessageReceived
queueBuffer
bufferLoad
query
...

Query as Pandas DataFrame

from perfetto.trace_processor import TraceProcessor
tp = TraceProcessor(trace='trace.perfetto-trace')

qr_it = tp.query('SELECT ts, name FROM slice')
qr_df = qr_it.as_pandas_dataframe()
print(qr_df.to_string())

Output

ts                   name
-------------------- ---------------------------
     261187017446933 eglSwapBuffersWithDamageKHR
     261187017518340 onMessageReceived
     261187020825163 queueBuffer
     261187021345235 bufferLoad
     261187121345235 query
     ...

Metric

from perfetto.trace_processor import TraceProcessor
tp = TraceProcessor(trace='trace.perfetto-trace')

cpu_metrics = tp.metric(['android_cpu'])
print(cpu_metrics)

Output

metrics {
  android_cpu {
    process_info {
      name: "/system/bin/init"
      threads {
        name: "init"
        core {
          id: 1
          metrics {
            mcycles: 1
            runtime_ns: 570365
            min_freq_khz: 1900800
            max_freq_khz: 1900800
            avg_freq_khz: 1902017
          }
        }
        core {
          id: 3
          metrics {
            mcycles: 0
            runtime_ns: 366406
            min_freq_khz: 1900800
            max_freq_khz: 1900800
            avg_freq_khz: 1902908
          }
        }
        ...
      }
      ...
    }
    ...
  }
}

Next steps

There are several options for exploring more of the trace analysis features Perfetto provides:

The trace conversion quickstart gives an overview on how to convert Perfetto traces to legacy formats to integrate with existing tooling.
The Trace Processor documentation gives more information about how to work with trace processor including details on how to write queries and how tables in trace processor are organized.
The metrics documentation gives a more in-depth look into metrics including a short walkthrough on how to build an experimental metric from scratch.
The SQL table reference gives a comprehensive guide to the all the available tables in trace processor.
The common tasks page gives a list of steps on how new metrics can be added to the trace processor.