Quickstart: SQL-based analysis and trace-based metrics

This quickstart explains how to use trace_processor as well as its Python API to programmatically query the trace contents through SQL and compute trace-based metrics.

Trace Processor

TraceProcessor is a multi-format trace importing and query engine based on SQLite. It comes both as a C++ library and as a standalone executable: trace_processor_shell (or just trace_processor).

Setup

# Download prebuilts (Linux and Mac only) curl -LO https://get.perfetto.dev/trace_processor chmod +x ./trace_processor # Start the interactive shell ./trace_processor trace.perfetto-trace # Start a local trace processor instance to replace wasm module in the UI ./trace_processor trace.perfetto-trace --httpd

NOTE: In HTTP mode the trace will be loaded into the trace_processor and the UI will connect and issue queries over TCP. This can allow arbitrary sized traces to be loaded since there are no memory constraints, unlike the WASM module. In addition, this can improve performance in the UI as it issues SQL queries.

See Trace Processor docs for the full TraceProcessor guide.

Sample queries

For more exhaustive examples see the SQL section of the various Data sources docs.

Slices

Slices are stackable events which have name and span some duration of time.

> SELECT ts, dur, name FROM slice ts dur name -------------------- -------------------- --------------------------- 261187017446933 358594 eglSwapBuffersWithDamageKHR 261187017518340 357 onMessageReceived 261187020825163 9948 queueBuffer 261187021345235 642 bufferLoad 261187121345235 153 query ...

Counters

Counters are events with a value which changes over time.

> SELECT ts, value FROM counter ts value -------------------- -------------------- 261187012149954 1454.000000 261187012399172 4232.000000 261187012447402 14304.000000 261187012535839 15490.000000 261187012590890 17490.000000 261187012590890 16590.000000 ...

Scheduler slices

Scheduler slices indicate which thread was scheduled on which CPU at which time.

> SELECT ts, dur, cpu, utid FROM sched ts dur cpu utid -------------------- -------------------- -------------------- -------------------- 261187012170489 267188 0 390 261187012170995 247153 1 767 261187012418183 12812 2 2790 261187012421099 220000 6 683 261187012430995 72396 7 2791 ...

Trace-based metrics

Trace Processor offers also a higher-level query interface that allows to run pre-baked queries, herein called "metrics". Metrics are generally curated by domain experts, often the same people who add the instrumentation points in the first place, and output structured JSON/Protobuf/text. Metrics allow to get a summarized view of the trace without having to type any SQL or having to load the trace in the UI.

The metrics` schema files live in the /protos/perfetto/metrics directory. The corresponding SQL queries live in /src/trace_processor/metrics.

Run a single metric

Let's run the android_cpu metric. This metrics computes the total CPU time and the total cycles (CPU frequency * time spent running at that frequency) for each process in the trace, breaking it down by CPU (core) number.

./trace_processor --run-metrics android_cpu trace.perfetto-trace android_cpu { process_info { name: "/system/bin/init" threads { name: "init" core { id: 1 metrics { mcycles: 1 runtime_ns: 570365 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902017 } } core { id: 3 metrics { mcycles: 0 runtime_ns: 366406 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902908 } } ... } ... } process_info { name: "/system/bin/logd" threads { name: "logd.writer" core { id: 0 metrics { mcycles: 8 runtime_ns: 33842357 min_freq_khz: 595200 max_freq_khz: 1900800 avg_freq_khz: 1891825 } } core { id: 1 metrics { mcycles: 9 runtime_ns: 36019300 min_freq_khz: 1171200 max_freq_khz: 1900800 avg_freq_khz: 1887969 } } ... } ... } ... }

Running multiple metrics

Multiple metrics can be flagged using comma separators to the --run-metrics flag. This will output a text proto with the combined result of running both metrics.

$ ./trace_processor --run-metrics android_mem,android_cpu trace.perfetto-trace android_mem { process_metrics { process_name: ".dataservices" total_counters { anon_rss { min: 19451904 max: 19890176 avg: 19837548.157829277 } file_rss { min: 25804800 max: 25829376 avg: 25827909.957489081 } swap { min: 9289728 max: 9728000 avg: 9342355.8421707246 } anon_and_swap { min: 29179904 max: 29179904 avg: 29179904 } } ... } ... } android_cpu { process_info { name: "/system/bin/init" threads { name: "init" core { id: 1 metrics { mcycles: 1 runtime_ns: 570365 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902017 } } ... } ... } ... }

JSON and binary output

The trace processor also supports binary protobuf and JSON as alternative output formats. This is useful when the intended reader is an offline tool.

Both single and multiple metrics are supported as with proto text output.

./trace_processor --run-metrics android_mem --metrics-output=binary trace.perfetto-trace <binary protobuf output> ./trace_processor --run-metrics android_mem,android_cpu --metrics-output=json trace.perfetto-trace { "android_mem": { "process_metrics": [ { "process_name": ".dataservices", "total_counters": { "anon_rss": { "min": 19451904.000000, "max": 19890176.000000, "avg": 19837548.157829 }, "file_rss": { "min": 25804800.000000, "max": 25829376.000000, "avg": 25827909.957489 }, "swap": { "min": 9289728.000000, "max": 9728000.000000, "avg": 9342355.842171 }, "anon_and_swap": { "min": 29179904.000000, "max": 29179904.000000, "avg": 29179904.000000 } }, ... }, ... ] } "android_cpu": { "process_info": [ { "name": "\/system\/bin\/init", "threads": [ { "name": "init", "core": [ { "id": 1, "metrics": { "mcycles": 1, "runtime_ns": 570365, "min_freq_khz": 1900800, "max_freq_khz": 1900800, "avg_freq_khz": 1902017 } }, ... ] ... } ... ] ... }, ... ] ... } }

Python API

The API can be run without requiring the trace_processor binary to be downloaded or installed.

Setup

$ pip install perfetto

NOTE: The API is only compatible with Python3.

Example functions

See the Python API section of Trace Processor (SQL) to get more details on all available functions.

Query

from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') qr_it = tp.query('SELECT name FROM slice') for row in qr_it: print(row.name)

Output

eglSwapBuffersWithDamageKHR onMessageReceived queueBuffer bufferLoad query ...

Query as Pandas DataFrame

from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') qr_it = tp.query('SELECT ts, name FROM slice') qr_df = qr_it.as_pandas_dataframe() print(qr_df.to_string())

Output

ts name -------------------- --------------------------- 261187017446933 eglSwapBuffersWithDamageKHR 261187017518340 onMessageReceived 261187020825163 queueBuffer 261187021345235 bufferLoad 261187121345235 query ...

Metric

from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') cpu_metrics = tp.metric(['android_cpu']) print(cpu_metrics)

Output

metrics { android_cpu { process_info { name: "/system/bin/init" threads { name: "init" core { id: 1 metrics { mcycles: 1 runtime_ns: 570365 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902017 } } core { id: 3 metrics { mcycles: 0 runtime_ns: 366406 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902908 } } ... } ... } ... } }

Next steps

There are several options for exploring more of the trace analysis features Perfetto provides: