Converting arbitrary timestamped data to Perfetto
In this guide, you'll learn how to:
- Convert your own timestamped data into the Perfetto trace format.
- Create custom tracks, slices, and counters.
- Visualize your custom data in the Perfetto UI.
If you have existing logs or timestamped data from your own systems, you don't need to miss out on Perfetto's powerful visualization and analysis capabilities. By converting your data into Perfetto's native protobuf-based trace format, you can create synthetic traces that can be opened in the Perfetto UI and queried with Trace Processor.
This page provides a guide on how to programmatically generate these synthetic traces.
The Basics: Perfetto's Trace Format
A Perfetto trace file (.pftrace
or .perfetto-trace
) is a sequence of
TracePacket messages, wrapped in a
root Trace message. Each TracePacket
can
contain various types of data.
For generating traces from custom data, the most common and flexible payload to
use within a TracePacket
is the
TrackEvent. TrackEvent
allows you to define:
- Tracks: A single sequence of events (slices or counter) over time. Corresponds to a single "swim-lane" in the Perfetto UI.
- Slices: Events with a name, start timestamp, and duration (e.g., function calls, tasks).
- Counters: Numeric values that change over time (e.g., memory usage, custom metrics).
- Flows: Arrows connecting related slices across different tracks.
Generating Traces Programmatically
The examples in this guide use Python and a helper class from the perfetto
Python library to demonstrate how to construct these protobuf messages. However,
the underlying principles and protobuf definitions are language-agnostic. You
can generate Perfetto traces in any programming language that has Protocol
Buffer support.
- Official Protobuf Libraries: Google provides official protobuf compilers and runtime libraries for languages like Java, C++, Python, Go, and more.
- Third-Party Libraries: Numerous third-party libraries also provide protobuf support for a wide range of languages.
Regardless of the language, the core task is to construct TracePacket
messages
according to the Perfetto
protobuf schemas
and serialize them into a binary file.
Python Script Template
For the Python examples in the following sections, we'll use a script template.
This script handles the basics of creating a trace file and serializing
TracePacket
messages. You'll fill in the populate_packets
function with the
specific logic for the type of trace data you want to create.
First, ensure you have the perfetto
library installed, which provides the
necessary protobuf classes and potentially a builder utility (like the
TraceProtoBuilder
class you've designed, or an equivalent from the library).
pip install perfetto
Here is the Python script template. Save this as trace_converter_template.py
or a similar name. Each subsequent example will show you what code to place
inside the populate_packets
function.
Click to expand/collapse Python code
#!/usr/bin/env python3
import uuid
from perfetto.trace_builder.proto_builder import TraceProtoBuilder
from perfetto.protos.perfetto.trace.perfetto_trace_pb2 import TrackEvent, TrackDescriptor, ProcessDescriptor, ThreadDescriptor
def populate_packets(builder: TraceProtoBuilder):
"""
This function is where you will define and add your TracePackets
to the trace. The examples in the following sections will provide
the specific code to insert here.
Args:
builder: An instance of TraceProtoBuilder to add packets to.
"""
# ======== BEGIN YOUR PACKET CREATION CODE HERE ========
# Example (will be replaced by specific examples later):
#
# packet = builder.add_packet()
# packet.timestamp = 1000
# packet.track_event.type = TrackEvent.TYPE_SLICE_BEGIN
# packet.track_event.name = "My Example Event"
# packet.track_event.track_uuid = 12345
#
# packet2 = builder.add_packet()
# packet2.timestamp = 2000
# packet2.track_event.type = TrackEvent.TYPE_SLICE_END
# packet2.track_event.track_uuid = 12345
#
# ======== END YOUR PACKET CREATION CODE HERE ========
# Remove this 'pass' when you add your code
pass
def main():
"""
Initializes the TraceProtoBuilder, calls populate_packets to fill it,
and then writes the resulting trace to a file.
"""
builder = TraceProtoBuilder()
populate_packets(builder)
output_filename = "my_custom_trace.pftrace"
with open(output_filename, 'wb') as f:
f.write(builder.serialize())
print(f"Trace written to {output_filename}")
print(f"Open with [https://ui.perfetto.dev](https://ui.perfetto.dev).")
if __name__ == "__main__":
main()
To use this template:
- Save the code above as a Python file (e.g.
trace_converter_template.py
). - For each example in the sections that follow (e.g., "Thread-scoped slices,"
"Counters"), copy the Python code provided in that section and paste it into
the
populate_packets
function in yourtrace_converter_template.py
file, replacing the example placeholder content. - Run the script:
python trace_converter_template.py
. This will generatemy_custom_trace.pftrace
.
The TraceProtoBuilder class (which is imported from perfetto
pip package)
helps manage the list of TracePacket
messages that form the Trace
. The
populate_packets
function is where you'll define the content of these packets
based on your specific data.
Creating Basic Timeline Slices
The most fundamental way to represent an activity in Perfetto is as a "slice." A slice is simply a named event that has a start time and a duration. Slices live on "tracks," which are visual timelines in the Perfetto UI. Essentially, slices are used in any situation where you want to say "a named activity was happening during this specific interval of time."
Common examples of what slices can represent include:
- The interval of time during which a particular function was executing.
- The interval of time spent waiting for a server to respond to a network request.
- The time it takes for a resource (like an image, a script, or a data file) to load.
- The duration of a specific phase in an application's lifecycle, like "parsing data" or "rendering frame."
To create slices from your custom data, you'll typically:
- Define a track where your slices will appear. This is done using a
TrackDescriptor
packet. For basic custom data, you can create a generic track that isn't tied to a specific process or thread. - For each event in your data, emit
TrackEvent
packets to mark the beginning and end of the slice.
Python Example
Let's say you have data representing tasks with a name, start time, and end time. Here's how you could convert them into Perfetto slices on a custom track. This first example will show distinct, non-nested slices and a single instant event.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
# Define a unique ID for this sequence of packets (generate once per trace producer)
TRUSTED_PACKET_SEQUENCE_ID = 1001 # Choose any unique integer
# Define a unique UUID for your custom track (generate a 64-bit random number)
CUSTOM_TRACK_UUID = 12345678 # Example UUID
# 1. Define the Custom Track
# This packet describes the track on which your events will be displayed.
# Emit this once at the beginning of your trace.
packet = builder.add_packet()
packet.track_descriptor.uuid = CUSTOM_TRACK_UUID
packet.track_descriptor.name = "My Custom Data Timeline"
# 2. Emit events for this custom track
# Example Event 1: "Task A"
packet = builder.add_packet()
packet.timestamp = 1000 # Start time in nanoseconds
packet.track_event.type = TrackEvent.TYPE_SLICE_BEGIN
packet.track_event.track_uuid = CUSTOM_TRACK_UUID # Associates with the track
packet.track_event.name = "Task A"
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
packet = builder.add_packet()
packet.timestamp = 1500 # End time in nanoseconds
packet.track_event.type = TrackEvent.TYPE_SLICE_END
packet.track_event.track_uuid = CUSTOM_TRACK_UUID
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# Example Event 2: "Task B" - a separate, non-nested task occurring later
packet = builder.add_packet()
packet.timestamp = 1600 # Start time in nanoseconds
packet.track_event.type = TrackEvent.TYPE_SLICE_BEGIN
packet.track_event.track_uuid = CUSTOM_TRACK_UUID
packet.track_event.name = "Task B"
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
packet = builder.add_packet()
packet.timestamp = 1800 # End time in nanoseconds
packet.track_event.type = TrackEvent.TYPE_SLICE_END
packet.track_event.track_uuid = CUSTOM_TRACK_UUID
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# Example Event 3: An instantaneous event
packet = builder.add_packet()
packet.timestamp = 1900 # Timestamp in nanoseconds
packet.track_event.type = TrackEvent.TYPE_INSTANT
packet.track_event.track_uuid = CUSTOM_TRACK_UUID
packet.track_event.name = "Milestone Y"
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Nested Slices (Hierarchical Activities)
Often, an activity or operation is made up of several sub-activities that must complete before the main activity can finish. Nested slices are perfect for representing these hierarchical relationships. The key rule is that child slices must start after their parent slice begins and finish before their parent slice ends.
This is very common for:
- Function execution: A function call (parent slice) contains calls to other functions (child slices).
- Structured concurrency: Operations like Kotlin Coroutines, where child coroutines are launched within the scope of a parent coroutine and must complete before the parent.
- Phases of a larger operation: A complex task like "Compiling Module" (parent) might have distinct phases like "Lexical Analysis," "Parsing," "Optimization," and "Code Generation" as nested child slices.
- UI rendering pipelines: A "RenderFrame" slice might encompass "Measure Pass," "Layout Pass," and "Draw Pass" as child slices.
- Request handling with sub-operations: A web server handling a "ProcessHTTPRequest" (parent) might have nested slices for "ParseHeaders," "AuthenticateUser," "FetchDataFromDB," and "RenderResponse."
The Perfetto UI will visually nest these slices, making the hierarchy clear.
Python Example
This example demonstrates creating multiple stacks of nested slices on a custom
track. The packets are emitted in timestamp order to correctly represent the
nesting. We'll define a small helper function add_event
inside
populate_packets
to reduce boilerplate.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
# Define a unique ID for this sequence of packets
TRUSTED_PACKET_SEQUENCE_ID = 2002 # Using a new ID for this example
# Define a unique UUID for this example's custom track
NESTED_SLICE_TRACK_UUID = 987654321 # Example UUID
# 1. Define the Custom Track for Nested Slices
# Emit this once at the beginning.
packet = builder.add_packet()
packet.track_descriptor.uuid = NESTED_SLICE_TRACK_UUID
packet.track_descriptor.name = "My Nested Operations Timeline"
# Helper function to add a TrackEvent packet
def add_event(ts, event_type, name=None):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = event_type
packet.track_event.track_uuid = NESTED_SLICE_TRACK_UUID
if name:
packet.track_event.name = name
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# --- Stack 1: Operation Alpha ---
add_event(ts=2000, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Operation Alpha")
add_event(ts=2050, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Alpha.LoadConfig")
add_event(ts=2150, event_type=TrackEvent.TYPE_SLICE_END) # Closes Alpha.LoadConfig
add_event(ts=2200, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Alpha.Execute")
add_event(ts=2250, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Alpha.Execute.SubX")
add_event(ts=2350, event_type=TrackEvent.TYPE_SLICE_END) # Closes Alpha.Execute.SubX
add_event(ts=2400, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Alpha.Execute.SubY")
add_event(ts=2500, event_type=TrackEvent.TYPE_SLICE_END) # Closes Alpha.Execute.SubY
add_event(ts=2800, event_type=TrackEvent.TYPE_SLICE_END) # Closes Alpha.Execute
add_event(ts=3000, event_type=TrackEvent.TYPE_SLICE_END) # Closes Operation Alpha
# --- Stack 2: Operation Beta (on the same track) ---
add_event(ts=3200, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Operation Beta")
add_event(ts=3250, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Beta.Initialize")
add_event(ts=3350, event_type=TrackEvent.TYPE_SLICE_END) # Closes Beta.Initialize
add_event(ts=3400, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Beta.Process")
add_event(ts=3700, event_type=TrackEvent.TYPE_SLICE_END) # Closes Beta.Process
add_event(ts=3800, event_type=TrackEvent.TYPE_SLICE_END) # Closes Operation Beta
# --- An independent slice after all stacks ---
add_event(ts=4000, event_type=TrackEvent.TYPE_SLICE_BEGIN, name="Cleanup")
add_event(ts=4100, event_type=TrackEvent.TYPE_SLICE_END) # Closes Cleanup
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Asynchronous Slices and Overlapping Events
Many systems deal with asynchronous operations where multiple activities can be in progress simultaneously and their lifetimes can overlap without strict nesting. Examples include:
- Network Requests: A process might issue multiple network requests concurrently.
- Broadcast Receivers (Android): An application can receive multiple broadcast intents. The handling of each can overlap.
- Wakelocks (Android/Linux): Multiple components can hold wakelocks simultaneously.
- File I/O Operations: A program might initiate several asynchronous read or write operations to different files.
In these scenarios, you cannot represent all these overlapping events on a
single track if you are using begin/end slice semantics, because
TYPE_SLICE_END
always closes the most recently opened slice on that specific
track.
The Perfetto way to model this is to assign each concurrent, potentially
overlapping operation to its own unique track (with a unique UUID). To
achieve visual grouping in the Perfetto UI for these related asynchronous
operations, you can give the TrackDescriptor
of each of these individual
operation tracks the same name
(e.g., "Network Connections" or "File
I/O"). The slices themselves on these tracks can have distinct names (e.g., "GET
/api/data", "Read /config.txt").
The Perfetto UI will typically group or visually merge tracks that have the same name.
Python Example
Imagine we are tracking active network connections. Each connection is an independent asynchronous event. We'll give all connection tracks the same name to encourage the UI to group them. We'll use helper functions to define tracks and add events.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script:
Click to expand/collapse Python code
TRUSTED_PACKET_SEQUENCE_ID = 3003
# Common name for all individual connection tracks for UI grouping
ASYNC_TRACK_GROUP_NAME = "HTTP Connections"
# Helper to define a new track with a unique UUID
def define_track(group_name):
track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
packet = builder.add_packet()
packet.track_descriptor.uuid = track_uuid
packet.track_descriptor.name = group_name
return track_uuid
# Helper to add a begin or end slice event to a specific track
def add_slice_event(ts, event_type, event_track_uuid, name=None):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = event_type
packet.track_event.track_uuid = event_track_uuid
if name:
packet.track_event.name = name
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# --- Network Connection 1 ---
conn1_track_uuid = define_track(ASYNC_TRACK_GROUP_NAME)
add_slice_event(ts=1000, event_type=TrackEvent.TYPE_SLICE_BEGIN, event_track_uuid=conn1_track_uuid, name="GET /data/config")
add_slice_event(ts=1500, event_type=TrackEvent.TYPE_SLICE_END, event_track_uuid=conn1_track_uuid)
# --- Network Connection 2 (Overlapping with Connection 1) ---
conn2_track_uuid = define_track(ASYNC_TRACK_GROUP_NAME)
add_slice_event(ts=1100, event_type=TrackEvent.TYPE_SLICE_BEGIN, event_track_uuid=conn2_track_uuid, name="POST /submit/form")
add_slice_event(ts=2000, event_type=TrackEvent.TYPE_SLICE_END, event_track_uuid=conn2_track_uuid)
# --- Network Connection 3 (Starts after 1 ends, overlaps with 2) ---
conn3_track_uuid = define_track(ASYNC_TRACK_GROUP_NAME)
add_slice_event(ts=1600, event_type=TrackEvent.TYPE_SLICE_BEGIN, event_track_uuid=conn3_track_uuid, name="GET /status/check")
add_slice_event(ts=2200, event_type=TrackEvent.TYPE_SLICE_END, event_track_uuid=conn3_track_uuid)
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Counters (Values Changing Over Time)
Counters are used to represent a numerical value that changes over time. They are excellent for tracking metrics or states that are not event-based but rather reflect a continuous or sampled quantity.
Common examples of what counters can represent include:
- Memory usage: Total memory consumed by a process, or specific memory pools.
- CPU frequency: The current operating frequency of a CPU core.
- Queue sizes: The number of outstanding requests in a network queue or tasks in a work queue.
- Battery percentage: The remaining battery charge.
- Resource limits: The current value of a resource like file descriptors or network bandwidth being utilized.
To create a counter track, you'll:
- Define a
TrackDescriptor
for your counter. This track needs auuid
, aname
, and importantly, itscounter
field should be populated. This tells Perfetto to treat this track as a counter. - Emit
TrackEvent
packets withtype: TYPE_COUNTER
. Each such packet should have atimestamp
and acounter_value
(which can be an integer or a double).
Python Example
Let's say we want to track the number of outstanding network requests over time.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
TRUSTED_PACKET_SEQUENCE_ID = 4004
# UUID for the counter track
OUTSTANDING_REQUESTS_TRACK_UUID = uuid.uuid4().int & ((1 << 63) - 1)
# 1. Define the Counter Track
packet = builder.add_packet()
track_desc = packet.track_descriptor
track_desc.uuid = OUTSTANDING_REQUESTS_TRACK_UUID
track_desc.name = "Outstanding Network Requests"
# To mark this as a counter track, set the 'counter' field as existing.
track_desc.counter.SetInParent()
# Helper to add a counter event
def add_counter_event(ts, value):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = TrackEvent.TYPE_COUNTER
packet.track_event.track_uuid = OUTSTANDING_REQUESTS_TRACK_UUID
packet.track_event.counter_value = value
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# 2. Emit counter values over time
add_counter_event(ts=1000, value=0)
add_counter_event(ts=1100, value=1) # One request starts
add_counter_event(ts=1200, value=2) # Second request starts
add_counter_event(ts=1300, value=3) # Third request starts
add_counter_event(ts=1400, value=2) # First request finishes
add_counter_event(ts=1500, value=2) # No change
add_counter_event(ts=1600, value=1) # Second request finishes
add_counter_event(ts=1700, value=0) # Third request finishes
add_counter_event(ts=1800, value=1) # New request starts
add_counter_event(ts=1900, value=0) # Last request finishes
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Flows (Connecting Causally Related Events)
Flows are used to visually connect slices that have an explicit causal or dependency relationship, especially when these slices occur on different tracks (like different threads or even different processes). They are crucial for understanding how an action in one part of a system triggers or enables an action in another.
Think of flows as drawing an arrow from a "cause" or "dispatch" event to an "effect" or "handling" event. Common scenarios include:
- A UI thread dispatches a task to a worker thread: a flow connects the dispatch slice to the execution slice on the worker.
- A service makes an RPC/IPC call to another service: a flow can link the client-side call initiation to the server-side request handling.
- An event is posted to a message queue and later processed: a flow can show the link from posting to processing.
In Perfetto's TrackEvent
model, you establish a flow by:
- Assigning one or more unique 64-bit
flow_id
s to theTrackEvent
s that are part of the flow. This ID acts as the link. - Typically, a
flow_id
is added to aTYPE_SLICE_BEGIN
orTYPE_SLICE_END
event to mark the origin or termination of a causal link from/to that slice. - The same
flow_id
is then added to anotherTrackEvent
(often aTYPE_SLICE_BEGIN
on a different track) to show the continuation or handling of that causally linked operation.
The Perfetto UI will draw arrows connecting the slices that share a common
flow_id
, making the dependency chain explicit.
Python Example
Let's model a simple system where a "Request Handler" track dispatches work to a "Data Processor" track. We'll use flows to link the request dispatch to its processing, and then link the processing completion back to the handler acknowledging completion.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
TRUSTED_PACKET_SEQUENCE_ID = 5005
# --- Define Custom Tracks ---
REQUEST_HANDLER_TRACK_UUID = uuid.uuid4().int & ((1 << 63) - 1)
DATA_PROCESSOR_TRACK_UUID = uuid.uuid4().int & ((1 << 63) - 1)
# Request Handler Track
packet = builder.add_packet()
packet.track_descriptor.uuid = REQUEST_HANDLER_TRACK_UUID
packet.track_descriptor.name = "Request Handler"
# Data Processor Track
packet = builder.add_packet()
packet.track_descriptor.uuid = DATA_PROCESSOR_TRACK_UUID
packet.track_descriptor.name = "Data Processor"
# Helper to add a slice event (BEGIN or END)
def add_slice_event(ts, event_type, event_track_uuid, name=None, flow_ids=None):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = event_type
packet.track_event.track_uuid = event_track_uuid
if name:
packet.track_event.name = name
if flow_ids:
for flow_id in flow_ids:
packet.track_event.flow_ids.append(flow_id)
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# --- Define unique flow IDs for the causal links ---
DISPATCH_TO_PROCESS_FLOW_ID = uuid.uuid4().int & ((1<<63)-1)
PROCESS_COMPLETION_FLOW_ID = uuid.uuid4().int & ((1<<63)-1)
# 1. Request Handler: Dispatch data processing (origin of the first flow)
add_slice_event(ts=1000, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=REQUEST_HANDLER_TRACK_UUID, name="DispatchProcessing",
flow_ids=[DISPATCH_TO_PROCESS_FLOW_ID])
add_slice_event(ts=1050, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=REQUEST_HANDLER_TRACK_UUID)
# 2. Data Processor: Process the data (flow from handler's dispatch)
# This slice's BEGIN event includes DISPATCH_TO_PROCESS_FLOW_ID, linking it.
# It also starts the PROCESS_COMPLETION_FLOW_ID from its BEGIN event.
add_slice_event(ts=1100, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=DATA_PROCESSOR_TRACK_UUID, name="ProcessDataItem",
flow_ids=[DISPATCH_TO_PROCESS_FLOW_ID, PROCESS_COMPLETION_FLOW_ID])
add_slice_event(ts=1300, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=DATA_PROCESSOR_TRACK_UUID)
# 3. Request Handler: Acknowledge completion (PROCESS_COMPLETION_FLOW_ID terminates here)
add_slice_event(ts=1350, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=REQUEST_HANDLER_TRACK_UUID, name="AcknowledgeCompletion",
flow_ids=[PROCESS_COMPLETION_FLOW_ID])
add_slice_event(ts=1400, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=REQUEST_HANDLER_TRACK_UUID)
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Grouping Tracks with Hierarchies
As traces become more complex, you might want to group related tracks together
to create a more organized and understandable visualization. Perfetto allows you
to define a parent-child relationship between tracks using the parent_uuid
field in the TrackDescriptor
.
This is useful when:
- You have a high-level component (parent track) that comprises several sub-components (child tracks), and you want to see them grouped in the UI.
- You want to create logical groupings for different types of asynchronous events or different sets of counters.
- You are representing a system with inherent hierarchical structures (e.g., a machine with multiple GPUs, each GPU having multiple engines).
A parent track can serve two main purposes:
- Pure Grouping: The parent track itself might not have any direct events (slices or counters) but acts solely as a container to group its child tracks in the UI.
- Summary Track: The parent track can also have its own slices or counters. These could represent an overview or a summary of the activity detailed in its child tracks, or an independent set of events related to the parent itself.
The Perfetto UI will typically render these as an expandable tree.
Python Example
Let's create a hierarchy:
- A "Main System" track, which will also have its own summary slice.
- Two child tracks of "Main System": "Subsystem A" and "Subsystem B".
- "Subsystem A" will further have its own child track, "Detail A.1".
- We'll then place slices on the parent "Main System" track, "Subsystem B", and on the deepest child track "Detail A.1".
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
TRUSTED_PACKET_SEQUENCE_ID = 6006
# --- Define Track UUIDs ---
main_system_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
subsystem_a_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
subsystem_b_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
detail_a1_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
# Helper to define a TrackDescriptor
def define_custom_track(track_uuid, name, parent_track_uuid=None):
packet = builder.add_packet()
desc = packet.track_descriptor
desc.uuid = track_uuid
desc.name = name
if parent_track_uuid:
desc.parent_uuid = parent_track_uuid
# Helper to add a slice event
def add_slice_event(ts, event_type, event_track_uuid, name=None):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = event_type
packet.track_event.track_uuid = event_track_uuid
if name:
packet.track_event.name = name
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# 1. Define the Track Hierarchy
define_custom_track(main_system_track_uuid, "Main System")
define_custom_track(subsystem_a_track_uuid, "Subsystem A", parent_track_uuid=main_system_track_uuid)
define_custom_track(subsystem_b_track_uuid, "Subsystem B", parent_track_uuid=main_system_track_uuid)
define_custom_track(detail_a1_track_uuid, "Detail A.1", parent_track_uuid=subsystem_a_track_uuid)
# 2. Emit slices on various tracks in the hierarchy
# Slice on the parent "Main System" track (summary/overall activity)
add_slice_event(ts=4800, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=main_system_track_uuid, name="System Initialization Phase")
add_slice_event(ts=7000, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=main_system_track_uuid)
# Slice on "Detail A.1" (child of "Subsystem A")
add_slice_event(ts=5000, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=detail_a1_track_uuid, name="Activity in A.1")
add_slice_event(ts=5500, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=detail_a1_track_uuid)
# Slice on "Subsystem B"
add_slice_event(ts=6000, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=subsystem_b_track_uuid, name="Work in Subsystem B")
add_slice_event(ts=6200, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=subsystem_b_track_uuid)
# Another slice on "Detail A.1"
add_slice_event(ts=5600, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=detail_a1_track_uuid, name="Further Activity in A.1")
add_slice_event(ts=5800, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=detail_a1_track_uuid)
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Track Hierarchies for Waterfall / Trace Views
Another powerful use of track hierarchies is to visualize the breakdown of a complex operation or request, similar to how "trace views" or "span views" are displayed in distributed tracing systems. This is useful when an operation involves sequential or parallel steps, potentially across different logical components, and you want to see the timing and relationship of these steps in a waterfall or Gantt-like chart.
In this model:
- A root track represents the entire end-to-end request or operation.
- Each major step, function call, or RPC call within that operation is represented as a child track parented under the root track (or under another step if it's a sub-sub-step).
- A slice on each child track shows the duration of that specific step.
- The
parent_uuid
field creates the hierarchy. The UI will then typically render these as an expandable tree, and the start/end times of the slices on these hierarchically arranged tracks create the "waterfall" effect.
Python Example: Service Request Breakdown
Let's imagine a frontend service makes a request that involves calls to two backend services: an Authentication Service and a Data Service. The Data Service call can only happen after the Authentication Service call completes.
Copy the following Python code into the populate_packets(builder)
function in
your trace_converter_template.py
script.
Click to expand/collapse Python code
TRUSTED_PACKET_SEQUENCE_ID = 7007
# --- Define Track UUIDs ---
root_request_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
auth_service_call_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
data_service_call_track_uuid = uuid.uuid4().int & ((1 << 63) - 1)
# UUID for an internal step within data_service_call
data_service_internal_step_track_uuid = uuid.uuid4().int & ((1<<63)-1)
# Helper to define a TrackDescriptor
def define_custom_track(track_uuid, name, parent_track_uuid=None):
packet = builder.add_packet()
desc = packet.track_descriptor
desc.uuid = track_uuid
desc.name = name
if parent_track_uuid:
desc.parent_uuid = parent_track_uuid
# Helper to add a slice event
def add_slice_event(ts, event_type, event_track_uuid, name=None):
packet = builder.add_packet()
packet.timestamp = ts
packet.track_event.type = event_type
packet.track_event.track_uuid = event_track_uuid
if name:
packet.track_event.name = name
packet.trusted_packet_sequence_id = TRUSTED_PACKET_SEQUENCE_ID
# 1. Define the Root Track for the overall request
define_custom_track(root_request_track_uuid, "Frontend Request: /api/user/profile")
# Add a slice for the total duration of the frontend request on its own track
add_slice_event(ts=10000, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=root_request_track_uuid, name="Total Request Duration")
# 2. Define child tracks for each service call (span) as children of the root request
define_custom_track(auth_service_call_track_uuid, "Call: AuthService.AuthenticateUser",
parent_track_uuid=root_request_track_uuid)
define_custom_track(data_service_call_track_uuid, "Call: DataService.GetUserData",
parent_track_uuid=root_request_track_uuid)
# 3. Emit slices on these service call tracks
# Auth Service Call
add_slice_event(ts=10100, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=auth_service_call_track_uuid, name="AuthService.AuthenticateUser")
add_slice_event(ts=10300, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=auth_service_call_track_uuid)
# Data Service Call (starts after Auth completes)
add_slice_event(ts=10350, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=data_service_call_track_uuid, name="DataService.GetUserData")
# Simulate an internal step within DataService.GetUserData, shown on its own sub-track
# This track will be a child of the "Call: DataService.GetUserData" track.
define_custom_track(data_service_internal_step_track_uuid, "Internal: QueryDatabase",
parent_track_uuid=data_service_call_track_uuid)
add_slice_event(ts=10400, event_type=TrackEvent.TYPE_SLICE_BEGIN,
event_track_uuid=data_service_internal_step_track_uuid, name="QueryDatabase")
add_slice_event(ts=10550, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=data_service_internal_step_track_uuid)
add_slice_event(ts=10600, event_type=TrackEvent.TYPE_SLICE_END, # End of DataService.GetUserData
event_track_uuid=data_service_call_track_uuid)
# End of the total frontend request
add_slice_event(ts=10700, event_type=TrackEvent.TYPE_SLICE_END,
event_track_uuid=root_request_track_uuid)
After running the script, opening the generated my_custom_trace.pftrace
in the
Perfetto UI will display the following output:
Next Steps
You've now seen how to convert various types of custom timestamped data into
Perfetto traces using Python and the TrackEvent
protobuf. With these
techniques, you can represent simple activities, nested operations, asynchronous
events, counters, flows, and create organized track hierarchies.
Once you have your custom data in the Perfetto trace format (.pftrace
file),
you can:
- Explore advanced
TrackEvent
features: For more detailed control over track and event appearance, interning, and other advanced capabilities of theTrackEvent
protobuf, refer to the Writing synthetic traces using TrackEvent protobufs reference page. - Visualize your trace: Open your generated
.pftrace
file in the Perfetto UI to explore your data on an interactive timeline. - Analyze with SQL: Use the Trace Processor to
query your custom trace data. Your custom tracks and events will populate
standard tables like
slice
,track
,counter
, etc.