Advanced Guide to Programmatic Trace Generation

This page serves as an advanced reference for programmatically creating Perfetto trace files. It builds upon the foundational concepts and examples presented in "Converting arbitrary timestamped data to Perfetto".

We assume you are familiar with:

• The basic structure of Perfetto traces (a Trace message containing a stream of TracePacket messages).
• Using the TrackEvent payload within TracePacket to create custom tracks with various types of slices (simple, nested, asynchronous), counters, and flows.
• The Python script template (trace_converter_template.py) for generating traces, and that the Python examples provided here are intended to be used within its populate_packets(builder) function.

This guide will currently focus on advanced TrackEvent features, such as:

• Associating your timeline data with operating system (OS) processes and threads for richer integration.
• Explicit track sorting and data interning for optimizing trace size and detail.

While TrackEvent is a primary method for representing timeline data, TracePacket is a versatile container. In the future, this guide may expand to cover other TracePacket payloads useful for synthetic trace generation.

The examples will continue to use Python, but the principles apply to any language with Protocol Buffer support. For complete definitions of all available fields, always refer to the official Perfetto protobuf sources, particularly TracePacket and its various sub-messages, including TrackEvent.

Associating Tracks with Operating System Concepts

While the "Converting arbitrary timestamped data to Perfetto" guide demonstrated creating generic custom tracks, you can provide more specific context to Perfetto by associating your tracks with operating system (OS) processes and threads. This allows Perfetto's UI and analysis tools to offer richer integration and better correlation with other system-wide data.

Associating Tracks with Processes

You can create a top-level track that represents an OS process. Any other custom tracks (which might contain slices or counters) can then be parented to this process track. This helps in:

• UI Grouping: Your custom tracks will appear under the specified process name and PID in the Perfetto UI, alongside any other data collected for that process (e.g., CPU scheduling, memory counters).
• Correlation: Events on your custom tracks can be more easily correlated with system-level activity related to that process.
• Clear Identification: Explicitly naming the process and providing its PID makes it unambiguous which process your custom data pertains to.

To define a process track, you populate the process field within its TrackDescriptor. At a minimum, you should provide a pid and ideally a process_name.

It is also recommended to add a timestamp to the TracePacket containing the process's TrackDescriptor. This is especially important when the trace contains data from other sources (e.g. scheduling information from the kernel). Unlike with "global" tracks, these track types may interact with other data sources and as such having a timestamp makes sure that Trace Processor can accurately sort the descriptor into the right place.

Python Example

Let's say you want to emit a custom counter (e.g. "Active DB Connections") and have it appear under a specific process named "MyDatabaseService" with PID 1234.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

If you only have symbolized function names, call add_frame(...) with just the interned function name ID: e.g. add_frame(packet.interned_data, FRAME_MAIN, FUNC_MAIN).

You can query process-associated counter data using SQL in the Perfetto UI's Query tab or with Trace Processor:

Once you have defined a process track, you can parent various other kinds of tracks to it. This includes tracks for specific threads within that process (see next section), as well as custom tracks for process-wide counters (as shown above) or groups of asynchronous operations related to this process (using the techniques for asynchronous slices described in the "Converting arbitrary timestamped data to Perfetto" guide).

Associating Tracks with Threads

You can create tracks that are explicitly associated with specific threads within an OS process. This is the most common way to represent thread-specific activity, such as function call stacks or thread-local counters.

Benefits:

• Correct UI Placement: When a thread track's pid and tid are specified in its TrackDescriptor, the Perfetto UI typically groups it under the corresponding process (identified by that pid). This helps organize the trace.
• Correlation with System Data: Perfetto can automatically correlate events on your thread track with system-level data for that thread, such as CPU scheduling slices.
• Clear Naming: You can provide a human-readable name for your thread.

To define a thread track:

1. Create a TrackDescriptor for the thread.
2. Populate its thread field, providing the pid of the process this thread belongs to and the unique tid of the thread. You should also set thread_name.
3. Optionally and encouraged, you can also define a separate TrackDescriptor for the parent process itself (using its process field and pid), though it's not strictly required for the thread track to be recognized as a thread of that PID. The UI often infers process groupings from PIDs present in thread tracks.

Similarly to process tracks, it is also recommended to add a timestamp to the TracePacket containing the thread's TrackDescriptor. This is especially important when the trace contains data from other sources (e.g. scheduling information from the kernel). Unlike with "global" tracks, these track types may interact with other data sources and as such having a timestamp makes sure that Trace Processor can accurately sort the descriptor into the right place.

Python Example: Thread-Specific Slices

This example defines a thread "MainWorkLoop" (TID 5678) belonging to process "MyApplication" (PID 1234). It then emits a couple of slices directly onto this thread's track. We also define a track for the process itself for clarity, though the thread track's association is primarily through its pid and tid fields.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

You can query thread-specific slices using SQL in the Perfetto UI's Query tab or with Trace Processor:

Advanced Track Customization

Beyond associating tracks with OS concepts, Perfetto offers ways to fine-tune how your tracks are presented and how data is encoded.

Controlling Track Sorting Order

By default, the Perfetto UI applies its own heuristics to sort tracks (e.g., alphabetically by name, or by track UUID). However, for complex custom traces, you might want to explicitly define the order in which sibling tracks appear under a parent. This is achieved using the child_ordering field on the parent TrackDescriptor and, for EXPLICIT ordering, the sibling_order_rank on the child TrackDescriptors.

This child_ordering setting on a parent track only affects its direct children.

Available child_ordering modes (defined in TrackDescriptor.ChildTracksOrdering):

• ORDERING_UNSPECIFIED: The default. The UI will use its own heuristics.
• LEXICOGRAPHIC: Child tracks are sorted alphabetically by their name.
• CHRONOLOGICAL: Child tracks are sorted based on the timestamp of the earliest TrackEvent that occurs on each of them. Tracks with earlier events appear first.
• EXPLICIT: Child tracks are sorted based on the sibling_order_rank field set in their respective TrackDescriptors. Lower ranks appear first. If ranks are equal, or if sibling_order_rank is not set, the tie-breaking order is undefined.

Note: The UI treats these as strong hints. While it generally respects these orderings, there are contexts in which the UI reserves the right not to show them in this order; generally this would be if the user explicitly requested this or if the UI has some special handling for these tracks.

Python Example: Demonstrating All Sorting Types

This example defines three parent tracks, each demonstrating a different child_ordering mode.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

Sharing Y-Axis Between Counters

When visualizing multiple counter tracks, it is often useful to have them share the same Y-axis range. This allows for easy comparison of their values. Perfetto supports this feature through the y_axis_share_key field in the CounterDescriptor.

All counter tracks that have the same y_axis_share_key and the same parent track will share their Y-axis range in the UI.

Python Example: Sharing Y-Axis

In this example, we create two counter tracks with the same y_axis_share_key. This will cause them to be rendered with the same Y-axis range in the Perfetto UI.

Adding a Track Description

You can add a human-readable description to any track to provide more context about the data it contains. In the Perfetto UI, this description appears in a popup when the user clicks the help icon next to the track's name. This is useful for explaining what a track represents, the meaning of its events, or how it should be interpreted, especially in complex custom traces.

To add a description, you simply set the optional description field in the track's TrackDescriptor.

Python Example

This example defines two tracks: one with a description field set and one without, to illustrate the difference in the UI.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

Advanced Event Writing

This section covers advanced TrackEvent features for specialized use cases, including data optimization techniques and event linking mechanisms.

Interning Data for Trace Size Optimization

Interning is a technique used to reduce the size of trace files by emitting frequently repeated strings (like event names or categories) only once in the trace. Subsequent references to these strings use a compact integer identifier (an "interning ID" or iid). This is particularly useful when you have many events that share the same name or other string-based attributes.

How it works:

1. Define Interned Data: In a TracePacket, you include an interned_data message. Inside this, you map your strings to iids. For example, you can define event_names where each entry has an iid (a non-zero integer you choose) and a name string. This packet establishes the mapping.

2. Reference by IID: In subsequent TrackEvents (within the same trusted_packet_sequence_id and before the interned state is cleared), instead of setting the name field directly, you set the corresponding name_iid field to the integer iid you defined.

3. Sequence Flags: The TracePacket.sequence_flags field is crucial:

   • SEQ_INCREMENTAL_STATE_CLEARED (value 1): Set this on a packet if the interning dictionary (and other incremental state) for this sequence should be considered reset before processing this packet's interned_data. This is often used on the first packet of a sequence that defines interned entries.
   • SEQ_NEEDS_INCREMENTAL_STATE (value 2): Set this on any packet that either defines new interned data entries OR uses iids that were defined in previous packets (within the current valid state of the sequence).

   A typical packet that initializes the interning dictionary for a sequence will set both flags: TracePacket.SEQ_INCREMENTAL_STATE_CLEARED | TracePacket.SEQ_NEEDS_INCREMENTAL_STATE. Packets that use these established interned entries (or add more entries to the existing valid dictionary) will set TracePacket.SEQ_NEEDS_INCREMENTAL_STATE.

Python Example: Interning Event Names

This example shows how to define an interned string for an event name and then use it multiple times.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

Interned Callstacks

The Getting Started guide covers inline callstacks for simple use cases. This section covers interned callstacks for efficiency when callstacks repeat or when you need binary mapping information for symbolization.

Interned callstacks define the callstack structure once in InternedData and reference it by ID from multiple events. At a minimum you only need to define frames, callstacks, and reference those callstacks from your events. The other pieces are optional and can be supplied when you have that information:

1. Build IDs and Mapping Paths → Mappings (binaries/libraries). You may skip this entirely if you do not have binary metadata.
2. Mappings → Frames (function + location). mapping_id, rel_pc, source_file_id, line_number, etc. are all optional—set only what makes sense for your data.
3. Frames → Callstacks (frame sequences)
4. Callstacks → Events (via callstack_iid)

Python Example: Interned Callstacks

This example demonstrates the complete workflow for interning callstacks, including mappings, frames, and callstacks. For minimal traces you can skip the mapping entries and populate frames with just function names (and whatever location details you have).

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

Notes:

• Sequence flags: Use SEQ_INCREMENTAL_STATE_CLEARED | SEQ_NEEDS_INCREMENTAL_STATE when defining interned data (for the first time); use only SEQ_NEEDS_INCREMENTAL_STATE when referencing it or defining more incremental data.
• Frame order: frame_ids are ordered outermost to innermost (same as inline callstacks).
• Reuse: Event 3 reuses CALLSTACK_1, demonstrating the efficiency gain.

After running the script, opening the generated trace in the Perfetto UI and doing an area selection will display the following output:

Linking Related Events with Correlation IDs

Correlation IDs provide a way to visually link slices that are part of the same logical operation, even when they are not causally connected. Unlike flows, which represent direct cause-and-effect relationships, correlation IDs group events that share a common context or belong to the same high-level operation.

Common use cases:

• GPU rendering: Link all slices involved in rendering the same frame across different GPU stages
• Distributed systems: Group all slices related to the same RPC request as it moves through different services
• Network processing: Connect all slices involved in processing the same network request through different kernel stages

Visual benefits: The Perfetto UI can use correlation IDs to assign consistent colors to related slices or highlight the entire correlated set when one slice is hovered, making it easier to track related operations across different tracks.

Relationship to flows:

• Use flows when events have a direct causal relationship (A triggers B)
• Use correlation IDs when events are part of the same logical operation but not directly connected
• You can use both together: flows for causal connections within a correlated group

Perfetto supports three types of correlation identifiers:

• correlation_id: A 64-bit unsigned integer (most efficient, recommended for most cases)
• correlation_id_str: A string value (most flexible, human-readable)
• correlation_id_str_iid: An interned string ID (see Interning Data for Trace Size Optimization above for details on interning)

Python Example

This example demonstrates correlation IDs using integer identifiers by simulating different stages of processing for two separate requests across multiple service tracks.

Copy the following Python code into the populate_packets(builder) function in your trace_converter_template.py script.

Controlling Track Merging

By default, the Perfetto UI merges tracks that share the same name. This is often the desired behavior for grouping related asynchronous events. However, there are scenarios where you need more explicit control. You can override this default merging logic using the sibling_merge_behavior and sibling_merge_key fields in the TrackDescriptor.

This allows you to:

• Prevent merging: Force tracks, even with the same name, to always be displayed separately.
• Merge by key: Force tracks to merge based on a custom key, regardless of their names.

The sibling_merge_behavior field can be set to one of the following values:

• SIBLING_MERGE_BEHAVIOR_BY_TRACK_NAME (the default): Merges sibling tracks that have the same name.
• SIBLING_MERGE_BEHAVIOR_NONE: Prevents the track from being merged with any of its siblings.
• SIBLING_MERGE_BEHAVIOR_BY_SIBLING_MERGE_KEY: Merges sibling tracks that have the same sibling_merge_key string.

Python Example: Preventing Merging

In this example, we create two tracks with the same name. By setting their sibling_merge_behavior to SIBLING_MERGE_BEHAVIOR_NONE, we ensure they are always displayed as distinct tracks in the UI.

Python Example: Merging by Key

In this example, we create two tracks with different names but the same sibling_merge_key. By setting their sibling_merge_behavior to SIBLING_MERGE_BEHAVIOR_BY_SIBLING_MERGE_KEY, we instruct the UI to merge them into a single visual track. The name of the merged group will be taken from one of the tracks (usually the one with the lower UUID).

Handling Large Traces with Streaming

All the examples so far have used the TraceProtoBuilder, which builds the entire trace in memory before writing it to a file. This is simple and effective for moderately sized traces, but can lead to high memory consumption if you are generating traces with millions of events.

For these scenarios, the StreamingTraceProtoBuilder is the recommended solution. It writes each TracePacket to a file as it's created, keeping memory usage minimal regardless of the trace size.

How it Works

The API for the streaming builder is slightly different:

1. Initialization: You initialize StreamingTraceProtoBuilder with a file-like object opened in binary write mode.
2. Packet Creation: Instead of builder.add_packet(), you call builder.create_packet() to get a new, empty TracePacket.
3. Packet Writing: After populating the packet, you must explicitly call builder.write_packet(packet) to serialize and write it to the file.

Python Example: Complete Streaming Script

Here is a complete, standalone Python script that demonstrates how to use the StreamingTraceProtoBuilder. It is based on the "Creating Basic Timeline Slices" example from the Getting Started guide.

You can save this code as a new file (e.g., streaming_converter.py) and run it.