summaryrefslogtreecommitdiffstats
path: root/tools/tracing
Commit message (Collapse)AuthorAgeFilesLines
* rtla/timerlat_hist: Add timerlat user-space supportDaniel Bristot de Oliveira2023-06-131-11/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat hist -u -c 1-3 -d 600 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:10:01 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 IRQ-003 Thr-003 Usr-003 0 477555 0 0 425287 0 0 474357 0 0 1 122385 7998 0 174616 1921 0 125412 3138 0 2 47 587376 492150 89 594717 447830 147 593463 454872 3 11 2549 101930 7 2682 145580 64 2530 138680 4 3 1954 2833 1 463 4917 11 548 4656 5 0 60 1037 0 138 1117 6 179 1130 6 0 26 1837 0 38 277 1 76 339 7 0 15 143 0 28 147 2 37 156 8 0 10 23 0 11 75 0 12 80 9 0 7 17 0 0 26 0 11 42 10 0 2 11 0 0 18 0 2 20 11 0 0 7 0 1 8 0 2 12 12 0 0 6 0 1 4 0 2 8 13 0 1 3 0 0 0 0 0 1 14 0 1 0 0 0 1 0 0 2 15 0 1 0 0 0 0 0 0 2 16 0 1 2 0 0 0 0 0 0 17 0 0 1 0 0 0 0 0 0 19 0 0 1 0 0 0 0 0 0 over: 0 0 0 0 0 0 0 0 0 count: 600001 600001 600001 600000 600000 600000 600000 600000 600000 min: 0 1 2 0 1 2 0 1 2 avg: 0 1 2 0 2 2 0 2 2 max: 4 16 19 4 12 14 7 12 15 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/b6a042d55003c4a67ff7dce28d96044b7044f00d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat_top: Add timerlat user-space supportDaniel Bristot de Oliveira2023-06-137-6/+474
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat top -u -d 600 -q Timer Latency 0 00:10:01 | IRQ Timer Latency (us) | Thread Timer Latency (us) | Ret user Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max | cur min avg max 0 #600001 | 0 0 0 3 | 2 1 2 9 | 3 2 3 15 1 #600001 | 0 0 0 2 | 2 1 2 13 | 2 2 3 18 2 #600001 | 0 0 0 10 | 2 1 2 16 | 3 2 3 20 3 #600001 | 0 0 0 7 | 2 1 2 10 | 3 2 3 11 4 #600000 | 0 0 0 16 | 2 1 2 41 | 3 2 3 58 5 #600000 | 0 0 0 3 | 2 1 2 10 | 3 2 3 13 6 #600000 | 0 0 0 5 | 2 1 2 7 | 3 2 3 10 7 #600000 | 0 0 0 1 | 2 1 2 7 | 3 2 3 10 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/758ad2292a0a1d884138d08219e1a0f572d257a2.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/hwnoise: Reduce runtime to 75%Daniel Bristot de Oliveira2023-06-131-1/+7
| | | | | | | | | | | | | | | | | | osnoise runs 100% of time by default. It makes sense because osnoise is preemptive. hwnoise checks preemption once a second, so it reduces system progress. Reduce runtime to 75% to avoid problems by default. I added a Fixes as it might avoid problems for first time users as it lands on distros. Link: https://lkml.kernel.org/r/af0b7113ffc00031b9af4bb40ef5889a27dadf8c.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Fixes: 1f428356c38d ("rtla: Add hwnoise tool") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Start the tracers after creating all instancesDaniel Bristot de Oliveira2023-06-134-17/+46
| | | | | | | | | | | | | | | | Group all start tracing after finishing creating all instances. The tracing instance starts first for the case of hitting a stop tracing while enabling other instances. The trace instance is the one with most valuable information. Link: https://lkml.kernel.org/r/67da7a703a56f75d7cd46568525145a65501a7e8.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat_hist: Add auto-analysis supportDaniel Bristot de Oliveira2023-06-131-2/+49
| | | | | | | | | | | | | | Add auto-analysis to timerlat hist, including the --no-aa option to reduce overhead and --dump-task. --aa-only was not added as it is already on timerlat top. Link: https://lkml.kernel.org/r/c2693f47ee83e659a7723fed8035f5d2534f528e.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat: Give timerlat auto analysis its own instanceDaniel Bristot de Oliveira2023-06-133-21/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the auto-analysis is attached to the timerlat top instance. The idea was to avoid creating another instance just for that, so one instance could be reused. The drawback is that, by doing so, the auto-analysis run for the entire session, consuming CPU time. On my 24 box CPUs for timerlat with a 100 us period consumed 50 % with auto analysis, but only 16 % without. By creating an instance for auto-analysis, we can keep the processing stopped until a stop tracing condition is hit. Once it happens, timerlat auto-analysis can use its own trace instance to parse only the end of the trace. By doing so, auto-analysis stop consuming cpu time when it is not needed. If the --aa-only is passed, the timerlat top instance is reused for auto analysis. Link: https://lkml.kernel.org/r/346b7168c1bae552a415715ec6d23c129a43bdb7.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Automatically move rtla to a house-keeping cpuDaniel Bristot de Oliveira2023-06-136-0/+87
| | | | | | | | | | | | | | | | | | | When the user sets -c <cpu-list> try to move rtla out of the <cpu-list>, even without an -H option. This is useful to avoid having rtla interfering with the workload. This works by removing <cpu-list> from rtla's current affinity. If rtla fails to move itself away it is not that of a problem as this is an automatic measure. Link: https://lkml.kernel.org/r/c54304d90c777310fb85a3e658d1449173759aab.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Change monitored_cpus from char * to cpu_set_tDaniel Bristot de Oliveira2023-06-135-87/+24
| | | | | | | | | | | | | | Use a cpumask instead of a char *, reducing memory footprint and code. No functional change, and in preparation for auto house-keeping. Link: https://lkml.kernel.org/r/54c46293261d13cb1042d0314486539eeb45fe5d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Add --house-keeping optionDaniel Bristot de Oliveira2023-06-136-9/+162
| | | | | | | | | | | | | | | | | | | | To avoid having rtla interfering with the measurement threads, add an option for the user to set the CPUs in which rtla should run. For instance: # rtla timerlat top -H 0 -c 1-7 Will place rtla in the CPU 0, while running the measurement threads in the CPU 1-7. Link: https://lkml.kernel.org/r/6a6c78a579a96ba8b02ae67ee1e0ba2cb5e03c4a.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Add -C cgroup supportDaniel Bristot de Oliveira2023-06-136-9/+282
| | | | | | | | | | | | | | | The -C option sets a cgroup to the tracer's threads. If the -C option is passed without arguments, the tracer's thread will inherit rtla's cgroup. Otherwise, the threads will be placed on the cgroup passed to the option. Link: https://lkml.kernel.org/r/cb051477331d292f17c08bf1d66f0e0384bbe5a5.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat: Fix "Previous IRQ" auto analysis' lineDaniel Bristot de Oliveira2023-04-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | The "Previous IRQ interference" line is misaligned and without a \n, breaking the tool's output: ## CPU 12 hit stop tracing, analyzing it ## Previous IRQ interference: up to 2.22 us IRQ handler delay: 18.06 us (0.00 %) IRQ latency: 18.52 us Timerlat IRQ duration: 4.41 us (0.00 %) Blocking thread: 216.93 us (0.03 %) Fix the output: ## CPU 7 hit stop tracing, analyzing it ## Previous IRQ interference: up to 8.93 us IRQ handler delay: 0.98 us (0.00 %) IRQ latency: 2.95 us Timerlat IRQ duration: 11.26 us (0.03 %) Link: https://lore.kernel.org/linux-trace-devel/8b5819077f15ccf24745c9bf3205451e16ee32d9.1679685525.git.bristot@kernel.org Fixes: 27e348b221f6 ("rtla/timerlat: Add auto-analysis core") Cc: Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/timerlat: Add auto-analysis only optionDaniel Bristot de Oliveira2023-04-251-5/+44
| | | | | | | | | | | | | | | | | | | | | | Parsing and formating timerlat data might consume a reasonable amount of CPU time on very large systems, or when timerlat has a too short period. Add an option to run timerlat with auto-analysis enabled while skipping the statistics parsing. In this mode, rtla timerlat periodically checks if the tracing is on, going to sleep waiting for the stop tracing condition to stop tracing, or for the tracing session to finish. If the stop tracing condition is hit, the tool prints the auto analysis. Otherwise, the tool prints the max observed latency and exit. The max observed latency is captured via tracing_max_latency. Link: https://lore.kernel.org/linux-trace-devel/4dc514d1d5dc353c537a466a9b5af44c266b6da2.1680106912.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Add .gitignore fileRong Tao2023-04-251-0/+1
| | | | | | | | | | Add .gitignore file to ignore the rtla binary. Link: https://lore.kernel.org/linux-trace-devel/tencent_3C22A3418CD06196C2E5A84AE3EBC2281206@qq.com Signed-off-by: Rong Tao <rongtao@cestc.cn> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* Merge tag 'trace-v6.3' of ↵Linus Torvalds2023-02-231-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing updates from Steven Rostedt: - Add function names as a way to filter function addresses - Add sample module to test ftrace ops and dynamic trampolines - Allow stack traces to be passed from beginning event to end event for synthetic events. This will allow seeing the stack trace of when a task is scheduled out and recorded when it gets scheduled back in. - Add trace event helper __get_buf() to use as a temporary buffer when printing out trace event output. - Add kernel command line to create trace instances on boot up. - Add enabling of events to instances created at boot up. - Add trace_array_puts() to write into instances. - Allow boot instances to take a snapshot at the end of boot up. - Allow live patch modules to include trace events - Minor fixes and clean ups * tag 'trace-v6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (31 commits) tracing: Remove unnecessary NULL assignment tracepoint: Allow livepatch module add trace event tracing: Always use canonical ftrace path tracing/histogram: Fix stacktrace histogram Documententation tracing/histogram: Fix stacktrace key tracing/histogram: Fix a few problems with stacktrace variable printing tracing: Add BUILD_BUG() to make sure stacktrace fits in strings tracing/histogram: Don't use strlen to find length of stacktrace variables tracing: Allow boot instances to have snapshot buffers tracing: Add trace_array_puts() to write into instance tracing: Add enabling of events to boot instances tracing: Add creation of instances at boot command line tracing: Fix trace_event_raw_event_synth() if else statement samples: ftrace: Make some global variables static ftrace: sample: avoid open-coded 64-bit division samples: ftrace: Include the nospec-branch.h only for x86 tracing: Acquire buffer from temparary trace sequence tracing/histogram: Wrap remaining shell snippets in code blocks tracing/osnoise: No need for schedule_hrtimeout range bpf/tracing: Use stage6 of tracing to not duplicate macros ...
| * tracing: Always use canonical ftrace pathRoss Zwisler2023-02-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The canonical location for the tracefs filesystem is at /sys/kernel/tracing. But, from Documentation/trace/ftrace.rst: Before 4.1, all ftrace tracing control files were within the debugfs file system, which is typically located at /sys/kernel/debug/tracing. For backward compatibility, when mounting the debugfs file system, the tracefs file system will be automatically mounted at: /sys/kernel/debug/tracing Many comments and Kconfig help messages in the tracing code still refer to this older debugfs path, so let's update them to avoid confusion. Link: https://lore.kernel.org/linux-trace-kernel/20230215223350.2658616-2-zwisler@google.com Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Reviewed-by: Mukesh Ojha <quic_mojha@quicinc.com> Signed-off-by: Ross Zwisler <zwisler@google.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Add hwnoise toolDaniel Bristot de Oliveira2023-02-135-16/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The hwnoise tool is a special mode for the osnoise top tool. hwnoise dispatches the osnoise tracer and displays a summary of the noise. The difference is that it runs the tracer with the OSNOISE_IRQ_DISABLE option set, thus only allowing only hardware-related noise, resulting in a simplified output. hwnoise has the same features of osnoise. An example of the tool's output: # rtla hwnoise -c 1-11 -T 1 -d 10m -q Hardware-related Noise duration: 0 00:10:00 | time is in us CPU Period Runtime Noise % CPU Aval Max Noise Max Single HW NMI 1 #599 599000000 138 99.99997 3 3 4 74 2 #599 599000000 85 99.99998 3 3 4 75 3 #599 599000000 86 99.99998 4 3 6 75 4 #599 599000000 81 99.99998 4 4 2 75 5 #599 599000000 85 99.99998 2 2 2 75 Link: https://lkml.kernel.org/r/2d6f49a6f3a4f8b51b2c806458b1cff71ad4d014.1675805361.git.bristot@kernel.org Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: Bagas Sanjaya <bagasdotme@gmail.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/timerlat: Add auto-analysis support to timerlat topDaniel Bristot de Oliveira2023-02-021-2/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, timerlat top displays the timerlat tracer latency results, saving the intuitive timerlat trace for the developer to analyze. This patch goes a step forward in the automaton of the scheduling latency analysis by providing a summary of the root cause of a latency higher than the passed "stop tracing" parameter if the trace stops. The output is intuitive enough for non-expert users to have a general idea of the root cause by looking at each factor's contribution percentage while keeping the technical detail in the output for more expert users to start an in dept debug or to correlate a root cause with an existing one. The terminology is in line with recent industry and academic publications to facilitate the understanding of both audiences. Here is one example of tool output: ----------------------------------------- %< ----------------------------------------------------- # taskset -c 0 timerlat -a 40 -c 1-23 -q Timer Latency 0 00:00:12 | IRQ Timer Latency (us) | Thread Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max 1 #12322 | 0 0 1 15 | 10 3 9 31 2 #12322 | 3 0 1 12 | 10 3 9 23 3 #12322 | 1 0 1 21 | 8 2 8 34 4 #12322 | 1 0 1 17 | 10 2 11 33 5 #12322 | 0 0 1 12 | 8 3 8 25 6 #12322 | 1 0 1 14 | 16 3 11 35 7 #12322 | 0 0 1 14 | 9 2 8 29 8 #12322 | 1 0 1 22 | 9 3 9 34 9 #12322 | 0 0 1 14 | 8 2 8 24 10 #12322 | 1 0 0 12 | 9 3 8 24 11 #12322 | 0 0 0 15 | 6 2 7 29 12 #12321 | 1 0 0 13 | 5 3 8 23 13 #12319 | 0 0 1 14 | 9 3 9 26 14 #12321 | 1 0 0 13 | 6 2 8 24 15 #12321 | 1 0 1 15 | 12 3 11 27 16 #12318 | 0 0 1 13 | 7 3 10 24 17 #12319 | 0 0 1 13 | 11 3 9 25 18 #12318 | 0 0 0 12 | 8 2 8 20 19 #12319 | 0 0 1 18 | 10 2 9 28 20 #12317 | 0 0 0 20 | 9 3 8 34 21 #12318 | 0 0 0 13 | 8 3 8 28 22 #12319 | 0 0 1 11 | 8 3 10 22 23 #12320 | 28 0 1 28 | 41 3 11 41 rtla timerlat hit stop tracing ## CPU 23 hit stop tracing, analyzing it ## IRQ handler delay: 27.49 us (65.52 %) IRQ latency: 28.13 us Timerlat IRQ duration: 9.59 us (22.85 %) Blocking thread: 3.79 us (9.03 %) objtool:49256 3.79 us Blocking thread stacktrace -> timerlat_irq -> __hrtimer_run_queues -> hrtimer_interrupt -> __sysvec_apic_timer_interrupt -> sysvec_apic_timer_interrupt -> asm_sysvec_apic_timer_interrupt -> _raw_spin_unlock_irqrestore -> cgroup_rstat_flush_locked -> cgroup_rstat_flush_irqsafe -> mem_cgroup_flush_stats -> mem_cgroup_wb_stats -> balance_dirty_pages -> balance_dirty_pages_ratelimited_flags -> btrfs_buffered_write -> btrfs_do_write_iter -> vfs_write -> __x64_sys_pwrite64 -> do_syscall_64 -> entry_SYSCALL_64_after_hwframe ------------------------------------------------------------------------ Thread latency: 41.96 us (100%) The system has exit from idle latency! Max timerlat IRQ latency from idle: 17.48 us in cpu 4 Saving trace to timerlat_trace.txt ----------------------------------------- >% ----------------------------------------------------- In this case, the major factor was the delay suffered by the IRQ handler that handles timerlat wakeup: 65.52 %. This can be caused by the current thread masking interrupts, which can be seen in the blocking thread stacktrace: the current thread (objtool:49256) disabled interrupts via raw spin lock operations inside mem cgroup, while doing write syscall in a btrfs file system. A simple search for the function name on Google shows that this is a legit case for disabling the interrupts: cgroup: Use irqsave in cgroup_rstat_flush_locked() lore.kernel.org/linux-mm/20220301122143.1521823-2-bigeasy@linutronix.de/ The output also prints other reasons for the latency root cause, such as: - an IRQ that happened before the IRQ handler that caused delays - The interference from NMI, IRQ, Softirq, and Threads The details about how these factors affect the scheduling latency can be found here: https://bristot.me/demystifying-the-real-time-linux-latency/ Link: https://lkml.kernel.org/r/3d45f40e630317f51ac6d678e2d96d310e495729.1675179318.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/timerlat: Add auto-analysis coreDaniel Bristot de Oliveira2023-02-023-0/+1005
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, timerlat displays a summary of the timerlat tracer results saving the trace if the system hits a stop condition. While this represented a huge step forward, the root cause was not that is accessible to non-expert users. The auto-analysis fulfill this gap by parsing the trace timerlat runs, printing an intuitive auto-analysis. Link: https://lkml.kernel.org/r/1ee073822f6a2cbb33da0c817331d0d4045e837f.1675179318.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | tools/tracing/rtla: osnoise_hist: display average with two-digit precisionAndreas Ziegler2023-01-311-2/+2
| | | | | | | | | | | | | | | | | | | | | | Calculate average value in osnoise-hist summary with two-digit precision to avoid displaying too optimitic results. Link: https://lkml.kernel.org/r/20230103103400.275566-3-br015@umbiko.net Signed-off-by: Andreas Ziegler <br015@umbiko.net> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | tools/tracing/rtla: osnoise_hist: use total duration for average calculationAndreas Ziegler2023-01-311-1/+4
|/ | | | | | | | | | | | | | | Sampled durations must be weighted by observed quantity, to arrive at a correct average duration value. Perform calculation of total duration by summing (duration * count). Link: https://lkml.kernel.org/r/20230103103400.275566-2-br015@umbiko.net Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Signed-off-by: Andreas Ziegler <br015@umbiko.net> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Fix exit status when returning from calls to usage()John Kacur2022-12-093-17/+13
| | | | | | | | | | | | | | | | | | | | rtla_usage(), osnoise_usage() and timerlat_usage() all exit with an error status. However when these are called from help, they should exit with a non-error status. Fix this by passing the exit status to the functions. Note, although we remove the subsequent call to exit after calling usage, we leave it in at the end of a function to suppress the compiler warning "control reaches end of a non-void function". Link: https://lkml.kernel.org/r/20221107144313.22470-1-jkacur@redhat.com Signed-off-by: John Kacur <jkacur@redhat.com> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Consolidate and show all necessary libraries that failed for buildingSteven Rostedt (Google)2022-08-101-26/+36
| | | | | | | | | | | | | When building rtla tools, if the necessary libraries are not installed (libtraceevent and libtracefs), show the ones that are missing in one consolidated output, and also show how to install them (at least for Fedora). Link: https://lore.kernel.org/all/CAHk-=wh+e1qcCnEYJ3JRDVLNCYbJ=0u+Ts5bOYZnY3mX_k-hFA@mail.gmail.com/ Link: https://lkml.kernel.org/r/20220810113918.5d19ce59@gandalf.local.home Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* tools/rtla: Build with EXTRA_{C,LD}FLAGSBen Hutchings2022-08-101-2/+2
| | | | | | | | | | | | To allow for distributions and other builders to apply hardening policy and other customisation, append EXTRA_CFLAGS and EXTRA_LDFLAGS to the corresponding variables. Link: https://lore.kernel.org/linux-trace-devel/YtLBshz0nMQ7530H@decadent.org.uk Signed-off-by: Ben Hutchings <benh@debian.org> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* tools/rtla: Fix command symlinksBen Hutchings2022-08-101-2/+2
| | | | | | | | | | | | | "ln -s" stores the next argument directly as the symlink target, so it needs to be a relative path. In this case, just "rtla". Link: https://lore.kernel.org/linux-trace-devel/YtLBXMI6Ui4HLIF1@decadent.org.uk Fixes: 0605bf009f18 ("rtla: Add osnoise tool") Fixes: a828cd18bc4a ("rtla: Add timerlat tool and timelart top mode") Signed-off-by: Ben Hutchings <benh@debian.org> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Fix tracer nameAlexandre Vicenzi2022-08-102-2/+2
| | | | | | | | | The correct tracer name is timerlat and not timelat. Link: https://lore.kernel.org/linux-trace-devel/20220808180343.22262-1-alexandre.vicenzi@suse.com Signed-off-by: Alexandre Vicenzi <alexandre.vicenzi@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Define syscall numbers for riscvAndreas Schwab2022-07-311-1/+1
| | | | | | | | | | RISC-V uses the same (generic) syscall numbers as ARM64. Link: https://lkml.kernel.org/r/mvma68wl2ul.fsf@suse.de Signed-off-by: Andreas Schwab <schwab@suse.de> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Fix double freeAndreas Schwab2022-07-311-2/+7
| | | | | | | | | | | | | Avoid double free by making trace_instance_destroy indempotent. When trace_instance_init fails, it calls trace_instance_destroy, but its only caller osnoise_destroy_tool calls it again. Link: https://lkml.kernel.org/r/mvmilnlkyzx.fsf_-_@suse.de Fixes: 0605bf009f18 ("rtla: Add osnoise tool") Signed-off-by: Andreas Schwab <schwab@suse.de> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Fix Makefile when called from -C tools/Daniel Bristot de Oliveira2022-07-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sedat Dilek reported an error on rtla Makefile when running: $ make -C tools/ clean [...] make[2]: Entering directory '/home/dileks/src/linux-kernel/git/tools/tracing/rtla' [...] '/home/dileks/src/linux-kernel/git/Documentation/tools/rtla' /bin/sh: 1: test: rtla-make[2]:: unexpected operator <------ The problem rm: cannot remove '/home/dileks/src/linux-kernel/git': Is a directory make[2]: *** [Makefile:120: clean] Error 1 make[2]: Leaving directory This occurred because the rtla calls kernel's Makefile to get the version in silence mode, e.g., $ make -sC ../../.. kernelversion 5.19.0-rc4 But the -s is being ignored when rtla's makefile is called indirectly, so the output looks like this: $ make -C ../../.. kernelversion make: Entering directory '/root/linux' 5.19.0-rc4 make: Leaving directory '/root/linux' Using 'grep -v make' avoids this problem, e.g., $ make -C ../../.. kernelversion | grep -v make 5.19.0-rc4 Thus, add | grep -v make. Link: https://lkml.kernel.org/r/870c02d4d97a921f02a31fa3b229fc549af61a20.1657747763.git.bristot@kernel.org Fixes: 8619e32825fd ("rtla: Follow kernel version") Reported-by: Sedat Dilek <sedat.dilek@gmail.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/utils: Use calloc and check the potential memory allocation failurejianchunfu2022-07-111-2/+3
| | | | | | | | | | | | Replace malloc with calloc and add memory allocating check of mon_cpus before used. Link: https://lkml.kernel.org/r/20220615073348.6891-1-jianchunfu@cmss.chinamobile.com Fixes: 7d0dc9576dc3 ("rtla/timerlat: Add --dma-latency option") Signed-off-by: jianchunfu <jianchunfu@cmss.chinamobile.com> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Remove procps-ng dependencyDaniel Bristot de Oliveira2022-05-264-24/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Daniel Wagner reported to me that readproc.h got deprecated. Also, while the procps-ng library was available on Fedora, it was not available on RHEL, which is a piece of evidence that it was not that used. rtla uses procps-ng only to find the PID of the tracers' workload. I used the procps-ng library to avoid reinventing the wheel. But in this case, reinventing the wheel took me less time than the time we already took trying to work around problems. Implement a function that reads /proc/ entries, checking if: - the entry is a directory - the directory name is composed only of digits (PID) - the directory contains the comm file - the comm file contains a comm that matches the tracers' workload prefix. - then return true; otherwise, return false. And use it instead of procps-ng. Link: https://lkml.kernel.org/r/e8276e122ee9eb2c5a0ba8e673fb6488b924b825.1652423574.git.bristot@kernel.org Cc: John Kacur <jkacur@redhat.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Tao Zhou <tao.zhou@linux.dev> Fixes: b1696371d865 ("rtla: Helper functions for rtla") Reported-by: Daniel Wagner <dwagner@suse.de> Reviewed-by: Daniel Wagner <dwagner@suse.de> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Fix __set_sched_attr error messageDaniel Bristot de Oliveira2022-05-261-1/+1
| | | | | | | | | | | | | | | | | rtla's function __set_sched_attr() was borrowed from stalld, but I forgot to update the error message to something meaningful for rtla. Update the error message from: boost_with_deadline failed to boost pid PID: STRERROR to a proper one: Failed to set sched attributes to the pid PID: STRERROR Link: https://lkml.kernel.org/r/a2d19b2c53f6512aefd1ee7f8c1bd19d4fc8b99d.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/eeded730413e7feaa13f946924bcf2cbf7dd9561.1650617571.git.bristot@kernel.org/ Fixes: b1696371d865 ("rtla: Helper functions for rtla") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Minor grammar fix for rtla READMEJohn Kacur2022-05-261-7/+5
| | | | | | | | | | | | | | | | | - Change to "The rtla meta-tool includes" - Remove an unnecessary "But, " - Adjust the formatting of the paragraph resulting from the changes. - Simplify the wording for the libraries and tools. Link: https://lkml.kernel.org/r/437f0accdde53713ab3cce46f3564be00487e031.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/20220408161012.10544-1-jkacur@redhat.com/ Cc: Daniel Bristot de Oliveria <bristot@kernel.org> Fixes: 79ce8f43ac5a ("rtla: Real-Time Linux Analysis tool") Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Don't overwrite existing directory modeJohn Kacur2022-05-261-1/+2
| | | | | | | | | | | | | | | | | The mode on /usr/bin is often 555 these days, but make install on rtla overwrites this with 755 Fix this by preserving the current directory if it exists. Link: https://lkml.kernel.org/r/8c294a6961080a1970fd8b73f7bcf1e3984579e2.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/20220402043939.6962-1-jkacur@redhat.com Cc: Daniel Bristot de Oliveria <bristot@redhat.com> Fixes: 79ce8f43ac5a ("rtla: Real-Time Linux Analysis tool") Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla: Avoid record NULL pointer dereferenceWan Jiabing2022-05-264-16/+20
| | | | | | | | | | | | | | | | | | | | | | Fix the following null/deref_null.cocci errors: ./tools/tracing/rtla/src/osnoise_hist.c:870:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/osnoise_top.c:650:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_hist.c:905:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_top.c:700:31-36: ERROR: record is NULL but dereferenced. "record" is NULL before calling osnoise_init_trace_tool. Add a tag "out_free" to avoid dereferring a NULL pointer. Link: https://lkml.kernel.org/r/ae0e4500d383db0884eb2820286afe34ca303778.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/20220408151406.34823-1-wanjiabing@vivo.com/ Cc: kael_w@yeah.net Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Fixes: 51d64c3a1819 ("rtla: Add -e/--event support") Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Wan Jiabing <wanjiabing@vivo.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* rtla/Makefile: Properly handle dependenciesDaniel Bristot de Oliveira2022-05-261-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Linus had a problem compiling RTLA, saying: "[...] I wish the tracing tools would do a bit more package checking and helpful error messages too, rather than just fail with: fatal error: tracefs.h: No such file or directory" Which is indeed not a helpful message. Update the Makefile, adding proper checks for the dependencies, with useful information about how to resolve possible problems. For example, the previous error is now reported as: $ make ******************************************** ** NOTICE: libtracefs version 1.3 or higher not found ** ** Consider installing the latest libtracefs from your ** distribution, e.g., 'dnf install libtracefs' on Fedora, ** or from source: ** ** https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ ** ******************************************** These messages are inspired by the ones used on trace-cmd, as suggested by Stevel Rostedt. Link: https://lore.kernel.org/r/CAHk-=whxmA86E=csNv76DuxX_wYsg8mW15oUs3XTabu2Yc80yw@mail.gmail.com/ Changes from V1: - Moved the rst2man check to the install phase (when it is used). - Removed the procps-ng lib check [1] as it is being removed. [1] a0f9f8c1030c66305c9b921057c3d483064d5529.1651220820.git.bristot@kernel.org Link: https://lkml.kernel.org/r/3f1fac776c37e4b67c876a94e5a0e45ed022ff3d.1651238057.git.bristot@kernel.org Cc: Ingo Molnar <mingo@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Suggested-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* Merge tag 'kbuild-v5.18-v2' of ↵Linus Torvalds2022-03-311-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - Add new environment variables, USERCFLAGS and USERLDFLAGS to allow additional flags to be passed to user-space programs. - Fix missing fflush() bugs in Kconfig and fixdep - Fix a minor bug in the comment format of the .config file - Make kallsyms ignore llvm's local labels, .L* - Fix UAPI compile-test for cross-compiling with Clang - Extend the LLVM= syntax to support LLVM=<suffix> form for using a particular version of LLVm, and LLVM=<prefix> form for using custom LLVM in a particular directory path. - Clean up Makefiles * tag 'kbuild-v5.18-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kbuild: Make $(LLVM) more flexible kbuild: add --target to correctly cross-compile UAPI headers with Clang fixdep: use fflush() and ferror() to ensure successful write to files arch: syscalls: simplify uapi/kapi directory creation usr/include: replace extra-y with always-y certs: simplify empty certs creation in certs/Makefile certs: include certs/signing_key.x509 unconditionally kallsyms: ignore all local labels prefixed by '.L' kconfig: fix missing '# end of' for empty menu kconfig: add fflush() before ferror() check kbuild: replace $(if A,A,B) with $(or A,B) kbuild: Add environment variables for userprogs flags kbuild: unify cmd_copy and cmd_shipped
| * kbuild: replace $(if A,A,B) with $(or A,B)Masahiro Yamada2022-02-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | $(or ...) is available since GNU Make 3.81, and useful to shorten the code in some places. Covert as follows: $(if A,A,B) --> $(or A,B) This patch also converts: $(if A, A, B) --> $(or A, B) Strictly speaking, the latter is not an equivalent conversion because GNU Make keeps spaces after commas; if A is not empty, $(if A, A, B) expands to " A", while $(or A, B) expands to "A". Anyway, preceding spaces are not significant in the code hunks I touched. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
* | rtla: Tools main loop cleanupDaniel Bristot de Oliveira2022-03-154-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I probably started using "do {} while();", but changed all but osnoise_top to "while(){};" leaving the ; behind. Cleanup the main loop code, making all tools use "while() {}" Changcheng Deng reported this problem, as reported by coccicheck: Fix the following coccicheck review: ./tools/tracing/rtla/src/timerlat_hist.c: 800: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/osnoise_hist.c: 776: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/timerlat_top.c: 596: 2-3: Unneeded semicolon Link: https://lkml.kernel.org/r/3c1642110aa87c396f5da4a037dabc72dbb9c601.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Reported-by: Changcheng Deng <deng.changcheng@zte.com.cn> Reported-by: Zeal Robot <zealci@zte.com.cn> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/timerlat: Add --dma-latency optionDaniel Bristot de Oliveira2022-03-154-4/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the --dma-latency to set /dev/cpu_dma_latency to the specified value, this aims to avoid having exit from idle states latencies that could be influencing the analysis. Link: https://lkml.kernel.org/r/72ddb0d913459f13217086dadafad88a7c46dd28.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/osnoise: Fix osnoise hist stop tracing messageDaniel Bristot de Oliveira2022-03-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | rtla osnoise hist is printing the following message when hitting stop tracing: printf("rtla timelat hit stop tracing\n"); which is obviosly wrong. s/timerlat/osnoise/ fixing the printf. Link: https://lkml.kernel.org/r/2b8f090556fe37b81d183b74ce271421f131c77b.1646247211.git.bristot@kernel.org Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Check for trace off also in the trace instanceDaniel Bristot de Oliveira2022-03-156-8/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the addition of --trigger option, it is also possible to stop the trace from the -t tracing instance using the traceoff trigger. Make rtla tools to check if the trace is stopped also in the trace instance, stopping the execution of the tool. Link: https://lkml.kernel.org/r/59fc7c6f23dddd5c8b7ef1782cf3da51ea2ce0f5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/trace: Save event histogram output to a fileDaniel Bristot de Oliveira2022-03-151-0/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The hist: trigger generates a histogram in the file sys/event/hist. If the hist: trigger is used, automatically save the histogram output of the event sys:event in the sys_event_hist.txt file. Link: https://lkml.kernel.org/r/b5c906af31d4e022ffe87fb0848fac5c089087c8.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Add --filter supportDaniel Bristot de Oliveira2022-03-154-10/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add --filter option. This option enables a trace event filtering of the previous -e sys:event argument. This option is available for all current tools. Link: https://lkml.kernel.org/r/509d70b6348d3e5bcbf1f07ab725ce08d063149a.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/trace: Add trace event filter helpersDaniel Bristot de Oliveira2022-03-152-0/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a set of helper functions to allow rtla tools to filter events in the trace instance. Link: https://lkml.kernel.org/r/12623b1684684549d53b90f4bf66fae44584fd14.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Add --trigger supportDaniel Bristot de Oliveira2022-03-154-10/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add --trigger option. This option enables a trace event trigger to the previous -e sys:event argument, allowing some advanced tracing options. For instance, in a system with CPUs 2:23 isolated, it is possible to get a stack trace of thread wakeup targeting those CPUs while running osnoise with the following command line: # osnoise top -c 2-23 -a 50 -e sched:sched_wakeup --trigger="stacktrace if target_cpu >= 2" This option is available for all current tools. Link: https://lkml.kernel.org/r/07d2983d5f71261d4da89dbaf02efcad100ab8ee.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/trace: Add trace event trigger helpersDaniel Bristot de Oliveira2022-03-152-1/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a set of helper functions to allow rtla tools to trigger event actions in the trace instance. Link: https://lkml.kernel.org/r/e0d31abe879a78a5600b64f904d0dfa8f76e4fbb.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla: Add -e/--event supportDaniel Bristot de Oliveira2022-03-154-11/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add -e/--event option. This option enables an event in the trace (-t) session. The argument can be a specific event, e.g., -e sched:sched_switch, or all events of a system group, e.g., -e sched. Multiple -e are allowed. It is only active when -t or -a are set. This option is available for all current tools. Link: https://lkml.kernel.org/r/6a3b753be9b1e811953995f7f21a86918ad13390.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/trace: Add trace events helpersDaniel Bristot de Oliveira2022-03-152-0/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a set of helper functions to allow the rtla tools to enable additional tracepoints in the trace instance. Link: https://lkml.kernel.org/r/932398b36c1bbaa22c7810d7a40ca9b8c5595b94.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/timerlat: Add the automatic trace optionDaniel Bristot de Oliveira2022-03-152-7/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the -a/--auto <arg in us> option. This option sets some commonly used options while debugging the system. It aims to help users produce reports in the field, reducing the number of arguments passed to the tool in the first approach to a problem. It is equivalent to setting osnoise/stop_tracing_total_us and print_stack with the argument, and saving the trace to timerlat_trace.txt file if the trace is stopped automatically. Link: https://lkml.kernel.org/r/92438f7ef132c731f538cebdf77850300afe04a5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
* | rtla/osnoise: Add the automatic trace optionDaniel Bristot de Oliveira2022-03-152-6/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the -a/--auto <arg in us> option. This option sets some commonly used options while debugging the system. It aims to help users produce reports in the field, reducing the number of arguments passed to the tool in the first approach to a problem. It is equivalent to setting osnoise/stop_tracing_us with the argument, setting tracing_thresh to 1 us, and saving the trace to osnoise_trace.txt file if the trace is stopped automatically. Link: https://lkml.kernel.org/r/ef04c961b227eb93a83cd0b54bfca45e1a381b77.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-04 13:05:14 +0000