Where a run spends its time

A pipeline’s wall-clock floor is not its total runtime — it is the longest chain of dependencies, because nothing on that chain can be run in parallel with the rest of it. pipeline_profile() reports both, and critical_path() is the walk underneath.

Two chains, one deliberately slow:

proj <- file.path(tempdir(), "profile-demo")
dir.create(proj, recursive = TRUE, showWarnings = FALSE)

writeLines(c(
  "library(targets)",
  "",
  "list(",
  "  tar_target(seed,   { Sys.sleep(0.2); 1 }),",
  "  tar_target(slow_a, { Sys.sleep(0.6); seed + 1 }),",
  "  tar_target(slow_b, { Sys.sleep(0.6); slow_a + 1 }),",
  "  tar_target(slow_c, { Sys.sleep(0.6); slow_b + 1 }),",
  "  tar_target(quick,  { Sys.sleep(0.1); seed * 10 }),",
  "  tar_target(report, paste(slow_c, quick))",
  ")"
), file.path(proj, "_targets.R"))

withr::with_dir(proj, targets::tar_make(callr_function = NULL))
#>+ seed dispatched
#>✔ seed completed [201ms, 51 B]
#>+ quick dispatched
#>✔ quick completed [100ms, 51 B]
#>+ slow_a dispatched
#>✔ slow_a completed [601ms, 50 B]
#>+ slow_b dispatched
#>✔ slow_b completed [601ms, 51 B]
#>+ slow_c dispatched
#>✔ slow_c completed [601ms, 51 B]
#>+ report dispatched
#>✔ report completed [0ms, 57 B]
#>✔ ended pipeline [2.3s, 6 completed, 0 skipped]

Read the pipeline, then profile it. check_outdated = FALSE skips the re-hashing pass; freshness is not what this example is about.

cfg <- tardoc::build_site_config(proj)
tardoc::setup_site_dirs(cfg)
#>Directories ready under: /tmp/Rtmp1vpW0n/profile-demo/tardoc
td <- tardoc::load_targets_data(cfg, check_outdated = FALSE)
#>Store found -- run metadata loaded.
#>Loaded 6 targets.
prof <- tardoc::pipeline_profile(td)

prof$slowest
name seconds
slow_a 0.601
slow_b 0.601
slow_c 0.601
seed 0.201
quick 0.100
prof$critical_path
#>[1] "seed"   "slow_a" "slow_b" "slow_c" "report"
c(total_seconds    = round(prof$total_seconds, 2),
  critical_seconds = round(prof$critical_seconds, 2))
#>   total_seconds critical_seconds 
#>             2.1              2.0 

The critical path is seed → slow_a → slow_b → slow_c → report. Its total is the floor on a full rebuild however many workers you give tar_make(): quick can run alongside the slow chain, so its time is free, and shortening it would save nothing. That is the number worth optimising, and the viewer draws it on the graph.

critical_path() works on any {name, seconds} frame and edge list, so you can run it against your own weights — estimated cost, memory, anything:

nodes <- data.frame(name = c("a", "b", "c", "d"), seconds = c(1, 10, 1, 1))
edges <- data.frame(from = c("a", "a", "b", "c"), to = c("b", "c", "d", "d"))
tardoc::critical_path(nodes, edges)
#>[1] "a" "b" "d"