integrate coverage support into pw presubmit [279161371]

Fixed

Feature Request

Status Update

No update yet.

Description

mo...@google.com

created issue #1

Apr 21, 2023 02:09PM

Integrate support for building with coverage into pw presubmit, and enable turning this on and off with buildbucket properties.

Comments

ap...@google.com <ap...@google.com> #2May 2, 2023 10:50PM

[Details redacted due to bug visibility]

Change-Id: I30c96e02e5a8c32e5738769899da50a24c1488e1

https://pigweed-internal-review.git.corp.google.com/44108

ap...@google.com <ap...@google.com> #3May 2, 2023 11:56PM

[Details redacted due to bug visibility]

Change-Id: I160232aec6f3a26c95216fa17027da61a36e9c61

https://pigweed-internal-review.git.corp.google.com/44109

ap...@google.com <ap...@google.com> #4May 4, 2023 02:53PM

Project: pigweed/pigweed
Branch: main

commit 19d66ff9f9f6b6a9f149f7c26f154a654bc25450
Author: Rob Mohr <mohrr@google.com>
Date: Thu May 04 14:53:15 2023

pw_presubmit: Add coverage support

Add coverage support to pw_presubmit. This involves explicitly setting
some GN variables, and, for incremental builds, passing into GN the list
of modified files. Only certain ninja targets support coverage, but with
some more work it may be possible to use coverage for arbitrary targets
in the future.

Among other things, pw_presubmit finds coverage reports in the build
directory and tars them up. This makes it easy to make them accessible
through the LUCI UI.

Also fixed some minor issues in docs.

Bug:

b/279161371

Change-Id: I5d57c3ca77228752cfa295e83c149ad7cbc93ced
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/144310
Reviewed-by: Taylor Cramer <cramertj@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Rob Mohr <mohrr@google.com>

M pw_build/docs.rst
M pw_presubmit/py/pw_presubmit/build.py
M pw_presubmit/py/pw_presubmit/pigweed_presubmit.py
M pw_presubmit/py/pw_presubmit/presubmit.py

https://pigweed-review.git.corp.google.com/144310

ap...@google.com <ap...@google.com> #5May 12, 2023 06:47PM

Project: infra/recipes
Branch: main

commit 6b0dcea985d1cc7614a3f4a0044168edd11d40b8
Author: Rob Mohr <mohrr@google.com>
Date: Fri May 12 18:44:28 2023

checkout: Save project name to json

Save the Gerrit project name to the changes JSON file that's passed into
`pw presubmit`.

Bug:

b/279161371

Change-Id: I46f496f282924ba27b3fea1c07e28eea555d59c4
Reviewed-on:

https://pigweed-review.googlesource.com/c/infra/recipes/+/146850
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Oliver Newman <olivernewman@google.com>
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>

M recipe_modules/checkout/api.py
M recipe_modules/checkout/test_api.py
M recipe_modules/checkout/tests/git.expected/ci.json
M recipe_modules/checkout/tests/git.expected/not_in_gerrit.json
M recipe_modules/checkout/tests/git.expected/other.json
M recipe_modules/checkout/tests/git.expected/trigger.json
M recipe_modules/checkout/tests/git.expected/try.json
M recipe_modules/checkout/tests/repo.expected/ci.json
M recipe_modules/checkout/tests/repo.expected/feature_branches_ci.json
M recipe_modules/checkout/tests/repo.expected/feature_branches_try.json
M recipe_modules/checkout/tests/repo.expected/feature_branches_try_multiple_features.json
M recipe_modules/checkout/tests/repo.expected/feature_branches_try_multiple_matches.json
M recipe_modules/checkout/tests/repo.expected/feature_branches_try_no_matching.json
M recipe_modules/checkout/tests/repo.expected/no_trigger.json
M recipe_modules/checkout/tests/repo.expected/prefix.json
M recipe_modules/checkout/tests/repo.expected/try-multiple-cqdeps.json
M recipe_modules/checkout/tests/repo.expected/try-multiple-onenotapplied.json
M recipe_modules/checkout/tests/repo.expected/try-multiple.json
M recipe_modules/checkout/tests/repo.expected/try.json
M recipe_modules/checkout/tests/repo.expected/try_manifest.json
M recipe_modules/checkout/tests/repo.expected/try_repo_not_in_manifest.json
M recipe_modules/checkout/tests/submodule.expected/submodule-ci.json
M recipe_modules/checkout/tests/submodule.expected/submodule-repeated-match.json
M recipe_modules/checkout/tests/submodule.expected/submodule-repeated-nomatch.json
M recipe_modules/checkout/tests/submodule.expected/submodule-repeated-twomatch.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-equivalent.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-excluded-missing.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-excluded.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-included-excluded.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-included.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-multiple-cqdeps.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-multiple-one-missing-one-forbidden-cqdeps.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-multiple-one-missing.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-multiple.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try-not-found.json
M recipe_modules/checkout/tests/submodule.expected/submodule-try.json
M recipes/build.expected/basic.json
M recipes/build.expected/skip_archive.json
M recipes/docs_builder.expected/docs-dry_run-build.json
M recipes/docs_builder.expected/docs-postsubmit.json
M recipes/docs_builder.expected/docs-presubmit.json
M recipes/envtest.expected/environment_variables.json
M recipes/envtest.expected/fail.json
M recipes/envtest.expected/pigweed.json
M recipes/envtest.expected/windows.json
M recipes/luci_config.expected/starlark.json
M recipes/pw_presubmit.expected/one_step.json
M recipes/pw_presubmit.expected/sign.json
M recipes/pw_presubmit.expected/two_steps.json
M recipes/recipes.expected/cq_try.json
M recipes/run_script.expected/run_script.json
M recipes/run_script.expected/variable_substitution.json
M recipes/target_to_cipd.expected/pw-presubmit.json
M recipes/target_to_cipd.expected/success.json
M recipes/tokendb_check.expected/addition.json
M recipes/tokendb_check.expected/no-change.json
M recipes/tokendb_check.expected/removal.json
M recipes/tokendb_check.expected/update.json
M recipes/tokendb_updater.expected/dry-run.json
M recipes/tokendb_updater.expected/simple.json
M recipes/txt_roller.expected/backwards.json
M recipes/txt_roller.expected/no-trigger.json
M recipes/txt_roller.expected/success.json
M recipes/update_python_versions.expected/simple.json
M recipes/xrefs.expected/dry_run.json
M recipes/xrefs.expected/kythe.json
M recipes/xrefs.expected/tryjob.json

https://pigweed-review.git.corp.google.com/146850

ap...@google.com <ap...@google.com> #6May 15, 2023 10:15PM

Project: pigweed/pigweed
Branch: main

commit 49dd927ef10583a2314c8b6bad724b69e53ada95
Author: Rob Mohr <mohrr@google.com>
Date: Mon May 15 22:15:14 2023

pw_presubmit: Add project to LuciTrigger

Bug:

b/279161371

Change-Id: I45fc257b5cc3f3e59eaefaac302e34476b6fa96b
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/146951
Commit-Queue: Rob Mohr <mohrr@google.com>
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>

M pw_presubmit/docs.rst
M pw_presubmit/py/pw_presubmit/presubmit.py

https://pigweed-review.git.corp.google.com/146951

ap...@google.com <ap...@google.com> #7Jun 6, 2023 05:07PM

Project: pigweed/pigweed
Branch: main

commit 64c72ae0b36b0c254db2b62589399b37564a421a
Author: Rob Mohr <mohrr@google.com>
Date: Tue Jun 06 17:06:33 2023

pw_presubmit: Simplify coverage tarball name

Bug:

b/279161371

Change-Id: I44ab46231386228788c9887ee871ae94458bcddb
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/150751
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Taylor Cramer <cramertj@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>

M pw_presubmit/py/pw_presubmit/build.py

https://pigweed-review.git.corp.google.com/150751

ap...@google.com <ap...@google.com> #8Jun 7, 2023 03:00PM

[Details redacted due to bug visibility]

Change-Id: If2610ee60a138e68b9614148de7ea0295f7eecaf

https://pigweed-internal-review.git.corp.google.com/46133

ap...@google.com <ap...@google.com> #9Jun 7, 2023 03:05PM

Project: infra/recipes
Branch: main

commit ad5ec4b0f394f5851f601701a1fb00337d278420
Author: Rob Mohr <mohrr@google.com>
Date: Wed Jun 07 15:04:37 2023

save_logs: Save coverage tarballs

Bug:

b/279161371

Change-Id: I17615828c7a613dbdd7501c4a30a3fbfa6bb185a
Reviewed-on:

https://pigweed-review.googlesource.com/c/infra/recipes/+/150801
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Oliver Newman <olivernewman@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>

M recipe_modules/environment/tests/full.expected/doctor-fail.json
M recipe_modules/environment/tests/full.expected/normal.json
M recipe_modules/environment/tests/full.expected/override-cas.json
M recipe_modules/environment/tests/full.expected/override-cipd.json
M recipe_modules/environment/tests/full.expected/windows.json
M recipe_modules/pw_presubmit/tests/full.expected/bad-json-steps.json
M recipe_modules/pw_presubmit/tests/full.expected/comment-always-disallowed-host.json
M recipe_modules/pw_presubmit/tests/full.expected/comment-always-no-cl.json
M recipe_modules/pw_presubmit/tests/full.expected/comment-always.json
M recipe_modules/pw_presubmit/tests/full.expected/comment-on-failure.json
M recipe_modules/pw_presubmit/tests/full.expected/long.json
M recipe_modules/pw_presubmit/tests/full.expected/medium.json
M recipe_modules/pw_presubmit/tests/full.expected/pigweed.json
M recipe_modules/pw_presubmit/tests/full.expected/step.json
M recipe_modules/pw_presubmit/tests/full.expected/substep.json
M recipe_modules/save_logs/api.py
M recipe_modules/save_logs/tests/full.expected/full.json
M recipes/docs_builder.expected/docs-postsubmit.json
M recipes/docs_builder.expected/docs-presubmit.json
M recipes/envtest.expected/environment_variables.json
M recipes/envtest.expected/fail.json
M recipes/envtest.expected/pigweed.json
M recipes/envtest.expected/windows.json
M recipes/pw_presubmit.expected/one_step.json
M recipes/pw_presubmit.expected/sign.json
M recipes/pw_presubmit.expected/two_steps.json
M recipes/target_to_cipd.expected/pw-presubmit.json

https://pigweed-review.git.corp.google.com/150801

ap...@google.com <ap...@google.com> #10Jun 22, 2023 10:49PM

[Details redacted due to bug visibility]

Change-Id: I9191afc3811d23012c771ce8b23467dedca02efc

https://pigweed-internal-review.git.corp.google.com/47130

ap...@google.com <ap...@google.com> #11Jul 14, 2023 07:37PM

Project: pigweed/pigweed
Branch: main

commit cd5a6cabd97e7bd202ab0591da54bcf2121fcb07
Author: Rob Mohr <mohrr@google.com>
Date: Fri Jul 14 19:37:06 2023

pw_env_setup: Add coverage utilities

Bug:

b/279161371

Change-Id: I1c2196c6752dc78fac7d8e60ca7d5750aac0fbcb
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/155810
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ted Pudlik <tpudlik@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed-service-accounts.iam.gserviceaccount.com>

M pw_env_setup/py/cipd_setup_update_test.py
A pw_env_setup/py/pw_env_setup/cipd_setup/coverage.json
M pw_env_setup/py/pw_env_setup/cipd_setup/upstream.json

https://pigweed-review.git.corp.google.com/155810

ap...@google.com <ap...@google.com> #12Jul 14, 2023 11:01PM

[Details redacted due to bug visibility]

Change-Id: I5c864a68888fd31e583a5fb00cb40460c3b1a668

https://pigweed-internal-review.git.corp.google.com/48448

ap...@google.com <ap...@google.com> #13Aug 3, 2023 05:20PM

Project: pigweed/pigweed
Branch: main

commit 4d6620db1bce9847fbadf0761ee98f8e9e8edf41
Author: Rob Mohr <mohrr@google.com>
Date: Thu Aug 03 17:18:50 2023

pw_presubmit: Upload coverage json to zoss

Bug:

b/279161371

Change-Id: I263721e460168e8f39956efab17d7a415fd42c5c
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/162090
Reviewed-by: Andrés Villegas <andresvi@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Rob Mohr <mohrr@google.com>
Reviewed-by: Armando Montanez <amontanez@google.com>

M pw_presubmit/py/pw_presubmit/build.py

https://pigweed-review.git.corp.google.com/162090

tp...@google.com <tp...@google.com> #14Oct 4, 2023 11:30PM

Reassigned to tp...@google.com.

As a first step, I tried to generate the coverage report locally via,

pw presubmit --step coverage

Unfortunately, this produces a cryptic gn error:

ERROR Unresolved dependencies.
//:extended_default(//pw_toolchain/default:default)
  needs //:stm32f429i_size_optimized(//pw_toolchain/default:default)

Looks like this may be related to http://pwrev/110391?

ap...@google.com <ap...@google.com> #15Oct 5, 2023 04:02PM

Project: pigweed/pigweed
Branch: main

commit 19fa3bf9556ca5441c325f809cbd55d7f06ca179
Author: Ted Pudlik <tpudlik@google.com>
Date: Thu Oct 05 16:00:16 2023

build: Fix extended default group

Before this change, you would get the following error when running gn
gen with non-default optimization levels:

$ gn gen out '--args=pw_C_OPTIMIZATION_LEVELS=["debug"]'
ERROR Unresolved dependencies.
//:extended_default(//pw_toolchain/default:default)
needs //:stm32f429i_size_optimized(//pw_toolchain/default:default)

Bug:

b/279161371

Change-Id: I54bed86a6c737e6d6299054eb9f71e9d2e5054ab
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/174574
Commit-Queue: Ted Pudlik <tpudlik@google.com>
Reviewed-by: Armando Montanez <amontanez@google.com>

M BUILD.gn

https://pigweed-review.git.corp.google.com/174574

ap...@google.com <ap...@google.com> #16Oct 20, 2023 07:48PM

[Details redacted due to bug visibility]

Change-Id: Iab264c20fd9af6b1419ea99d8bc4882aaa8be82c

https://pigweed-internal-review.git.corp.google.com/54349

tp...@google.com <tp...@google.com> #17Oct 30, 2023 10:35PM

Status: we have a WIP CL (http://pwrev/175070), but uploading incremental coverage from it to Kalypsi failed (see b/261779031). Need some help from the Kalypsi team to resolve this, it looks like potentially a configuration error on their side (we're uploading to the incremental bucket, but the data is parsed as if it was absolute?).

Message last modified on Oct 30, 2023 10:36PM

ap...@google.com <ap...@google.com> #18Oct 30, 2023 11:40PM

Project: pigweed/pigweed
Branch: main

commit f48b7c0db427b8f7ecd5901cf20e3a9a51744804
Author: Ted Pudlik <tpudlik@google.com>
Date: Mon Oct 30 23:36:13 2023

pw_presubmit: Kalypsi-based coverage upload

This reimplements the coverage support in pw_presubmit to use Kalypsi
rather than Zoss.

Bug:

b/279161371

Change-Id: I396623d5fc5e6a18c192941214f09d7dfe86312e
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/175070
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rob Mohr <mohrr@google.com>
Commit-Queue: Ted Pudlik <tpudlik@google.com>

M pw_env_setup/py/pw_env_setup/cipd_setup/luci.json
M pw_presubmit/py/pw_presubmit/build.py
M pw_presubmit/py/pw_presubmit/pigweed_presubmit.py

https://pigweed-review.git.corp.google.com/175070

ap...@google.com <ap...@google.com> #19Nov 7, 2023 07:31PM

Project: pigweed/pigweed
Branch: main

commit 3880f6a72ed1fa5a6198ffb9a1dab1b8387d3c5e
Author: Ted Pudlik <tpudlik@google.com>
Date: Tue Nov 07 19:28:20 2023

pw_presubmit: Correct coverage ref

Based on

b/261779031#comment12

.

Bug:

b/279161371

Change-Id: I8b8e2d53117da677e477c52fe891df642b64db44
Reviewed-on:

https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/179452
Commit-Queue: Ted Pudlik <tpudlik@google.com>
Reviewed-by: Rob Mohr <mohrr@google.com>

M pw_presubmit/py/pw_presubmit/pigweed_presubmit.py

https://pigweed-review.git.corp.google.com/179452

tp...@google.com <tp...@google.com> Nov 28, 2023 03:41PM

Marked as fixed.

Issue 279161371

Description

Issue summary

Comments

ap...@google.com <ap...@google.com> #2May 2, 2023 10:50PM

ap...@google.com <ap...@google.com> #3May 2, 2023 11:56PM

ap...@google.com <ap...@google.com> #4May 4, 2023 02:53PM

ap...@google.com <ap...@google.com> #5May 12, 2023 06:47PM

ap...@google.com <ap...@google.com> #6May 15, 2023 10:15PM

ap...@google.com <ap...@google.com> #7Jun 6, 2023 05:07PM

ap...@google.com <ap...@google.com> #8Jun 7, 2023 03:00PM

ap...@google.com <ap...@google.com> #9Jun 7, 2023 03:05PM

ap...@google.com <ap...@google.com> #10Jun 22, 2023 10:49PM

ap...@google.com <ap...@google.com> #11Jul 14, 2023 07:37PM

ap...@google.com <ap...@google.com> #12Jul 14, 2023 11:01PM

ap...@google.com <ap...@google.com> #13Aug 3, 2023 05:20PM

tp...@google.com <tp...@google.com> #14Oct 4, 2023 11:30PM

ap...@google.com <ap...@google.com> #15Oct 5, 2023 04:02PM

ap...@google.com <ap...@google.com> #16Oct 20, 2023 07:48PM

tp...@google.com <tp...@google.com> #17Oct 30, 2023 10:35PM

ap...@google.com <ap...@google.com> #18Oct 30, 2023 11:40PM

ap...@google.com <ap...@google.com> #19Nov 7, 2023 07:31PM

tp...@google.com <tp...@google.com> Nov 28, 2023 03:41PM

Add comment

Issue metadata