Skip to content

chore(scripts/js-packages-release): new deployment pattern#21122

Closed
y3rsh wants to merge 16 commits intoedgefrom
AUTH-2832-publish-npm-packages-1
Closed

chore(scripts/js-packages-release): new deployment pattern#21122
y3rsh wants to merge 16 commits intoedgefrom
AUTH-2832-publish-npm-packages-1

Conversation

@y3rsh
Copy link
Copy Markdown
Member

@y3rsh y3rsh commented Mar 26, 2026

Overview

Adds a unified GitHub Packages release-prep workflow under scripts/js-packages-release for coordinated releases of the JS packages in this dependency chain. The implementation is now TypeScript-based (.mts) and covers registry preflight checks, ordered build orchestration, manifest rewriting, GitHub Actions automation for js-packages-release@* tags, documentation, focused Vitest coverage, and a Cursor skill to support the workflow.

This PR also prepares step-generation to be published by updating its build targets and package metadata so it ships built lib artifacts with the expected entrypoints and exports for the release flow.

Test Plan and Hands on Testing

  • make -C scripts/js-packages-release clean
  • make -C scripts/js-packages-release build
  • make -C scripts/js-packages-release format
  • make -C scripts/js-packages-release lint
  • make -C scripts/js-packages-release test
  • make -C scripts/js-packages-release build-packages
  • make -C scripts/js-packages-release build-packages-manifests-only VERSION=9999.0.0-beta.1
  • make -C scripts/js-packages-release publish-current
  • make -C scripts/js-packages-release publish-ci VERSION=9999.0.0-beta.1
  • Verify .github/workflows/js-packages-release.yaml is aligned with the TypeScript toolchain and runs

Out of scope

  • the actual npm publish step to GitHub Packages
  • validating the live tag-triggered workflow in GitHub Actions
  • removing the legacy publish jobs from older workflows
  • components-testing parity for all four packages

Changelog

  • Replaces the earlier Python release-prep implementation with a TypeScript .mts toolchain under scripts/js-packages-release
  • Adds TypeScript source modules for shared release metadata, preflight checks, manifest rewriting, and ordered package builds
  • Adds Vitest coverage for version parsing, validation rules, and manifest rewrite behavior
  • Updates the js-packages-release GitHub Actions workflow to use the repo's JS setup instead of uv
  • Updates README, implementation notes, and the Cursor skill to match the TypeScript workflow and current make targets
  • Updates step-generation so it can be built and published with lib-based package outputs and loader-compatible exports
  • Updates path-based step-generation imports in tests where needed for the new package shape

Risk assessment

  • Attention level: medium, because this changes release tooling, CI behavior, and package build assumptions across multiple interdependent packages.
  • Trade-offs, coupling, side effects: The flow depends on a fixed package order, manifest rewriting, and tag conventions. Mistakes here could cause build failures, partial release states, or mismatched internal package versions.
  • Blast radius: GitHub Packages release automation, the scripts/js-packages-release workflow, and consumers of @opentrons/shared-data, @opentrons/step-generation, @opentrons/components, and @opentrons/protocol-visualization.
  • How tests and hands-on checks reduce regression risk: The release-prep make targets were exercised end to end, the TypeScript sources pass lint and format checks, focused Vitest coverage validates parsing and manifest logic, and the build-only and manifest-only flows were manually verified before any real publish step is introduced.

@y3rsh y3rsh requested review from jerader and sanni-t March 26, 2026 16:34
@y3rsh y3rsh self-assigned this Mar 26, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.55%. Comparing base (3e258a7) to head (b90c3a4).
⚠️ Report is 50 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #21122      +/-   ##
==========================================
- Coverage   57.62%   57.55%   -0.07%     
==========================================
  Files        3924     3922       -2     
  Lines      321830   321716     -114     
  Branches    46358    46358              
==========================================
- Hits       185441   185163     -278     
- Misses     136166   136325     +159     
- Partials      223      228       +5     
Flag Coverage Δ
app 45.37% <ø> (+0.20%) ⬆️
protocol-designer 19.95% <ø> (ø)
step-generation 5.69% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...esigner/src/steplist/test/generateSubsteps.test.ts 58.17% <ø> (ø)

... and 146 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@jerader jerader left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI passes, code changes seem reasonable and easy to follow what you did. nice!

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new unified scripts/npmjs-publish/ toolchain (uv + Typer/Rich) to coordinate versioning, builds, and npm registry preflight checks for the monorepo’s JS packages, plus CI automation for tag-driven runs. It also updates @opentrons/step-generation packaging metadata/targets to support publishing built lib/ artifacts and updates a few Protocol Designer tests to import from the package entrypoint.

Changes:

  • Add scripts/npmjs-publish/ Python CLIs (publish.py, build_packages.py) with manifest rewrite helpers, uv/ruff/pytest config, and unit tests.
  • Add a dedicated GitHub Actions workflow for PR validation and tag-driven preflight/build orchestration.
  • Update step-generation build targets and package.json entrypoints/exports; adjust some Protocol Designer test imports accordingly.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
step-generation/package.json Switches entrypoints/exports to lib/ outputs to prepare the package for publishing.
step-generation/Makefile Adds build-ts, lib, and pack targets to produce lib/ artifacts.
scripts/npmjs-publish/pyproject.toml Defines uv-managed Python project + ruff configuration for the new tooling.
scripts/npmjs-publish/uv.lock Locks Python dependencies for reproducible runs via uv.
scripts/npmjs-publish/publish_core.py Centralizes package order + semver/tag parsing helpers.
scripts/npmjs-publish/publish.py Implements npm registry preflight CLI with summaries and interactive mode.
scripts/npmjs-publish/build_packages.py Implements ordered monorepo build + optional manifest rewrite CLI.
scripts/npmjs-publish/manifests.py Rewrites package.json versions and internal dependency pins.
scripts/npmjs-publish/github_summary.py Appends markdown output to GITHUB_STEP_SUMMARY in Actions.
scripts/npmjs-publish/tests/test_publish.py Adds unit tests for version/tag parsing + preflight logic.
scripts/npmjs-publish/tests/test_manifests.py Adds unit tests for manifest rewrite behavior.
scripts/npmjs-publish/tests/test_build_packages.py Adds unit tests for build CLI behavior and flags.
scripts/npmjs-publish/README.md Documents local + CI usage for the new release tooling.
scripts/npmjs-publish/PLAN.md Captures roadmap/status notes for the publish workflow migration.
scripts/npmjs-publish/Makefile Adds convenience targets for setup/lint/test/preflight/build flows.
.github/workflows/npmjs-publish.yaml Adds workflow to lint/test on PRs and run tag-based preflight/build.
protocol-designer/src/ui/steps/actions/tests/actions.test.ts Updates a test import to use @opentrons/step-generation package entrypoint.
protocol-designer/src/steplist/test/generateSubsteps.test.ts Updates a test import to use @opentrons/step-generation package entrypoint.
protocol-designer/src/file-data/tests/createFile.test.ts Updates a test import to use @opentrons/step-generation package entrypoint.
.cursor/skills/npmjs-publish/SKILL.md Adds Cursor skill documenting conventions for the new tooling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/npmjs-publish.yaml Outdated
Comment thread step-generation/package.json Outdated
Copy link
Copy Markdown
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we sure that a bunch of python is the right thing to use for preflight tests for npm publishes? is there not a bunch of node stuff that would be a lighter integration burden and maybe some helpful packages that do some of it for us? this will definitely work, but it's a bit eyebrow raising in the same way that adding a bunch of js to process pypi publishes would be.

also, can we strip this down a little? this is a whooole lotta code for this flow because of things like interactive modes, colorized table outputs, etc etc. it makes it really hard to tell what it actually does, and the little table of what each file is for mixes internal detail, external usage information, and semantic intent in a way that makes it pretty unhelpful.

finally i don't really get what the intent of the npm-publish@ tag prefix thing is. is the idea that after we do a big release process - say, 9.0.0, so we've got refs/tags/v9.0.0 - we push an npm-publish@ tag that matches that? or is independent of that? and that has a separate version track that determines what's published, and runs on just whatever git ref is pushed currently? Makes sense, but can we document that?

Comment thread scripts/npmjs-publish/publish_core.py Outdated

import semver

PACKAGES: tuple[str, ...] = (
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is in a very specific dependency order, right? maybe put that in a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya I went a little heavy. Switched it to ts. Tag deployment pattern like everything else. You push js-packages-release@semver tag and all 4 packages are built and released together since they must be in sync. I stuck to semver as we might want it to be meaningful at some point.

@y3rsh y3rsh requested review from jerader and sfoster1 March 27, 2026 22:25
@y3rsh y3rsh changed the title chore(scripts/npmjs-publish): new deployment pattern chore(scripts/scripts/js-packages-release): new deployment pattern Apr 1, 2026
@y3rsh y3rsh changed the title chore(scripts/scripts/js-packages-release): new deployment pattern chore(scripts/js-packages-release): new deployment pattern Apr 7, 2026
@y3rsh y3rsh closed this Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants