Skip to content

feat(platform): add migrate project:project for same-org CD-core copy - #364

Open
dewan-ahmed wants to merge 7 commits into
harness:mainfrom
dewan-ahmed:feat/project-copy-cd-core
Open

dewan-ahmed wants to merge 7 commits into
harness:mainfrom
dewan-ahmed:feat/project-copy-cd-core

Conversation

@dewan-ahmed

Copy link
Copy Markdown

Fixes #362

Problem: There is no CLI workflow to copy a Harness project's CD configuration to another project. Operators have to recreate secrets, connectors, services, pipelines, and related resources by hand.

Solution: harness migrate project:project plans a same-org copy by default and writes only with --apply.

harness migrate project:project \
  --from src \
  --to target \
  --org ORG \
  --resources cd-core

harness migrate project:project \
  --from src \
  --to target \
  --org ORG \
  --resources cd-core \
  --apply
  • Dry-run is the default. --apply creates the destination project if it is missing (--to is identifier and name) and then copies CD-core resources.
  • Identical destination resources are skipped. Differing content aborts before any write. A later create failure stops the run; re-run --apply to continue.
  • Inline SecretText is created with placeholder 123456 (source values are never read). Reference secrets copy manager + path. Git-backed entities are skipped and the command exits nonzero. Triggers are created disabled. Delegates are checked, not installed.

Out of scope for this PR (follow-ups, not this diff):

  • --resources other than cd-core
  • cross-org / cross-account copy
  • copying git-backed entities, delegates, Code, GitOps, feature flags
  • copying real secret values

Test plan

  • go test ./modules/platform/... (plan does not write; conflict writes nothing; identical rerun skips; git-backed skip is incomplete; reference-secret path conflict; template dependency order; stable-template errors fail closed; mutating POST is not retried; mid-copy failure + resume)
  • Live plan: Cosmos sourceproject → targetproject (cd-core)
  • Live --apply: destination project created; github-pat, githubconnector, build_harness_cli copied; destination name is --to
  • Second live --apply: matching resources reported skip (inline secrets not reset to 123456)
  • Destination pipeline YAML projectIdentifier rewritten to targetproject
  • Destination GitHub connector usable after updating the copied PAT (placeholder 123456 is expected)
  • Optional maintainer replay: apply against a source that also has services, environments, templates, and triggers

How to replay locally

task build
./bin/harness migrate project:project \
  --from SOURCE --to DEST --org ORG --resources cd-core
./bin/harness migrate project:project \
  --from SOURCE --to DEST --org ORG --resources cd-core --apply

Then update each listed inline secret and harness list pipeline|connector|secret --org ORG --project DEST.

augmentcode Bot and others added 7 commits October 27, 2026 13:29
Extend migrate project:project from secret planning to a same-org copy of
pipelines and their platform dependencies, creating the destination project
when needed and leaving inline secret values for the user to replace.

Co-authored-by: Cursor <cursoragent@cursor.com>
Service, environment, and infrastructure creates must send JSON with
identity fields and a yaml string. Raw YAML posts return 400 from NG.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep --dry-run from applying, reject --out before any writes, rewrite
only matching source scope identifiers, and fail closed on truncated or
unrecognized list pages.

Co-authored-by: Cursor <cursoragent@cursor.com>
Git-backed skips now fail the command, reference secrets conflict on path, templates copy in dependency order, and stable-template errors stop the write. Transport retries only read calls, detects not-found from API codes, and refreshes SSO tokens before requests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sawka-harness

Copy link
Copy Markdown
Collaborator

cosmos review, grammar-review

@BlueLabelscotch

Copy link
Copy Markdown
Collaborator

cosmos review

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

Review completed with 5 suggestions.

Reviewed commit: a30e485

if err := c.planProject(); err != nil {
return err
}
if err := c.planSecrets(); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

run queues all secrets before any connectors, but planSecrets accepts reference secrets whose unqualified secretManagerIdentifier names a project-scoped external secret-manager connector. For a new destination, creating such a secret fails before its manager connector is copied, and every rerun stops at the same point.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

func rewriteTree(v any, opts rewriteOpts) {
switch n := v.(type) {
case map[string]any:
if cur, ok := n["projectIdentifier"].(string); ok && cur == opts.SourceProject {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

rewriteTree remaps projectIdentifier, but a type: Pipeline stage targets its child through spec.org, spec.project, and spec.pipeline. When parent and child both belong to --from, the copied parent therefore still executes the child in the source project, using the source project's configuration and secrets rather than the copied resources.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

c.conflictResult("template", unit.reportID)
continue
}
c.keep(Result{Kind: "template", Identifier: unit.reportID, Action: ActionSkip, Reason: "destination matches"})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

If template creation succeeds but updateStableTemplate fails, the rerun takes this YAML-only ActionSkip branch and never retries the failed stable promotion. With an older destination version still stable, Copy can then report copied while template references without an explicit version resolve to the wrong version.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

if strings.Contains(path, "/list") || strings.HasSuffix(path, "/listDelegates") {
return true
}
return strings.Contains(body, `"filterType"`)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

isReadPost classifies any POST body containing literal "filterType" as read-only, including a pipeline creation whose multiline shell script contains that JSON key. After a lost response or transient gateway error, exchange can therefore replay a successful resource creation and stop the migration on an already-existing-resource error, despite the no-retry contract for mutating POSTs.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

return a.yamlList(project, "environment", "prod", "environment:\n identifier: prod\n name: prod\n orgIdentifier: fixture_org\n projectIdentifier: source\n")
case strings.HasPrefix(path, "/ng/api/environmentsV2/") && path != "/ng/api/environmentsV2/serviceOverrides" && method == http.MethodGet:
return a.storedYAML(http.MethodPost+" /ng/api/environmentsV2 "+project, "environment")
case path == "/ng/api/infrastructures" || path == "/ng/api/environmentsV2/serviceOverrides":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep Code Review Agent🐛

copyAPI.serve always returns empty inventories for infrastructures and service overrides, and no other test exercises their planners. This leaves the new creation, scope-rewrite, destination-conflict, and rerun paths for these resource types without behavioral coverage, even though --resources cd-core includes them.

Severity: medium


🤖 Was this useful? React with 👍 or 👎

@BlueLabelscotch

Copy link
Copy Markdown
Collaborator

cosmos grammar-review

@augmentcode

augmentcode Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

CLI Grammar Review · session (Harness team members only) · a30e485

CLI grammar changes (beta) · Advisory only, FYI, not required changes.

  • 🟡 New harness migrate project:project: --from, --to, --resources, --dry-run, --apply
    • --dry-run (no-op default) and --apply overlap; --dry-run=false is rejected without --apply. Consider --apply alone.
    • Existing --dry-run commands (7) write by default; here the default is a plan. Consider documenting this inversion in docs/verbs.md.

Overall: 🟡 Recommend manual review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Harness Migrate project workflow.

3 participants