Skip to content

chore: reset migration versions to 0 and drop old compatibility code - #9623

Merged
rasswanth-s merged 6 commits into
devfrom
rasswanth/migrations-remove-stale
Nov 20, 2026
Merged

rasswanth-s merged 6 commits into
devfrom
rasswanth/migrations-remove-stale

Conversation

@rasswanth-s

@rasswanth-s rasswanth-s commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Resets the migration system to a clean start. Every protocol and versioned object in syft, syft-job and syft-dataset is back at version 0, and the code that kept older releases (up to 0.10.1) working is removed. Older clients are refused with a clear message.

Changes

  • Versions reset to 0: all three protocols, their minimum supported versions, and every versioned object. Jobs live under v0/, and dataset collections use the v0 variant.
  • Old versions removed: older object classes and migrations, old job and dataset layouts, and the frozen release artifacts and fixtures of earlier releases.
  • Older releases refused clearly: a peer on an older release is reported as unsupported, and peering still completes. Logging in on an old SyftBox offers a fresh start. MIN_SUPPORTED_SYFT_VERSION is 0.10.2.
  • Other compatibility code removed: pre-hash job receipts, the old folder-wide review grant, the requests/ back-fill, '@' job names, skip_job_names and other deprecated options (full list in CHANGELOG.md).
  • Protocol 0 stays open until 1.0.0: a release freezes nothing, so a change to a version-0 object needs no new version or bump. The three export_release_artifact.py scripts and their release-workflow steps are removed until then, and docs/release.md warns about it and says how to restore them.
  • Kept for future versions: the syft-migration framework, protocol negotiation, and the release-fixture generators for use from 1.0.0.

Testing

  • Every unit CI suite passes. The one failure, syft-rds test_dataset_only_mock_data_uploaded, also fails on dev.
  • New tests cover refusing an unsupported peer and logging in on an old SyftBox.
  • The migration suites pass in a clean checkout, and actionlint is clean on the three release workflows without the freeze step.
  • Local runs: a stale setuptools build/ folder in a source checkout still holds the old history files and breaks local job runs. Remove it with rm -rf build.

Asana task

Every protocol (syft, syft-job, syft-dataset) and every versioned object
goes back to version 0. Older versions, migrations, layouts, release
artifacts and fixtures are removed, along with compatibility code for
releases up to 0.10.1. Older clients are refused with a clear reason,
and a login on an old SyftBox offers a fresh start.

The migration framework, protocol negotiation and the release-fixture
tooling stay, ready for the next version.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The two test accounts are shared with every other run, including runs of
other releases. A SyftBox that an unsupported release wrote now stops the
login with an offer to start fresh, so the login tests remove both
accounts' SyftBoxes first, as the sync tests already do.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@pjwerneck pjwerneck left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rasswanth-s There's a blocker here. Protocol v0 must stay open and the only protocol until we have a real 1.0.0 release, but the next package release is going to freeze protocol v0 and bump to 1, even if it's <1.0.0.

You have to modify the three export_release_artifact.py scripts so they are disabled and do nothing for now. We can also just delete them and restore later, when we decide to really freeze protocol v0 for a 1.0.0 version. Your call. Just make sure docs/release.md has a warning on that.

@koenvanderveen Agreed?

The next release would have frozen protocol 0, so any later change to a
version-0 object would need protocol 1, long before 1.0.0. Until the
team freezes protocol 0 for 1.0.0, a release freezes nothing:

- remove the three export_release_artifact.py scripts, the "Freeze
  release artifacts" step of the three release workflows, and the
  export-release-artifacts recipe; a script that did nothing would
  still fail the release at its git add, as the history folders no
  longer exist
- docs/release.md warns that nothing is frozen before 1.0.0 and says
  how to restore the scripts from git history; the manual
  release-fixture step waits for 1.0.0 as well

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@rasswanth-s

Copy link
Copy Markdown
Collaborator Author

@pjwerneck Thanks, agreed. Fixed in 154e2f2.

  • Removed the three export_release_artifact.py scripts rather than disabling them. The history folders no longer exist, so a script that did nothing would still have failed the release at its git add step.
  • Removed the "Freeze release artifacts" step from cd-monorepo.yml, cd-syft-job.yml and cd-syft-dataset.yml, and the just export-release-artifacts recipe.
  • docs/release.md now warns that nothing is frozen before 1.0.0, says why, and says how to restore the scripts from git history when we freeze protocol 0.
  • Also paused the manual release-fixture step: fixtures would freeze v0 the same way, because later releases must still read them.

@pjwerneck pjwerneck left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rasswanth-s The blocker is solved, so I'm approving. I recommend doing a pass with the new CLAUDE.md and dev-guidelines before merging.

rasswanth-s and others added 2 commits November 19, 2026 11:54
- DatasetConfigV0 keeps its own folder method names. Folding V1 into V0
  had brought V1's names, including the misspelled
  private_all_datatset_folder().
- When the DO restores its own collections on connect, a collection in a
  layout this client does not read is now skipped with a warning, as the
  DS side already does, instead of silently. An owner upgrading from
  0.10.1 now sees why a dataset is missing.
- unsupported_peer_reason() imports SYFT_VERSION at the top; there is no
  circular import.
- Restore the three syft-job codec guards (unique codec versions, every
  codec registered in JobStorage, every protocol covered once). Deleting
  the file with the old-layout tests had removed them too.
- The two login handlers share _delete_all_state() instead of a copied
  delete block.
- Test names drop articles and padding words, per
  dev-guidelines/guidelines.md. The fake Drive download helper is
  _download_file_returning().
- The two login tests move from test_unsupported_peer.py to
  test_login_unsupported_state.py; the version file they share moves to
  tests/unit/utils.py.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@rasswanth-s
rasswanth-s merged commit 4855755 into dev Nov 20, 2026
21 checks passed
@rasswanth-s
rasswanth-s deleted the rasswanth/migrations-remove-stale branch November 20, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants