Skip to content

✨ [Feature] 用 token-shingle 比较更新,并独立提示 userscript 元数据变更 - #1832

Draft
cyfung1031 wants to merge 3 commits into
mainfrom
docs/rfc-token-shingle-update-similarity-20261107
Draft

cyfung1031 wants to merge 3 commits into
mainfrom
docs/rfc-token-shingle-update-similarity-20261107

Conversation

@cyfung1031

@cyfung1031 cyfung1031 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Checklist / 检查清单

  • Fixes mentioned issues / 修复已提及的问题(无关联 issue)
  • Code reviewed by human / 代码通过人工检查(待维护者审核)
  • Changes tested / 已完成针对性测试

背景与实现

Replace ScriptCat's old character-based string-similarity-js update score with a pairwise token-shingle comparison inspired by Greasy Fork. Unlike a site-wide similar-script search, this computes a fingerprint of the installed and proposed versions during the existing update check; no global index or storage migration is needed.

  • Executable code: js-tokens tokenization; ignore whitespace and ordinary comments; 5-token overlapping shingles excluding structural punctuation; length-prefixed SHA-256-derived 64-bit hashes; bottom-256 unique hashes; cutoff-aware sampled Jaccard with token-count cap. Identical nonempty executable tokens score 100%. Missing/empty executable code is explicitly unscorable rather than a misleading numeric similarity.
  • Userscript metablock (meaningful, not throwaway comments): independently compare parsed // ==UserScript== directives, including repeated ordered values and unknown/ScriptCat-specific tags. Classify permission, scope, dependency, injection, and update-source changes as sensitive; preserve localized display directives; detect missing/invalid headers and ==UserConfig== changes.
  • Update safety and UI: carry structured comparison data through the service-worker full cache and lightweight update-list delivery. Desktop/mobile batch-update cards expose a clickable metadata diff, while retaining the new-@connect badge and existing update actions. A sensitive metadata change forces the major-change risk even with 100% code similarity. Version-only metadata changes remain represented in the version diff without duplicate warning clutter.
  • Scoring: new heuristic bands <0.50 major, 0.50–<0.90 noticeable, >=0.90 tiny, with metadata/unscorable overrides. Do not interpret the score as a percentage of lines copied or proof of semantic equivalence.
  • Scope: existing checkSilenceUpdate policy for newly added @connect is preserved. Extending silent-update policy for other metadata is a separate security decision. Common-snippet highlighting is a possible follow-up, not part of this numeric-score replacement.
  • Dependencies/docs: add js-tokens, remove string-similarity-js, update the design document and translation keys in all 10 locales.

验证 / Verification

  • ✅ Focused Vitest: scorer, script-service cache delivery, batch-update logic, desktop/mobile components, and hooks — 228 tests passed; an additional focused rerun including the large/minified fingerprint case — 195 tests passed.
  • ✅ Isolated control test BatchActionsBar.test.tsx — 6 passed.
  • ✅ pnpm run lint:ci passed (Prettier, TypeScript, locale/issue-template validation, and ESLint); production pnpm run build succeeded with bundle-size/Monaco warnings.
  • ⚠️ Broad pnpm run test:ci is not green in this environment (142 files failed in the concurrent run). A retry with two workers was also not green (57 files failed, 326 passed; 147 individual failures), including unrelated UI suites, while their isolated control run passed. This is not claimed as a complete repository-wide test pass.
  • ⏳ Human review, any real-browser smoke verification, and real-world threshold calibration remain pending.

See design and implementation details, Greasy Fork scorer, and Greasy Fork's introduction.

@cyfung1031 cyfung1031 changed the title 📄 [RFC] 用 token-shingle 替换更新相似度,并独立识别 userscript 元数据变更 ✨ [Feature] 用 token-shingle 比较更新,并独立提示 userscript 元数据变更 Nov 7, 2026
@cyfung1031
cyfung1031 marked this pull request as ready for review November 7, 2026 02:42
@cyfung1031
cyfung1031 marked this pull request as draft November 7, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant