Repository navigation
fix: support source re-ingestion and removal - #30
Conversation
francescobianco
left a comment
francescobianco
left a comment
There was a problem hiding this comment.
Thank you @Aaqibhafeezkhan, this is a substantial contribution and it closes one of the most important gaps in gknowledge.
Until now, ingestion was a one-way street: if a document was updated, ingested with the wrong model, or simply did not belong in the project, the only fix was to start a new knowledge base. With this PR the knowledge base becomes something you can actually maintain over time, which is the whole point of the "evolving knowledge base" idea.
What makes it solid:
- Provenance-aware cleanup.
_detach_sourcedeletes only the pages that the source alone supported, while pages shared with other sources are kept and just lose the[[source]]link. A shared concept page never disappears because one of its sources was removed. - No stale state left behind. Jobs, review items,
mentions.jsonentries, the index, the overview and the log are all updated, so the wiki, the database and the cache stay consistent. - Safe by design.
_managed_pathrefuses paths outsidesources/andknowledge/, both actions are blocked while an ingestion is running, and Remove asks for confirmation with a destructive-styled dialog. - Tests for both flows, including the tricky case of a page shared between two sources.
Great work, and thank you again!
|
@Aaqibhafeezkhan since you now know the source→page lifecycle better than anyone, here are the open issues that build directly on this PR, in case you want to keep going:
None of this is required for merging #30. They're just the places where your work pays off most. And of course, the invitation to join https://github.com/openapi/contributors stands 🙂 |
Thanks @francescobianco Will check them out :) |
francescobianco
left a comment
francescobianco
left a comment
There was a problem hiding this comment.
Approved, thanks @Aaqibhafeezkhan! 🙌 Don't forget to add yourself to https://github.com/openapi/contributors (see my note on #31).
|
@francescobianco raised a PR to add myself on https://github.com/openapi/contributors |
…F support Generated pages rarely go wrong because the model invents something; they go wrong because the source moved on. Every source already stores the hash it was ingested from, so drift is a hash comparison: - core/lint.py: ChangeDetector reports ingested sources whose original file no longer matches, with the pages that cite them (hashes memoised by size and mtime). - IngestionManager.refresh_source(): take the new version, update hash and size, re-ingest (builds on reingest_source from openapi#30). - UI: changed sources are marked "changed since ingest"; Re-ingest becomes "Update"; a toast on open and when the window regains focus. - MCP: stale_sources tool, the lint step of Karpathy's pattern. Flatpak: - make start-flatpak runs the dev loop inside org.gnome.Platform, whose GTK/libadwaita differ from the host's; documented in CONTRIBUTING. - Vendor pypdf 6.19.0: the 0.2.2 bundle cannot import PDFs. - README no longer says "Flatpak planned": the Pages remote installs. Suggested by a reader on dev.to (FromTheArchitect, comment 3i4d5). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Closes #18