[pve-devel] Plan for (invasive) shrink of pve-manager git repository

[Thomas Lamprecht]

Am 26/05/2023 um 11:45 schrieb Thomas Lamprecht:
> I'll use the git filter-repo [0] tool, a replacement for filter-branch with
> better UX and less potential for getting it wrong, to rewrite the history,
> filtering out any problematic artefact or directory.
> 
> For this I'll use the following file-list
> 
> www/ext6
> www/ext5
> www/ext4
> www/touch
> po
> glob:*.zip
> 
> used as inverted match via the following command:
> 
> git filter-repo --invert-paths --paths-from-file
> ~/pve-manager-inverted-filter-paths
> 
> Then, I'd rename the current "pve-manager.git" hosted at git.proxmox.com to
> "pve-manager-legacy.git", so it will still be able as reference for ancient
> history, providing the possibility to build pre PVE 5 pve-manager packages
> (why ever one would want/needs to do that).
> 
> A new repo, with the same name "pve-manager.git", would then get created and
> the now cleaned up git repo pushed to it.

Above has been carried out now.

Old repo is still available here:
https://git.proxmox.com/?p=pve-manager-legacy.git;a=summary


If you fetch in an existing pve-manager.git repository you'll see something like:
>From git://git.proxmox.com/git/pve-manager
 + f548e4fca...4a8501a8b master     -> origin/master  (forced update)
 + 40ccc11c4...d26a7c43e stable-3   -> origin/stable-3  (forced update)
 + 08ba4d2dd...789b4067b stable-4   -> origin/stable-4  (forced update)
 + d0ec33c69...b80838a2f stable-5   -> origin/stable-5  (forced update)
 + 6ba2c0bcf...b31a318d0 stable-6   -> origin/stable-6  (forced update)

For re-aligning your local master branch you can do a hard-reset, BUT check
if you got any local commits yet (move them over to another branch with e.g.
`git checkout -b feature-to-re-apply-on-master`

git checkout master
git reset --hard origin/master

Then re-create your active development branches freshly from the master
and cherry-pick the relevant patches from the old branch.

After that you can delete the old branches.

> # Fallout
> 
> This naturally has some fallout for developers currently working patch series,
> similar to any force-push (which we normally avoid at all cost).
> 
> Rebasing won't work IIUC, but as the source file layout won't change, you can
> simply use "git cherry-pick <rev-range>" if you have the before filter and
> after filter remotes & branches in the same git repo.  Otherwise, one can
> always use "git format-patch -o ~/patches/ <rev-range>" in the old repo to
> export patches cleanly, and then use "git am -3 ~/patches/*.patch" in the new
> repo.

FWIW, I migrated over my branches, and cherry-picking worked well.

> 
> Note that git commit hash references inside commit messages of pve-manager will
> get rewritten, so here won't notice anything.  Commit references from other
> repos are naturally untouched, but pve-manager being a leave package means that
> it won't have that many in other repos.

Note that above is wrong, my test for that was misguided, but filter-repo does
check for this and outputs it as "suboptimal-issues" file (see below), it luckily
ain't that many as we only (relatively) recently began to track stuff like "Fixes"
in there.

> I'll safe a copy of the old -> new commit reference map that git filter-repo
> produces, ensuring we got full transparency.

This is publicly available here:

https://pve.proxmox.com/pve-manager-filter-repo-result/

Most interesting will be the "commit-map" file.
In the "ref-map" I marked those branches which I did not copy over, mostly some
ancient hot fix branches; OTOH, all stable-X branches *got* copied over.

Again, sorry for any trouble and headache this may cause, if you have specific
question (or see something that is off) -> ask me (e.g., reply to this mail on
the list)

cheers,
 Thomas