rfinnie/turku-api
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Default to link-dest on large-file backups #1

Open
Ghost wants to merge 1 commit from spads-spads/main into main
pull from: spads-spads/main
merge into: rfinnie:main
Conversation 1 Commits 1 Files changed 1 +10
Ghost commented 2021-12-02 12:30:31 +00:00 (Migrated from codeberg.org)
Copy link

We don't use attic for large file backups, and find that link-dest makes a much better default for these.

We don't use attic for large file backups, and find that link-dest makes a much better default for these.
rfinnie commented 2021-12-03 03:43:10 +00:00 (Migrated from codeberg.org)
Copy link

Leaving snapshot_mode null unless explicitly set by the source was deliberate in my original schema, and I don't think the API should be programatically setting it. (A source's in-db config is basically just a copy of what the agent is sending to it.)

The variable handling of what snapshot mode is picked is already part of turku-storage process_ping() (which is way too long, ugh; check line ~173). It currently uses mode "none" if either large_rotating_files or large_modifying_files are set, if no source snapshot_mode is defined.

Switching turku-storage's handling of large_rotating_files to link-dest makes sense (but only when combined with --fuzzy from your other PR, which seems fine). However, in my mind large_modifying_files was envisioned for a situation where, say, you have a source with a 100GB daily database dump. You probably don't want to default to any sort of snapshotting for that (unless you have lots of disk, in which case you could explicitly set snapshot_mode: link_dest on the source).

In summary, I think:

  • This logic shouldn't be in turku-api
  • turku-storage's ping handling should change snapshot_mode = "none" to snapshot_mode = "link-dest" for large_rotating_files
  • large_modifying_files logic shouldn't be changed from default snapshot_mode = "none"
  • rfinnie/turku-storage#1 should be merged regadless

(Oh, and BTW, attic mode was removed long, long ago, and everyone was happy)

Leaving snapshot_mode null unless explicitly set by the source was deliberate in my original schema, and I don't think the API should be programatically setting it. (A source's in-db config is basically just a copy of what the agent is sending to it.) The variable handling of what snapshot mode is picked is already part of turku-storage `process_ping()` (which is way too long, ugh; check line ~173). It currently uses mode "none" if either `large_rotating_files` or `large_modifying_files` are set, if no source `snapshot_mode` is defined. Switching turku-storage's handling of `large_rotating_files` to link-dest makes sense (but only when combined with --fuzzy from your other PR, which seems fine). However, in my mind `large_modifying_files` was envisioned for a situation where, say, you have a source with a 100GB daily database dump. You probably don't want to default to any sort of snapshotting for that (unless you have lots of disk, in which case you could explicitly set `snapshot_mode: link_dest` on the source). In summary, I think: - This logic shouldn't be in turku-api - turku-storage's ping handling should change `snapshot_mode = "none"` to `snapshot_mode = "link-dest"` for `large_rotating_files` - `large_modifying_files` logic shouldn't be changed from default `snapshot_mode = "none"` - rfinnie/turku-storage#1 should be merged regadless (Oh, and BTW, attic mode was removed long, long ago, and everyone was happy)
This pull request has changes conflicting with the target branch.
  • turku_api/views.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin spads-spads/main:spads-spads/main
git switch spads-spads/main

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff spads-spads/main
git switch spads-spads/main
git rebase main
git switch main
git merge --ff-only spads-spads/main
git switch spads-spads/main
git rebase main
git switch main
git merge --no-ff spads-spads/main
git switch main
git merge --squash spads-spads/main
git switch main
git merge --ff-only spads-spads/main
git switch main
git merge spads-spads/main
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug
Something isn't working

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
question
Further information is requested

  
wontfix
This will not be worked on

No labels
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
rfinnie/turku-api!1
No description provided.