Path to this page:
./
sysutils/bup,
Highly efficient file backup system based on the git packfile format
Branch: CURRENT,
Version: 0.33.5,
Package name: bup-0.33.5,
Maintainer: gdtbup is a program that backs things up. bup has a few advantages over other
backup software:
It uses a rolling checksum algorithm (similar to rsync) to split large files
into chunks. The most useful result of this is you can backup huge virtual
machine (VM) disk images, databases, and XML files incrementally, even though
they're typically all in one huge file, and not use tons of disk space for
multiple versions.
It uses the packfile format from git (the open source version control system),
so you can access the stored data even if you don't like bup's user interface.
Unlike git, it writes packfiles directly (instead of having a separate garbage
collection/repacking stage) so it's fast even with gratuitously huge amounts of
data. bup's improved index formats also allow you to track far more filenames
than git (millions) and keep track of far more objects (hundreds or thousands of
gigabytes).
Data is "automagically" shared between incremental backups without having to
know which backup is based on which other one - even if the backups are made
from two different computers that don't even know about each other. You just
tell bup to back stuff up, and it saves only the minimum amount of data needed.
Required to run:[
archivers/par2] [
lang/perl5] [
devel/py-readline] [
lang/python27] [
www/py-tornado] [
devel/git-base]
Required to build:[
net/rsync] [
pkgtools/cwrappers]
Master sites:
Filesize: 485.368 KB
Version history: (Expand)
- (2024-12-11) Updated to version: bup-0.33.5
- (2024-08-26) Updated to version: bup-0.33.4
- (2023-12-30) Updated to version: bup-0.33.3
- (2023-12-09) Updated to version: bup-0.33.2
- (2023-08-14) Updated to version: bup-0.32.1nb1
- (2023-06-19) Updated to version: bup-0.32.1
CVS history: (Expand)
2024-12-11 02:19:15 by Greg Troxel | Files touched by this commit (3) | |
Log message:
sysutils/bup: Update to 0.33.5
This is a micro but it fixes serious bugs so if you use get/gc you
really need to update!
Summary of upstream NEWS:
Notable changes in 0.33.5 since 0.33.4 (incomplete)
===================================================
May require attention
---------------------
* Problems have been discovered that could have allowed the creation
of incomplete trees or commits, for example `bup save` or `bup get`
could have created saves with missing data. This should no longer
be possible, but any existing incomplete trees might also be re-used
by `bup get` (for example), and so represent a continuing hazard.
Note that if you've never used `bup gc` or `bup get`, then we don't
currently believe your repositories could have been affected.
You can detect whether you've been affected by running
`bup-validate-object-links(1)`. If it doesn't report any broken
links (as `no HASH for PARENT_HASH`), then you can stop here, the
repository should be fine. You can also run `bup midx -af` first,
which may speed up the validation.
If it does report broken links, then you should run `bup gc --ignore
missing` to completion before making any further additions to the
repository. But first, if you have other repositories that might
still contain the missing objects, then you may want to try to
retrieve them. See "repopulate missing objects" below for details.
`bup gc --ignore-missing` will eliminate some of the hazards and
report at least one of the paths to each missing object.
If `gc` doesn't report any broken paths (missing objects), then you
can stop here, the repository should be fine.
If `gc` does report broken paths, then you should clear the related
indexes, e.g. `bup index --clear` or `bup on HOST index --clear`,
etc.
If you don't rely on `bup get` (e.g. if you only `save`) then
clearing the index(es) should ensure that new saves will be complete
(though existing broken saves will remain structurally broken for
now).
If you do rely on `bup get`, and if `gc` reports broken paths, then
there's not yet an easy way to ensure `get` won't continue to re-use
incomplete trees when building new saves. For now, you could start
a new repository (and save the old one for future repairs).
You can try to repopulate missing objects from the source (or some
other) repository. To do so, you can collect a list of missing
objects via `bup validate-object-links`:
bup validate-object-links | tee validate-out
grep -E '^no ' validate-out | cut -d' ' -f 2 | sort -u > missing-objects
sed -e 's/^/--unnamed git:/' missing-objects > unnamed-objects
and then try to retrieve the missing objects from another repository
via `bup get`. For example, perhaps:
xargs bup get --source repo --unnamed --ignore-missing < unnamed-objects
or
xargs bup on HOST get --unnamed --ignore-missing < unnamed-objects
After that, you can run `bup validate-object-links` to see whether
you were able to fix all of the broken references (i.e. whether it
still reports missing objects).
If you have enough missing objects, it's possible xargs might split
the argument list between `--unnamed` and its argument, causing
`get` to fail. If so, you can just specify an even numbered value
for `xargs -n`, for example `xargs -n 64 bup get ...`.
On most systems, you can choose a much larger `n`.
If you would just like to validate some saves, you can now run `bup
validate-ref-links SAVE...` which should be much more efficient than
attempting a restore or joining the saves to /dev/null.
We're also working on a command that will repair the structure of
any existing broken trees so that commands like restore will still
be able to work with them.
See issue/missing-objects.md for a detailed explanation of the
problem. If you have pandoc and graphviz dot installed, this will
be rendered to issue/missing-objects.html which you can open in a
browser, or you can find it
[here](https://bup.github.io/issue/missing-objects.html).
General
-------
* `bup validate-object-links` has been added. This command scans the
objects in the repository and reports any "broken links" it finds,
i.e. any links from a tree or commit in the repository to an object
that doesn't exist.
* `bup validate-ref-links` has been added. This command traverses
repository references (e.g. saves) and logs paths to missing
objects, i.e. references from a tree or commit to an object that
doesn't exist in the repository. At the moment, it will report at
least one path to each missing object; it does not attempt to find
all of the paths.
* `bup gc` now provides `--ignore-missing` which allows a `gc`
operation to continue after encountering objects that are missing
from the repository.
* `bup join` now reports the path to any missing object it encounters.
Bugs
----
* `bup gc` should no longer risk leaving the repository with
incomplete tree or commit objects -- trees or commits with
references to objects that are no longer in the repository.
Previously this could happen because the collection was
probabilistic with respect to all object types, and so it could
leave (completely orphaned) vestigial commits or trees that referred
to objects that had been removed. It could also do this if the
`--threshold` caused it to keep a parent in one "live enough" pack,
while discarding a descendant in a pack that doesn't cross the
threshold.
These objects can cause serious trouble because they can be re-used
as-is (without noticing that they are incomplete) by other commands
like `bup get`.
* `bup get` should no longer be able to leave the repository with
incomplete trees or commits if it's interrupted at the wrong time.
Previously it fetched objects "top down", and so if it was
interrupted after the parent tree/commit was written to the
repository, but before all the children were written, then the
repository would be left with an incomplete tree.
* `bup` should always ignore midx files that refer to missing indexes.
Previously it might not notice when objects had disappeared (via
`gc`) which could, in particular, cause remote/client operations
like a remote save to decide that the repository already contained
data that it did not.
* `bup midx` `--auto` and `--force` now delete midx files that refer
to missing indexes.
* `bup gc` should no longer throw bloom close-related exceptions when
interrupted.
Build system
------------
* [Graphviz](https://graphviz.org) `dot` is optional, but must be
available in order to render the figures referred to by
issue/missing-objects.md.
Thanks to (at least)
====================
Greg Troxel, Johannes Berg, and Rob Browning
|
2024-08-26 15:57:02 by Greg Troxel | Files touched by this commit (2) |
Log message:
sysutils/bup: Update to 0.33.4
Notable changes in 0.33.4 since 0.33.3 (incomplete)
===================================================
May require attention
---------------------
* The `par2` command (invoked by `bup fsck -g`) may generate empty
recovery files if interrupted (say via C-c). To mitigate this, bup
now runs `par2` in a temporary directory, and only moves the
recovery files into place if the generation succeeds. It will also
look for any empty par2 files, or incomplete sets, associated with
packfiles that it has been asked to examine. If found, they will
provoke an error. See also
https://github.com/Parchive/par2cmdline/issues/84
* Previously, any `bup on REMOTE ...` commands that attempted to read
from standard input (for example `bup on HOST split < something` or
`bup on HOST split --git-ids ...`) would read nothing instead of the
original content of standard input. Now those commands will either
reject the invocation, or just crash when they attempt to read a
closed stream.
* When building bup, if your version of GNU make is not named `make`,
prefer invoking it directly (e.g. `gmake -j4 check`) instead of
relying on the ./Makefile to redirect to GNU make. Some non-GNU
makes now break this redirection. For example, if you include a
`-j` option in your build command one of them adds a `-J` to
`MAKEFLAGS` that GNU make rejects. In future releases, we're likely
to drop the redirection entirely.
Bugs
----
* `bup-config(5)` has been added, and mentions that at the moment `bup
on REMOTE ...` incorrectly reads the `pack.packSizeLimit` from the
`REMOTE` repository.
* `bup fsck` no longer requires a repository via `BUP_DIR`, `-d`,
etc. when paths are provided on the command line.
Thanks to (at least)
====================
Ed Maste, Greg Troxel, Johannes Berg, and Rob Browning
|
2024-08-12 19:45:52 by Greg Troxel | Files touched by this commit (1) |
Log message:
sysutils/bup: Pass explicit reference to python-config
Otherwise, if python 3.11 and 3.12 are both installed, and pkgsrc is
building for 3.12, bup's configure might find 3.11.
|
2023-12-30 16:35:22 by Greg Troxel | Files touched by this commit (1) |
Log message:
sysutils/bup: Remove local patches from distinfo
|
2023-12-30 16:34:49 by Greg Troxel | Files touched by this commit (2) |
Log message:
sysutils/bup: Update to 0.33.3
Upstream NEWS:
bugfixes
performance improvements (significant, but in specific
circumstances, and low risk)
|
2023-12-09 02:46:30 by Greg Troxel | Files touched by this commit (3) | |
Log message:
sysutils/bup: Update to 0.33.2
Upstream NEWS less some bugfixes and minor improvements:
Notable changes in 0.33 since 0.32
Python 3 is now required, in particular 3.7 or newer.
The version of Python is no longer determined by the value of PYTHON
during ./configure. It is now determined by the version of
python-config selected during configuration. The BUP_PYTHON_CONFIG
environment variable can be set during configuration to select a
particular executable (e.g. BUP_PYTHON_CONFIG=python3.9-config
./configure). See the README for further details.
The bup executable is now a binary rather than a Python
script. Practically speaking, this means that the Python version is
determined by the libpython that the executable is linked against
(selected via python-config), and it means that bup will show up as
"bup" in process listings rather than a python invocation.
The output of bup ls for multiple paths includes each path before
its contents, more closely matching the system ls.
The bup ftp ls command should now respect the working directory (set
by cd), and that directory has been added to the prompt.
The index-cache is no longer included in the bup midx --dir default.
Performance may have improved after the relocation of a number of
operations from Python to C, and a number of improvements to the
VFS.
Bugs
Bup should now respect the current umask, directory sgid bit,
etc. when creating new files (e.g. new packfiles). Previously, it
would create them via mkstemp, which would cause them to be readable
only by the current user.
Bup should now be able to handle repositories containing signed
commits.
bup tag has been changed to rely on git update-ref instead of
writing the (loose) refs directly so that it can handle packed refs
correctly.
bup save should be more careful about noting an error and skipping
paths whose type (link, regular file, ...) has changed since
indexing.
There should no longer be a narrow window where save could store
conflicting symlink targets or conflicting sizes (in the metadata
record and the packfile symlink blob) for a path if the filesystem
changed at just the wrong time during a save.
bup fuse should no longer become unusable after an unfinished read
of commits associated with tags. The underlying VFS issue may have
affected other commands too.
Bup's packfile names should now match git's. Previously bup computed
the SHA1 from just the sorted object list (cf. git-index-pack(1)),
but git appears to include the entire pack file in the hash.
Passing idx files to bup midx along with --dir should now work
correctly.
Resource usage (e.g. memory) may have improved, given a number of
changes intended to make bup release resources more carefully and
sooner.
Build and install
The build process should now work on systems where make isn't GNU
Make (the Makefile automatically redirects to GNUmakefile).
Thanks to (at least)
Abdel Said, Arthur Ward, Bas Stottelaar, Brian Minton, Greg Troxel,
Johannes Berg, Mark Hewitt, Muh Muhten, and Rob Browning
|
2023-08-14 07:25:36 by Thomas Klausner | Files touched by this commit (1247) |
Log message:
*: recursive bump for Python 3.11 as new default
|
2023-06-19 00:35:59 by Greg Troxel | Files touched by this commit (2) | |
Log message:
sysutils/bup: Update to 0.32.1
This is a bug-fix micro update.
|