./sysutils/py-borgbackup, Deduplicating backup program with compression and encryption

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 1.1.9, Package name: py37-borgbackup-1.1.9, Maintainer: bsiegert

BorgBackup (short: Borg) is a deduplicating backup program. Optionally,
it supports compression and authenticated encryption.

The main goal of Borg is to provide an efficient and secure way to
backup data. The data deduplication technique used makes Borg suitable
for daily backups since only changes are stored. The authenticated
encryption technique makes it suitable for backups to not fully trusted

Required to run:
[devel/py-setuptools] [devel/py-cython] [devel/py-msgpack] [archivers/lz4] [lang/python37]

Required to build:
[pkgtools/cwrappers] [devel/py-setuptools_scm]

Master sites:

SHA1: 6028a5b867b0a7d87e1ca67b1e4f60a15e048bfd
RMD160: b40a6bf068d557bd7359ca77da53417fe403092e
Filesize: 3386.972 KB

Version history: (Expand)

CVS history: (Expand)

   2019-02-11 19:40:37 by Benny Siegert | Files touched by this commit (2) | Package updated
Log message:
Update py-borgbackup to 1.1.9.

Version 1.1.9 (2019-02-10)


- security fix: configure FUSE with "default_permissions", #3903
  "default_permissions" is now enforced by borg by default to let the
  kernel check uid/gid/mode based permissions.
  "ignore_permissions" can be given to not enforce \ 
- make "hostname" short, even on misconfigured systems, #4262
- fix free space calculation on macOS (and others?), #4289
- config: quit with error message when no key is provided, #4223
- recover_segment: handle too small segment files correctly, #4272
- correctly release memoryview, #4243
- avoid diaper pattern in configparser by opening files, #4263
- add "# cython: language_level=3" directive to .pyx files, #4214
- info: consider part files for "This archive" stats, #3522
- work around Microsoft WSL issue #645 (sync_file_range), #1961

New features:

- add --rsh command line option to complement BORG_RSH env var, #1701
- init: --make-parent-dirs parent1/parent2/repo_dir, #4235


- add archive name to check --repair output, #3447
- check for unsupported msgpack versions
- shell completions:

  - new shell completions for borg 1.1.9
  - more complete shell completions for borg mount -o
  - added shell completions for borg help
  - option arguments for zsh tab completion
- docs:

  - add FAQ regarding free disk space check, #3905
  - update BORG_PASSCOMMAND example and clarify variable expansion, #4249
  - FAQ regarding change of compression settings, #4222
  - add note about BSD flags to changelog, #4246
  - improve logging in example automation script
  - add note about files changing during backup, #4081
  - work around the backslash issue, #4280
  - update release workflow using twine (docs, scripts), #4213
  - add warnings on repository copies to avoid future problems, #4272
- tests:

  - fix the homebrew 1.9 issues on travis-ci, #4254
  - fix duplicate test method name, #4311
   2019-01-18 22:39:43 by Benny Siegert | Files touched by this commit (2) | Package updated
Log message:
Update borgbackup to 1.1.8.

Version 1.1.8 (2018-12-09)


- enforce storage quota if set by serve-command, #4093
- invalid locations: give err msg containing parsed location, #4179
- list repo: add placeholders for hostname and username, #4130
- on linux, symlinks can't have ACLs, so don't try to set any, #4044

New features:

- create: added PATH::archive output on INFO log level
- read a passphrase from a file descriptor specified in the
  BORG_PASSPHRASE_FD environment variable.

Version 1.1.7 (2018-08-11)

Compatibility notes:

- added support for Python 3.7


- cache lock: use lock_wait everywhere to fix infinite wait, see #3968
- don't archive tagged dir when recursing an excluded dir, #3991
- py37 argparse: work around bad default in py 3.7.0a/b/rc, #3996
- py37 remove loggerDict.clear() from tearDown method, #3805
- some fixes for bugs which likely did not result in problems in practice:

  - fixed logic bug in platform module API version check
  - fixed xattr/acl function prototypes, added missing ones

New features:

- init: add warning to store both key and passphrase at safe place(s)
- BORG_HOST_ID env var to work around all-zero MAC address issue, #3985
- borg debug dump-repo-objs --ghost (dump everything from segment files,
  including deleted or superceded objects or commit tags)
- borg debug search-repo-objs (search in repo objects for hex bytes or strings)

Other changes:

- add Python 3.7 support
- updated shell completions
- call socket.gethostname only once
- locking: better logging, add some asserts
- borg debug dump-repo-objs:

  - filename layout improvements
  - use repository.scan() to get on-disk order
- docs:

  - update installation instructions for macOS
  - added instructions to install fuse via homebrew
  - improve diff docs
  - added note that checkpoints inside files requires 1.1+
  - add link to tempfile module
  - remove row/column-spanning from docs source, #4000 #3990
- tests:

  - fetch less data via os.urandom
  - add py37 env for tox
  - travis: add 3.7, remove 3.6-dev (we test with -dev in master)
   2018-06-12 16:22:25 by Benny Siegert | Files touched by this commit (3) | Package updated
Log message:
Update py-borgbackup to 1.1.6.

Version 1.1.6 (2018-06-11)

Compatibility notes:

- When upgrading from borg 1.0.x to 1.1.x, please note:

  - read all the compatibility notes for 1.1.0*, starting from 1.1.0b1.
  - borg upgrade: you do not need to and you also should not run it.
  - borg might ask some security-related questions once after upgrading.
    You can answer them either manually or via environment variable.
    One known case is if you use unencrypted repositories, then it will ask
    about a unknown unencrypted repository one time.
  - your first backup with 1.1.x might be significantly slower (it might
    completely read, chunk, hash a lot files) - this is due to the
    --files-cache mode change (and happens every time you change mode).
    You can avoid the one-time slowdown by using the pre-1.1.0rc4-compatible
    mode (but that is less safe for detecting changed files than the default).
    See the --files-cache docs for details.
- 1.1.6 changes:

  - also allow msgpack-python 0.5.6.


- fix borg exception handling on ENOSPC error with xattrs, #3808
- prune: fix/improve overall progress display
- borg config repo ... does not need cache/manifest/key, #3802
- debug dump-repo-objs should not depend on a manifest obj
- pypi package:

  - include .coveragerc, needed by tox.ini
  - fix package long description, #3854

New features:

- mount: add uid, gid, umask mount options
- delete:

  - only commit once, #3823
  - implement --dry-run, #3822
- check:

  - show progress while rebuilding missing manifest, #3787
  - more --repair output
- borg config --list <repo>, #3612

Other changes:

- update msgpack requirement, #3753
- update bundled zstd to 1.3.4, #3745
- update bundled lz4 code to 1.8.2, #3870
- docs:

  - describe what BORG_LIBZSTD_PREFIX does
  - fix and deduplicate encryption quickstart docs, #3776
- vagrant:

  - FUSE for macOS: upgrade 3.7.1 to 3.8.0
  - exclude macOS High Sierra upgrade on the darwin64 machine
  - remove borgbackup.egg-info dir in fs_init (after rsync)
  - use pyenv-based build/test on jessie32/62
  - use local 32 and 64bit debian jessie boxes
  - use "vagrant" as username for new xenial box
- travis OS X: use xcode 8.3 (not broken)
   2018-04-21 14:37:46 by Thomas Klausner | Files touched by this commit (1)
Log message:
py-borgbackup: add patch for previous
   2018-04-21 14:36:17 by Thomas Klausner | Files touched by this commit (2) | Package updated
Log message:
py-borgbackup: accept msgpack.

   2018-04-17 12:40:56 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
py-borgbackup: update to 1.1.5.

Version 1.1.5 (2018-04-01)

Compatibility notes:

- 1.1.5 changes:

  - require msgpack-python >= 0.4.6 and < 0.5.0.
    0.5.0+ dropped python 3.4 testing and also caused some other issues because
    the python package was renamed to msgpack and emitted some FutureWarning.


- create --list: fix that it was never showing M status, #3492
- create: fix timing for first checkpoint (read files cache early, init
  checkpoint timer after that), see #3394
- extract: set rc=1 when extracting damaged files with all-zero replacement
  chunks or with size inconsistencies, #3448
- diff: consider an empty file as different to a non-existing file, #3688
- files cache: improve exception handling, #3553
- ignore exceptions in scandir_inorder() caused by an implicit stat(),
  also remove unneeded sort, #3545
- fixed tab completion problem where a space is always added after path even
  when it shouldn't
- build: do .h file content checks in binary mode, fixes build issue for
  non-ascii header files on pure-ascii locale platforms, #3544 #3639
- borgfs: fix patterns/paths processing, #3551
- config: add some validation, #3566
- repository config: add validation for max_segment_size, #3592
- set cache previous_location on load instead of save
- remove platform.uname() call which caused library mismatch issues, #3732
- add exception handler around deprecated platform.linux_distribution() call
- use same datetime object for {now} and {utcnow}, #3548

New features:

- create: implement --stdin-name, #3533
- add chunker_params to borg archive info (--json)
- BORG_SHOW_SYSINFO=no to hide system information from exceptions

Other changes:

- updated zsh completions for borg 1.1.4
- files cache related code cleanups
- be more helpful when parsing invalid --pattern values, #3575
- be more clear in secure-erase warning message, #3591
- improve getpass user experience, #3689
- docs build: unicode problem fixed when using a py27-based sphinx
- docs:

  - security: explicitly note what happens OUTSIDE the attack model
  - security: add note about combining compression and encryption
  - security: describe chunk size / proximity issue, #3687
  - quickstart: add note about permissions, borg@localhost, #3452
  - quickstart: add introduction to repositories & archives, #3620
  - recreate --recompress: add missing metavar, clarify description, #3617
  - improve logging docs, #3549
  - add an example for --pattern usage, #3661
  - clarify path semantics when matching, #3598
  - link to offline documentation from README, #3502
  - add docs on how to verify a signed release with GPG, #3634
  - chunk seed is generated per repository (not: archive)
  - better formatting of CPU usage documentation, #3554
  - extend append-only repo rollback docs, #3579
- tests:

  - fix erroneously skipped zstd compressor tests, #3606
  - skip a test if argparse is broken, #3705
- vagrant:

  - xenial64 box now uses username 'vagrant', #3707
  - move cleanup steps to fs_init, #3706
  - the boxcutter wheezy boxes are 404, use local ones
  - update to Python 3.5.5 (for binary builds)
   2018-01-23 10:50:18 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
py-borgbackup: fix runtime problem

msgpack distribution changed name from 'msgpack-python' to 'msgpack'.

   2018-01-07 15:42:08 by Benny Siegert | Files touched by this commit (2) | Package updated
Log message:
Update borgbackup to 1.1.4.


- check: data corruption fix: fix for borg check --repair malfunction, #3444.
  See the more detailled notes close to the top of this document.
- delete: also delete security dir when deleting a repo, #3427
- prune: fix building the "borg prune" man page, #3398
- init: use given --storage-quota for local repo, #3470
- init: properly quote repo path in output
- fix startup delay with dns-only own fqdn resolving, #3471

New features:

- added zstd compression. try it!
- added placeholder {reverse-fqdn} for fqdn in reverse notation
- added BORG_BASE_DIR environment variable, #3338

Other changes:

- list help topics when invalid topic is requested
- fix lz4 deprecation warning, requires lz4 >= 1.7.0 (r129)
- add parens for C preprocessor macro argument usages (did not cause malfunction)
- exclude broken pytest 3.3.0 release
- updated fish/bash completions
- init: more clear exception messages for borg create, #3465
- docs:

  - add auto-generated docs for borg config
  - don't generate HTML docs page for borgfs, #3404
  - docs update for lz4 b2 zstd changes
  - add zstd to compression help, readme, docs
  - update requirements and install docs about bundled lz4 and zstd
- refactored build of the compress and crypto.low_level extensions, #3415:

  - move some lib/build related code to setup_{zstd,lz4,b2}.py
  - bundle lz4 1.8.0 (requirement: >= 1.7.0 / r129)
  - bundle zstd 1.3.2 (requirement: >= 1.3.0)
  - blake2 was already bundled
  - rename BORG_LZ4_PREFIX env var to BORG_LIBLZ4_PREFIX for better consistency:
    we also have BORG_LIBB2_PREFIX and BORG_LIBZSTD_PREFIX now.
  - add prefer_system_lib* = True settings to setup.py - by default the build
    will prefer a shared library over the bundled code, if library and headers
    can be found and meet the minimum requirements.

Pre-1.1.4 potential data corruption issue

A data corruption bug was discovered in borg check --repair, see issue #3444.

This is a 1.1.x regression, releases < 1.1 (e.g. 1.0.x) are not affected.

To avoid data loss, you must not run borg check --repair using an unfixed version
of borg 1.1.x. The first official release that has the fix is 1.1.4.

Package maintainers may have applied the fix to updated packages of 1.1.x (x<4)
though, see the package maintainer's package changelog to make sure.

If you never had missing item metadata chunks, the bug has not affected you
even if you did run borg check --repair with an unfixed version.

When borg check --repair tried to repair corrupt archives that miss item metadata
chunks, the resync to valid metadata in still present item metadata chunks
malfunctioned. This was due to a broken validator that considered all (even valid)
item metadata as invalid. As they were considered invalid, borg discarded them.
Practically, that means the affected files, directories or other fs objects were
discarded from the archive.

Due to the malfunction, the process was extremely slow, but if you let it
complete, borg would have created a "repaired" archive that has lost a \ 
lot of items.
If you interrupted borg check --repair because it was so strangely slow (killing
borg somehow, e.g. Ctrl-C) the transaction was rolled back and no corruption \ 

The log message indicating the precondition for the bug triggering looks like:

    item metadata chunk missing [chunk: 001056_bdee87d...a3e50d]

If you never had that in your borg check --repair runs, you're not affected.

But if you're unsure or you actually have seen that, better check your archives.
By just using "borg list repo::archive" you can see if all expected \ 
items are listed.