./sysutils/py-borgbackup, Deduplicating backup program with compression and encryption

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 1.1.2, Package name: py36-borgbackup-1.1.2, Maintainer: bsiegert

BorgBackup (short: Borg) is a deduplicating backup program. Optionally,
it supports compression and authenticated encryption.

The main goal of Borg is to provide an efficient and secure way to
backup data. The data deduplication technique used makes Borg suitable
for daily backups since only changes are stored. The authenticated
encryption technique makes it suitable for backups to not fully trusted

Required to run:
[devel/py-setuptools] [devel/py-cython] [devel/py-msgpack] [archivers/lz4]

Required to build:
[pkgtools/cwrappers] [devel/py-setuptools_scm]

Master sites:

SHA1: 1fef77e343442db2daf621eb3769bc831ca91b9c
RMD160: 1c31ef1d0f983aee673e580d7fba06bec3c4fd50
Filesize: 19395.054 KB

Version history: (Expand)

CVS history: (Expand)

   2017-11-12 11:52:42 by Benny Siegert | Files touched by this commit (6) | Package updated
Log message:
Update borgbackup to 1.1.2.

Full changelog (long) at
https://github.com/borgbackup/borg/blob … anges.rst.

When upgrading from borg 1.0.x to 1.1.x, please note:

- borg might ask some security-related questions once after upgrading. You can
  answer them either manually or via environment variable. One known case is if
you use unencrypted repositories, then it will ask about a unknown unencrypted
repository one time.
- your first backup with 1.1.x might be significantly slower (it might
  completely read, chunk, hash a lot files) - this is due to the --files-cache
mode change (and happens every time you change mode). You can avoid the
one-time slowdown by using the pre-1.1.0rc4-compatible mode (but that is less
safe for detecting changed files than the default). See the --files-cache docs
for details.
   2017-08-06 10:34:02 by Thomas Klausner | Files touched by this commit (1)
Log message:
   2017-08-05 12:25:46 by Benny Siegert | Files touched by this commit (3) | Package updated
Log message:
Update borgbackup to 1.0.11.
(I am not 100% sure about the PLIST changes, I'd appreciate for someone
to verify.)

Version 1.0.11 (2017-07-21)

Bug fixes:

- use limited unpacker for outer key (security precaution), #2174
- fix paperkey import bug

Other changes:

- change --checkpoint-interval default from 600s to 1800s, #2841.
  this improves efficiency for big repositories a lot.
- docs: fix OpenSUSE command and add OpenSUSE section
- tests: add tests for split_lstring and paperkey
- vagrant:

  - fix openbsd shell
  - backport cpu/ram setup from master
  - add stretch64 VM

Version 1.0.11rc1 (2017-06-27)

Bug fixes:

- performance: rebuild hashtable if we have too few empty buckets, #2246.
  this fixes some sporadic, but severe performance breakdowns.
- Archive: allocate zeros when needed, #2308
  fixes huge memory usage of mount (8 MiB × number of archives)
- IPv6 address support
  also: Location: more informative exception when parsing fails
- borg single-file binary: use pyinstaller v3.2.1, #2396
  this fixes that the prelink cronjob on some distros kills the
  borg binary by stripping away parts of it.
- extract:

  - warning for unextracted big extended attributes, #2258
  - also create parent dir for device files, if needed.
  - don't write to disk with --stdout, #2645
- archive check: detect and fix missing all-zero replacement chunks, #2180
- fix (de)compression exceptions, #2224 #2221
- files cache: update inode number, #2226
- borg rpc: use limited msgpack.Unpacker (security precaution), #2139
- Manifest: use limited msgpack.Unpacker (security precaution), #2175
- Location: accept //servername/share/path
- fix ChunkIndex.__contains__ assertion  for big-endian archs (harmless)
- create: handle BackupOSError on a per-path level in one spot
- fix error msg, there is no --keep-last in borg 1.0.x, #2282
- clamp (nano)second values to unproblematic range, #2304
- fuse / borg mount:

  - fix st_blocks to be an integer (not float) value
  - fix negative uid/gid crash (they could come into archives e.g. when
    backing up external drives under cygwin), #2674
  - fix crash if empty (None) xattr is read
  - do pre-mount checks before opening repository
  - check llfuse is installed before asking for passphrase
- borg rename: expand placeholders, #2386
- borg serve: fix forced command lines containing BORG_ env vars
- fix error msg, it is --keep-within, not --within
- fix borg key/debug/benchmark crashing without subcommand, #2240
- chunker: fix invalid use of types, don't do uint32_t >> 32
- document follow_symlinks requirements, check libc, #2507

New features:

- added BORG_PASSCOMMAND environment variable, #2573
- add minimal version of in repository mandatory feature flags, #2134

  This should allow us to make sure older borg versions can be cleanly
  prevented from doing operations that are no longer safe because of
  repository format evolution. This allows more fine grained control than
  just incrementing the manifest version. So for example a change that
  still allows new archives to be created but would corrupt the repository
  when an old version tries to delete an archive or check the repository
  would add the new feature to the check and delete set but leave it out
  of the write set.
- borg delete --force --force to delete severely corrupted archives, #1975

Other changes:

- embrace y2038 issue to support 32bit platforms
- be more clear that this is a "beyond repair" case, #2427
- key file names: limit to 100 characters and remove colons from host name
- upgrade FUSE for macOS to 3.5.8, #2346
- split up parsing and filtering for --keep-within, better error message, #2610
- docs:

  - fix caskroom link, #2299
  - address SSH batch mode, #2202 #2270
  - improve remote-path description
  - document snapshot usage, #2178
  - document relative path usage, #1868
  - one link per distro in the installation page
  - development: new branching model in git repository
  - kill api page
  - added FAQ section about backing up root partition
  - add bountysource badge, #2558
  - create empty docs.txt reequirements, #2694
  - README: how to help the project
  - note -v/--verbose requirement on affected options, #2542
  - document borg init behaviour via append-only borg serve, #2440
  - be clear about what buzhash is used for (chunking) and want it is not
    used for (deduplication)- also say already in the readme that we use a
    cryptohash for dedupe, so people don't worry, #2390
  - add hint about chunker params to borg upgrade docs, #2421
  - clarify borg upgrade docs, #2436
  - quickstart: delete problematic BORG_PASSPRHASE use, #2623
  - faq: specify "using inline shell scripts"
  - document pattern denial of service, #2624
- tests:

  - remove attic dependency of the tests, #2505
  - travis:

    - enhance travis setuptools_scm situation
    - install fakeroot for Linux
  - add test for borg delete --force
  - enable remote tests on cygwin (the cygwin issue that caused these tests
    to break was fixed in cygwin at least since cygwin 2.8, maybe even since
  - remove skipping the noatime tests on GNU/Hurd, #2710
  - fix borg import issue, add comment, #2718
  - include attic.tar.gz when installing the package
    also: add include_package_data=True

Version 1.0.10 (2017-02-13)

Bug fixes:

- Manifest timestamps are now monotonically increasing,
  this fixes issues when the system clock jumps backwards
  or is set inconsistently across computers accessing the same repository, #2115
- Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on
  py.test >= 3.0, #2112

New features:

- "key export" can now generate a printable HTML page with both a QR \ 
code and
  a human-readable "paperkey" representation (and custom text) through the
  ``--qr-html`` option.

  The same functionality is also available through `paperkey.html \ 
  which is the same HTML page generated by ``--qr-html``. It works with existing
  "key export" files and key files.

Other changes:

- docs:

  - language clarification - "borg create --one-file-system" option \ 
does not respect
    mount points, but considers different file systems instead, #2141
- setup.py: build_api: sort file list for determinism

Version 1.0.10rc1 (2017-01-29)

Bug fixes:

- borg serve: fix transmission data loss of pipe writes, #1268
  This affects only the cygwin platform (not Linux, BSD, OS X).
- Avoid triggering an ObjectiveFS bug in xattr retrieval, #1992
- When running out of buffer memory when reading xattrs, only skip the
  current file, #1993
- Fixed "borg upgrade --tam" crashing with unencrypted repositories. Since
  :ref:`the issue <tam_vuln>` is not relevant for unencrypted repositories,
  it now does nothing and prints an error, #1981.
- Fixed change-passphrase crashing with unencrypted repositories, #1978
- Fixed "borg check repo::archive" indicating success if \ 
"archive" does not exist, #1997
- borg check: print non-exit-code warning if --last or --prefix aren't fulfilled
- fix bad parsing of wrong repo location syntax
- create: don't create hard link refs to failed files,
  mount: handle invalid hard link refs, #2092
- detect mingw byte order, #2073
- creating a new segment: use "xb" mode, #2099
- mount: umount on SIGINT/^C when in foreground, #2082

Other changes:

- binary: use fixed AND freshly compiled pyinstaller bootloader, #2002
- xattr: ignore empty names returned by llistxattr(2) et al
- Enable the fault handler: install handlers for the SIGSEGV, SIGFPE, SIGABRT,
  SIGBUS and SIGILL signals to dump the Python traceback.
- Also print a traceback on SIGUSR2.
- borg change-passphrase: print key location (simplify making a backup of it)
- officially support Python 3.6 (setup.py: add Python 3.6 qualifier)
- tests:

  - vagrant / travis / tox: add Python 3.6 based testing
  - vagrant: fix openbsd repo, #2042
  - vagrant: fix the freebsd64 machine, #2037 #2067
  - vagrant: use python 3.5.3 to build binaries, #2078
  - vagrant: use osxfuse 3.5.4 for tests / to build binaries
    vagrant: improve darwin64 VM settings
  - travis: fix osxfuse install (fixes OS X testing on Travis CI)
  - travis: require succeeding OS X tests, #2028
  - travis: use latest pythons for OS X based testing
  - use pytest-xdist to parallelize testing
  - fix xattr test race condition, #2047
  - setup.cfg: fix pytest deprecation warning, #2050
- docs:

  - language clarification - VM backup FAQ
  - borg create: document how to backup stdin, #2013
  - borg upgrade: fix incorrect title levels
  - add CVE numbers for issues fixed in 1.0.9, #2106
- fix typos (taken from Debian package patch)
- remote: include data hexdump in "unexpected RPC data" error message
- remote: log SSH command line at debug level
- API_VERSION: use numberspaces, #2023
- remove .github from pypi package, #2051
- add pip and setuptools to requirements file, #2030
- SyncFile: fix use of fd object after close (cosmetic)
- Manifest.in: simplify, exclude \*.{so,dll,orig}, #2066
- ignore posix_fadvise errors in repository.py, #2095
  (works around issues with docker on ARM)
- make LoggedIO.close_segment reentrant, avoid reentrance
   2017-01-05 17:33:16 by Thomas Klausner | Files touched by this commit (2) | Package updated
Log message:
Updated py-borgbackup to 1.0.9.

Version 1.0.9 (2016-12-20)

Security fixes:

    A flaw in the cryptographic authentication scheme in Borg
    allowed an attacker to spoof the manifest. See Pre-1.0.9 manifest
    spoofing vulnerability above for the steps you should take.
    borg check: When rebuilding the manifest (which should only be
    needed very rarely) duplicate archive names would be handled
    on a âfirst come first serveâ basis, allowing an attacker to
    apparently replace archives.

Bug fixes:

    borg check:
        rebuild manifest if it’s corrupted
        skip corrupted chunks during manifest rebuild
    fix TypeError in integrity error handler, #1903, #1894
    fix location parser for archives with @ char (regression introduced in \ 
1.0.8), #1930
    fix wrong duration/timestamps if system clock jumped during a create
    fix progress display not updating if system clock jumps backwards
    fix checkpoint interval being incorrect if system clock jumps

Other changes:

        add python3-devel as a dependency for cygwin-based installation
        clarify extract is relative to current directory
        FAQ: fix link to changelog
        markup fixes
        test_get_(cache|keys)_dir: clean env state, #1897
        get back pytest’s pretty assertion failures, #1938
    setup.py build_usage:
        fixed build_usage not processing all commands
        fixed build_usage not generating includes for debug commands

Version 1.0.9rc1 (2016-11-27)

Bug fixes:

    files cache: fix determination of newest mtime in backup set
    (which is used in cache cleanup and led to wrong âAâ [added]
    status for unchanged files in next backup), #1860.
    borg check:
        fix incorrectly reporting attic 0.13 and earlier archives as corrupt
        handle repo w/o objects gracefully and also bail out early if repo is \ 
completely empty, #1815.
    fix tox/pybuild in 1.0-maint
    at xattr module import time, loggers are not initialized yet

New features:

    borg umount <mountpoint> exposed already existing umount code
    via the CLI api, so users can use it, which is more consistent
    than using borg to mount and fusermount -u (or umount) to
    un-mount, #1855.
    implement borg create –noatime –noctime, fixes #1853

Other changes:

        display README correctly on PyPI
        improve cache / index docs, esp. files cache docs, fixes #1825
        different pattern matching for –exclude, #1779
        datetime formatting examples for {now} placeholder, #1822
        clarify passphrase mode attic repo upgrade, #1854
        clarify –umask usage, #1859
        clarify how to choose PR target branch
        clarify prune behavior for different archive contents, #1824
        fix PDF issues, add logo, fix authors, headings, TOC
        move security verification to support section
        fix links in standalone README (:ref: tags)
        add link to security contact in README
        add FAQ about security
        move fork differences to FAQ
        add more details about resource usage
    tests: skip remote tests on cygwin, #1268
        allow OS X failures until the brew cask osxfuse issue is fixed
        caskroom osxfuse-beta gone, it’s osxfuse now (3.5.3)
        upgrade OSXfuse / FUSE for macOS to 3.5.3
        remove llfuse from tox.ini at a central place
        do not try to install llfuse on centos6
        fix fuse test for darwin, #1546
        add windows virtual machine with cygwin
        Vagrantfile cleanup / code deduplication
   2017-01-03 14:23:05 by Jonathan Perkin | Files touched by this commit (52)
Log message:
Use "${MV} || ${TRUE}" and "${RM} -f" consistently in \ 
post-install targets.
   2016-11-16 14:21:14 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
Updated py-borgbackup to 1.0.8.

Version 1.0.8 (2016-10-29)

Bug fixes:

    RemoteRepository: Fix busy wait in call_many, #940

New features:

    implement borgmajor/borgminor/borgpatch placeholders, #1694 {borgversion} \ 
was already there (full version string). With the new placeholders you can now \ 
also get e.g. 1 or 1.0 or 1.0.8.

Other changes:

    avoid previous_location mismatch, #1741

    due to the changed canonicalization for relative pathes in PR #1711 / #1655 \ 
(implement /./ relpath hack), there would be a changed repo location warning and \ 
the user would be asked if this is ok. this would break automation and require \ 
manual intervention, which is unwanted.

    thus, we automatically fix the previous_location config entry, if it only \ 
changed in the expected way, but still means the same location.

        deployment.rst: do not use bare variables in ansible snippet
        add clarification about append-only mode, #1689
        setup.py: add comment about requiring llfuse, #1726
        update usage.rst / api.rst
        repo url / archive location docs + typo fix
        quickstart: add a comment about other (remote) filesystems

    vagrant / tests:
        no chown when rsyncing (fixes boxes w/o vagrant group)
        fix fuse permission issues on linux/freebsd, #1544
        skip fuse test for borg binary + fakeroot
        ignore security.selinux xattrs, fixes tests on centos, #1735

Version 1.0.8rc1 (2016-10-17)

Bug fixes:

    fix signal handling (SIGINT, SIGTERM, SIGHUP), #1620 #1593 Fixes e.g. \ 
leftover lock files for quickly repeated signals (e.g. Ctrl-C Ctrl-C) or lost \ 
connections or systemd sending SIGHUP.
    progress display: adapt formatting to narrow screens, do not crash, #1628
    borg create --read-special - fix crash on broken symlink, #1584. also \ 
correctly processes broken symlinks. before this regressed to a crash (5b45385) \ 
a broken symlink would've been skipped.
    process_symlink: fix missing backup_io() Fixes a \ 
chmod/chown/chgrp/unlink/rename/... crash race between getting dirents and \ 
dispatching to process_symlink.
    yes(): abort on wrong answers, saying so, #1622
    fixed exception borg serve raised when connection was closed before \ 
reposiory was openend. add an error message for this.
    fix read-from-closed-FD issue, #1551 (this seems not to get triggered in \ 
1.0.x, but was discovered in master)
    hashindex: fix iterators (always raise StopIteration when exhausted) (this \ 
seems not to get triggered in 1.0.x, but was discovered in master)
    enable relative pathes in ssh:// repo URLs, via /./relpath hack, #1655
    allow repo pathes with colons, #1705
    update changed repo location immediately after acceptance, #1524
    fix debug get-obj / delete-obj crash if object not found and remote repo, #1684
    pyinstaller: use a spec file to build borg.exe binary, exclude osxfuse dylib \ 
on Mac OS X (avoids mismatch lib <-> driver), #1619

New features:

    add "borg key export" / "borg key import" commands, \ 
#1555, so users are able to backup / restore their encryption keys more easily.

    Supported formats are the keyfile format used by borg internally and a \ 
special "paper" format with by line checksums for printed backups. For \ 
the paper format, the import is an interactive process which checks each line as \ 
soon as it is input.

    add "borg debug-refcount-obj" to determine a repo objects' \ 
referrer counts, #1352

Other changes:

    add "borg debug ..." subcommands (borg debug-* still works, but \ 
will be removed in borg 1.1)
    setup.py: Add subcommand support to build_usage.
    remote: change exception message for unexpected RPC data format to indicate \ 
dataflow direction.
    improved messages / error reporting:
        IntegrityError: add placeholder for message, so that the message we give \ 
appears not only in the traceback, but also in the (short) error message, #1572
        borg.key: include chunk id in exception msgs, #1571
        better messages for cache newer than repo, #1700
    vagrant (testing/build VMs):
        upgrade OSXfuse / FUSE for macOS to 3.5.2
        update Debian Wheezy boxes, #1686
        openbsd / netbsd: use own boxes, fixes misc rsync installation and \ 
fuse/llfuse related testing issues, #1695 #1696 #1670 #1671 #1728
        add docs for "key export" and "key import" commands, \ 
        fix inconsistency in FAQ (pv-wrapper).
        fix second block in "Easy to use" section not showing on \ 
GitHub, #1576
        add bestpractices badge
        link reference docs and faq about BORG_FILES_CACHE_TTL, #1561
        improve borg info --help, explain size infos, #1532
        add release signing key / security contact to README, #1560
        add contribution guidelines for developers
        development.rst: add sphinx_rtd_theme to the sphinx install command
        adjust border color in borg.css
        add debug-info usage help file
        internals.rst: fix typos
        setup.py: fix build_usage to always process all commands
        added docs explaining multiple --restrict-to-path flags, #1602
        add more specific warning about write-access debug commands, #1587
        clarify FAQ regarding backup of virtual machines, #1672
        work around fuse xattr test issue with recent fakeroot
        simplify repo/hashindex tests
        travis: test fuse-enabled borg, use trusty to have a recent FUSE
        re-enable fuse tests for RemoteArchiver (no deadlocks any more)
        clean env for pytest based tests, #1714
        fuse_mount contextmanager: accept any options
   2016-08-28 17:48:37 by Thomas Klausner | Files touched by this commit (112)
Log message:
Remove unnecessary PLIST_SUBST and FILES_SUBST that are now provided
by the infrastructure.

Mark a couple more packages as not ready for python-3.x.
   2016-08-20 00:34:14 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
Updated py-borgbackup to 1.0.7.

Important note about pre-1.0.4 potential repo corruption

Some external errors (like network or disk I/O errors) could lead to
corruption of the backup repository due to issue #1138.

A sign that this happened is if "E" status was reported for a file that can
not be explained by problems with the source file. If you still have logs from
"borg create -v --list", you can check for "E" status.

Here is what could cause corruption and what you can do now:

1) I/O errors (e.g. repo disk errors) while writing data to repo.

This could lead to corrupted segment files.


    # check for corrupt chunks / segments:
    borg check -v --repository-only REPO

    # repair the repo:
    borg check -v --repository-only --repair REPO

    # make sure everything is fixed:
    borg check -v --repository-only REPO

2) Unreliable network / unreliable connection to the repo.

This could lead to archive metadata corruption.


    # check for corrupt archives:
    borg check -v --archives-only REPO

    # delete the corrupt archives:
    borg delete --force REPO::CORRUPT_ARCHIVE

    # make sure everything is fixed:
    borg check -v --archives-only REPO

3) In case you want to do more intensive checking.

The best check that everything is ok is to run a dry-run extraction::

    borg extract -v --dry-run REPO::ARCHIVE

Version 1.0.7 (2016-08-19)

Security fixes:

- borg serve: fix security issue with remote repository access, #1428
  If you used e.g. --restrict-to-path /path/client1/ (with or without trailing
  slash does not make a difference), it acted like a path prefix match using
  /path/client1 (note the missing trailing slash) - the code then also allowed
  working in e.g. /path/client13 or /path/client1000.

  As this could accidentally lead to major security/privacy issues depending on
  the pathes you use, the behaviour was changed to be a strict directory match.
  That means --restrict-to-path /path/client1 (with or without trailing slash
  does not make a difference) now uses /path/client1/ internally (note the
  trailing slash here!) for matching and allows precisely that path AND any
  path below it. So, /path/client1 is allowed, /path/client1/repo1 is allowed,
  but not /path/client13 or /path/client1000.

  If you willingly used the undocumented (dangerous) previous behaviour, you
  may need to rearrange your --restrict-to-path pathes now. We are sorry if
  that causes work for you, but we did not want a potentially dangerous
  behaviour in the software (not even using a for-backwards-compat option).

Bug fixes:

- fixed repeated LockTimeout exceptions when borg serve tried to write into
  a already write-locked repo (e.g. by a borg mount), #502 part b)
  This was solved by the fix for #1220 in 1.0.7rc1 already.
- fix cosmetics + file leftover for "not a valid borg repository", #1490
- Cache: release lock if cache is invalid, #1501
- borg extract --strip-components: fix leak of preloaded chunk contents
- Repository, when a InvalidRepository exception happens:

  - fix spurious, empty lock.roster
  - fix repo not closed cleanly

New features:

- implement borg debug-info, fixes #1122
  (just calls already existing code via cli, same output as below tracebacks)

Other changes:

- skip the O_NOATIME test on GNU Hurd, fixes #1315
  (this is a very minor issue and the GNU Hurd project knows the bug)
- document using a clean repo to test / build the release

Version 1.0.7rc2 (2016-08-13)

Bug fixes:

- do not write objects to repository that are bigger than the allowed size,
  borg will reject reading them, #1451.

  Important: if you created archives with many millions of files or
  directories, please verify if you can open them successfully,
  e.g. try a "borg list REPO::ARCHIVE".
- lz4 compression: dynamically enlarge the (de)compression buffer, the static
  buffer was not big enough for archives with extremely many items, #1453
- larger item metadata stream chunks, raise archive item limit by 8x, #1452
- fix untracked segments made by moved DELETEs, #1442

  Impact: Previously (metadata) segments could become untracked when deleting data,
  these would never be cleaned up.
- extended attributes (xattrs) related fixes:

  - fixed a race condition in xattrs querying that led to the entire file not
    being backed up (while logging the error, exit code = 1), #1469
  - fixed a race condition in xattrs querying that led to a crash, #1462
  - raise OSError including the error message derived from errno, deal with
    path being a integer FD

Other changes:

- print active env var override by default, #1467
- xattr module: refactor code, deduplicate, clean up
- repository: split object size check into too small and too big
- add a transaction_id assertion, so borg init on a broken (inconsistent)
  filesystem does not look like a coding error in borg, but points to the
  real problem.
- explain confusing TypeError caused by compat support for old servers, #1456
- add forgotten usage help file from build_usage
- refactor/unify buffer code into helpers.Buffer class, add tests
- docs:

  - document archive limitation, #1452
  - improve prune examples

Version 1.0.7rc1 (2016-08-05)

Bug fixes:

- fix repo lock deadlocks (related to lock upgrade), #1220
- catch unpacker exceptions, resync, #1351
- fix borg break-lock ignoring BORG_REPO env var, #1324
- files cache performance fixes (fixes unneccessary re-reading/chunking/
  hashing of unmodified files for some use cases):

  - fix unintended file cache eviction, #1430
  - implement BORG_FILES_CACHE_TTL, update FAQ, raise default TTL from 10
    to 20, #1338

  - cache partially read data chunks (performance), #965, #966
  - always create a root dir, #1125
- use an OrderedDict for helptext, making the build reproducible, #1346
- RemoteRepository init: always call close on exceptions, #1370 (cosmetic)
- ignore stdout/stderr broken pipe errors (cosmetic), #1116

New features:

- better borg versions management support (useful esp. for borg servers
  wanting to offer multiple borg versions and for clients wanting to choose
  a specific server borg version), #1392:

  - add BORG_VERSION environment variable before executing "borg \ 
serve" via ssh
  - add new placeholder {borgversion}
  - substitute placeholders in --remote-path

- borg init --append-only option (makes using the more secure append-only mode
  more convenient. when used remotely, this requires 1.0.7+ also on the borg
  server), #1291.

Other changes:

- Vagrantfile:

  - darwin64: upgrade to FUSE for macOS 3.4.1 (aka osxfuse), #1378
  - xenial64: use user "ubuntu", not "vagrant" (as usual), #1331
- tests:

  - fix fuse tests on OS X, #1433
- docs:

  - FAQ: add backup using stable filesystem names recommendation
  - FAQ about glibc compatibility added, #491, glibc-check improved
  - FAQ: 'A' unchanged file; remove ambiguous entry age sentence.
  - OS X: install pkg-config to build with FUSE support, fixes #1400
  - add notes about shell/sudo pitfalls with env. vars, #1380
  - added platform feature matrix
- implement borg debug-dump-repo-objs

Version 1.0.6 (2016-07-12)

Bug fixes:

- Linux: handle multiple LD_PRELOAD entries correctly, #1314, #1111
- Fix crash with unclear message if the libc is not found, #1314, #1111

Other changes:

- tests:

  - Fixed O_NOATIME tests for Solaris and GNU Hurd, #1315
  - Fixed sparse file tests for (file) systems not supporting it, #1310
- docs:

  - Fixed syntax highlighting, #1313
  - misc docs: added data processing overview picture

Version 1.0.6rc1 (2016-07-10)

New features:

- borg check --repair: heal damaged files if missing chunks re-appear (e.g. if
  the previously missing chunk was added again in a later backup archive),
  #148. (*) Also improved logging.

Bug fixes:

- sync_dir: silence fsync() failing with EINVAL, #1287
  Some network filesystems (like smbfs) don't support this and we use this in
  repository code.
- borg mount (FUSE):

  - fix directories being shadowed when contained paths were also specified,
  - raise I/O Error (EIO) on damaged files (unless -o allow_damaged_files is
    used), #1302. (*)
- borg extract: warn if a damaged file is extracted, #1299. (*)
- Added some missing return code checks (ChunkIndex._add, hashindex_resize).
- borg check: fix/optimize initial hash table size, avoids resize of the table.

Other changes:

- tests:

  - add more FUSE tests, #1284
  - deduplicate fuse (u)mount code
  - fix borg binary test issues, #862
- docs:

  - changelog: added release dates to older borg releases
  - fix some sphinx (docs generator) warnings, #881


(*) Some features depend on information (chunks_healthy list) added to item
metadata when a file with missing chunks was "repaired" using all-zero
replacement chunks. The chunks_healthy list is generated since borg 1.0.4,
thus borg can't recognize such "repaired" (but content-damaged) files \ 
if the
repair was done with an older borg version.

Version 1.0.5 (2016-07-07)

Bug fixes:

- borg mount: fix FUSE crash in xattr code on Linux introduced in 1.0.4, #1282

Other changes:

- backport some FAQ entries from master branch
- add release helper scripts
- Vagrantfile:

  - centos6: no FUSE, don't build binary
  - add xz for redhat-like dists

Version 1.0.4 (2016-07-07)

New features:

- borg serve --append-only, #1168
  This was included because it was a simple change (append-only functionality
  was already present via repository config file) and makes better security now
  practically usable.
- BORG_REMOTE_PATH environment variable, #1258
  This was included because it was a simple change (--remote-path cli option
  was already present) and makes borg much easier to use if you need it.
- Repository: cleanup incomplete transaction on "no space left" condition.
  In many cases, this can avoid a 100% full repo filesystem (which is very
  problematic as borg always needs free space - even to delete archives).

Bug fixes:

- Fix wrong handling and reporting of OSErrors in borg create, #1138.
  This was a serious issue: in the context of "borg create", errors like
  repository I/O errors (e.g. disk I/O errors, ssh repo connection errors)
  were handled badly and did not lead to a crash (which would be good for this
  case, because the repo transaction would be incomplete and trigger a
  transaction rollback to clean up).
  Now, error handling for source files is cleanly separated from every other
  error handling, so only problematic input files are logged and skipped.
- Implement fail-safe error handling for borg extract.
  Note that this isn't nearly as critical as the borg create error handling
  bug, since nothing is written to the repo. So this was "merely" \ 
  error reporting.
- Add missing error handler in directory attr restore loop.
- repo: make sure write data hits disk before the commit tag (#1236) and also
  sync the containing directory.
- FUSE: getxattr fail must use errno.ENOATTR, #1126
  (fixes Mac OS X Finder malfunction: "zero bytes" file length, access \ 
- borg check --repair: do not lose information about the good/original chunks.
  If we do not lose the original chunk IDs list when "repairing" a file
  (replacing missing chunks with all-zero chunks), we have a chance to \ 
  the file back into its original state later, in case the chunks re-appear
  (e.g. in a fresh backup). Healing is not implemented yet, see #148.
- fixes for --read-special mode:

  - ignore known files cache, #1241
  - fake regular file mode, #1214
  - improve symlinks handling, #1215
- remove passphrase from subprocess environment, #1105
- Ignore empty index file (will trigger index rebuild), #1195
- add missing placeholder support for --prefix, #1027
- improve exception handling for placeholder replacement
- catch and format exceptions in arg parsing
- helpers: fix "undefined name 'e'" in exception handler
- better error handling for missing repo manifest, #1043
- borg delete:

  - make it possible to delete a repo without manifest
  - borg delete --forced allows to delete corrupted archives, #1139
- borg check:

  - make borg check work for empty repo
  - fix resync and msgpacked item qualifier, #1135
  - rebuild_manifest: fix crash if 'name' or 'time' key were missing.
  - better validation of item metadata dicts, #1130
  - better validation of archive metadata dicts
- close the repo on exit - even if rollback did not work, #1197.
  This is rather cosmetic, it avoids repo closing in the destructor.

- tests:

  - fix sparse file test, #1170
  - flake8: ignore new F405, #1185
  - catch "invalid argument" on cygwin, #257
  - fix sparseness assertion in test prep, #1264

Other changes:

- make borg build/work on OpenSSL 1.0 and 1.1, #1187
- docs / help:

  - fix / clarify prune help, #1143
  - fix "patterns" help formatting
  - add missing docs / help about placeholders
  - resources: rename atticmatic to borgmatic
  - document sshd settings, #545
  - more details about checkpoints, add split trick, #1171
  - support docs: add freenode web chat link, #1175
  - add prune visualization / example, #723
  - add note that Fnmatch is default, #1247
  - make clear that lzma levels > 6 are a waste of cpu cycles
  - add a "do not edit" note to auto-generated files, #1250
  - update cygwin installation docs
- repository interoperability with borg master (1.1dev) branch:

  - borg check: read item metadata keys from manifest, #1147
  - read v2 hints files, #1235
  - fix hints file "unknown version" error handling bug
- tests: add tests for format_line
- llfuse: update version requirement for freebsd
- Vagrantfile:

  - use openbsd 5.9, #716
  - do not install llfuse on netbsd (broken)
  - update OSXfuse to version 3.3.3
  - use Python 3.5.2 to build the binaries
- glibc compatibility checker: scripts/glibc_check.py
- add .eggs to .gitignore

Version 1.0.3 (2016-05-20)

Bug fixes:

- prune: avoid that checkpoints are kept and completed archives are deleted in
  a prune run), #997
- prune: fix commandline argument validation - some valid command lines were
  considered invalid (annoying, but harmless), #942
- fix capabilities extraction on Linux (set xattrs last, after chown()), #1069
- repository: fix commit tags being seen in data
- when probing key files, do binary reads. avoids crash when non-borg binary
  files are located in borg's key files directory.
- handle SIGTERM and make a clean exit - avoids orphan lock files.
- repository cache: don't cache large objects (avoid using lots of temp. disk
  space), #1063

Other changes:

- Vagrantfile: OS X: update osxfuse / install lzma package, #933
- setup.py: add check for platform_darwin.c
- setup.py: on freebsd, use a llfuse release that builds ok
- docs / help:

  - update readthedocs URLs, #991
  - add missing docs for "borg break-lock", #992
  - borg create help: add some words to about the archive name
  - borg create help: document format tags, #894