./net/py-s3cmd, Command line tool for managing Amazon S3 and CloudFront services

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 2.4.0, Package name: py312-s3cmd-2.4.0, Maintainer: pkgsrc-users

S3cmd lets you copy files from/to Amazon S3 (Simple Storage Service) using a
simple to use command line client. Supports rsync-like backup, GPG encryption,
and more. Also supports management of Amazon's CloudFront content delivery
network.


Required to run:
[textproc/py-expat] [lang/python27]

Required to build:
[pkgtools/cwrappers]

Master sites:

Filesize: 141.578 KB

Version history: (Expand)


CVS history: (Expand)


   2024-11-11 08:29:31 by Thomas Klausner | Files touched by this commit (862)
Log message:
py-*: remove unused tool dependency

py-setuptools includes the py-wheel functionality nowadays
   2024-01-12 06:32:52 by Adam Ciarcinski | Files touched by this commit (3) | Package updated
Log message:
py-s3cmd: updated to 2.4.0

s3cmd-2.4.0

* Added "setversioning" command for versioning configuration \ 
(Kuan-Chun Wang)
* Added "settagging", "gettagging", and \ 
"deltagging" commands for bucket/object tagging (Kuan-Chun Wang)
* Added "setobjectretention" and "setobjectlegalhold" \ 
commands (Etienne Adam/Withings SAS)
* Added "setownership" and "setblockpublicaccess" commands
* Added "cfinval" command to request Cloudfront to invalidate paths
* Added "--keep-dirs" option to have the folder structure preserved on \ 
remote side
* Added --skip-destination-validation option for "setnotification" \ 
command (Kuan-Chun Wang)
* Added "--max-retries" flag and "max_retries" config option
* Added FIPS support (Michael Roth)
* Added "object ownership" and block public access" values to \ 
"info" command output for buckets
* Added to "ls" command a "DIROBJ" tag for directory objects \ 
in S3 remote
* Added server profiles to run-tests.py to skip tests depending on the server type
* Fixed "TypeError: sequence item 1: expected str instance, bytes \ 
found" error with Python 3.12
* Fixed a missing return for "object_batch_delete" of S3.py  (James Hewitt)
* Fixed "object is not callable" error because of md5 FIPS test
* Fixed "compute_content_md5 is not defined" error for \ 
"setversioning"
* Fixed list objects to use NextMarker when only prefixes are returned (Albin Parou)
* Fixed upload to not retry when an S3 compatible server is full
* Fixed recursive delete of objects named with whitespace
* Fixed the mime type when uploading directories to be \ 
"application/x-directory"
* Fixed "string indices must be integers" error for sync when in \ 
dry-run mode
* Fixed SignatureDoesNotMatch error when modifying an object on Cloudflare R2
* Fixed Cloudfront invalidation issue for paths with wildcard or special characters
* Fixed Cloudfront crash because of error reporting for retries
* Fixed Cloudfront "unable to parse URL" error
* Improved the handling of "empty" files on the remote side to sync \ 
with local folders
* Improved "abortmp" command by requiring an object to avoid bad \ 
accidents when using Ceph (Joshua Haas)
* Improved file download by retrying when encountering SlowDown or \ 
TooManyRequests errors (Robin Geiger)
* Improved error messages in case of connection error or host unreachable
* Improved error messages to be more explicit for upload errors after retries
* Improved remote2local attributes setting code
* Improved remote2local with more explicit error messages when setting attributes
* Improved remote2local output messages by using the "mkdir" prefix \ 
instead of "make dir"
* Improved the SortedDict class
* Improved run-test.py by using "--include" when calling Curl instead \ 
of "-include" (Matthew James Kraai)
* Improved GitHub CI by enabling pip cache in actions/setup-python (Anton Yakutovich)
* Improved GitHub CI by adding a "codespell" check on push and PRs \ 
(Yaroslav Halchenko)
* Updated GitHub CI tests to use more recent versions of Minio and Python
* Upgraded GitHub actions (Anton Yakutovich)
* Cleanup and update of copyright headers, docs, comments and setup.py
* Cleanup to fix "invalid escape sequence" syntax warnings
* Many other bug fixes and cleanups
   2023-08-05 08:06:07 by Adam Ciarcinski | Files touched by this commit (3) | Package updated
Log message:
py-s3cmd: updated to 2.3.0

s3cmd-2.3.0      - 2022-10-03
===============
* Added "getnotification", "setnotification", and \ 
"delnotification" commands for notification policies (hrchu)
* Added support for AWS_STS_REGIONAL_ENDPOINTS
* Added ConnectionRefused [111] exit code to handle connection errors (Salar \ 
Nosrati-Ershad)
* Added support for IMDSv2. Should work automatically on ec2 (Anthony Foiani)
* Added --list-allow-unordered to list objects unordered. Only supported by Ceph \ 
based s3-compatible services
* Fixed --exclude dir behavior for python >= 3.6 (Daniil Tararukhin)
* Fixed Cloudfront invalidate retry issue (Yuan-Hsiang Lee)
* Fixed 0 byte cache files crashing s3cmd
* Fixed --continue behavior for the "get" command
* Fixed unicode issue with fixbucket
* Fixed CannotSendRequest and ConnectionRefusedError errors at startup
* Fixed error reporting for object info when the object does not exist
* Fixed "setup.py test" to do nothing to avoid failure that could be \ 
problematic for distribution packaging
* Improved expire command to use Rule/Filter/Prefix for LifecycleConfiguration
* Improved PASS/CHECK/INCLUDE/EXCLUDE debug log messages
* Improved setup.py with python 3.9 and 3.10 support info(Ori Avtalion)
* Many other bug fixes

s3cmd-2.2.0      - 2021-09-27
===============
* Added support for metadata modification of files bigger than 5 GiB
* Added support for remote copy of files bigger than 5 GiB using MultiPart copy \ 
(Damian Martinez, Florent Viard)
* Added progress info output for multipart copy and current-total info in output \ 
for cp, mv and modify
* Added support for all special/foreign character names in object names to \ 
cp/mv/modify
* Added support for SSL authentication (Aleksandr Chazov)
* Added the http error 429 to the list of retryable errors
* Added support for listing and resuming of multipart uploads of more than 1000 parts
* Added time based expiration for idle pool connections in order to avoid random \ 
broken pipe errors
* Added support for STS webidentity authentication (ie AssumeRole and \ 
AssumeRoleWithWebIdentity) (Samskeyti, Florent Viard)
* Added support for custom headers to the mb command
* Improved MultiPart copy to preserve acl and metadata of objects
* Improved the server errors catching and reporting for cp/mv/modify commands
* Improved resiliency against servers sending garbage responses
* Improved remote copy to have consistent copy of metadata in all cases: \ 
multipart or not, aws or not
* Improved security by revoking public-write acl when private acl is set
* Improved speed when running on an EC2 instance
* Reduced connection_max_age to 5s to avoid broken pipes as AWS closes https \ 
conns after around 6s
* Ensure that KeyboardInterrupt are always properly raised
* Changed sized of multipart copy chunks to 1 GiB
* Fixed ValueError when using more than one ":" inside add_header in \ 
config file
* Fixed extra label issue when stdin used as source of a MultiPart upload
* Fixed remote copy to allow changing the mime-type (ie content-type) of the new \ 
object
* Fixed remote_copy to ensure that meta-s3cmd-attrs will be set based on the \ 
real source and not on the copy source
* Fixed deprecation warnings due to invalid escape sequences (Karthikeyan \ 
Singaravelan)
* Fixed getbucketinfo that was broken when the bucket lifecycle uses the filter \ 
element (Liu Lan)
* Fixed RestoreRequest XML namespace URL
* Fixed PARTIAL exit code that was not properly set when needed for object_get
* Fixed a possible inifinite loop when a file is truncated during hashsum or upload
* Fixed report_exception wrong error when LANG env var was not set
* Fixed wrong wiki url in error messages (Alec Barrett)
* Py3: Fixed an AttributeError when using the "files-from" option
* Py3: Fixed compatibility issues due to the removal of getchildren() from \ 
ElementTree in python 3.9
* Py3: Fixed compatibility issues due to the removal of encodestring() in python 3.9
* Fixed a crash when the AWS_ACCESS_KEY env var is set but not AWS_SECRET_KEY
* Cleanup of check_md5 (Riccardo Magliocchetti)
* Removed legacy code for dreamhost that should be necessary anymore
* Migrated CI tests to use github actions (Arnaud Jaffre)
* Improved README with a link to INSTALL.md (Sia Karamalegos)
* Improved help content (Dmitrii Korostelev, Roland Van Laar)
* Improvements for setup and build configurations
* Many other bug fixes

s3cmd-2.1.0      - 2020-04-07
===============
* Changed size reporting using k instead of K as it a multiple of 1024
* Added "public_url_use_https" config to generate public url using https
* Added option to make connection pooling configurable and improvements (Arto \ 
Jantunen)
* Added support for path-style bucket access to signurl (Zac Medico)
* Added docker configuration and help to run test cases with multiple Python \ 
versions (Doug Crozier)
* Relaxed limitation on special chars for --add-header key names
* Fixed all regions that were automatically converted to lower case (Harshavardhana)
* Fixed size and alignment of DU and LS output reporting
* Fixes for SignatureDoesNotMatch error when host port 80 or 443 is specified, \ 
due to stupid servers
* Fixed the useless retries of requests that fail because of ssl cert checks
* Fixed a possible crash when a file disappears during cache generation
* Fixed unicode issues with IAM
* Fixed unicode errors with bucked Policy/CORS requests
* Fixed unicode issues when loading aws_credential_file
* Fixed an issue with the tenant feature of CephRGW. Url encode bucket_name for \ 
path-style requests
* Fixed signature v2 always used when bucket_name had special chars
* Allow to use signature v4 only, even for commands without buckets specified
* Fixed small open file descriptor leaks.
* Py3: Fixed hash-bang in headers to not force using python2 when \ 
setup/s3cmd/run-test scripts are executed directly.
* Py3: Fixed unicode issues with Cloudfront
* Py3: Fixed http.client.RemoteDisconnected errors
* Py3: Fixed 'dictionary changed size during iteration' error when using a cache-file
* Py3: Fixed the display of file sizes (Vlad Presnyak)
* Py3: Python 3.8 compatibility fixes (Konstantin Shalygin)
* Py2: Fixed unicode errors sometimes crashing remote2remote sync
* Added s3cmd.egg-info to .gitignore (Philip Dubé)
* Improved run-test script to not use hard-coded bucket names
* Renamed INSTALL to INSTALL.md and improvements (Nitro, Prabhakar Gupta)
* Improved the restore command help (Hrchu)
* Updated the storage-class command help with the recent aws s3 classes
* Fixed typo in the --continue-put help message (Pengyu Chen)
* Fixed typo
* Improvements for setup and build configurations
* Many other bug fixes

s3cmd-2.0.2      - 2018-07-15
===============
* Fixed unexpected timeouts encountered during requests or transfers due to AWS \ 
strange connection short timeouts
* Fixed a throttle issue slowing down too much transfers in some cases
* Added support for $AWS_PROFILE
* Added clarification comment for the socket_timeout configuration value OS limit
* Avoid distutils usage at runtime (Matthias Klose)
* Python 2 compatibility: Fixed import error of which with fallback code \ 
(Gianfranco Costamagna)
* Fixed Python 3 bytes string encoding when getting IAM credentials (Alexander \ 
Allakhverdiyev)
* Fixed handling of config tri-state bool values (like acl_public) (Brian C. Lane)
* Fixed V2 signature when restore command is used (Jan Kasiak)
* Fixed setting full_control on objects with public read access (Matthew Vernon)
* Fixed a bug when only one path is supplied with Cloudfront. (Mikael Svensson)
* Fixed signature errors with 'modify' requests (Radek Simko)
* Fixed - Fix setacl command exception (Robert Moucha)
* Fixed error reporting if deleting a source object failed after a move
* Many other bug fixes

Important info: AWS S3 doesn't allow anymore uppercases and underscores in \ 
bucket names since march 1, 2018

s3cmd-2.0.1      - 2017-10-21
===============
* Support for Python 3 is now stable
* Fixed signature issues due to upper cases in hostname
* Improved support for Minio Azure gateway (Julien Maitrehenry, Harshavardhana)
* Added signurl_use_https option to use https prefix for signurl (Julien Recurt)
* Fixed a lot of remaining issues and regressions for Python 3
* Fixed --configure option with Python 3
* Fixed non string cmdline parameters being ignored
* Windows support fixes
* Don't force anymore to have a / on last parameter for the "modify" \ 
command
* Removed the python3 support warning
* Detect and report error 403 in getpolicy for info command
* Added a specific error message when getting policy by non owner
* Many other bug fixes

s3cmd-2.0.0      - 2017-06-26
===============
* Added support for Python 3 (Shaform, Florent Viard)
* Added getlifecycle command (Daniel Gryniewicz)
* Added --cf-inval for invalidating multiple CF distributions (Joe Mifsud)
* Added --limit to "ls" and "la" commands to return the \ 
specified number of objects (Masashi Ozawa)
* Added --token-refresh and --no-token-refresh and get the access token from the \ 
environment (Marco Jakob)
* Added --restore-priority and --restore-days for S3 Glacier (Robert Palmer)
* Fixed requester pays header with HEAD requests  (Christian Rodriguez)
* Don't allow mv/cp of multiple files to single file (Guy Gur-Ari)
* Generalize wildcard certificate forgiveness (Mark Titorenko)
* Multiple fixes for SSL connections and proxies
* Added support for HTTP 100-CONTINUE
* Fixes for s3-like servers
* Big cleanup and many unicode fixes
* Many other bug fixes
   2022-01-22 15:34:18 by Thomas Klausner | Files touched by this commit (2)
Log message:
py-s3cmd: fix PLIST
   2021-10-26 13:07:15 by Nia Alarie | Files touched by this commit (958)
Log message:
net: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes

Not committed (merge conflicts...):

net/radsecproxy/distinfo

The following distfiles could not be fetched (fetched conditionally?):

./net/citrix_ica/distinfo citrix_ica-10.6.115659/en.linuxx86.tar.gz
./net/djbdns/distinfo dnscache-1.05-multiple-ip.patch
./net/djbdns/distinfo djbdns-1.05-test28.diff.xz
./net/djbdns/distinfo djbdns-1.05-ignoreip2.patch
./net/djbdns/distinfo djbdns-1.05-multiip.diff
./net/djbdns/distinfo djbdns-cachestats.patch
   2021-10-07 16:43:07 by Nia Alarie | Files touched by this commit (962)
Log message:
net: Remove SHA1 hashes for distfiles
   2020-01-26 18:32:28 by Roland Illig | Files touched by this commit (981)
Log message:
all: migrate homepages from http to https

pkglint -r --network --only "migrate"

As a side-effect of migrating the homepages, pkglint also fixed a few
indentations in unrelated lines. These and the new homepages have been
checked manually.
   2019-11-03 12:45:59 by Roland Illig | Files touched by this commit (255)
Log message:
net: align variable assignments

pkglint -Wall -F --only aligned --only indent -r

No manual corrections.