Path to this page:
Subject: CVS commit: pkgsrc/net/gallery-dl
From: Leonardo Taccari
Date: 2022-07-17 18:19:04
Message id: 20220717161904.9592AFB1A@cvs.NetBSD.org
Log Message:
gallery-dl: Update to 1.22.4
Changes:
## 1.22.4 - 2022-07-15
### Additions
- [instagram] add `pinned` metadata field
- [itaku] categorize sections by group
- [khinsider] extract `platform` metadata
- [tumblr] support `/blog/view` URLs
- [twitter] implement `strategy` option
- [twitter] add `count` metadata field
- [formatter] implement `O` format specifier
- [postprocessor:mtime] add `value` option
- add `--no-postprocessors` command-line option
- implement `format-separator` option
### Changes
- [pinterest] handle section pins with separate extractors
- [postprocessor:ugoira] enable `mtime` by default
### Fixes
- [bunkr] fix extraction
- [hentaifoundry] fix metadata extraction
- [itaku] fix user caching
- [itaku] fix `date` parsing
- [kemonoparty] ensure all files have an `extension`
- [komikcast] update domain
- [mangakakalot] update domain
- [newgrounds] only attempt to login if necessary
- [newgrounds] prevent exception on empty results
- [nozomi] reduce memory consumption during searches
- [pixiv] fix default `background` filenames
- [sankaku] rewrite file URLs to s.sankakucomplex.com
- [slideshare] fix `description` extraction
- [twitter] ignore previously seen Tweets
- [twitter] unescape HTML entities in `content`
- [weibo] handle invalid or broken status objects
- [postprocessor:zip] ensure target directory exists
- make `brotli` an *optional* dependency
- limit path length for `--write-pages` output on Windows
### Removals
- [foolfuuka] remove archive.wakarimasen.moe
## 1.22.3 - 2022-06-28
### Changes
- [twitter] revert strategy changes for user URLs
- update default User-Agent headers
## 1.22.2 - 2022-06-27
### Additions
- [cyberdrop] add fallback URLs
- [horne] add support for horne.red
- [itaku] add `gallery` and `image` extractors
- [poipiku] add `user` and `post` extractors
- [skeb] add `following` extractor
- [twitter] implement `expand` option
- [twitter] implement `csrf` option
- [unsplash] add `collection_title` and `collection_id` metadata fields
- [weibo] support `tabtype=video` listings
- [formatter] implement slice operator as format specifier
- support cygwin/BSD/etc for `--cookies-from-browser`
### Fixes
- [instagram] improve metadata generated by `_parse_post_api()`
- [instagram] fix `tag` extractor
- [instagram] automatically invalidate expired login sessions
- [twitter] fix pagination for conversion tweets
- [twitter] improve `"replies": "self"`
- [twitter] improve strategy for user URLs
- [vk] take URLs from `*_src` entries
- [weibo] fix URLs generated by `user` extractor
- [weibo] fix retweets
- [downloader:ytdl] update `_set_outtmpl()`
- [formatter] fix `!j` conversion for non-serializable types
- [snap] Fix missing libslang dependency
## 1.22.1 - 2022-06-04
### Additions
- [gfycat] add support for collections
- [instagram] support specifying users by ID
- [paheal] extract more metadata
- [reddit] add `home` extractor
- [weibo] support usernames in URLs
- [weibo] support `livephoto` and `gif` files
- [weibo] add support for several different `tabtype` listings
- [postprocessor:metadata] write to stdout by setting filename to "-"
- implement `output.ansi` option
- support user-defined `output.mode` settings
### Changes
- [readcomiconline] remove default `browser` setting
- [weibo] switch to desktop API
- fix command-line argument name of `--cookies-from-browser`
### Fixes
- [bunkr] change domain to `app.bunkr.is`
- [deviantart] fix folder listings with `"pagination": "manual"`
- [gofile] fix 401 Unauthorized errors
- [hypnohub] move to gelbooru_v02 instances
- [instagram] fix and update extractors
- [nozomi] remove slashes from search terms
- [pixiv] include `.gif` in background fallback URLs
- [sankaku] extend URL patterns
- [subscribestar] fix `date` metadata
## 1.22.0 - 2022-05-25
### Additions
- [gelbooru_v01] add `favorite` extractor
- [instagram] add `tagged_users` to keywords for stories
- [lolisafe] implement `domain` option
- [naverwebtoon] support (best)challenge comics
- [nijie] support /history_nuita.php listings
- [pixiv] provide more data when `metadata` is enabled
- [shopify] support several more sites by default
- [twitter] extract alt texts as `description`
- [twitter] recognize vxtwitter URLs
- [weasyl] implement `metadata` option
- implement `--cookies-from-browser`
- implement `output.colors` options
- implement string literals in replacement fields
- support using extended format strings for archive keys
### Changes
- [foolfuuka] match 4chan filenames
- [pixiv] implement `include` option
- provide `avatar`/`background` downloads as separate extractors
- [twitter] use a better strategy for user URLs
- [twitter] disable `cards` by default
- delay directory creation
- flush writes to stdout/stderr
- build executables on GitHub Actions with Python 3.10
### Fixes
- [artstation] use `"browser": "firefox"` by default
- [imgur] prevent exception with empty albums
- [instagram] report redirects to captcha challenges
- [khinsider] fix metadata extraction
- [mangafox] send Referer headers
- [mangahere] send Referer headers
- [mangasee] use randomly generated PHPSESSID cookie
- [pixiv] make retrieving ugoira metadata non-fatal
- [readcomiconline] update deobfuscation code
- [realbooru] fix extraction
- [vk] handle photos without width/height info
- [vk] fix user ID extraction
- [webtoons] extract real episode numbers
- create missing directories for archive files
- detect circular references with `-K`
- replace "\f" in `--filename` arguments with a form feed character
### Removals
- [gelbooru_v01] remove tlb.booru.org from supported domains
## 1.21.2 - 2022-04-27
### Additions
- [deviantart] implement `pagination` option
- [pixiv] implement `background` option
- [postprocessor:ugoira] report ffmpeg/mkvmerge errors
### Fixes
- [cyberdrop] match cyberdrop.to URLs
- [e621] fix 403 errors
- [issuu] fix extraction
- [mangadex] download from available chapters despite `externalUrl`
- [photovogue] update domain and api endpoint
- [sexcom] add fallback for empty files
- [twitter] improve syndication video selection
- [twitter] fix various syndication issues
- [vk] fix extraction
- [weibo] fix infinite retries for deleted accounts
- [postprocessor:ugoira] use compatible paths with mkvmerge
- [postprocessor:ugoira] do not auto-select the `image2` demuxer
## 1.21.1 - 2022-04-08
### Additions
- [gofile] add gofile.io extractor
- [instagram] add `previews` option
- [kemonoparty] add `duplicates` option
- [pinterest] add extractor for created pins
- [pinterest] support multiple files per pin
- [telegraph] Add telegra.ph extractor
- [twitter] add `syndication` option
- [twitter] accept fxtwitter.com URLs
- [downloader:http] support using an arbitrary method and sending POST data
- [postprocessor:metadata] implement archive options
- [postprocessor:ugoira] add `mtime` option
- [postprocessor:ugoira] support setting timecodes with `mkvmerge`
- [formatter] support evaluating f-string literals
- add `--ugoira-conv-copy` command-line option
- implement a `contains()` function for filter statements
### Fixes
- [aryion] provide correct `date` metadata independent of DST
- [furaffinity] fix search result pagination
- [hitomi] update and fix metadata extraction
- [kissgoddess] extract all images
- [mangasee] unescape manga names
- [newgrounds] update and fix pagination
- [newgrounds] warn about age-restricted posts
- [pinterest] do not force `m3u8_native` for video downloads
- [twibooru] fix posts without `name`
- [unsplash] replace dash with space in search API queries
- [postprocessor:mtime] fix timestamps from datetime objects
- fix yet another bug in `_check_cookies()`
- fix loading/storing cookies without domain
## 1.21.0 - 2022-03-14
### Additions
- [fantia] add `num` enumeration index
- [fantia] support "Blog Post" content
- [imagebam] add support for /view/ paths
- [kemonoparty] match beta.kemono.party URLs
- [kissgoddess] add `gallery` and `model` extractors
- [mememuseum] add `tag` and `post` extractors
- [newgrounds] add `post_url` metadata field
- [patreon] add `image_large` file type
- [toyhouse] support `art` listings
- [twibooru] add extractors for searches, galleries, and posts
- [postprocessor:metadata] implement `mtime` option
- [postprocessor:mtime] add `event` option
- add fish shell completion
- add `timedelta` class to global namespace in filter expressions
### Changes
- [seiga] require authentication with `user_session` cookie
- remove username & password login due to 2FA
- refactor proxy support
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
### Fixes
- [bunkr] fix mp4 downloads
- [fanbox] fetch data for each individual post
- [hentaicosplays] send `Referer` header
- [imagebam] set `nsfw_inter` cookie
- [kemonoparty] limit default filename length
- [mangadex] fix chapters without `translatedLanguage`
- [newgrounds] fix video descriptions
- [skeb] add `sent-requests` option
- [slideshare] fix extraction
- [subscribestar] unescape attachment URLs
- [twitter] fix handling of 429 Too Many Requests responses
- [twitter] warn about age-restricted Tweets
- [twitter] handle Tweets with "softIntervention" entries
- [twitter] update query hashes
- fix another bug in `_check_cookies()`
## 1.20.5 - 2022-02-14
### Additions
- [furaffinity] add `layout` option
- [lightroom] add Lightroom gallery extractor
- [reddit] support standalone submissions on personal user pages
- [redgifs] support i.redgifs.com URLs
- [wallpapercave] add extractor for images and search results
- add `signals-ignore` option
### Changes
- [danbooru] merge `danbooru` and `e621` extractors
- support `atfbooru`
- remove support for old e621 tag search URLs
### Fixes
- [furaffinity] improve new/old layout detection
- [imgbox] fix ImgboxExtractor
- [inkbunny] rename search parameters to their API equivalents
- [kemonoparty] handle files without names
- [twitter] fix extraction
- [vk] fix infinite pagination loops
- [downloader:ytdl] make `ImportError`s non-fatal
## 1.20.4 - 2022-02-06
### Additions
- [e621] add `favorite` extractor
- [hitomi] add `format` option
- [kohlchan] add Kohlchan extractors
- [sexcom] add `pins` extractor
- [twitter] add `warnings` option
- add ability to disable TLS 1.2
- add examples for custom gelbooru instances
### Fixes
- [bunkr] fix mp4 downloads
- [gelbooru] improve and fix pagination
- [hitomi] "fix" 403 errors
- [kemonoparty] fix downloading smaller text files
- [patreon] disable TLS 1.2 by default
- [twitter] restore errors for protected timelines etc
- [twitter] restore `logout` functionality
- [twitter] provide fallback URLs for card images
- [weibo] update pagination code
Files: