Subject: CVS commit: pkgsrc/security/amass
From: Leonardo Taccari
Date: 2023-03-19 18:45:02
Message id: 20230319174503.1E0AEFA90@cvs.NetBSD.org

Log Message:
amass: Update to 3.22.1

Changes:
v3.22.1
## Changelog
* moved the reverse DNS sweeping into the scripting engine

v3.22.0
## Changelog
* updates to the enumeration pipeline
* updated the date elements in the word list
* updated the ShadowServer and TeamCymru data source integrations
* updated the Farsight DNSDB integration
* updated copyright and licensing information
* updated AlienVault, and removed the networksdb and umbrella data sources
* started to switch to the new HTTP request function
* removed the Google Certificate Transparency data source integration
* removed support for Snapcraft and Cloudflare
* performance improvements to the scripting engine
* moved all active techniques to the scripting engine
* discoveries are now written directly to the data store
* added the grep.app search API as a data source
* Removed the Twitter integration and associated dependencies
* Removed all references to the ioutil deprecated package

v3.21.2
## Changelog
* Updated the caffix/resolve package

v3.21.1
## Changelog
* Fixed the Docker workflow

v3.21.0
## Changelog
* Updated the Docker workflow
* Additional reliability and consistency from the DNS resolution

v3.20.0
## Changelog
* Updated the project to Go version 1.18
* Updated testing packages
* Updated User Guide
* Updated SearX instances list
* Small fix for AbuseIPDB data source
* Removed ineffective words for Alterations
* Ported FOFA to lua script implementation
* Numerous data source integration enhancements
* Make Pastebin data-source only available with an API key
* Improved HackerOne datasource and renamed PSBDMP to Pastebin
* Improved GitLab data source and ported Searchcode source to API usage
* FullHunt integration now requires an API key
* Fixed ThreatBook and incorrect indent in some data source scripts
* Fixed Censys data-source script
* BeVigil Source Added
* ArchiveIt data source optimization
* Added support for commercial BufferOver API key
* Added Yandex datasource and adjusted rate limit of Gists
* Added Netlas as a datasource and removed IPv4Info
* Added Google, PSBDMP and PublicWWW as datasources
* Added DNSHistory as a datasource
* Added ASNLookup and BidDataCloud as data sources

v3.19.3
## Changelog
* Fixes for the intel subcommand
* Added links for Joona and Alexis in the testimonial

v3.19.2
## Changelog
* Added the testimonial provided by the Visma Red Team
* Removed the dns subcommand
* Fixed issue #807 that caused passive enumerations to halt
* Fixed #808 by removing the integration instructions no longer supported
* Added support to 'viz' command to accept an output file prefix
* Added a unit test for data source response caching

v3.19.1
## Changelog
* Updated the caffix/resolve package dependency
* Improved how DNS names are being filtered
* Add common words for alterations

v3.19.0
## Changelog
* Initial integration of the BGP.Tools data source
* Fixed issue #490 causing the timeout to hang
* Enable SBOM
* Changed Lua memory allocation options
* Added the mtime function to the scripting environment
* Added BGPTools to the list of data sources

v3.18.3
## Changelog
* Updates to the Snapcraft build config
* Enum pipeline optimizations and event bus removal fixes

v3.18.2
## Changelog
* Disabling SBOM generation

v3.17.1
## Changelog
* Additional DNS related improvements
* Added LeakIX to the list of data sources

v3.17.0
## Changelog
* Updated all filters to Stable Bloom Filters
* Removed support for sharing discoveries
* New SRV record for Citrix Receiver
* Implemented rate limit detection for Github datasource
* Fixed issue #781 so users specifying resolvers do not download public DNS info
* Fixed issue #746 with SetDataSources error return value
* Fixed Snapcraft config file parsing issue
* Fix Digitorus datasource
* Discontinued use of the in-memory graph database
* Added the LeakIX data source
* Added new configurations when using DNS
* Fixed the CleanName bug

v3.16.0
## Changelog
* Updated the BufferOver data source implementation to use a paid service
* Updated the Amass installation process in documentation
* The ReconDev data source is no longer available
* CommonCrawl data source interface changed

v3.15.2
## Changelog
* Passive mode saves names in the graph database

v3.15.1
## Changelog
* Fixed Gexf test

v3.15.0
## Changelog
* max-depth option for brute-forcing
* Updates to the new max depth recursive brute forcing feature
* Improvements to graph database migration performance
* FullHunt integration that resolves issue #740
* Fix to ASN information caching with large impact on performance
* Additional improvements to the enumeration engine
* Adding MaxDepth option to Brute-force

v3.14.3
## Changelog
* Started using Go native file embedding
* Removed the use of the asnlist.txt file
* Improvements to the enumeration engine workflow

v3.14.2
## Changelog
* Verify that addresses in CheckAddresses are actually valid
* Put common crawl into a separate category
* Optimized the enumeration engine
* Improve test coverage of loadBruteForceSettings to 100%
* Fixed #707 bug in the Google CT data source
* Enumeration engine performance improvements
* Add help subcommand

v3.14.1
## Changelog
* Updated various data source integrations
* Ported ShadowServer and TeamCymru to script implementations
* Fixed #707 issue with Google certificate transparency
* Change file pointer to use STDOUT for "-" with JSON output
* Added the socket Lua module and updated DNS resolve

v3.14.0
## Changelog
* Using the new stringset implementation
* Scraping CSV export page for compact data (Riddler)
* Quake data source bug fix
* Fixed the call to new_addr
* Fix for failure to build the resolver pool
* Exclude www from DuckDuckGo search result
* Enhanced web crawling feature
* Better URL protocol detection
* Added logging for all requests to data sources
* Added checking for output to help fix #566
* Added UKWebArchive as a data source
* Added Paradigm to external projects
* Added Maltiverse, Greynoise as data sources and some rate limit adjustments
* Added IPdata data source and some fixes
* Added HyperStat and removed Whoisology data source
* Added HAW (Croatian web archive) as a data source
* Added Github Gists as a data source and renamed Anubis
* Added Detectify and N45HT
* Added CertDetails as a data source
* Added Ahrefs and Quake as data sources
* Added AbuseIPDB, AskDNS, SpyOnWeb as data sources
* Added ARIN, DNSlytics, Spamhaus as data sources

v3.13.4
## Changelog
* Updated the IP2ASN data
* Reduced the DNS query rate provided to wildcard detection
* Increased rate data is provided to the pipeline
* Improved Spyse integration and fixed #643
* Added WhoisXML API to the information sharing community

v3.13.3
## Changelog
* improvement: update spyse api script
* Updates to data source scripts
* Update to project management file
* Reliability and speedup related to wildcard detection
* Increased script request processing
* Added system config file fallback feature and closed #495
* Added more baseline resolvers

v3.13.2
## Changelog
* Removed Alternate DNS from the resolvers list
* Fixed a synchronization error in the ASNCache

v3.13.1
## Changelog
* Utilizing DNS wildcard records discovered during zone transfers
* Updated and added unit testing for the ASNCache
* Initial implementations for DuckDuckGo, Hunter, and IPinfo data sources
* Fixed bug removing half of the usable DNS resolvers
* Checks added to ensure infra information is resolved
* Added the source_count field as recommended by Shodan

v3.13.0
## Changelog
* Updates to documentation for the info sharing feature
* Updated trusted DNS resolver data
* Updated and extended the WhoisXML API implementation
* Store enumeration context as soon as possible on Start
* Sorted SRV records alphabetically
* Initial enumeration findings share process implementation
* Increased the queue fill rate
* Improved some datasource scripts and removed PasteBin from datasources
* Improved Baidu datasource script
* Added the share flag to the enum subcommand options
* Added more missing SRV records
* Added a feature for detecting and removing false positives

v3.12.3
## Changelog
* Updated enumeration pipeline parameters
* Support the AS prefix in ASNs
* Remove unused options related to DNS resolvers
* Reduced the load that output has on the graph db
* No minimum rate limit for data source scripts
* Increased the rate of name alterations
* Increased potential DNS resolver rates

v3.12.2
## Changelog
* Updated dependencies

v3.12.1
## Changelog
* Reduced pipeline memory consumption #603
* Integrated changes to the data pipeline implementation

v3.12.0
## Changelog
* Moved the graph database implementation to another repo
* Changes resulting in performance improvements

Files:
RevisionActionfile
1.37modifypkgsrc/security/amass/Makefile
1.10modifypkgsrc/security/amass/distinfo
1.5modifypkgsrc/security/amass/go-modules.mk