./mail/rspamd, Fast, free and open-source spam filtering system

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 1.8.0nb1, Package name: rspamd-1.8.0nb1, Maintainer: bartosz.kuzma

Rspamd is fast, modular and lightweight spam filter. It is designed to work
with big ammount of mail and can be easily extended with own filters written in
lua.


Required to run:
[textproc/icu] [security/gnupg2] [mail/gmime] [devel/fann] [devel/glib2] [devel/ragel] [devel/lua-lpeg] [databases/hiredis] [lang/lua53]

Required to build:
[pkgtools/cwrappers]

Master sites:

SHA1: 918331a060246d921c89ffc42e99feed0335a89a
RMD160: 2a263670bf68b22792f52e39b2fb44a6c2e42da5
Filesize: 4161.444 KB

Version history: (Expand)


CVS history: (Expand)


   2018-10-12 14:49:28 by Roy Marples | Files touched by this commit (3)
Log message:
Avoid calling ERR string init functions as this doesn't work with
OpenSSL-1.1 and should happen automatically with this version.
   2018-10-07 22:10:57 by Filip Hajny | Files touched by this commit (6) | Package updated
Log message:
mail/rspamd: Update to 1.8.0.

- [Feature] Add arguments schemas to processors and extractors
- [Feature] Add functional selectors library
- [Feature] Add generic selector to reputation module
- [Feature] Add more ratelimits: by digest, by attachments data, by
  filenames
- [Feature] Add preliminary stop words detection support
- [Feature] Add pure Lua debugm function
- [Feature] Add schema validation for Redis settings
- [Feature] Add selectors combine function
- [Feature] Add some recursion protection to lua logger
- [Feature] Add support for Lua API tracing
- [Feature] Allow to apply schema to arguments
- [Feature] Allow to get dkim signing data directly from HTTP headers
- [Feature] Allow to reuse existing authentication results
- [Feature] Cache selectors results in re runtime
- [Feature] Implement new text tokenizer based on libicu
- [Feature] Integrate selectors framework to multimap
- [Feature] Relax FORGED_RECIPIENTS
- [Feature] Support (almost) all html entities
- [Feature] Support adding and deletion of recipients in the milter
  block
- [Feature] Support gathering HTTP body from fragments in lua_http
- [Feature] Support multi flag in regexp and glob maps
- [Feature] Support selectors in ratelimit module
- [Feature] Support selectors in settings
- [Feature] Use khash in HTML parser
- [Feature] Use pure Lua debugm function
- [Fix] Add fail-safety for destroying sessions
- [Fix] Allow to add result-less fake DNS records
- [Fix] Another try to fix race conditions on config unload
- [Fix] Call Lua callback on DNS timeouts
- [Fix] Deprecate task:inc_dns_req as it is redundant
- [Fix] Do not allow events deletions on cleanup
- [Fix] Do not try to process skipped messages
- [Fix] Fix HTTP requests with no body
- [Fix] Fix another cleanup race condition
- [Fix] Fix bug in processing of pcre regexps
- [Fix] Fix byte array allocation in the pool
- [Fix] Fix crashes on task cleanup
- [Fix] Fix dynamic buckets in ratelimits
- [Fix] Fix endless loop when waiting for Rspamd to stop
- [Fix] Fix lua_util.str_split in case of delimiters set
- [Fix] Fix more issues with watching of async events
- [Fix] Fix stop words detection and loading logic
- [Fix] Fix various corner cases for language detection
- [Fix] Fix watchers in lua_tcp
- [Fix] Fix words decay algorithm
- [Fix] Implement watchers replacement to handle nested calls
- [Fix] Save faked code into fake dns record
- [Fix] Show the proper frame when using lua_util.debugm
- [Fix] Use fake dns records in tests
- [Fix] Use unicode replacements for HTML entities
- [Fix] fixed "cannot find dependency on symbol 1" issue when using
  replaced symbols in spamassassin rules
- [Fix] partition_id is not available in old versions of CH
- [Project] Add implicit conversion logic to selectors
- [Project] Add initial support for selectors in regexps
- [Project] Add method concept
- [Project] Further changes in unicode operations
- [Project] Implement Clickhouse migrations
- [Project] Implement implicit conversions to userdata
- [Project] Implement insert method
- [Project] Implement selectors registration for regular expressions
- [Project] Implement selectors support in re_cache
- [Project] Improve language detector: cleanup unused files,
  categorize
- [Project] Migrate CH data to a fat table
- [Project] Rework selectors logic
- [Project] Start Clickhouse utilities library
- [Project] Start unicode rework
- [Project] coroutine threaded model for API calls: thread pool
- [Rework] Move phishtank to a DNS based service
- [Rework] Rework Clickhouse plugin to use the new API
- [Rework] Rework language detector
- [Rework] Rework utf content processing in text parts
- [WebUI] Add progress bar for AJAX requests
- [WebUI] Avoid errors table reinitialization
- [WebUI] Avoid history table reinitialization
- [WebUI] Avoid throughput summary table reinitialization
- [WebUI] Destroy summary table on disconnect
- [WebUI] Fix "auth" request URL
- [WebUI] Fix disabling and hiding controls on page reload
- [WebUI] Fix maps loading from neighbours
- [WebUI] Fix symbols sorting by score
- [WebUI] Fix tables destroying
- [WebUI] Fix throughput data consolidation
- [WebUI] Fix upload buttons disabling
   2018-09-07 13:54:16 by Filip Hajny | Files touched by this commit (1)
Log message:
mail/rspamd: Clean up temp testing flags
   2018-09-07 13:19:07 by Filip Hajny | Files touched by this commit (6) | Package updated
Log message:
mail/rspamd: Update to 1.7.9.

- Fix missing config files (pkg/53577).

The most important features and fixes

- Ratelimits are reworked and now work as intended (and documented)
- Clickhouse module supports data retention policies
- Reworked C modules to avoid global contexts (simplifies leaks
  detection on reload)
- Reputation plugin now supports SPF records reputation
- WebUI code is now even more conformant to the modern JS standards
- Maps are now distributed remotely with local file safety fallback to
  allow faster maps update without waiting for a new release
- Antivirus module checks attachments only (as decoded content) in
  attachments_only mode to improve AV performance by hiding the mime
  content from them

Full list of the meaningful changes

- [CritFix] Fix caseless comparison of equal length strings
- [Feature] Add HTTP basic auth support to elastic and clickhouse
  plugins
- [Feature] Add SPF selector to reputation
- [Feature] Add support of the fallback backends for HTTP maps
- [Feature] Allow to print full mime structure when extracting mime
  data
- [Feature] Allow to split symbols in reputation plugin
- [Feature] Check attachments only on AV scanners in attachments_only
  mode
- [Feature] Disable all SSL checks if ssl_no_verify flag is set
- [Feature] Implement parsing of scoped IPv6 addresses
- [Feature] Improve rspamc counters output
- [Fix] Add sanity checks when expanding SPF macros
- [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
- [Fix] Avoid one extra byte writing
- [Fix] Deal with direct hash table
- [Fix] Detect empty text part as text, not HTML
- [Fix] Do not reduce map watch timeout for mixed http/file maps
- [Fix] Fix HTML part detection heuristic
- [Fix] Fix double free in redirectors cleanup
- [Fix] Fix legacy history handling in the controller
- [Fix] Fix messages insertion
- [Fix] Fix sending string method
- [Fix] Fix statconver command line arguments
- [Fix] Fixed argument checking for being null
- [Fix] Fixed issues reported by luacheck
- [Fix] Freeze updates queue when do actual storage update
- [Fix] HTTP map hash is per-backend and not per-map
- [Fix] Plug memory leak in fuzzy updates
- [Fix] Prefer 'MTA-Name' when producing authentication results
- [Fix] Replace bad unicode sequences instead of stopping on them
- [Fix] Set classifier version on learning
- [Project] Reworked ratelimits
- [Project] Apply topological sorting for symbols in Rspamd
- [Project] Remove global contexts from C modules
- [Project] Move performance critical hash tables to khash
- [WebUI] Avoid unused indexes
- [WebUI] Do not execute on_success callback
- [WebUI] Fix history reset for "All SERVERS" (#2346)
- [WebUI] Fix query URL for selected server
- [WebUI] Fix symbols display in legacy history,
- [WebUI] Hide symbols order selector for legacy history
- [WebUI] Refactor query functions into one
- [WebUI] Remove previously-attached event handlers
- [WebUI] Save symbols to the selected server
- [WebUI] Unify arguments of query functions
- [WebUI] Use common query functions to get graph data
- [WebUI] Use common query functions to save symbols
   2018-08-22 11:48:07 by Thomas Klausner | Files touched by this commit (3558)
Log message:
Recursive bump for perl5-5.28.0
   2018-07-31 14:27:03 by Filip Hajny | Files touched by this commit (4) | Package updated
Log message:
mail/rspamd: Update to 1.7.8

1.7.8: 12 Jul 2018
- [Feature] Add more extended statistics about fuzzy updates
- [Feature] Add more non-conformant Received headers support
- [Feature] Add preliminary function to get fuzzy hashes from text in
  Lua
- [Feature] Allow to configure AV module rejection message
- [Feature] Implement fuzzy hashes extraction in mime tool
- [Feature] Improve WHITE_ON_WHITE rule
- [Feature] Improve integer -> string conversion
- [Feature] Reuse maps in multimap module more aggressively
- [Fix] Avoid race condition in skip map as pool lifetime is not
  enough
- [Fix] Eliminate all specific C plugins pools
- [Fix] Fix DKIM check rule if DNS is unavailable
- [Fix] Fix build where ucontext is defined in ucontext.h
- [Fix] Fix crash in base url handling
- [Fix] Fix descriptors leak in sqlite3 locking code
- [Fix] Fix messages quarantine
- [Fix] Fix padded numbers printing
- [Fix] Fix race condition on maps reinit
- [Fix] Fix regexp functions when no data is passed
- [Fix] Fix specific urls extraction
- [Fix] Fix styles propagation
- [Fix] Improve resetting of the limit buckets
- [Fix] Initialize sqlite3 properly
- [Fix] Work with broken resolvers in resolv.conf
- [Project] Implement HTTP maps caching
- [Project] Refresh fuzzy hashes when matched
- [Project] Add logic to deduplicate fuzzy updates queue
- [WebUI] Add missed declarations
- [WebUI] Avoid using "undefined" property
- [WebUI] Do not accept passwords containing control characters
- [WebUI] Do not redeclare variables
- [WebUI] Enable strict mode,
- [WebUI] Fix variable assignment
- [WebUI] Initialize variables at declaration
- [WebUI] Remove duplicated path from RequireJS config
- [WebUI] Remove unused block
- [WebUI] Remove unused variable
- [WebUI] Remove unused variables
- [WebUI] Use self-explanatory notation
- [WebUI] Use type-safe equality operators

1.7.7: 02 Jul 2018
- [CritFix] Check NM part of pubkey to match it with rotating keypairs
- [CritFix] Do not overwrite PID of the main process
- [CritFix] Fix maps after reload
- [CritFix] Fix maps race conditions on reload
- [CritFix] Fix shmem leak in encrypting proxy mode
- [Feature] Add a concept of ignored symbols to avoid race conditions
- [Feature] Add ability to print bayes tokens in rspamadm mime
- [Feature] Add method to get statistical tokens in Lua API
- [Feature] Add preliminary mime stat command
- [Feature] Add rspamadm mime tool
- [Feature] Add urls extraction tool
- [Feature] Address ZeroFont exploit
- [Feature] Allow rspamadm mime to process multiple files
- [Feature] Allow to extract words in `rspamadm mime`
- [Feature] Allow to print mime part data
- [Feature] Allow to show HTML structure on extraction
- [Feature] Distinguish IP failures from connection failures
- [Feature] Improve output for mime command
- [Feature] Improve styles propagation
- [Feature] Main process crash will now cleanup all children
- [Feature] Preload file and static maps in main process
- [Feature] Print stack trace on crash
- [Feature] Process font size in HTML parser
- [Feature] Propagate content length of invisible tags
- [Feature] Read ordinary file maps in chunks to be more safe on
  rewrites
- [Feature] Support base tag in HTML
- [Feature] Support more size suffixes when parsing HTML styles
- [Feature] Support opacity style
- [Fix] Another fix for nested composites
- [Fix] Fill nm id in keypairs cache code
- [Fix] Fix colors alpha channel handling
- [Fix] Fix destruction logic
- [Fix] Fix double free
- [Fix] Fix maps preload logic
- [Fix] Fix nested composites process
- [Fix] Fix proxying of Exim connections
- [Fix] Fix reload crash
- [Fix] Fix rspamadm -l command
- [Fix] Update ed25519 signing schema
- [WebUI] Stop using "const" declaration
- [WebUI] Update RequireJS to 2.3.5

1.7.6: 15 Jun 2018
- [CritFix] Fix multiple neural networks support
- [Feature] Add decryption function to keypair command
- [Feature] Add gzip compression for HTTP requests in elastic module
- [Feature] Add gzip methods to lua util
- [Feature] Add maps based on Top Level Domains
- [Feature] Add pubkey checks for dkim_signing
- [Feature] Add support of fake DNS records
- [Feature] Add tool to encrypt files
- [Feature] Allow to add symbols using settings directly
- [Feature] Allow to match private and public keys for DKIM signatures
- [Feature] Allow to set task flags via settings
- [Feature] Allow to specify fake DNS address from the config
- [Feature] Implement signatures verification using rspamadm keypair
- [Feature] Implement signing using `rspamadm keypair`
- [Feature] Improve error reporting for DKIM key access issues
- [Feature] Provide $HOSTNAME variable in UCL
- [Feature] Rework levenshtein distance computation
- [Feature] Split message parsing and processing
- [Feature] Support ED25519 DKIM signatures
- [Feature] Support encrypted configs in UCL
- [Feature] Suppress duplicate warning on very large radix tries
- [Feature] Use OSB to combine header names
- [Fix] Cleanup maps data on shutdown
- [Fix] Fix '~' behaviour in composites
- [Fix] Fix HTTP maps updates
- [Fix] Fix NIST signatures
- [Fix] Fix RFC822 comments when processing a mime address
- [Fix] Fix double free
- [Fix] Fix dynamic settings application
- [Fix] Fix for CommuniGate Pro maillist
- [Fix] Fix keypair creation method to actually create keypair...
- [Fix] Fix matching patterns with no paths
- [Fix] Fix memory leak in parsing comments
- [Fix] Fix parsing of urls with numeric password
- [Fix] Fix plugins intialisation in configwizard
- [Fix] Fix potential crash on reload
- [Fix] Fix potential race condition for a finished HTTP connections
- [Fix] Fix race-condition leak on processes reload
- [Fix] Fix signing in openssl mode
- [Fix] Free language detector structures
- [Fix] Relax alignment requirements
- [Fix] Send DMARC reports compressed
- [Fix] Try to fix leak in dmarc module
- [Fix] Try to plug memory leak in metric exporter
- [Project] Convert rspamadm subcommands to Lua
- [WebUI] Display smtp sender/recipient in history
- [WebUI] Fix elements disabling in "Symbols" tab
- [WebUI] Limit recipients list in history column to 3
- [WebUI] Match envelope and mime addresses following in arbitrary
  order
- [WebUI] Update column header
- [WebUI] Wrap addresses in history

1.7.5: 18 May 2018
- [Conf] Add MSBL proposed return codes
- [Conf] Add additional groups for policies
- [CritFix] Do not use volatile Lua strings as UCL keys
- [Feature] Add ability to add fuzzy hashes to headers
- [Feature] Add function to extract most meaningful urls
- [Feature] Add rule to block mixed text and encrypted parts
- [Feature] Allow multiple groups for symbols
- [Feature] Allow to disable lua squeezing logic
- [Feature] Allow to get multipart children in Lua
- [Feature] Allow to insert multiple headers from milter headers
- [Feature] Allow to print scores in subject and further extensions
- [Feature] Be more error-prone in squeezed rules
- [Feature] Support multiple return codes in emails module
- [Feature] Use EMA for calculating averages
- [Feature] Use common jit cache for all regexps
- [Feature] support for CommuniGate Pro self-generated messages
- [Fix] Allow to have multiple values for headers as arrays
- [Fix] Do not open sockets for disabled workers
- [Fix] Fix AuthservId
- [Fix] Fix base64 folding in Lua API
- [Fix] Fix build on non-x86 platforms
- [Fix] Fix cached maps logic
- [Fix] Fix compatibility with old maps query logic
- [Fix] Fix crash if skip_map is used
- [Fix] Fix importing static maps from UCL
- [Fix] Fix parsing of unix sockets
- [Fix] Fix raw_mime regexp on HTML part with no text content
- [Fix] Fix tables logging
- [Fix] Fix vertical tab handling in libucl
- [Fix] Try to fix frequency counters
- [Fix] Use better sharding for ip_score
- [Fix] Use multiple results from SURBL DNS reply
- [Fix] When doing AV scan select a different server for retransmit
   2018-07-20 05:34:33 by Ryo ONODERA | Files touched by this commit (705)
Log message:
Recursive revbump from textproc/icu-62.1
   2018-05-16 18:49:44 by Filip Hajny | Files touched by this commit (5) | Package updated
Log message:
mail/rspamd: Update to 1.7.4.

* [Conf] Major stock config updates:
    - Workers are now specified in a new format worker "type" { ... }
    - Enable fuzzy worker to simplify local fuzzy storages configuration
    - Bind all workers to localhost by default to avoid security flaws
* [Conf] Make more sane fuzzy_check default settings
* [CritFix] Fix ucl escape for bad symbols
* [Feature] Add failure symbol for AV module
* [Feature] Add lazy expiration mode for new classifier schema
* [Feature] Add preliminary version of maps stats plugin
* [Feature] Allow to block fuzzy requests from specific networks
* [Feature] Allow to change `expire` of live statistics
* [Feature] Distinguish AV failure from clean result
* [Feature] Further improvements of language detector by using khash
* [Feature] Further optimization of the lang_detection
* [Feature] Implement cluster-aware bayes expiry
* [Feature] Implement exclude patterns in rspamc
* [Feature] Implement glob maps in addition to regexp maps
* [Feature] Implement map statistics function for lua API
* [Feature] Implement stop symbols for Clickhouse collection
* [Feature] Support recipients separated by commas
* [Feature] Try harder to upload scripts to the Redis server
* [Feature] Upgrade t1ha distribution
* [Feature] use_domain_sign_inbound
* [Feature] Use scores from maps if `symbols_set` is not defined
* [Fix] Add resolving version of radix map helper
* [Fix] Check URL before adding implicit prefix
* [Fix] Do not check pid/state when using PRNG
* [Fix] Fix CentOS logrotate script for systemd
* [Fix] Fix slash + dot in urls
* [Fix] Fix systemd version of the logrotate script
* [Fix] Propagate key when import implicit array from Lua
* [Fix] Strip spaces from map keys and values
* [Fix] Try to fix a specific case when processing milter protocol
* [Fix] Try to fix crash when a tcp connection cannot be set
* [Fix] Typo use_domain_local --> use_domain_sign_local
* [Fix] Various fixes to once_received module
* [Project] Store hits counters for map elements