Log message:
syslog-ng*: update to 3.38.1
snmp module untested since net-snmp doesn't currently build on NetBSD-current
Changes:
3.38.1
======
## Highlights
### Sneak peek into syslog-ng v4.0
syslog-ng v4.0 is right around the corner.
This release (v3.38.1) contains all major changes, however, they are
currently all hidden behind a feature flag.
To enable and try those features, you need to specify `@version: 4.0` at the
top of the configuration file.
You can find out more about the 4.0 changes and features
[here](https://github.com/syslog-ng/syslog-ng/blob/master/NEWS-4.0.md).
Read our practical introduction to typing at
[syslog-ng-future.blog](https://syslog-ng-future.blog/syslog-ng-4-progress-3-38-1-release/).
## Features
* `grouping-by()`: added `inject-mode(aggregate-only)`
This inject mode will drop individual messages that make up the correlation
context (`key()` groups) and would only yield the aggregate messages
(e.g. the results of the correlation).
([#3998](https://github.com/syslog-ng/syslog-ng/pull/3998))
* `add-contextual-data()`: add support for type propagation, e.g. set the
type of name-value pairs as they are created/updated to the value returned
by the template expression that we use to set the value.
The 3rd column in the CSV file (e.g. the template expression) now supports
specifying a type-hint, in the format of "type-hint(template-expr)".
Example line in the CSV database:
selector-value,name-value-pair-to-be-created,list(foo,bar,baz)
([#4051](https://github.com/syslog-ng/syslog-ng/pull/4051))
* `$(format-json)`: add --key-delimiter option to reconstruct JSON objects
using an alternative structure separator, that was created using the
key-delimiter() option of json-parser().
([#4093](https://github.com/syslog-ng/syslog-ng/pull/4093))
* `json-parser()`: add key-delimiter() option to extract JSON structure
members into name-value pairs, so that the names are flattened using the
character specified, instead of dot.
Example:
Input: {"foo":{"key":"value"}}
Using json-parser() without key-delimiter() this is extracted to:
foo.key="value"
Using json-parser(key-delimiter("~")) this is extracted to:
foo~key="value"
This feature is useful in case the JSON keys contain dots themselves, in
those cases the syslog-ng representation is ambigious.
([#4093](https://github.com/syslog-ng/syslog-ng/pull/4093))
## Bugfixes
* Fixed buffer handling of syslog and timestamp parsers
Multiple buffer out-of-bounds issues have been fixed, which could cause
hangs, high CPU usage, or other undefined behavior.
([#4110](https://github.com/syslog-ng/syslog-ng/pull/4110))
* Fixed building with LibreSSL
([#4081](https://github.com/syslog-ng/syslog-ng/pull/4081))
* `network()`: Fixed a bug, where syslog-ng halted the input instead of \
skipping a character
in case of a character conversion error.
([#4084](https://github.com/syslog-ng/syslog-ng/pull/4084))
* `redis()`: Fixed bug where using redis driver without the `batch-lines` \
option caused program crash.
([#4114](https://github.com/syslog-ng/syslog-ng/pull/4114))
* `pdbtool`: fix a SIGABRT on FreeBSD that was triggered right before pdbtool
exits. Apart from being an ugly crash that produces a core file,
functionally the tool behaved correctly and this case does not affect
syslog-ng itself.
([#4037](https://github.com/syslog-ng/syslog-ng/pull/4037))
* `regexp-parser()`: due to a change introduced in 3.37, named capture groups
are stored indirectly in the LogMessage to avoid copying of the value. In
this case the name-value pair created with the regexp is only stored as a
reference (name + length of the original value), which improves performance
and makes such name-value pairs use less memory. One omission in the
original change in 3.37 is that syslog-ng does not allow builtin values to
be stored indirectly (e.g. $MESSAGE and a few of others) and this case
causes an assertion to fail and syslog-ng to crash with a SIGABRT. This
abort is now fixed. Here's a sample config that reproduces the issue:
regexp-parser(patterns('(?<MESSAGE>.*)'));
([#4043](https://github.com/syslog-ng/syslog-ng/pull/4043))
* set-tag: fix cloning issue when string literal were used (see #4062)
([#4065](https://github.com/syslog-ng/syslog-ng/pull/4065))
* `add-contextual-data()`: fix high memory usage when using large CSV files
([#4067](https://github.com/syslog-ng/syslog-ng/pull/4067))
## Other changes
* The `json-c` library is no longer bundled in the syslog-ng source tarball
Since all known OS package managers provide json-c packages nowadays, the json-c
submodule has been removed from the source tarball.
The `--with-jsonc=internal` option of the `configure` script has been removed
accordingly, system libraries will be used instead. For special cases, the JSON
support can be disabled by specifying `--with-jsonc=no`.
([#4078](https://github.com/syslog-ng/syslog-ng/pull/4078))
* platforms: Dropped support for ubuntu-impish as it became EOL
([#4088](https://github.com/syslog-ng/syslog-ng/pull/4088))
|
Log message:
syslog-ng: update to 3.34.1
Based on package provided by Peter Czanik in PR 56406.
Changes below are for 3.34.1 only, the other 16 releases before
that are not summarized.
3.34.1
======
## Highlights
* `regexp-parser()`: new parser that can parse messages with regular expressions
* `redis()`: `workers()` and batching support
The Redis driver now support the `workers()` option, which specifies the
number of parallel workers, and the `batch-lines()` option.
This could drastically increase the throughput of the Redis destination driver.
* `mqtt()`: TLS and WebSocket Secure support
The MQTT destination now supports TLS and WSS.
## Features
* `system()` source: added support for NetBSD
* `stats`: new statistics counter
* `mqtt()`: username/password authentication
* `mqtt()`: new option `http-proxy()` for specifying HTTP/HTTPS proxy for \
WebSocket connections
([#3747](https://github.com/syslog-ng/syslog-ng/pull/3747))
* `syslog-ng-ctl`: new flag for pruning statistics
* `disk-buffer()`: added a new option to reliable disk-buffer: `qout-size()`.
## Bugfixes
* `network(), syslog()`: fixed network sources on NetBSD
* `disk-buffer()`: fixed a very rare case, where the reliable disk-buffer never \
resumed
after triggering `flow-control`.
* `disk-buffer()`: fixed a rare memory leak that occurred when `mem-buf-length()`
or `mem-buf-size()` was configured incorrectly
* `redis()`: fixed command errors that were not detected and marked as \
successful delivery
|
Log message:
Update sysutils/syslog-ng and modules to 3.8.1.
3.8.1
Library updates
- Kafka-client updated to version to 0.9.0.0
- Minimal required version of hiredis is set to 0.11.0 to avoid
possible deadlocks
- Minimal version of libdbi is set to 0.9.0
Improvements and features
- Added the long-waited disk-buffer.
- date-parser ported from incubator to upstream
- New template functions: min, max, sum, average
- Added Apache-accesslog-parser
- Added loggly destination
- Added logmatic destination
- Added template function for supporting CEF.
- cURL-based HTTP destination driver added (implemented in C
programming language)
- SELinux policy installer script now has support for Red Hat
Enterprise Linux/CentOS/ Oracle Linux 5, 6 and 7.
- Implemented add-contextual-data: With add-context-data syslog-ng
can use an external database file to append custom name-value
pairs on incoming logs (to enrich messages).
Program destination/source drivers
- Added inherit-environment configuration option to program source
and destination.
- Added keep-alive option to program destination (afprog).
Java drivers
- HTTP destination: Added the ability to use templates in both url
and message.
- ElasticSearch Destination driver: Support 2.2.x series of
ElasticSearch (transport and node mode).
MongoDB destination driver
- Replaced submodule limongo-client with mongo-c-driver.
- Additional support for previous syntax used by libmongo-client
before we started using mongo-c-driver and its URI syntax
exclusively.
Riemann destination driver
- Use cert-file() and key-file() options to match afsocket
keywords as the same way as afsocket drivers use these options.
Rewrite rules
- Introduced template options in rewrite rules.
- Added unset operation to make it possible to unset a specific
name-value pair for a logmessage.
Parsers
- kvformat: make it possible to specify name-value separator
- linux-audit-scanner: recognize a0-a9* as fields to be decoded
- csv-parser has been refactored, extended with new dialect and
prefix options.
PatternDB
- added groupingby() parser that can perform simple correlation on
log messages
- added create-context action
- Added NLSTRING parser that captures a string until the following
end-of-line
Miscellaneous features
- syslog-debun (debug bundle script for syslog-ng) has been
improved
Bugfixes
- geoip-parser: When default database if not specified, syslog-ng
crashed.
- Added support for multiple drivers with the same name in
syslog-ng config.
- Fixed aack counting logic for junctions that have branches that
modify the LogMessage.
- Fixed a potential crash for code that uses log_msg_clear() in
production (e.g. syslog-parser()).
- Fixed potential crash in reload logic
- system(): use string comparison instead of numeric in PID
rewrite
- Support encoding on glib compiled with libiconv
- pdbtool: Fix the ordering of the debug-info list in PatternDB
- afprog: Don't kill our own process group
- Handle option names with hyphen (-) characters in java scls
- dnscache performance improved
- Fixed IPv6 parser in patterndb.
- Fixed journald program name flapping
- Fixed create-dirs() inheritance in file destinations
- Fixed pass-unix-credentials() global inheritance in afunix
- Fixed create-dirs() global inheritance in afunix
- Fixed byteorder handling on bigendian systems in netmask6 filter
- Fixed flow-control issue when overflow queue is full (suspending
source by setting the window size to 0).
- Log HTTP response error codes in HTTPDestination (Java).
- Fixed potential leaks related $(sanitize) argument parsing in
basicfuncs.
- Fixed a memory leak in python debugger
- Fixed a use-after-free bug in templates.
- Fixed a memory leak around reload in netmask6 filter.
- Fixed a memory leak in LogProtoBufferedServer in case the
encoding() option is used.
- configure: don't override $enable_python while executing
pkg-config
- Fixed BSD timestamp parsing in syslog-format.
- Fixed a SIGPIPE bug in program destination.
- Error handling has been improved in AMQP destination.
- value-pairs performance improvements, memleak fixes
- Various issues around UTF-8 support fixed.
- Fixed integer overflow in numerical operations template function
- Fixed an integer underflow in afsocket.
- Fixed numerical comperisons issues around filters.
- Fixed kernel log message time drift on Linux.
- Take CRLF sequences equivalent to an LF in patterndb.
- When syslog-ng failed to insert data into Redis, it has crashed.
- When device file is set as a file destination then syslog-ng
will not try to change the permission of the device file.
- Various fixes around config file parsing:
3.7.3
Improvements
- Updated Python package requirements.
- Can now compile without MongoDB.
- Added eventlog to the list of required pkg-config packages.
- Basic FreeBSD and HP-UX support of syslog debug bundle generator
by improving POSIX shell compatibility.
- Keep the program destination open between configuration reloads.
- system-source now uses keep-timestamp(no) for Linux kernel log.
The time source used by /dev/kmsg is not updated after system
SUSPEND/RESUME.
Fixes
- Fix a SIGSEGV when a Redis command returns an error.
- Resolve deadlock in logwriter triggered by suppress()
- Mitigate possible deadlock in patterndb
- Fixed global inheritance of pass-unix-credentials() and
create-dirs().
- Certain compilers complained about an undefined symbol when
setting keep-alive(yes).
- For certain use cases, afsocket would not handle procfs read
errors due to an integer underflow.
- Enhanced Java version check and the handling of
SyslogNgInternalLogger (used by Kafka), the FATAL loglevel and
getLocationInformation().
- When a big amount of kernel log was produced in a very short
time, the syslog-ng process sometimes entered into a spin and
stop processing messages.
|