2018-03-04 16:52:21 by Benny Siegert | Files touched by this commit (95) | |
Log message:
Revbump all Go packages after Go 1.10 update.
|
2018-02-27 13:54:02 by Filip Hajny | Files touched by this commit (2) |
Log message:
sysutils/beats: Update to 6.2.2.
==== Bugfixes
Affecting all Beats
- Add logging when monitoring cannot connect to Elasticsearch.
- Fix infinite loop when event unmarshal fails in Kubernetes
pod watcher.
Filebeat
- Fix a conversion issue for time related fields in the Logstash
module for the slowlog fileset.
|
2018-02-07 17:31:34 by Filip Hajny | Files touched by this commit (2) | |
Log message:
sysutils/beats: Update to 6.2.0.
==== Breaking changes
Affecting all Beats
- The log format may differ due to logging library changes.
- The default value for pipelining is reduced to 2 to avoid high
memory in the Logstash beats input.
Auditbeat
- Split the audit.kernel and audit.file metricsets into their own
modules
named auditd and file_integrity, respectively. This change requires
existing users to update their config.
- Renamed file_integrity module fields.
- Renamed auditd module fields.
Metricbeat
- Rename `golang.heap.system.optained` field to
`golang.heap.system.obtained`.
- De dot keys in jolokia/jmx metricset to prevent collisions.
==== Bugfixes
Auditbeat
- Fixed an issue where the proctitle value was being truncated.
- Fixed an issue where values were incorrectly interpretted as hex
data.
- Fixed parsing of the `key` value when multiple keys are present.
- Fix possible resource leak if file_integrity module is used with
config
reloading on Windows or Linux.
Filebeat
- Fix variable name for `convert_timezone` in the system module.
Metricbeat
- Fix error `datastore '*' not found` in Vsphere module.
- Fix error `NotAuthenticated` in Vsphere module.
- Fix mongodb session consistency mode to allow command execution on
secondary nodes.
- Fix kubernetes `state_pod` `status.phase` so that the active phase
is returned instead of `unknown`.
- Fix error collecting network_names in Vsphere module.
- Fix process cgroup memory metrics for memsw, kmem, and kmem_tcp.
- Fix kafka OffsetFetch request missing topic and partition
parameters.
Packetbeat
- Fix mysql SQL parser to trim `\r` from Windows Server
`SELECT\r\n\t1`.
==== Added
Affecting all Beats
- Adding a local keystore to allow user to obfuscate password
- Add autodiscover for kubernetes.
- Add Beats metrics reporting to Xpack.
- Update the command line library cobra and add support for zsh
completion
- Update to Golang 1.9.2
- Moved `ip_port` indexer for `add_kubernetes_metadata` to all beats.
- `ip_port` indexer now index both IP and IP:port pairs.
- Add the ability to write structured logs.
- Use structured logging for the metrics that are periodically logged
via the
`logging.metrics` feature.
- Improve Elasticsearch output metrics to count number of dropped and
duplicate (if event ID is given) events.
- Add the abilility for the add_docker_metadata process to enrich
based on process ID.
- The `add_docker_metadata` and `add_kubernetes_metadata` processors
are now GA, instead of Beta.
- Update go-ucfg library to support top level key reference and cyclic
key reference for the
keystore
Auditbeat
- Auditbeat is marked as GA, no longer Beta.
- Add support for BLAKE2b hash algorithms to the file integrity
module.
- Add support for recursive file watches.
Filebeat
- Add Osquery module.
- Add stream filtering when using `docker` prospector.
Metricbeat
- Add ceph osd_df to metricbeat
- Add field network_names of hosts and virtual machines.
- Add experimental system/raid metricset.
- Add a dashboard for the Nginx module.
- Add experimental mongodb/collstats metricset.
- Update the MySQL dashboard to use the Time Series Visual Builder.
- Add experimental uwsgi module.
- Docker and Kubernetes modules are now GA, instead of Beta.
- Support haproxy stats gathering using http (additionaly to tcp
socket).
- Support to optionally 'de dot' keys in http/json metricset to
prevent collisions.
Packetbeat
- Configure good defaults for `add_kubernetes_metadata`.
|
2018-01-22 20:54:56 by Filip Hajny | Files touched by this commit (2) |
Log message:
Update sysutils/beats to 6.1.2.
6.1.2
Auditbeat
- Add an error check to the file integrity scanner to prevent a panic
when there is an error reading file info via lstat.
Filebeat
- Switch to docker prospector in sample manifests for Kubernetes
deployment
|
2017-12-18 15:37:02 by Filip Hajny | Files touched by this commit (2) |
Log message:
Update sysutils/beats to 6.1.0.
=== Beats version 6.1.0
==== Breaking changes
Auditbeat
- Changed `audit.file.path` to be a multi-field so that path is
searchable.
Metricbeat
- Rename `heap_init` field to `heap.init` in the Elasticsearch module.
- Rename `http.response.status_code` field to `http.response.code` in
the HTTP module.
==== Bugfixes
Affecting all Beats
- Remove ID() from Runner interface
- Correctly send configured `Host` header to the remote server.
- Change add_kubernetes_metadata to attempt detection of namespace.
- Avoid double slash when join url and path
- Fix console color output for Windows.
- Fix logstash output debug message.
- Fix isolation of modules when merging local and global field
settings.
Filebeat
- Add support for adding string tags
- Fix race condition when limiting the number of harvesters running in
parallel
- Fix relative paths in the prospector definitions.
- Fix `recursive_globe.enabled` option.
Metricbeat
- Change field type of http header from nested to object
- Fix the fetching of process information when some data is missing
under MacOS X.
- Change `MySQL active connections` visualization title to `MySQL
total connections`.
- Fix `ProcState` on Linux and FreeBSD when process names contain
parentheses.
- Fix incorrect `Mem.Used` calculation under linux.
Packetbeat
- Fix http status phrase parsing not allow spaces.
- Fix http parse to allow to parse get request with space in the URI.
Winlogbeat
- Fix the registry file. It was not correctly storing event log names,
and upon restart it would begin reading at the start of each event log.
==== Added
Affecting all Beats
- Support dashboard loading without Elasticseach
- Changed the hashbang used in the beat helper script from `/bin/bash`
to `/usr/bin/env bash`.
- Changed beat helper script to use `exec` when running the beat.
- Fix reloader error message to only print on actual error
- Add support for enabling TLS renegotiation.
- Add Azure VM support for add_cloud_metadata processor
- Add `output.file.permission` config option.
- Refactor add_kubernetes_metadata to support autodiscovery
- Improve custom flag handling and CLI flags usage message.
- Add number_of_routing_shards config set to 30
- Set log level for kafka output.
- Move TCP UDP start up into `server.Start()`
- Update to Golang 1.9.2
Auditbeat
- Add support for SHA3 hash algorithms to the file integrity module.
- Add dashboards for Linux audit framework events (overview,
executions, sockets).
Filebeat
- Add PostgreSQL module with slowlog support.
- Add Kafka log module.
- Add support for `/var/log/containers/` log path in
`add_kubernetes_metadata` processor.
- Remove error log from runnerfactory as error is returned by API.
- Add experimental Docker `json-file` prospector .
- Add experimental Docker autodiscover functionality.
- Add option to convert the timestamps to UTC in the system module.
- Add Logstash module support for main log and the slow log, support
the plain text or structured JSON format
Metricbeat
- Add graphite protocol metricbeat module.
- Add http server metricset to support push metrics via http.
- Make config object public for graphite and http server
- Add system uptime metricset.
- Add experimental `queue` metricset to RabbitMQ module.
- Add additional php-fpm pool status kpis for Metricbeat module
- Add etcd module.
- Add ip address of docker containers to event.
- Add ceph osd tree information to Metricbeat
- Add basic Logstash module.
- Add dashboard for Windows service metricset.
- Add experimental Docker autodiscover functionality.
- Add Windows service metricset in the windows module.
- Update gosigar to v0.6.0.
Packetbeat
- Add support for decoding the TLS envelopes.
=== Beats version 6.0.1
==== Bugfixes
Affecting all Beats
- Fix documentation links in README.md files.
- Fix `add_docker_metadata` dropping some containers.
Heartbeat
- Fix the "HTTP up status" visualization.
Metricbeat
- Fix map overwrite in docker diskio module.
- Fix connection leak in mongodb module.
- Fix the include top N processes feature for cases where there are
fewer processes than N.
|
2017-12-13 13:44:54 by Filip Hajny | Files touched by this commit (2) |
Log message:
Update sysutils/beats to 6.0.1.
Affecting all Beats
- Fix documentation links in README.md files.
- Fix add_docker_metadata dropping some containers.
Heartbeat
- Fix the "HTTP up status" visualization.
Metricbeat
- Fix map overwrite in docker diskio module.
- Fix connection leak in mongodb module.
- Fix the include top N processes feature for cases where there
are fewer processes than N.
|
2017-11-20 16:55:16 by Filip Hajny | Files touched by this commit (3) | |
Log message:
Update sysutils/beats to 6.0.0.
=== Beats version 6.0.0
==== Breaking changes
Affecting all Beats
- The log directory (`path.log`) for Windows services is now set to
`C:\ProgramData\[beatname]\logs`.
- The _all field is disabled in Elasticsearch 6.0.
- Fail if removed setting output.X.flush_interval is explicitly
configured.
- Rename the `/usr/bin/beatname.sh` script (e.g. `metricbeat.sh`) to
`/usr/bin/beatname`.
- Beat does not start if elasticsearch index pattern was modified but
not the template name and pattern.
- Fail if removed setting output.X.flush_interval is explicitly
configured.
- Rename `kubernetes` processor to `add_kubernetes_metadata`.
- Rename `.full.yml` config files to `*.reference.yml`.
- The `scripts/import_dashboards` is removed from packages. Use the
`setup` command instead.
- Change format of the saved kibana dashboards to have a single JSON
file for each dashboard
- Rename `configtest` command to `test config`.
- Remove setting `queue_size` and `bulk_queue_size`.
- Remove setting `dashboard.snapshot` and `dashboard.snapshot_url`.
- Beats can no longer be launched from Windows Explorer (GUI), command
line is required.
Filebeat
- Rename `input_type` field to `prospector.type`
- The `@metadata.type` field, added by the Logstash output, is now
hardcoded to `doc` and will be removed in future versions.
Metricbeat
- Change all `system.cpu..pct` metrics to be scaled by the number of
CPU cores.
- Remove filters setting from metricbeat modules.
- Added `type` field to filesystem metrics.
Heartbeat
- Renamed the heartbeat RPM/DEB name to `heartbeat-elastic`.
Packetbeat
- Remove not-working `runoptions.uid` and `runoptions.gid` options in
Packetbeat.
- Remove the already unsupported `pf_ring` sniffer option.
Auditbeat
- Changed file metricset config to make `file.paths` a list instead of
a dictionary.
==== Bugfixes
Affecting all Beats
- Fix data race accessing watched containers.
- Do not require template if index change and template disabled
- Fix missing ACK in redis output.
- Fix the `/usr/bin/beatname` script to accept `-d ""` as a parameter.
- Combine `fields.yml` properties when they are defined in different
sources.
- Keep Docker & Kubernetes pod metadata after container dies while
they are needed by processors.
- Fix `fields.yml` lookup when using `export template` with a custom
`path.config` param.
- Remove runner creation from every reload check
- Fix add_kubernetes_metadata matcher registry lookup.
- Register kubernetes `field_format` matcher and remove logger in
`Encode` API
- Fix go plugins not loaded when beat starts
- Add support for `initContainers` in `add_kubernetes_metadata`
processor.
- Eliminate deprecated _default_ mapping in 6.x
- Fix pod name indexer to use both namespace, pod name to frame index
key
- Don't stop with error loading the ES template if the ES output is
not enabled.
- Fix race condition in internal logging rotator.
- Normalize all times to UTC to ensure proper index naming.
- Fix issue with loading dashboards to ES 6.0 when .kibana index did
not already exist.
- Fix importing the dashboards when the limit for max open files is
too low.
- Fix configuration documentation for kubernetes processor
- Fix misspelling in `add_locale` configuration option for
abbreviation.
Filebeat
- Fix machine learning jobs setup for dynamic modules.
- Fix default paths for redis 4.0.1 logs on macOS
- Fix Filebeat not starting if command line and modules configs are
used together.
- Fix double `@timestamp` field when JSON decoding was used.
- Fix issue where the `fileset.module` could have the wrong value.
- Fix race condition on harvester stopping with reloading enabled.
- Fix recursive glob config parsing and resolution across restarts.
- Allow string characters in user agent patch version (NGINX and
Apache)
- Fix grok pattern in filebeat module system/auth without hostname.
Winlogbeat
- Removed validation of top-level config keys.
Metricbeat
- Use `beat.name` instead of `beat.hostname` in the Host Overview
dashboard.
- Fix the loading of 5.x dashboards.
- Fix a memory allocation issue where more memory was allocated than
needed in the windows-perfmon metricset.
- Don't start metricbeat if external modules config is wrong and
reload is disabled
- The MongoDB module now connects on each fetch, to avoid stopping the
whole Metricbeat instance if MongoDB is not up when starting.
- Fix kubernetes events module to be able to index time fields
properly.
- Fixed `cmd_set` and `cmd_get` being mixed in the Memcache module.
- Added missing mongodb configuration file to the `modules.d` folder.
- Fix wrong MySQL CRUD queries timelion visualization
- Add new metrics to CPU metricsset
- Fix issue affecting Windows services timing out at startup.
- Fix incorrect docker.diskio.total metric calculation.
- Vsphere module: used memory field corrected.
- Set correct format for percent fields in memory module.
- Fix a debug statement that said a module wrapper had stopped when it
hadn't.
- Use MemAvailable value from /proc/meminfo on Linux 3.14.
- Fix panic when events were dropped by filters.
- Add filtering to system filesystem metricset to remove relative
mountpoints like those from Linux network namespaces.
- Remove unnecessary print statement in schema apis.
- Fix type of field `haproxy.stat.check.health.last`.
Heartbeat
- Fix monitor.name being empty by default.
- Fix wrong event timestamps.
Packetbeat
- Fix missing length check in the PostgreSQL module.
- Fix panic in ACK handler if event is dropped on blocked queue
- Update flow timestamp on each packet being received.
- Enabled /proc/net/tcp6 scanning and fixed ip v6 parsing.
- Enable memcache filtering only if a port is specified in the config
file.
Auditbeat
- Fix `file.max_file_size` config option for the audit file metricset.
==== Added
Affecting all Beats
- Enable flush timeout by default.
- Add @metadata.version to events send to Logstash.
- Add setting to enable/disable the slow start in logstash output.
- Update init scripts to use the `test config` subcommand instead of
the deprecated `-configtest` flag.
- Get by default the credentials for connecting to Kibana from the
Elasticsearch output configuration.
- Added `cloud.id` and `cloud.auth` settings, for simplifying using
Beats with the Elastic Cloud.
- Add lz4 compression support to kafka output.
- Add newer kafka versions to kafka output.
- Configure the index name when loading the dashboards and the index
pattern.
- New cli subcommands interface.
- Allow source path matching in `add_docker_metadata` processor.
- Add support for analyzers and multifields in fields.yml.
- Add support for JSON logging.
- Add `test output` command, to test Elasticsearch and Logstash output
settings.
- Introduce configurable event queue settings: queue.mem.events,
queue.mem.flush.min_events and queue.mem.flush.timeout.
- Enable pipelining in Logstash output by default.
- Added 'result' field to Elasticsearch QueryResult struct for
compatibility with 6.x Index and Delete API responses.
- The sample dashboards are now included in the Beats packages.
- Add `pattern` option to be used in the fields.yml to specify the
pattern for a number field.
- Upgraded to Golang 1.8.3.
- Added the possibility to set Elasticsearch mapping template settings
from the Beat configuration file.
- Add a variable to the SysV init scripts to make it easier to change
the user.
- Add the option to write the generated Elasticsearch mapping template
into a file.
- Add `instance_name` in GCE add_cloud_metadata processor.
- Add `add_docker_metadata` processor.
- Add `logging.files` `permissions` option.
Filebeat
- Add Kubernetes manifests to deploy Filebeat.
- Changed the number of shards in the default configuration to 3.
- Don't start filebeat if external modules/prospectors config is wrong
and reload is disabled
- Add `filebeat.registry_flush` setting, to delay the registry
updates.
- Add experimental Redis module.
- Nginx module: use the first not-private IP address as the remote_ip.
- Load Ingest Node pipelines when the Elasticsearch connection is
established, instead of only once at startup.
- Add support for loading Xpack Machine Learning configurations from
the modules, and added sample configurations for the Nginx module.
- Add udp prospector type.
- Enabled Cgo which means libc is dynamically compiled.
- Add Beta module config reloading mechanism
- Remove spooler and publisher components and settings.
- Added ability to sort harvested files.
- Add experimental Redis slow log prospector type.
Winlogbeat
- Changed the number of shards in the default configuration to 3.
- Add the ability to use LevelRaw if Level isn't populated in the
event XML.
Metricbeat
- Add Kubernetes manifests to deploy Metricbeat.
- Auto-select a hostname (based on the host on which the Beat is
running) in the Host Overview dashboard.
- Add `filesystem.ignore_types` to system module for ignoring
filesystem types.
- Add support to exclude labels from kubernetes pod metadata.
- Add random startup delay to each metricset to avoid the thundering
herd problem.
- Add the ability to configure audit rules to the kernel module.
- Add the ability to configure kernel's audit failure mode.
- Add experimental Aerospike module.
- Vsphere module: collect custom fields from virtual machines.
- Add `test modules` command, to test modules expected output.
- Add `processors` setting to metricbeat modules.
- Support `npipe` protocol (Windows) in Docker module.
- Add macOS implementation of the system diskio metricset.
- Add process_summary metricset that records high level metrics about
processes.
- Add `kube-state-metrics` based metrics to `kubernetes` module
- Add debug logging to Jolokia JMX metricset.
- Add events metricset for kubernetes metricbeat module
- Change Metricbeat default configuration file to be better optimized
for most users.
- Add experimental RabbitMQ module.
- Add Kibana dashboard for the Kubernetes modules.
Heartbeat
- Changed the number of shards in the default configuration to 1.
- Enabled Cgo which means libc is dynamically compiled.
Packetbeat
- Changed the number of shards in the default configuration to 3.
Auditbeat
- Changed the number of shards in the default configuration to 3.
- Add support for receiving audit events using a multicast socket.
- Added `file.hash_types` config option for controlling the hash
types.
- Added the ability to specify byte unit suffixes to
`file.max_file_size`.
- Add file integrity metricset to the audit module.
==== Deprecated
Affecting all Beats
- The `@metadata.type` field, added by the Logstash output, is
deprecated, hardcoded to `doc` and will be removed in future
versions.
Filebeat
- The `filebeat.config_dir` option is deprecated. Use
`filebeat.config.prospector` options instead.
- Deprecate `input_type` prospector config. Use `type` config option
instead.
=== Beats version 5.6.4
- Fix race condition in internal logging rotator.
- Add support for enabling TLS renegotiation.
- Add setting to enable/disable the slow start in logstash output.
- Packetbeat: Fix missing length check in the PostgreSQL module.
|
2017-11-02 16:30:50 by Filip Hajny | Files touched by this commit (1) |
Log message:
sysutils/beats: Provide a better hint to where libpcap is. May fix
the build in some situations.
|
2017-10-17 13:31:36 by Filip Hajny | Files touched by this commit (2) |
Log message:
Update sysutils/beats to 5.6.3.
No changes - sync version with databases/elasticsearch.
|
2017-09-27 14:40:44 by Filip Hajny | Files touched by this commit (2) |
Log message:
Update sysutils/beats to 5.6.2.
=== Beats version 5.6.0
==== Breaking changes
Affecting all Beats
- The _all.norms setting in the Elasticsearch template is no longer
disabled.
==== Bugfixes
Filebeat
- Fix issue where the `fileset.module` could have the wrong value.
Packetbeat
- Update flow timestamp on each packet being received.
Metricbeat
- Fix a debug statement that said a module wrapper had stopped when it
hadn't.
- Use MemAvailable value from /proc/meminfo on Linux 3.14.
- Fix panic when events were dropped by filters.
==== Added
Affecting all Beats
- Add option to the import_dashboards script to load the dashboards via
Kibana API.
Filebeat
- Add support for loading Xpack Machine Learning configurations from the
modules, and added sample configurations for the Nginx module.
- Add ability to parse nginx logs exposing the X-Forwarded-For header
instead of the remote address.
Metricbeat
- Add `filesystem.ignore_types` to system module for ignoring filesystem
types.
==== Deprecated
Affecting all Beats
- Loading more than one output is deprecated and will be removed in 6.0.
|