Log Message:
ntopng: updated to 6.2

ntopng 6.2 (August 2024)

Breakthroughs

Major code optimizations and reduction of locks
Huge memory footprint reduction (by more than half)
Huge improvements to SNMP polling
MITRE alerts classification
New Security report
Replay historical flows on a virtual interface
Support for ClickHouse Cloud and TLS towards ClickHouse/SQLite
Cisco QoS MIB poller
New translations: Korean, Spanish and French
Support for influxdb v.2 (with v.1 compatible buckets)
CheckMK alerts export through Event Console (syslog)
New WeChat Alert endpoint
Add more filtering capabilities to the Reports
New UI table component

Improvements

Add flow_risk and host_risk remediations.
Add VLAN rules
Add drops/flows and probes info to view interface
Add exporters limits to ntopng licenses
Add extensions for asset inventory
Add feature sorting flows by protocol
Add flows and drops ts to netflow/sflow exporters
Add info to nprobes and exporters pages
Add interface to SNMP topology map
Add localhost to ipaddress expection lists
Add mac address to the hosts page
Add missing DHCP mappings
Add mitre_info to alerts in ClickHouse
Add NAT info to ClickHouse and ECS
Add SIP status call
Add the ability to set custom alert score
Add uuid_num and unique_source_id to exporters and probes
Add various filters to Historical/Alerts pages
Add L2TP decapsulation
Add sankey to probes/exporters page
Add support for flow source
Add --disable-purge for debug purposes Added average flow throughtput in flows
Add support for Ethernet-over-IP tunnel support
Add SNMP interface and device usage page and timeseries
Add detection of interfaces going down/up when open in pcap mode
Add host name discovered with DHCP
Add blacklist charts
Add SNMP Trap support
Add QoS page to snmp
Add sankey to probes/exporters page
Add support for MAC addresses in traffic profiles
Add smcroute integration.
Add traffic profiles rules.
Add TCP flow connection state
Add SNMP interface speed configuration
Add report editor
Add support for ModBUS Scattered Holding Register Read
Add filtering ability to report page
Add JE malloc support
Improve cloud support
Implement NetFlow polling device using coroutines
Implement flow traffic account in pcap interfaces when reading traffic from a \ 
pcap interface.
Implement mitre_table_info inside database
Implement TLS swap heuristic similar to SSH
Improve host pool reload latency
Improve performance in SNMP device listing
Improve SNMP various performances and reworked interfaces page
Modify Lua allocator to avoid allocating small blocks and using ^2 blocks size \ 
to reduce heap fragmentation
Reduced memory and trhead usage Added missing HTTP server thread naming Added \ 
--limit-resources to tell ntopng to reduce memory usage (useful for systems with \ 
limited resources)
Rework periodic discovery code
Rework flow exporters lua stats
Rework interface polling with snmpbulk
Rework flow exporters host rules
Rework timeseries backend and added support to bar charts
Rework throughput calculation for flow-based interfaces: it is no longer \ 
calculated periodically but only when a new flow update is received
Update the dashboard with the editing component feature.
Add support for interfaces of different datalink with pcap (e.g. -i ethX,tunY...)

Changes

Add ntopng to group systemd-journal
Add download of journalctl logs for the last day
Add hostnames to custom queries
Add mapping between db fields and netflow
Add usage of proto.ndpi_confidence in flow_details.
Add SNMP import functionality for CSV files
Add limit on DB interface flows accoring to the flow cache
Add Major and Minor connection states
Add percentage and * as exporter device option in Flow Exporter rules + minor fixes.
Add option to backup redis (ntopng-utils-manage-config -a backup -r)
Add percentage and * as exporter device option in Flow Exporter rules + minor fixes.
Add check for avoiding crash with hosts with no MAC
Add trigger period action on shell script
Add exporters limits to ntopng licenses
Add memory boundaries checks
Add switch between normal and per minute traffic ts
Add icon in flows that indicate when the flow has swapped directions
Add flow exporter top chart
Add autosearch when opening edit application page
Add topk chart to conversations
Add support for ModBUS Scattered Holding Register Read
Add host location to flow page
Add limitations for max number of polled SNMP devices
Add check for preventing false positive for long lived connections on top of \ 
protocols that can take a while
Add SNMP usage page
Add thpt charts to historical flows
Add garbage collector calls
Add startup flush for ntopng.trace_error.alert_queue
Add Bootstrap 5 tooltip support
Add check to avoid memory issues (heap overflow) during DHCP packet dissection
Add check for avoid setting the interface in non-blocking mode when used with \ 
pcap files
Change the severity of the old blacklisted flow to critical
Change the labels from 'Downlink Usage' and 'Uplink Usage' to 'In Usage' and \ 
'Out Usage'
Changed score level for various Alerts.
Cleaned up flow throughout calculation
Disabled flow swap for UDP flows that might lead to false positives
Disable download image button on Safari.
Enable the editing of blacklist URL.
Enable interface name search.
Enable search in the SNMP interfaces page.
Make sort/delete persistent. Compute component_id on server side.
nmap command path is now computed at runtime
Packet padding is no longer accounted in flow traffic
Prevents non-admin users to pause interfaces
Report templates can now be defined in multiple paths
Reduced table retention
Remove additional http header
Remove sflowdev timeseries and unified to flowdev
Remove outdated unahandled flows that was casing fiscrepancies in flow accounting
Remove useless work when shutting down
Run nmap setcap only when we're outside a container
Update doc with all the latest features.

nEdge

Add option to enable external captive portal auth
Add Keep Src Address flag.
Add MAC and IP Address to radius interim-update
Add new fields to radius accounting
Add code to delete expired flows in ntopng still present in conntrack
Add check for offloaded flows with uncompleted protocol detection that have \ 
observed too many packets (updated via conntrack)
Implement remote radius authentication for local users (toggle)
Handle broadcast forwarding
Optimized std::map to reduce memory usage
Remove keep_src_address
Remove the hardcoded testing value for traffic_quota_ratio.
Remove alerts no longer necessary as they have been replaced by local traffic rules
Fix broadcast forwarding
Fix Daily Traffic Quota and Daily Time Quota column style.
Fix incorrect delta calculation
Fix repeater config modal reset
Fix the apply button in repeaters modal.
Fix progress bar.
Fix editing on repeater-config modal by removing unnecessary variable.
Fix the enable_nat and enable_iface toggles
Fix the alignment of column_key icons on the host_details/flows page.
Fix Daily Traffic Quota and Daily Time Quota column style.
Fix the alignment of column_info icons.

Fixes

Fix top visited websites leak (growing undefinitely) and cpu load (sorting on \ 
every decoded site)
Fix aggregated live flows exporter filter.
Fix L7 Protocol usage & empty table statement using the view interface in \ 
Server Ports Analysis page
Fix pcap extraction for unprivileged users
Fix chown group
Fix TCP Flow Reset check.
Fix TCPFlowReset check.
Fix free on uninitialized pointers
Fix the creation of the all_alerts_view in the ClickHouse cluster SQL script.
Fix the partition parameter in the ClickHouse cluster SQL database schema.
Fix a bug related to removing CVEs when a scan is in progress and make minor \ 
optimizations.
Fix the formatting of 0 percentage.
Fix access to released memory in UT hash iteration
Fix navigation from server ports analysis chart view to table view.
Fix where on aggregated queries (interface id was ignored)
Fix invalid packet count with fragemented traffic
Fix info field cut after 256 characters
Fix crash and memory leak introduced
Fix missing fields in TLS alerts
Fix invalid application protocol accounting in network interfaces due to partial \ 
nDPi detection
Fix pcap download
Fix bug in UDP scan
Fix counter polling
Fix SSH flow swap heuristic
Fix segmentation fault on Stratosphere lab blacklist loading
Fix pcap polling on macOS and FreeBSD Fixes handling of interface pause (idle) \ 
on pcap interfaces
Fix SQL injection description
Fix copy not working on alert description
Fix string info cut due to buffer size
Fix invalid host rename when using HTTP proxies
Fix reset counters does not reset sent/rcvd bytes/packets
Fix attempt to index nil value
Fix some performance issues in the new flow page
Fix timeseries queries not working with serialize by mac
Fix incorrect check on TOS
Fix thpt historical flow chart
Fix historical flow charts
Fix duplicated entries in radius
Fix service map learning not reset at startup
Fix circular dependencies
Fix tooltip not working
Fix active monitoring alert discarded with no pool selected
Fix incorrect hosts number
Fix issue with host pools assignment
Fix remote access alert not triggered
Fix SNMP topology map and added to all snmp devices
Fix SNMP v3 import not working
Fix topology map not correctly working
Fix various translation to It, JP an other languages.
Fix various issue with application reloading
Fix various issues in SNMP Chart
Fix bytes per minute SNMP Serie not added
Fix shell script execution on alerts engaged
Fix crash when sorting hosts in low memory conditions
Fix domain name extraction from the info column.
Fix colors in dygraph plotters
Fix throughput values in local traffic rules.
Fix wrong source type in exporters report
Fix emergency recipient toast not configured
Fix location not correctly set in case of aggregation
Fix unknown filter applied even when not filtered
Fix schema id switch in influx
Fix Heap-buffer-overflow in IEC104
Fix influxdb top stats
Fix timeseries charts timezone and removed no more used files
Fix FreeBSD packaging issues with VulScan
Fix incorrect total calculation
Fix various issues on the exporter pages
Fix historical aggregated flow issue with timestamp lower than the last day
Fix various lua memory issues
CentOS 7 fixes
Workaround for a memory leak on windows for a bug on the pthread library
Various OT fixes