Log Message:
libtrace: updated to 4.0.10

libtrace 4.0.10:

New features
Added new API function (trace_get_errstr()) which will map a given libtrace \ 
error number to a printable error message.

Bug fixes
Fixed SIOCGSTAMP undeclared error when building against newer Linux kernels.
Fixed corruption bug when running multiple concurrent etsilive: input processes.

Improvements
Bumped TTL of nDAG multicast group joining messages to 4, so they can be routed \ 
outside of the immediate subnet (i.e. through the host when libtrace is run \ 
within a container).

libtrace 4.0.9:

Bug fixes
Fixed traceanon build error on systems that did not have libcrypto installed.
Fixed DPDK detection in configure when the DPDK package was installed on either \ 
Debian buster and Ubuntu disco.
Updated DPDK code to compile against more recent DPDK releases, such as 18.11.
Fixed segmentation fault when failing to open a DAG device.
Fixed issue where a pcapng packet that does not match any of our known data \ 
types ends up having an uninitialised data type.
Fix some compilation errors when using DPDK on FreeBSD (may still be linking \ 
problems if you have built DPDK using the ports tree, though).
Fix infinite decoding loop if libpacketdump sees an SCTP option with a length of \ 
zero.

libtrace 4.0.8:

New features
traceanon is now capable of anonymising RADIUS traffic within packet traces. The \ 
anonymisation will obfuscate the data within AVPs that can be considered \ 
'sensitive', including user names, IP addresses and password hashes. Counter \ 
fields such as byte and packet counters are by default untouched, but traceanon \ 
can be configured to anonymise those as well if required.
traceanon can now be configured using a YAML configuration file, instead of CLI \ 
arguments. This change is due to the increased number of configuration options \ 
introduced by the RADIUS anonymisation feature. Instructions on how to write a \ 
configuration file can be found on the traceanon manpage, as well as on this \ 
wiki page.

Bug fixes
Fixed bug where ndag multicast sockets would bind to all addresses on an \ 
interface, rather than just the address of the multicast group.
Fixed segfault that can occur when pausing a trace input that has not been able \ 
to create its per packet processing threads for some reason.

libtrace 4.0.7:

New features
Added new API functions for exploring meta-data that is either attached to a \ 
specific packet or included in a trace as separate records (e.g. ERF provenance \ 
or pcap-ng meta-data). Many meta-data fields have a specific accessor function \ 
that can be called directly (e.g. trace_get_interface_fcslen()). You can also \ 
use trace_get_section() to get an array containing all meta-data items within a \ 
particular section, which will allow you to get access to any fields for which \ 
we have not implemented direct access functions.
Added new API functions to instant decoding all of the post-layer 2, pre-layer 3 \ 
headers in a packet so you can now easily explore any / all VLAN, MPLS, etc. \ 
headers in a packet without having to effectively re-implement \ 
trace_get_layer3() in your own code. See trace_get_layer2_headers() for more \ 
details.
Added support for both reading and writing TZSP sniffing streams.

Bug fixes
Fixed uninitialised bytes in message structure sent via trace_post_reporter -- \ 
thanks to Mark Weiman for fixing this.
Fixed build errors caused by attempting to #include pcap-int.h.
Fixed bug where a corrupt ERF record could cause a libtrace program to become \ 
un-haltable.
Fixed bug in error tracking when creating a fanout socket for the ring and int \ 
formats.
Fixed potential segfault when halting a libtrace program that was reading from a \ 
ring: input.
Fixed uninitialised mutex when copying a packet.

Improvements
Improved parallel performance by skipping some needless per-packet sanity checks.

libtrace 4.0.6:

New features
Added write support for pcapng: format.

Bug fixes
Fixed incorrect counting of input sources when using etsilive: for reading packets.
Fixed bug where trace_event() API was ignoring all received packets.
Fixed bug where tracereplay would segfault.
Fixed packet corruption bug in tracesplit when using the "jump to IP \ 
header" mode.
Fixed bug where we could end up trying to close a NULL pcap output.
Fixed build problems when building with dpdk enabled.
Fixed bug that was causing recvmmsg detection to fail at configure time.
Fixed bug where ETSI live sockets created later on are uninitialised.
Fixed memory leak when using BPF filters with ring: inputs.
Fixed a variety of potential crashes and buffer overflows revealed by Perry's \ 
fuzzing experiments

Improvements
Replaced numerous internal assertion checks with error return values instead, \ 
i.e. instead of a libtrace function assert failing and crashing your program, it \ 
will now return an error (or set the error status on the trace) and allow the \ 
user to deal with the error however they want.
Similarly, tidied up some of the error messaging to be clearer about what has \ 
gone wrong and added a variety of new error types.
Improved ring: read performance when used with the parallel API by reading \ 
multiple packets per function call.
Added option to report numbers of dropped and missing packets (cumulative) in \ 
tracertstats.
Ported traceends and tracetopends to use the parallel API.
Improvements to ndag packet reading performance.
libtrace 4.0.5:

Bug fixes
Fixed bug where clients would obtain an exclusive lock on an nDAG multicast group.
Fixed bogus payload length calculations on outgoing packets when the IP length \ 
field is filled in by the NIC prior to sending.
Fixed bug where any non-negative return value other than zero from a pstart \ 
callback would be treated as an error.
Fixed bug where packets that have been invalidated by a call to trace_ppause() \ 
are still treated as valid.
Fixed bug where parallel ring: inputs would assert fail when the input is halted.
Reduced likelihood of dropping packets on an ndag: input during initialisation phase.
Fixed build error for DPDK format due to missing header file.
Fix race condition that can occur when two threads attempt to call \ 
trace_create() or trace_create_dead() at the same time.

Improvements
Improved etsilive: decoding performance.
Avoid invalidating packets received via ring: following a pause until the trace \ 
is restarted.
Added caching for packet framing length.

libtrace 4.0.4:

NOTE: libwandio 4.0.0 is required to build this version of libtrace. Older \ 
versions of libwandio will not work.

New Features
Added trace_increment_packet_refcount() and trace_decrement_packet_refcount() \ 
functions to the parallel API. These functions can be used to track references \ 
to a libtrace packet across multiple threads, so that a shared packet can be \ 
released once all threads have finished with it. Packets where the reference \ 
count is decremented to zero are automatically released.
Add new built-in data structure: simple circular buffer.
Added new format for receiving and decoding packets encapsulated in the ETSI \ 
Lawful Intercept streaming format (requires libwandder).
Added support for decoding ETSI Lawful Intercept records to libpacketdump \ 
(requires libwandder).
Add trace_flush_output() API function to force a libtrace output to dump any \ 
buffered output to disk. Flushed files may still not be properly readable \ 
afterwards, but this will help give the appearance that the output file is \ 
growing in situations where the output rate is slow.

Bug Fixes
Fixed bug in ndag: which was causing the stream to be treated as inactive when \ 
there are buffered records available.
Fixed build errors caused by pthread_attr_setaffinity_np() being a glibc-only \ 
extension -- thanks to Tim Dawson for contributing this patch.
Fixed bug where uninitialised internal message queues were being destroyed -- \ 
thanks to EaseTheWorld for reporting this.
Fixed lack of error being returned when a user tries to change the number of \ 
perpkt threads on a paused trace.
Fixed problems in tracereplay caused by trying to replay packets with no \ 
contents (e.g. meta-data records).
Fix bug where packets received via a ring: interface were being released twice.
Fix rounding error in trace_event_trace() which would cause sleep intervals to \ 
be rounded down to zero.
Fix rounding error in pcapng_get_timespec() which would cause all packet \ 
timestamps to be truncated to the previous second.
Fix deadlock when calling trace_pstop() on a trace that has already been stopped.
Fix bug where two concurrent ring: inputs would be assigned to the same fanout \ 
group, causing the second input to fail to start.
Fixed errors in manpages for tracesplit, traceanon and tracemerge (regarding the \ 
correct names for the various compression methods) -- thanks to Hendrik \ 
Leppelsack for reporting this problem.
Fixed some uninitialised memory errors when valgrinding a parallel libtrace program.
Fixed potential buffer overruns in pcapng reading code.
Fixed bug that was preventing trace_pstop() from working as intended on pcapint:.
Fixed potential build errors relating to the absence (or not) of strndup(), \ 
strncasecmp() and snprintf().

Improvements
Updated DPDK support to be able to compile against DPDK 18.02.1
tracereplay is now able to reduce inter-packet gaps in the replayed stream by a \ 
user-specified speedup factor, so the trace can be replayed faster but with the \ 
same relative gaps between packets.
libtrace 4.0.3:

New Features
Added new API function: trace_get_perpkt_thread_id(), which allows callers to \ 
get the ID number of the packet processing thread that they are currently in.
Message Queue data structure API is now publicly exported.
Toeplitz Hash API is now publicly exported.
Added dpdkndag: capture format, which allows a libtrace program to capture and \ 
parse nDAG records that are intercepted on a DPDK-capable interface.
Moved trace_prepare_packet() into the external API.

Bug Fixes
Fixed bug where captures from GRE tunnel interfaces would fail due to unknown \ 
ARPHRD type.
Fixed problems when reading ERF provenance records from a DAG or ERF source -- \ 
thanks to Anthony Coddington at Endace for resolving this issue.
Fixed bug where nDAG packets could be corrupted if all of the receive buffers \ 
are full.
Fixed assertion failure when libwandio fails but does not set errno to a useful \ 
value -- thanks to Robert Zeh for patching this bug.
Fixed minor memory leak when a user does not provide a hash function when \ 
calling trace_set_hasher().
Fixed missing pthread_spinlock.h error that occurred whenever a user tried to \ 
include message_queue.h or ring_buffer.h.
Fixed bug where some key data structures were not initialised when doing DPDK output.
Fixed bug where DPDK memory buffers were too small to hold a full packet, \ 
causing payload to be truncated.
Fixed uninitialised write index in format_ndag, which could cause some nDAG \ 
captures to appear corrupt.

Improvements
Updated dag: format to use the 64 bit API -- this means that we can support \ 
capture on DAG streams that have large amounts of memory attached.
Improved nDAG performance by avoiding unnecessary calls to recvmmsg when there \ 
is no data available on the socket.
Improved nDAG performance by caching the byteswapped versions of some frequently \ 
accessed fields.
tracertstats will now handle SIGINT and SIGTERM signals cleanly.

libtrace 4.0.2:

New Features
Added ability to read pcapng trace files (and convert them into other formats).
Added input format for receiving and processing packets emitted by an nDAG \ 
multicaster.

Bug Fixes
Fixed bug that would cause the IPv6 fragment offset to be calculated incorrectly.
Fixed return value bug with pcap_write_packet().
Fixed bad assertion failure when halting parallel programs with SIGINT.
Fixed compilation issues caused by mismatched BPF presence macros when \ 
pcap-bpf.h is missing.
Fixed libpacketdump bug where it was reading past the end of captured IPv6 headers.
Fixed several issues in the libpacketdump parser for SCTP.
Fixed assertion failure in traceanon if the cryptopan key is too short.
Fixed compilation error with traceanon if libssl version >= 1.1.0.
Fix bug where the wrong parallel read function would be used in some specific \ 
configurations.
DPDK shared libraries are now correctly detected by the configure script.

libtrace 4.0.1:

New Features
DPDK support has been extended to cover the most recent stable release.
Added ability to parse SIT (IPv6 within IPv4) packets inside SLL.
Added trace_clear_statistics() API function.
Added support for IPv6 in PPP.
Added native support for bidirectional and balanced hashing to DPDK inputs.

Bug Fixes
Fixed bug where ring: and int: parallel inputs would not respond to \ 
trace_pstop() on older kernels.
Fixed bug where trace_interrupt() would not trigger on busy inputs (including files).
Fixed bug where DPDK inputs would cause the event API to hang.
Fixed bug where ring: and int: parallel formats could end up repeatedly polling.
Fixed performance issue with tracertstats when used on live formats.
Fixed bug where libtrace's default hasher was always sending packets to the same \ 
thread.
Fixed race conditions when using parallel API to read from a file format.
Fixed bug where the ordered combiner would appear to send packets to the \ 
reporter thread out of order, due to the packet ordering being based on a \ 
non-monotonic clock.
Fixed bug where trace_get_payload_from_gre() would not correctly parse PPTP GRE.

Improvements
Received packet counters are now valid for pcap inputs.
Improved performance by removing mutex from packet reading code.
Don't install extra header files directly into /usr/local/include; these are now \ 
installed into a libtrace-specific directory. This should resolve some namespace \ 
collision issues with some of our poorly-named header files.

libtrace 4.0.0:

New Features
New licensing -- Libtrace now uses the LGPL v3 rather than GPL v2, so it is now \ 
possible for people to link against libtrace without having to make their own \ 
code available under the GPL.
All new parallel API, written by Richard Sanger, that makes it easy to split \ 
packet processing tasks over multiple threads. If a capture format has support \ 
for native parallelism, e.g. DPDK, DAG streams, parallel libtrace will take \ 
advantage of these. The parallel API is contained and documented in \ 
"libtrace_parallel.h" -- include this header file to access the \ 
parallel API.
The previous single-threaded API is still supported, so all of your old libtrace \ 
programs should compile and run against libtrace 4 without modification.
Libwandio is no longer built in to libtrace and is now its own separate library. \ 
You can download libwandio from \ 
http://research.wand.net.nz/software/libwandio.php . Thanks to Alistair King for \ 
helping remove libwandio from libtrace.
New API function: trace_strip_packet(), which attempts to remove any VLAN, MPLS \ 
or other layer 2.5 headers from a captured packet.
Converted traceanon, tracertstats and tracestats tools to use the new parallel API.

Bug Fixes
Fixed bug where libpacketdump would print ICMP checksums in the wrong place.
Fixed inability to correctly parse ERF records that contained extension headers.
Fixed problem where traceanon wouldn't handle keyboard interrupts nicely.
Fixed memory leak if we fail to guess the format for an input trace (Thanks to \ 
Vincenzo Caruso for reporting this bug).
Fixed double free when destroying a DAG input.

Bugs squashed since the beta release:
Fixed bug that prevented multiple ring: or int: parallel inputs from being used \ 
on a single host concurrently.
Fixed memory leak when using a heavily filtered RT input.
Fixed bug where the ordered combiner would emit packets out of order.
Fixed bug where thread message queues were not being destroyed when the parent \ 
trace was destroyed.
Fixed race condition when modifying BPF headers on FreeBSD 9 systems.
Use default DPDK device driver thresholds instead of our previously hard-coded \ 
values.
Fixed potential infinite loop when parsing extended RadioTap headers.
Fixed bad decoding of RadioTap headers with extended presence.
Fixed bug where pausing a pcap: trace file would cause any resumption to return \ 
to the start of the file rather than resuming from where it left off.
Fixed segfault when destroying a packet associated with a trace has reached EOF.
Fixed memory management in trace_construct_packet (Thanks to Perry Lorier for \ 
submitting code to do this).
Fixed bug where pcap file descriptors were being leaked (Thanks to Tomas Konir \ 
for reporting this bug).
Fixed bug where trace_create_packet() would segfault if the system runs out of \ 
memory.

Improvements
Added BPF filtering option to traceanon.
Use libcrypto for traceanon IP address encryption rather than our own rijndael \ 
implementation. This adds a dependency on libcrypto, but should result in faster \ 
encryption operations.
Added a --jump option to tracesplit which can be used to strip any headers \ 
preceding the Nth layer 3 header; useful for decapsulating tunnelled IP traffic \ 
(Thanks to Perry Lorier for adding this feature).