./www/logswan, Fast Web log analyzer using probabilistic data structures

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 1.07, Package name: logswan-1.07, Maintainer: pkgsrc-users

Logswan is a fast Web log analyzer using probabilistic data structures. It is
targeted at very large log files, typically APIs logs. It has constant memory
usage regardless of the log file size, and takes approximatively 4MB of RAM.

Unique visitors counting is performed using two HyperLogLog counters (one for
IPv4, and another one for IPv6), providing a relative accuracy of 0.10%.

Project design goals include : speed, memory-usage efficiency, and keeping the
code as simple as possible.


Required to run:
[net/GeoIP] [textproc/jansson]

Required to build:
[pkgtools/cwrappers]

Master sites:

SHA1: d12e2f480549eac8801e36600647f72cff3dd72a
RMD160: b6a6bb17764096b1b5bdad739d0770d141aca9d1
Filesize: 18.259 KB

Version history: (Expand)


CVS history: (Expand)


   2017-02-15 21:51:57 by Maya Rashish | Files touched by this commit (2) | Package updated
Log message:
logswan: update to 1.07. Add simple smoketest.

Changes since 1.06:
    Harmonize arrays names
    Remove array of months, it's currently unused and will likely remain so
    Simplify internal JSON array and object names
    Use OpenBSD style(9) for function prototypes and declarations
    Revert back to using strtok, at least for now
    Do not use EXIT_SUCCESS and EXIT_FAILURE macros anymore
    Fix implicit function declaration rrror on NetBSD (Thanks Maya Rashish)
    Remove now useless variables initialization and unnecessary includes
    Do not add an extra new line when displaying usage or version
   2017-01-12 18:53:01 by Maya Rashish | Files touched by this commit (1) | Package updated
Log message:
Add note to drop hack next update, it was fixed upstream.
   2016-12-18 19:49:39 by Maya Rashish | Files touched by this commit (2) | Package updated
Log message:
logswan: update to 1.06.

Changed license to 2-clause BSD (from 3-clause)
Minor code shuffling.

From fcambus (author)
   2016-09-11 17:59:29 by Kamil Rytarowski | Files touched by this commit (3)
Log message:
Reset maintainer to pkgsrc-users@NetBSD.org

Change requested by Frederic Cambus.
   2016-07-26 19:15:37 by Kamil Rytarowski | Files touched by this commit (3) | Package updated
Log message:
Update www/logswan to 1.05

Package changes:

- Trimmed package description
- Update MAINTAINER + added missing trailing slash for HOMEPAGE

Upstream ChangeLog:
Logswan 1.05 (2016-02-25)

- Documentation update (notes on measuring Logswan memory usage)
- Add additional include directories for compat functions + dependencies
  to avoid using relative path in includes
- Check that *lineBuffer is not NUL before attempting to parse log line
- Perform GeoIP lookup and HLL add in the same if block
- Increment IPv4 and IPv6 hits counters individually and conditionally
- Use CMake to check if the system has OpenBSD's pledge available and
  link pledge conditionally using a null implementation when compiled
  on non OpenBSD systems
- Adding an array of months, for the upcoming split log functionalities

Patch provided by the upstream developer Frederic Cambus
   2016-01-11 08:11:20 by Kamil Rytarowski | Files touched by this commit (3) | Package updated
Log message:
Update www/logswan to 1.04

Package changes:
- follow upstream with the rule what's installed of the files
- use newly added -DGEOIPDIR in the place of older string replace function
- stop installing an example output as a configuration file

Upstream changelog:
Logswan 1.04 (2016-01-10)

- Moving global variables into main
- Using 'size_t' instead of 'int' for array indexes in for loops
- Using 'uint32_t' for all non 'uint64_t' integers
- Do not increment hits and processed lines counter for each parsed line,
  compute total only once when everything is parsed
- Setting 'CMAKE_BUILD_TYPE' to 'Release' and formatting fixes
- Sanitize CMake script to build under NetBSD (Thanks Kamil Rytarowski)
- Initializing some uninitialized variables
- Renaming 'DATADIR' variables to 'GEOIPDIR'

Logswan 1.03 (2016-01-01)

- Remove header display and do not print name of processed file
- Print results to stderr instead of stdout
- Output JSON data to stdout instead of creating a new file
- Define GeoIP databases path in CMakeLists.txt
- Adding log file name in the JSON output
- Removing some hardcoded values and replacing them with constants
  defined in config.h
- Breaking the loop when a match is found in the request parser
- Using enumeration constants instead of macros
- Process GeoIP continent information
- Re-ordering protocols and methods with more common occurences on top of
  the list, allowing to break earlier when iterating through the array
- Adding support for reading logs from standard input
- Renaming 'definitions' files to 'config'
- Increasing countries array size, as an attempt to be future-proof
- Initial support for using pledge() on OpenBSD
- Documentation updates (HLL precision, Features list, GeoIP databases)
- Updated JSON output example
- Added a manual page

Logswan 1.02 (2015-11-02)

- Renaming 'resource' variable to 'request' in the 'logLine' struct
- Do not attempt to parse empty date tokens
- Do not attempt to parse empty request tokens (Thanks Brian Carpenter for
reporting the issue)
   2015-11-04 03:47:43 by Alistair G. Crooks | Files touched by this commit (758)
Log message:
Add SHA512 digests for distfiles for www category

Problems found locating distfiles:
	Package haskell-cgi: missing distfile haskell-cgi-20001206.tar.gz
	Package nginx: missing distfile array-var-nginx-module-0.04.tar.gz
	Package nginx: missing distfile encrypted-session-nginx-module-0.04.tar.gz
	Package nginx: missing distfile headers-more-nginx-module-0.261.tar.gz
	Package nginx: missing distfile nginx_http_push_module-0.692.tar.gz
	Package nginx: missing distfile set-misc-nginx-module-0.29.tar.gz
	Package nginx-devel: missing distfile echo-nginx-module-0.58.tar.gz
	Package nginx-devel: missing distfile form-input-nginx-module-0.11.tar.gz
	Package nginx-devel: missing distfile lua-nginx-module-0.9.16.tar.gz
	Package nginx-devel: missing distfile nginx_http_push_module-0.692.tar.gz
	Package nginx-devel: missing distfile set-misc-nginx-module-0.29.tar.gz
	Package php-owncloud: missing distfile owncloud-8.2.0.tar.bz2

Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden).  All existing
SHA1 digests retained for now as an audit trail.
   2015-10-01 23:38:44 by Kamil Rytarowski | Files touched by this commit (1)
Log message:
Use new official domain webpage.