./wip/py-logtools, Log files analysis/filtering framework written in Python

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 0.8.7, Package name: py312-logtools-0.8.7, Maintainer: pkgsrc-users

logtools encompasses of a few easy-to-use, easy to configure command-line tools,
typically used in conjunction with Apache logs (but not only).

The idea is to standardize log parsing and filtering using a simple, coherent
configuration methodology and UNIX/POSIX-like command-line interface (STDIN
input streaming, command piping etc.), so as to create a consistent environment
for creating reports, charts and other such log mining artifacts that are
typically employed in a Website context.

logtools can be used either programmatically from within a python program, or as
a set of atomic command-line utilities. See below for description of the command
-line tools.


Required to run:
[devel/py-setuptools] [time/py-dateutil] [databases/py-sqlalchemy] [databases/py-sqlsoup] [textproc/py-acora] [lang/python37] [textproc/py-ptable]

Required to build:
[pkgtools/cwrappers]

Master sites:

RMD160: 0aa22d0d721b8ee76a79afc79944ea0abb965f51
Filesize: 37.466 KB

Version history: (Expand)

CVS history: (Expand)


   2015-06-27 01:03:28 by othyro | Files touched by this commit (3) 
Log message:
Update to 0.8.7. Adds a flattenjson module.
   2014-02-13 01:58:05 by othyro | Files touched by this commit (2) 
Log message:
Added textproc/py-acora as a dependency. Bump PKGREVISION to ensure this
dependency is pulled in.
   2014-02-03 11:51:00 by othyro | Files touched by this commit (4) 
Log message:
Updated to latest release, 0.8.6. Fixed DEPENDS. From TODO:

* Requires py-dateutil>=2.1, which is limited to python33 in pkgsrc.
* Requires py-acora, which isn't packaged and broken. Fixed upstream. Awaiting
  new release.

Use REPLACE_PYTHON by including lang/python/application.mk and drop
REPLACE_INTERPRETER.env. From ChangeLog:

version 0.8.6
Tweaks to the 'percentiles' utility - Added optional -N argument to force
tailing specific number of lines (samples), removed hard-coded 10000 lines
limitation, changed sort to use -g (general numeral sort) and enforce
LC_NUMERIC locale of en_US.utf-8 to correctly handle exponents (e.g '414e-05')

version 0.8.5
Removed pygooglechart dependency as its optional and currently seems
package is broken.

version 0.8.4
Added the triggered-regexp script
Added the logtail utility and CLI

version 0.8.3
Added explicit error message when --format is missing to AccessLog log parser
Added uWSGIParser. Can now be used with logparse --parser uWSGIParser.
Field names (for key-based -f lookup): 'ip', 'timestamp', 'method',
'path', 'bytes', 'processing_time'.
Added unit-test for the uWSGIParser
Better exception handling for the logparse utility, now reports failure to match
used-specified --fields in more accurate manner, separate from failures in \ 
actual parsing of the logline.

version 0.8.2
Updated dependecy on sqlsoup (separated out of sqlalchemy). Thanks to
Robert Churchill for pointing it out.
Update SqlAlchemy dependency version to latest stable release

version 0.8.1
    Fixes to logfilter to properly support --parser -based filtering
    Updated documentation and examples in README
    Cleanup and error checking to percentiles utility, fixed to work with osx
    mktemp
   2012-12-26 18:23:34 by othyro | Files touched by this commit (4) 
Log message:
Import py27-logtools as wip/py-logtools.

logtools encompasses of a few easy-to-use, easy to configure command-line tools,
typically used in conjunction with Apache logs (but not only).

The idea is to standardize log parsing and filtering using a simple, coherent
configuration methodology and UNIX/POSIX-like command-line interface (STDIN
input streaming, command piping etc.), so as to create a consistent environment
for creating reports, charts and other such log mining artifacts that are
typically employed in a Website context.

logtools can be used either programmatically from within a python program, or as
a set of atomic command-line utilities. See below for description of the command
-line tools.