./textproc/miller, Command-line CSV processor

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 6.9.0nb12, Package name: miller-6.9.0nb12, Maintainer: pkgsrc-users

Miller is like sed, awk, cut, join, and sort for name-indexed data
such as CSV.

With Miller, you get to use named fields without needing to count
positional indices.

This is something the Unix toolkit always could have done, and
arguably always should have done. It operates on key-value-pair
data while the familiar Unix tools operate on integer-indexed
fields: if the natural data structure for the latter is the array,
then Miller's natural data structure is the insertion-ordered hash
map. This encompasses a variety of data formats, including but not
limited to the familiar CSV. (Miller can handle positionally-indexed
data as a special case.)

Master sites:

Filesize: 10792.943 KB

Version history: (Expand)

CVS history: (Expand)


   2024-04-05 21:14:14 by Benny Siegert | Files touched by this commit (161) | Package updated
Log message:
Revbump all Go packages after go121 update
   2024-02-07 15:51:04 by Benny Siegert | Files touched by this commit (156) | Package updated
Log message:
Revbump all Go packages after go121 update
   2024-01-10 20:14:43 by Benny Siegert | Files touched by this commit (152) | Package updated
Log message:
Revbump all Go packages after go121 update
   2023-12-05 20:46:19 by Benny Siegert | Files touched by this commit (146) | Package updated
Log message:
Revbump all Go packages after go121 update
   2023-11-10 16:45:25 by Benny Siegert | Files touched by this commit (152) | Package updated
Log message:
Revbump all Go packages after go121 update
   2023-10-29 15:48:24 by Benny Siegert | Files touched by this commit (152) 
Log message:
Revbump all Go packages because go121 is now the default
   2023-10-15 14:04:35 by Benny Siegert | Files touched by this commit (155) | Package updated
Log message:
Revbump all Go packages after go120 security update
   2023-09-10 15:06:18 by Yorick Hardy | Files touched by this commit (4) | Package updated
Log message:
textproc/miller: update to miller-6.9.0

Changes:

6.9.0
-----

Nanosecond timestamps, ZSTD compression, improved data-error handling,
and more

New features

Support for nanosecond-resolution timestamps:

 *  Add DSL functions for integer nanoseconds since the epoch by
    @johnkerl in #1326
 *  Add %N and %O for strfntime by @johnkerl in #1334
 *  Add %s format specifier for strftime by @johnkerl in #1335
 *  Requested on issue #1152 by @derekmahar
 *  See also \ 
https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#time-functions

Statistical functions from the stats verb may now be computed over
arbitrary arrays and maps:

 *  New DSL functions for summary stats over arrays / maps by
    @johnkerl in #1364
 *  Requested on issue #1345 by @janxkoci
 *  See also \ 
https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#stats-functions

Additional control over filenames for the split verb:

 *  Filename options for split by @sloanlance in #1366
 *  Requested on issue #1365 by @sloanlance

Support for details of data-computation errors beyond the current
(error):

 *  Fatal-on-data-error mlr -x option by @johnkerl in #1373
 *  See also \ 
https://miller.readthedocs.io/en/6.9.0/reference-dsl-errors/#handling-for-data-errors
 *  Requested on issue #1106 by @honzajde

New verbs and DSL functions:

 *  New sub, gsub, and ssub verbs by @johnkerl in #1361, requested by
    @janxkoci on #1356. See also:
        https://miller.readthedocs.io/en/6.9.0/reference-verbs/#sub
        https://miller.readthedocs.io/en/6.9.0/reference-verbs/#gsub
        https://miller.readthedocs.io/en/6.9.0/reference-verbs/#ssub
 *  New contains DSL function by @johnkerl in #1374
        \ 
https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#contains

Other updates:

 *  Support ZSTD compression in-process by @johnkerl in #1360, from
    issue #1342 by @aborruso
        See also \ 
https://miller.readthedocs.io/en/6.9.0/reference-main-compressed-data/
 *  Support comments in mlr -s files by @johnkerl in #1359, from
    issue #1343 by @janxkoci
 *  Add empty-key check to mlr check by @johnkerl in #1330, from
    issue #1050 by @aborruso

Bug fixes

 *  Do wildcard globbing on Windows by @johnkerl in #1362, from
    issue #1341 by @coolbq
 *  Treat empty like absent in + - * by @johnkerl in #1371, from
    issue #1001 by @Poshi
 *  Can't use ${field_name} if it contains UTF-8 characters also
    encodeable as Latin-1 by @johnkerl in #1363, from issue #1358
    by @clemente
 *  Typofix in uif/uof percentiles by @johnkerl in #1375

Documentation updates

 *  Update readthedocs notes in the how-to-release page by @johnkerl
    in #1308
 *  Fix mlr grep docs re OFS/OPS by @johnkerl in #1309
 *  Update Fedora link by @bkmgit in #1339
 *  Small typos in documentation of mlr nest by @johnkerl in #1352,
    from issue #1350 by @kusalananda

Internal

 *  Update 2015-era Python sketch to Python 3 by @johnkerl in #1372
 *  Remove redundant nil check by @Juneezee in #1367
 *  Bump actions/checkout from 3.5.2 to 3.5.3 by @dependabot in #1319
 *  Bump github/codeql-action from 2.3.6 to 2.13.4 by @dependabot in #1318
 *  Bump golang.org/x/term from 0.8.0 to 0.9.0 by @dependabot in #1321
 *  Bump goreleaser/goreleaser-action from 4.2.0 to 4.3.0 by @dependabot in #1320
 *  Bump golang.org/x/text from 0.9.0 to 0.10.0 by @dependabot in #1322
 *  Bump golang.org/x/text from 0.10.0 to 0.11.0 by @dependabot in #1337
 *  Bump golang.org/x/sys from 0.9.0 to 0.10.0 by @dependabot in #1336
 *  Bump golang.org/x/term from 0.9.0 to 0.10.0 by @dependabot in #1338
 *  Bump golang.org/x/sys from 0.10.0 to 0.11.0 by @dependabot in #1347
 *  Bump golang.org/x/text from 0.11.0 to 0.12.0 by @dependabot in #1349
 *  Bump actions/setup-go from 4.0.1 to 4.1.0 by @dependabot in #1351
 *  Bump goreleaser/goreleaser-action from 4.3.0 to 4.4.0 by @dependabot in #1354
 *  Bump golang.org/x/term from 0.10.0 to 0.11.0 by @dependabot in #1348
 *  Bump actions/checkout from 3.5.3 to 3.6.0 by @dependabot in #1369

New Contributors

 *  @bkmgit made their first contribution in #1339
 *  @Juneezee made their first contribution in #1367
 *  @sloanlance made their first contribution in #1366

6.8.0
-----

New case verb, index DSL function, and more

New features

New case verb:

 *  Unify the case verb, and add options by @johnkerl in #1306
 *  Add new upcase and downcase verbs by @johnkerl in #1217

New index DSL function:

 *  index DSL function by @johnkerl in #1247

Enhancements:

 *  Add mlr step -a rprod for running products by @johnkerl in #1228
 *  Add optional second base argument to int DSL function
    by @johnkerl in #1244
 *  Implement --csv-trim-leading-space flag by @johnkerl in #1272
 *  New mlr json-parse -k flag by @johnkerl in #1291
 *  Let mlr help take pre-flags, such as --always-color
    by @johnkerl in #1292
 *  Values-only -a option for mlr grep by @johnkerl in #1305

Bugfixes

 *  Fix bug on DSL comment with no final newline by @johnkerl in #1216
 *  Better error message on unparseable TZ environment variable
    by @johnkerl in #1249
 *  Fix typo by @dnicolson in #1252
 *  Treat data-file numbers with leading + as numeric
    by @johnkerl in #1269
 *  Fix precedence of coalesce operators ?? and ??? by @johnkerl
    in #1270

Documentation

 *  Docs re tail -f and --records-per-batch 1 by @johnkerl in #1218
 *  Fix issue links in README-dev.md by @kcwu in #1248

Miscellaneous

 *  Include tools in the release tarball by @skitt in #1221
 *  Run go mod tidy by @skitt in #1220

Dependencies

Go:

 *  Bump minimum compiler version from Go 1.15 to 1.18 by @johnkerl in #1246

Others:

 *  Bump github/codeql-action from 2.2.5 to 2.2.6 by @dependabot in #1230
 *  Bump actions/cache from 3.2.6 to 3.3.1 by @dependabot in #1229
 *  Bump github/codeql-action from 2.2.6 to 2.2.7 by @dependabot in #1232
 *  Bump actions/setup-go from 3.5.0 to 4.0.0 by @dependabot in #1233
 *  Bump actions/checkout from 3.3.0 to 3.4.0 by @dependabot in #1234
 *  Bump github/codeql-action from 2.2.7 to 2.2.8 by @dependabot in #1242
 *  Bump actions/checkout from 3.4.0 to 3.5.0 by @dependabot in #1245
 *  Bump golang.org/x/term from 0.0.0-20210927222741-03fcf44c2211 to 0.6.0 by \ 
@dependabot in #1222
 *  Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 by @dependabot in #1243
 *  Bump github/codeql-action from 2.2.8 to 2.2.9 by @dependabot in #1250
 *  Bump codespell-project/actions-codespell from \ 
9c63fddd79f483308bfaea379a505dcd361b5d1d to \ 
57beb9f38f49d773d641ac555d1565c3b6a59938 by @dependabot in #1253
 *  Bump golang.org/x/term from 0.6.0 to 0.7.0 by @dependabot in #1256
 *  Bump github/codeql-action from 2.2.9 to 2.2.10 by @dependabot in #1259
 *  Bump github/codeql-action from 2.2.10 to 2.2.11 by @dependabot in #1261
 *  Bump actions/checkout from 3.5.0 to 3.5.1 by @dependabot in #1263
 *  Bump actions/checkout from 3.5.1 to 3.5.2 by @dependabot in #1264
 *  Bump github/codeql-action from 2.2.11 to 2.2.12 by @dependabot in #1265
 *  Bump github/codeql-action from 2.2.12 to 2.3.0 by @dependabot in #1274
 *  Bump github/codeql-action from 2.3.0 to 2.3.1 by @dependabot in #1277
 *  Bump github/codeql-action from 2.3.1 to 2.3.2 by @dependabot in #1279
 *  Bump codespell-project/actions-codespell from \ 
57beb9f38f49d773d641ac555d1565c3b6a59938 to \ 
94259cd8be02ad2903ba34a22d9c13de21a74461 by @dependabot in #1282
 *  Bump github/codeql-action from 2.3.2 to 2.3.3 by @dependabot in #1284
 *  Bump golang.org/x/term from 0.7.0 to 0.8.0 by @dependabot in #1285
 *  Bump actions/setup-go from 4.0.0 to 4.0.1 by @dependabot in #1294
 *  Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 by @dependabot in #1295
 *  Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 by @dependabot in #1296
 *  Bump github/codeql-action from 2.3.3 to 2.3.4 by @dependabot in #1299
 *  Bump github/codeql-action from 2.3.4 to 2.3.5 by @dependabot in #1300
 *  Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 by @dependabot in #1301
 *  Bump github/codeql-action from 2.3.5 to 2.3.6 by @dependabot in #1303

New Contributors

 *  @kcwu made their first contribution in #1248
 *  @dnicolson made their first contribution in #1252