./devel/fq, File parsing tool like jq, but for binary files

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 0.13.0, Package name: fq-0.13.0, Maintainer: pkgsrc-users

Tool, language and decoders for inspecting binary data.

In most cases fq works the same way as jq but instead of reading
JSON it reads binary data. The result is a JSON compatible structures
where each value has a bit range, symbolic interpretations and know
how to be presented in a useful way.


Master sites:

Filesize: 17450.726 KB

Version history: (Expand)


CVS history: (Expand)


   2024-04-14 17:31:10 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
fq: update to 0.11.0.

# 0.11.0

New iNES/NES 2.0 ROM decoder (thanks @mlofjard) and basic JPEG 2000
format support. jq language improvements and fixes from gojq. And
as always various decoder improvements and fixes.

## Changes

- Add `string_truncate` option to configure how to truncate long strings when \ 
displaying a decode value tree. `dd`, `dv` etc set truncate length to zero to \ 
not truncate. #919
- gojq updates from upstream:
  - Implement `ltrim`, `rtrim`, and `trim` functions
  - Fix object construction with duplicate keys (`{x:0,y:1} | {a:.x,a:.y}`)
  - Fix `halt` and `halt_error` functions to stop the command execution immediately
  - Fix variable scope of binding syntax (`"a" as $v | def f: $v; \ 
"b" as $v | f`)
  - Fix `ltrimstr` and `rtrimstr` functions to emit error on non-string input
  - Fix `nearbyint` and `rint` functions to round ties to even
  - Improve parser to allow `reduce`, `foreach`, `if`, `try`-`catch` syntax as \ 
object values
  - Remove `pow10` in favor of `exp10`, define `scalbn` and `scalbln` by `ldexp`
- Fix issue using decode value with `ltrimstr`/`rtrimstr`.

## Format changes

- `fit`
  - Skip array fields on pre read messages. #878
  - Fixed subfield referencing fields below self in message. #877
- `jp2c` New JPEG 2000 codestream decoder. #928
- `icc_profile` Strip whitespace in header and tag strings. #912
- `mp4`
  - Add `jp2c`, `jp2h`, `ihdr` `jP` JPEG 2000 related boxes support. #928
  - Add `thmb` box support. #897
  - Turns out for qt brand `hdlr` component name might be zero bytes. #896
- `nes` New iNES/NES 2.0 ROM decoder (thanks @mlofjard). #893
   2024-04-05 21:14:14 by Benny Siegert | Files touched by this commit (161) | Package updated
Log message:
Revbump all Go packages after go121 update
   2024-02-13 19:19:06 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
fq: update to 0.10.0.

# 0.10.0

Adds support for various LevelDB formats (thanks @mikez) and Garmin Flexible and \ 
Interoperable Data Transfer format (FIT) (thanks @mlofjard).

And as usual some small fixes and dependency updates.

## Changes

- On macOS fq now reads init script from `~/.config/fq` in addition to \ 
`~/Library/Application Support/fq`. #871
- Switch readline module from own fork to https://github.com/ergochat/readline #854
- Updated gojq fork. Notable changes from upstream below. #844
  - Fix pre-defined variables to be available in initial modules
  - Fix object construction with duplicate keys

## Format changes

- `aac_frame` Decode instance tag and common window flag. #859
- `fit` Add support for Garmin Flexible and Interoperable Data Transfer decoder. \ 
Thanks @mlofjard #874
  - This format is used by various GPS tracking devices to record location, \ 
speed etc.
  - Example of converting position information to KML:
  ```jq
  # to_kml.jq
  # convert locations in a fit structure to KML
  def to_kml:
    ( [ .data_records[].data_message as {position_lat: $lat, position_long: $long}
      | select($lat and $long)
      | [$long, $lat, 0]
      | join(",")
      ]
    | join(". ")
    | { kml: {"@xmlns":"http://earth.google.com/kml/2.0"
      , Document: {Placemark: {LineString: {coordinates: .}}}}
      }
    | to_xml({indent:2})
    );
  ```
  Usage:
  ```sh
  # -L to add current directory to library path
  # -r for raw string output
  # 'include "to_ml";' to include to_kml.jq
  # to_kml calls to_kml function
  $ fq -L . -r 'include "to_kml"; to_kml' file.fit > file.kml
  ```

- `hevc_sps` Fix some incorrect profile_tier_level decoding. #829
- `html` Fix issue parsing elements including SOLIDUS "/". #870
  - Upstream issue https://github.com/golang/go/issues/63402
- `mpeg_es` Support ES_ID_Inc and decode descriptors for IOD tags
- `leveldb_descriptor`, `leveldb_log`, `leveldb_table` Add support for LevelDB. \ 
Thanks @mikez #824
  - This format is used by many database backends and applications like Google \ 
chrome.
- `pcapng` Decode all section headers instead of just the first. #860
- `png` Fix incorrect decoding of type flags. #847
- `hevc_sps` Fix incorrect decoding of profile_tier_level. #829
- `tls` Fix field name typos. #839
- `mp4`
  - Don't try decode samples for a track that has an external reference. #834
  - Use box structure instead of track id to keep track for sample table data. #833
  - `ctts` box v0 sample offset seems to be signed in practice but not in spec. #832
- `webp` Decode width, height and flags for lossless WebP. #857
   2024-02-07 15:51:04 by Benny Siegert | Files touched by this commit (156) | Package updated
Log message:
Revbump all Go packages after go121 update
   2024-01-10 20:14:43 by Benny Siegert | Files touched by this commit (152) | Package updated
Log message:
Revbump all Go packages after go121 update
   2023-12-05 20:46:19 by Benny Siegert | Files touched by this commit (146) | Package updated
Log message:
Revbump all Go packages after go121 update
   2023-11-25 20:03:16 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
fq: update to 0.9.0.

# 0.9.0

## Changes

- Make synthetic values not look like decode values with a zero range. #777
- Bit ranges are now displayed using exclusive end notation to be more \ 
consistent. For example if a field at byte offset 5 is 8 bit in size using the \ 
`<byte>[.<bits>]` notation it is now shown as `5-6`, before it was \ 
shown as `5-5.7`. See usage documentation for more examples. #789
- Improve colors when using a light background. Thanks @adedomin for reporting. #781
- Better `from_jq` error handling. Thanks @emanuele6 for reporting. #788
- Updated gojq fork. Notable changes from upstream below. #808
  - Adds `pick/1` function. Outputs value and nested values for given input \ 
expression.
  - Implements comment continuation with backslash.
- Updated gopacket to 1.2.1. Notable changes from upstream below. #815
  - fix(ip4defrag): allow final fragment to be less than 8 octets by @niklaskb
  - refactor: don't fill empty metadata slots by @smira
  - refactor: optimize port map to switch statement by @smira

## Decoder changes

- `avi`
  - Some general clean up fixes. Thanks to Marion Jaks at mediathek.at for \ 
testing and motivation.
  - Add extended chunks support and `decode_extended_chunks` option. This is \ 
trailing chunks used by big AVI files. #786
  - Add type, handler, compression (video) and format_tag (audio) per stream. #775
  - Properly use sample size field when decoding samples. #776
- `exif` (and `tiff`)
  - Handle broken last next ifd offset by treating it as end marker. #804
- `gzip`
  - Correctly handle multiple members. Thanks @TomiBelan for the bug report and \ 
assistance. #795
  - Now gzip is modelled as a struct with a `members` array and a `uncompressed` \ 
field that is the concatenation of the uncompressed members.
- `macho`
  - Properly respect endian when decoding some flag fields. #796
  - Move formatted timestamp to description so that numeric value is easier to \ 
access. #797
- `matroska`
  - Support decoding EBML date type. #787
- `protobuf`
  - No need to use synthetic fields for string and bytes. #800
- `webp`
  - Refactor to use common RIFF decoder and also decode VP8X, EXIF, ICCP and XMP \ 
chunks. #803
- `zip` Better timestamp support and fixes
  - Fix incorrect MSDOS time/date decoding and add extended timestamp support. \ 
Also remodel time/date to be a struct with raw values, components and a \ 
synthetics UTC unixtime guess. Thanks @TomiBelan for the bug report and \ 
assistance. #793
   2023-11-10 16:45:25 by Benny Siegert | Files touched by this commit (152) | Package updated
Log message:
Revbump all Go packages after go121 update