./news/sfeed, RSS and Atom parser

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 1.8, Package name: sfeed-1.8, Maintainer: leot

RSS and Atom parser (and some format programs).

Required to build:

Master sites:

Filesize: 65.757 KB

Version history: (Expand)

CVS history: (Expand)

   2023-04-28 15:42:01 by Leonardo Taccari | Files touched by this commit (3) | Package updated
Log message:
sfeed: Update to 1.8

* sfeed_update: fail early if creating a temporary directory or status file fails.
* sfeed_atom, sfeed_json, sfeed_mbox:
  Fix reading past the buffer with an escaped NUL byte (\ NUL).
  Note that this could not happen with output from sfeed itself. Only if it was
* sfeed_curses: fix (very hard to trigger) memleak when getline() returns EOF
  for lazyloaded items.
* sfeed parser:
  * Improve parsing RFC2822 obsolete short year.
  * Use errno ENOMEM instead of EOVERFLOW.
    This matches the behaviour of setting errno for malloc/calloc on the following
    systems too: glibc, musl libc, OpenBSD libc.
  * date to unix timestamp: fix incorrect int type to long.
    Found while testing sfeed on 16-bit MS-DOS with Open Watcom (for \ 
"fun" :)).
* Makefile: remove duplicate CPPFLAGS for sfeed_curses

* sfeed_json: add JSON output format tool. This formats the TSV data to JSON.
  It uses a subset of JSON Feed 1.1: https://www.jsonfeed.org/version/1.1/

* sfeed_atom: save a few bytes in the output by removing the type="text"
  attribute, because for Atom the default for the type is text.
* Slightly reduce stack size for translating XML entities.
  A numeric entity could use 5 bytes, so use a round number of 8 bytes.

* sfeed: simplify time calculation and make it slightly easier to read.
  This also fixes a calculation (possibly a compiler bug) with Open Watcom 1.9.
* Remove the sfeed name in some outputs ("branding").

* Fix some typos and improve code comments, in particular about the time parsing.
* README: sfeed_download example: change youtube-dl to yt-dlp:
  this is an active maintained fork.
* README: add error checking for temporary files in the examples.

Aside from the above changes there have been lots of testing on different and
strange systems and the test-cases have been expanded to cover some cases.
These tests are in a separate repo.

Some tested strange systems:

* SerenityOS: gophers://codemadness.org/9/paste/sfeed-serenityos.webm partially
  works.  The core base utilities like sort are limited and non-POSIX though, so
  sfeed_update doesn't work directly there.

* MS-DOS (16-bit and 32-bit extender) using Open Watcom 1.9. Works fine using a
  32-bit extender.  No 2038-support with 64-bit time_t though, just _unsigned_
  32-bit :)

Tested these less-used systems or compilers can be fun and also discover
(pedantic) bugs.

While adding JSON Feed output support (sfeed_json) I also wrote a parser for it
to read the data.  It can convert from JSON Feed to Atom. It can also convert
directly from JSON Feed to sfeed(5). It is available at:


* Add SCO keys for next, prior (CSI I and CSI G).
  Tested on DragonFlyBSD (cons25 console).
* Add SUN keys support.
  Tested on OpenIndiana.

* Remove unnecesary PATH_MAX restricting the path length.
  This also makes it compile cleanly on GNU/Hurd.

* Man page and documentation improvements.
   2022-08-17 17:21:12 by Leonardo Taccari | Files touched by this commit (2)
Log message:
sfeed: Update to 1.6

* Code cleanups: unused code and refactored the code for reading read items
  from a plain-text URL file list.
* Fix typos in the code and documentation.
* Improve clarity of some code comments.
* Add setsid example to the README.
   2022-05-28 20:42:30 by Leonardo Taccari | Files touched by this commit (2) | Package updated
Log message:
sfeed: Update to 1.5

* sfeed_curses: interrupt waitpid while interactive child program is running
  This now handles SIGTERM on sfeed_curses while an interactive child program is
* sfeed_curses: close stdin before spawning a plumb program in non-interactive
  mode, which is more intuitive: the program doesn't seem to hang when it expects
  input in this case since there is no way to send input anyway.
* Properly escape backslashes in the man pages (thanks adc!).
* Documentation improvements to the man pages and a progress indicator example
  script for sfeed_update.

* Improve read and write error handling in streams in many programs.
* sfeed_update:
  * Improve error handling and return a non-zero status if any feed fails.
    NOTE: this changes the way of using:
        sfeed_update && pkill -SIGHUP sfeed_curses
  * Separate errors to stderr and "OK" messages to stdout.
* sfeed_curses:
  * Avoid processes becoming a zombie (plumb, pipe, yank).
  * Line editor: temporarily disable the mouse when searching with /.
  * Improved signal handling.
  * Improved waiting on processes and status handling.
* sfeed_html/sfeed_frames: add dark mode support to the example stylesheet.
* sfeed_opml_export: use a control-character separator for converting the list.
* General code cleanup improvements.
* Compatibility: reduce the assumption the builtin libc locale is ASCII-compatible
  noticed on OpenBSD 3.8 where iscntrl detected C1 as control-characters,
  breaking UTF-8.
* General documentation improvements.
* Makefile: rebuild sfeed_curses if the same theme changes.
* README: improve error handling in downloader example when a feed fails.
   2022-03-07 23:47:34 by Leonardo Taccari | Files touched by this commit (2)
Log message:
sfeed: Update to 1.3

* Fix a compiler warning with some curses implementations, like NetBSD curses.
* sfeed_curses: add keybinds for the home key and the default home and end key
  for urxvt.
* sfeed_curses: fix a redraw when reloading a file with a feed file read from
  stdin and using an URL file and changing this URL file externally.
* sfeed_curses: cast character for SFEED_AUTOCMD to unsigned char to allow
  character sequences outside the ASCII range.

* README: add an example script to count new and unread items. This can be
  useful for some statusbar indicator (asked about by e-mail).
* Small code-style, comments and documentation improvements and fixes.
   2022-02-10 12:35:16 by Leonardo Taccari | Files touched by this commit (2)
Log message:
sfeed: Update to 1.2

* sfeed parser: extend the time range to atleast 64-bit (long long).
* sfeed parser: allow leap second like 23:59:60 (RFC2822 Section 3.3).
* sfeed_curses: pedantic fix for UB with an empty URL file (qsort and bsearch).
* sfeed_curses: fix a compile warning with tparm() on some systems.

* sfeed_atom: add category field(s).
* sfeed_curses: add feature to go to the next bold row and previous bold row
with J and K.  This is useful to quickly jump to the next new item or new feed.
Thanks to IanJ for the suggestion and feedback!
* sfeed_content: allow to set HTML converter with env variable $SFEED_HTMLCONV.
For example: SFEED_HTMLCONV="w3m -I UTF-8 -O UTF-8 -T text/html -dump"
* Add compile-time option to improve output on dumb non-UTF8 terminals

* Documentation improvements to the man page and a sfeed_download example in
the README: this is a parallel downloader/extractor example script.
* Code-style improvements.
   2021-12-18 15:14:24 by Leonardo Taccari | Files touched by this commit (3)
Log message:
sfeed: Update to 1.1

 * The formatting program sfeed_curses is now merged in the sfeed repo.
   This is a program for viewing items in a curses UI.
   sfeed can still be built without it if its considered bloat with:
         make SFEED_CURSES="".
 * sfeed_mbox: escape the link and enclosure text when using HTML content
   with the option SFEED_MBOX_CONTENT="1"
 * sfeed_mbox: add link as base URL for HTML content.
 * sfeed_mbox: use 64-bit for the checksum number on 32-bit systems.
 * sfeed_content: optimizations to the awk script for the UTF-8 locale and
   string replacements.
 * sfeed_content: add link as base URL for HTML content.
 * Documentation improvements and more examples in the man page.
 * Small code-style improvements.
   2021-10-26 13:09:37 by Nia Alarie | Files touched by this commit (27)
Log message:
news: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes
   2021-10-07 16:45:00 by Nia Alarie | Files touched by this commit (27)
Log message:
news: Remove SHA1 hashes for distfiles