./graphics/libraw, Raw decoding/processing library

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 0.21.1nb1, Package name: libraw-0.21.1nb1, Maintainer: pkgsrc-users

LibRaw is a library for reading RAW files obtained from digital photo
cameras (CRW/CR2, NEF, RAF, DNG, and others).

LibRaw is based on the source codes of the dcraw utility, where part of
drawbacks have already been eliminated and part will be fixed in future.
The users of the library are provided with API to be built into their
software programs.


Required to run:
[graphics/jpeg] [graphics/lcms2]

Required to build:
[pkgtools/cwrappers]

Master sites:

Filesize: 1600.06 KB

Version history: (Expand)


CVS history: (Expand)


   2023-04-11 08:11:02 by Adam Ciarcinski | Files touched by this commit (83)
Log message:
revbump for jasper 4.0.0
   2023-03-16 09:38:51 by Thomas Klausner | Files touched by this commit (3) | Package updated
Log message:
libraw: update to 0.21.1.

 LibRaw 0.21.1-Release
 * fixed typo in panasonic metadata parser

 LibRaw 0.21-Release

 * Multiple fixes inspired by oss-fuzz project

 LibRaw 0.21-Beta1

 == Camera format support ==
 * Phase One/Leaf IIQ-S v2 support
 * Canon CR3 filmrolls
 * Canon CRM (movie) files
 * Tiled bit-packed (and 16-bit unpacked) DNGs
 * (non-standard) Deflate-compressed integer DNG files are allowed

== Camera support ==

 * Canon EOS R3, R7 and R10
 * Fujifilm X-H2S, X-T30 II
 * OM System OM-1
 * Leica M11
 * Sony A7-IV (ILCE-7M4)
 * DJI Mavic 3
 * Nikon Z9: standard compression formats only

 == Multiple (resolution) thumbnails support ==

 * New imgdata.thumbs_list data item with data fields:

   int thumbcount: thumbnail count
   libraw_thumbnail_item_t thumblist[LIBRAW_THUMBNAIL_MAXCOUNT]: list  of thumbnails

   libraw_thumbnail_item_t fields:

     enum LibRaw_internal_thumbnail_formats tformat: internal thumbnail format
        (this is not PPM/JPEG, but internal type related to thumbnail \ 
reading/unpacking method)

     ushort twidth, theight: thumbnail image size.

     ushort tflip: image rotation (see notes below)

     unsigned tlength: on-disk data size (not uncompressed for compressed formats)

     unsigned tmisc: bit depth and channel count:  (color << 5) | bitsperpixel

     INT64 toffset: Thumbnail data offset in file

  Notes:
    -  Only TIFF-based and CR3 files are parsed for thumbnail list, other \ 
formats will have
       thumbcount = 1 (or 0 if no thumbnail found in file).

    -  Thumbnail image size may be unknown (not recorded in metadata), in this \ 
case twidth and theight are zero.
       Usually small(er) thumbnails will always have twidth/theight filled, \ 
while largest one may have these fields set to zero.

    -  Thumbnail rotation (tflip) is filled only for TIFF-based RAWs (if \ 
Orientation tag is not present in IFD, default zero value is used)
       For non-tiff images, tflip is initialized to 0xffff (not known...)

    - There is no code to select thumbnail based on some criteria (e.g. minimal \ 
resolution), it is left to user to implement such a criteria, if needed
      (see unpack_thumb_ex() call below)

    - If you want to get largest possible thumbnail: just use old unpack_thumb() \ 
call to get it.

  * new API call:  LibRaw::unpack_thumb_ex(int i):
    Unpacks i-th thumbnail if it exists into imgdata.thumbnail structure
    i should be non-zero and less then imgdata.thumbs_list.thumbcount.

  * samples/simple_dcraw.cpp: new -E command line switch to extract all \ 
thumbnails from input file(s)

 == (Experimental) RawSpeed "version 3" (develop branch) support. ==

   Build with -DUSE_RAWSPEED3, see details in RawSpeed3/README.md

   LibRaw::capabilities will set bit LIBRAW_CAPS_RAWSPEED3 if compiled w/
   RawSpeed-v3 support

   If file was processed (or tried to process) via RawSpeed-v3, these bits
   are raised in imgdata.process_warnings:
    LIBRAW_WARN_RAWSPEED3_PROCESSED - processed via RawSpeed v3
    LIBRAW_WARN_RAWSPEED3_PROBLEM - not processed (due to exception in RawSpeed \ 
library)
    LIBRAW_WARN_RAWSPEED3_UNSUPPORTED - unsupported file
    LIBRAW_WARN_RAWSPEED3_NOTLISTED - file not listed in cameras.xml data

  RawSpeed-v3 support is controlled via libraw_decoder_info.t->flags
  (LIBRAW_DECODER_TRYRAWSPEED3 bit).
  The bit is set for:
    - lossless jpeg decoder
    - Canon sRAW decoder
    - lossless compressed and packed-compressed DNG decoders
    - Pentax decoder
    - Nikon (compressed) decoder
    - Phase One Compressed and Hasselblad/Compressed decoders
    - bitpacked decoder
    - Panasonic (old 12-bit compression) decoder
    - Olympus decoder
    - Sony ARW (v1 and v2) decoders
    - Samsung (version 1) decoder

 * (Experimental) Fine control of RawSpeed version selection/use.
   To enable, build LibRaw with -DUSE_RAWSPEED_BITS
   LibRaw::capabilities will set bit LIBRAW_CAPS_RAWSPEED_BITS if compiled
   with this flag.
   If enabled:
     imgdata.rawparams.use_rawspeed becomes bit-field (instead of 0/1 off/on)
     with bits:
        LIBRAW_RAWSPEEDV1_USE - use RawSpeed Version 1 (if compiled  with)
        LIBRAW_RAWSPEEDV1_FAILONUNKNOWN - do not process unknown files
             (not listed in RawSpeed v1 camera definitions file)
        LIBRAW_RAWSPEEDV1_IGNOREERRORS - ignore mirror decoding errors (out of \ 
range data,etc)

        LIBRAW_RAWSPEEDV3_USE - Use RawSpeed Version 3 (if compiled with)
        LIBRAW_RAWSPEEDV3_FAILONUNKNOWN - do not process unknown files
        LIBRAW_RAWSPEEDV3_IGNOREERRORS - ignore minor decoding errors

 == Misc changes/improvements ==
 * New compile time define LIBRAW_OWN_SWAB
   to use on platforms without swab() in C/C++ runtime

 * New imgdata.rawparams.options bit: \ 
LIBRAW_RAWOPTIONS_CANON_IGNORE_MAKERNOTES_ROTATION
   If set: image orientation is set based on TIFF/IFD0:Orientation tag,
   makernotes orientation data is ignored

 * Nikon makernotes: read NEFCompression tag for HE/HE* files

 * Nikon orientation tag: more fixed offsets for known cameras

 * Adobe DNG SDK 1.6 support (meaning, just an additional patch for GPR SDK)

 * removed LibRaw::memerr(),  memory allocation failures are already handled
   in LibRaw_memmgr::*alloc w/ exception thrown

 * removed memory error callback, out-of-memory error should be handled via
   returned error code check, all related parameters (e.g. LibRaw constructor
   option to not set such callback) is also removed

 * imgdata.params.adjust_maximum_thr is settable via C-API \ 
libraw_set_adjust_maximum_thr(..)

 * New rawoptions/processing flags for DNG processing if compiled
   with Adobe DNG SDK:
   LIBRAW_RAWOPTIONS_DNG_STAGE2_IFPRESENT,LIBRAW_RAWOPTIONS_DNG_STAGE3_IFPRESENT
   If these flag(s) are set: Stage2/Stage3 processing will be performed
   only if OpcodeList2/OpcodeList3 tags are present in the input DNG file
   Old (hard) flags (LIBRAW_RAWOPTIONS_DNG_STAGE2 and
   LIBRAW_RAWOPTIONS_DNG_STAGE3) are not removed and will force Stage2/3
   processing if set (regardless of input file tags)

 * New imgdata.rawparams.options bit: LIBRAW_RAWOPTIONS_DNG_ADD_MASKS
   If set: DNG Transparency Masks will be extracted (if selected via shot_select)

 * New decoder flag: LIBRAW_DECODER_UNSUPPORTED_FORMAT

 == Bugs fixed ==

 * Fixed possible out-of-buffer read in Nikon orientation tag parser

 * Windows datastream: fixed logic errors (not showing up in real life)

 * Out-of-range read-only array access in postprocessing if output_color is set \ 
to 0 (raw color)

 * Minolta Z2 was not recognized correctly on 32-bit systems

 * Fixed possible buffer overflow in Kodak C330 decoder

 * dcraw_process(): check for buffer allocation results to avoid NULL deref
   2021-10-26 12:47:26 by Nia Alarie | Files touched by this commit (800)
Log message:
graphics: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes
   2021-10-07 16:13:27 by Nia Alarie | Files touched by this commit (800)
Log message:
graphics: Remove SHA1 hashes for distfiles
   2021-09-28 12:28:37 by Nia Alarie | Files touched by this commit (8)
Log message:
Recursive revbump for graphics/libraw.
   2021-09-28 11:59:24 by Nia Alarie | Files touched by this commit (7) | Package updated
Log message:
libraw: Update to 0.20.2

2020-10-14 Alex Tutubalin <lexa@lexa.ru>

 * LibRaw 0.20.2
 Reverted 0.20.1 change:
   - const buffer for open_buffer() and open_bayer() calls
 Because of 0.20.0 ABI break

2020-10-14 Alex Tutubalin <lexa@lexa.ru>

 * LibRaw 0.20.1
 * Improvements:
   - exif callback is called on EXIF GPS and EXIF Interop IFDs
   - open_bayer call documented
   - Canon (ColorDatsSubver==32): parse Specular White instead of hardcoded value
 * Fixes for normal files processing:
   - Olympus XZ-1: do not provide linear_max (it is wrong in metadata)
   - Nikon Z cameras: added space in camera list
   - raw-identify: fixed wb-preset print
   - Pentax Optio 33WR: maker index was incorrect
   - dcraw_emu: corrected help line for -6 option.
   - raw-identify: corrected range check for color matrices print
   - use_camera_matrix option: fixed a bug introduced when making
     compiler more happy.

 * Fixes for damaged/special crafted files processing:
   - Fix for truncated CR3 files parsing
   - DNG metadata merger: all color loops are limited to MIN(4,colors)
   - Check for marings: should be less than raw image size
   - Check for xmpdata present in Samsung Lens ID assignment
   - Check for column range in leaf_hdr decoder
   - Additional checks in Hasselblad model parser
   - Fuji rotate: better limits check
   - DNG files: limit tiff_samples

 * Not fixes, but makes ASAN/compilers/etc happy:
   - corrected GPS EXIF output
   - const buffer for open_buffer() and open_bayer() calls

2020-07-23 Alex Tutubalin <lexa@lexa.ru>

 * LibRaw 0.20

== Camera Format support ==
  Canon CR3
  GoPro (via GPR SDK)
  Panasonic 14-bit
  Fujifilm compressed/16bit
  Rapsberry Pi RAW+JPEG format (if USE_6BY9RPI defined)
  Foveon X3F support changed: it is supported only if USE_X3FTOOLS defined
  at build (see below for 'Imported code policy changed')

== Camera support (+59, 1131 total) ==
  Canon: PowerShot G5 X Mark II, G7 X Mark III, SX70 HS,
         EOS R, EOS RP, EOS 90D, EOS 250D,  EOS M6 Mark II, EOS M50, EOS M200
         EOS 1DX Mark III (lossless files only)
  DJI Mavic Air, Osmo Action
  FujiFilm GFX 100, X-A7, X-Pro3, X100V, X-T4 (uncompressed/lossless compressed \ 
only), X-T200
  GoPro Fusion, HERO5, HERO6, HERO7, HERO8
  Hasselblad L1D-20c, X1D II 50C
  Leica D-LUX7, Q-P, Q2, V-LUX5, C-Lux / CAM-DC25, SL2, M10 Monochrom
  Nikon D780, Z50, P950
  Olympus TG-6, E-M5 Mark III, E-PL10, E-M1 Mark III,
  Panasonic DC-FZ1000 II, DC-G90, DC-S1, DC-S1R, DC-S1H, DC-TZ95
  PhaseOne IQ4 150MP
  Ricoh GR III
  Sony A7R IV, A9 II, ILCE-6100, ILCE-6600, RX0 II, RX100 VII
  Zenit M

also multiple smartphones (the tested ones are listed in LibRaw::cameraList)

== Source code re-arranged ==
 * dcraw.c is not used in the generation and build processes
 * dcraw_common.cpp and libraw_cxx.cpp are split into multiple code chunks
   placed in separate subfolders (decoders/ for raw data decoders,
   metadata/ for metadata parsers, etc)
 * dcraw_common.cpp and libraw_cxx.cpp remain to preserve existing
   build environments (these files are now just a bunch of #include directives).
 * It is possible to build LibRaw
    a)without postprocessing functions (dcraw_process() and  called function)
    b)without postprocessing and LibRaw::raw2image() call (and called function).
   It may be useful to reduce library memory/code footprint.
   See Makefile.devel.nopp and Makefile.devel.noppr2i for the list of source
   files needed to build reduced/stripped library.

== Normalized make/model ==

There is a huge number of identical cameras sold under different names,
depending on the market (e.g. multiple Panasonic or Canon models)
and even some identical cameras sold under different brands
(Panasonic -> Leica, Sony -> Hasselblad).

To reduce clutter, a normalization mechanism has been implemented in LibRaw:

In imgdata.idata:
    char normalized_make[64]; - primary vendor name (e.g. Panasonic for
    Leica re-branded cameras)
    char normalized_model[64]; - primary camera model name
    unsigned maker_index; - primary vendor name in indexed form (enum
       LibRaw_cameramaker_index, LIBRAW_CAMERAMAKER_* constant).
These fields are always filled upon LibRaw::open_file()/open_buffer() calls.

const char* LibRaw::cameramakeridx2maker(int index): converts maker_index
to normalized_make.

We recommend that you use these normalized names in a variety of data tables
(color profiles, etc.) to reduce the number of duplicate entries.

New vendor index values will be added strictly to the end of the
LibRaw_cameramaker_index table, ensuring that the numbers assigned to
vendors that are already known to LibRaw will not change.

== DNG frame selection  ==

 DNG frames selection code re-worked:
  - by default all frames w/ the NewSubfileType tag equal to 0
  (high-res image) are added to the list of available images (selection
  performed via imgdata.params.shot_select field, as usual)
  - the special case for Fuju SuperCCD (SamplesPerPixel == 2) works as
  before: shot_select=1 will extract second sub-image.
  - Additional flags to imgdata.params.raw_processing_options:
     LIBRAW_PROCESSING_DNG_ADD_ENHANCED - will add Enhanced DNG frame
     (NewSubfileType == 16) to the list of available frames
     LIBRAW_PROCESSING_DNG_ADD_PREVIEWS - will add previews
     (NewSubfileType == 1) to the list.

  - By default, DNG frames are not reordered and are available in same order
  as in DNG (LibRaw traverses IFD/Sub-IFD trees in deep-first order).
  To prioritize the largest image, set LIBRAW_PROCESSING_DNG_PREFER_LARGEST_IMAGE
  bit in imgdata.params.raw_processing_options.

 - DNG Stage2/Stage3 processing via DNG SDK (request via flags in
 raw_processing_options)

== Imported code policy disclaimer ==

  We've changed the policy regarding 3rd party code imported into LibRaw.

  We (like other authors of open-source RAW parsers) gladly import support
  code for various RAW formats from other projects (if the license allows it).
  This is done to expand camera support.
  Unfortunately, not all imported code can tolerate truncated or otherwise
  damaged raw files, as well as arbitrary conditions or arbitrary data;
  not all authors handle rejecting unexpected input well.

  LibRaw is now widely used in various projects, including ImageMagick, which,
  in turn, is often used on web sites to process any input images, including
  arbitrary data from unknown users.
  This opens up wide possibilities for exploiting the various vulnerabilities
  present in the code borrowed from other projects into LibRaw. In order to
  avoid such security risks, - the borrowed code will no longer compile
  by default.
  We are not able to support it in general case, and the authors refuse
  to add code to reject unexpected input.

  Thus, if you use some kind of camera for which the support is disabled
  by default, you need to recompile LibRaw for your specific case.

  Formats currently affected:
   X3F (Foveon) file format.
    Code is imported from Kalpanika X3F tools: https://github.com/Kalpanika/x3f
    To turn the support on, define USE_X3FTOOLS

   Rapsberry Pi RAW+JPEG format.
     Code is imported from https://github.com/6by9/dcraw/,
     To turn the support on, define USE_6BY9RPI
  Format support is indicated via LibRaw::capabilities() call with flags:
    LIBRAW_CAPS_X3FTOOLS - Foveon support
    LIBRAW_CAPS_RPI6BY9 - RPi RAW+JPEG support

== GoPro .gpr format support ==
 GoPro  format supported via open-source GPR SDK
 See README.GoPro.txt for details.

== Windows support/Windows unicode (wchar_t*) filenames support ==

* (old) LibRaw's WIN32 external define split into 3 defines to fine tune
  compiler/api compatibility:
  LIBRAW_WIN32_DLLDEFS - use to compile DLLs (__dllimport/__dllexport attributes)
  LIBRAW_WIN32_UNICODEPATHS - indicates that runtime has calls/datatypes for \ 
wchar_t filenames
  LIBRAW_WIN32_CALLS - use Win32 calls where appropriative (binary mode for \ 
files, LibRaw_windows_datastream, _snprintf instead of snprintf, etc).

  If the (old) WIN32 macro is defined at compile time, all three new defines are \ 
defined in libraw.h
  If not, these defines are defined based on compiler version/libc++ defines

* LibRaw::open_file(wchar_t*) is always compiled in under Windows, but
  if LIBRAW_WIN32_UNICODEPATHS (see above) is not defined, this call will
  return LIBRAW_NOT_IMPLEMENTED.
  Use (LibRaw::capabilities() & LIBRAW_CAPS_UNICODEPATHS) on runtime
  to check that this call was really implemented (or check for #ifdef \ 
LIBRAW_WIN32_UNICODEPATHS after #include <libraw.h>)

== LibRaw*datastream simplified ==

  * tempbuffer_open, subfile_open are not used, so removed from
  LibRaw_abstract_datastream and derived classes.

  * jpeg_src() call implemented using ->read() call and own buffering
  (16k buffer).
  * buffering_off() call added. It should be used in derived classes
    to switch from buffered reads to unbuffered.

== minor/unsorted changes ==
   * new flag LIBRAW_WARN_DNGSDK_PROCESSED to indicate decoder used
   *  LibRaw::open() call, max_buf_size special meaning:
       == 1 => open using bigfile_datastream
       == 2 => open using file_datastream
   * Add support for zlib during configure
   * Fixed multiple problems found by OSS-Fuzz
   * Lots of changes in imgdata.makernotes (hope someone will document it)
   * DNG SDK could be used (if enabled) to unpack multi-image DNG files.
   * DNG whitelevel calculated via BitsPerSample if not set via tags.
   * DNG: support for LinearDNG w/ BlackLevelRepeat.. pattern
   * Generic Arri camera format replaced w/ list of specific camera models in \ 
supported cameras list.
   * new samples/rawtextdump sample: allows one to dump (small selection) of RAW \ 
data in text format.
   * samples/raw-identify:
     * +M/-M params (same as in dcraw_emu)
     * -L <file-w-filelist> parameter to get file list from a file
     * -m paramerer to use mmap'ed IO.
     * -t parameter for timing
   * samples/dcraw_emu: fixed +M handling
   * better support for Nikon Coolscan 16-bit NEF files.
   * Visual Studio project files: re-generated to .vcxproj (Visual Studio 2019), \ 
different
     intermediate folders for different sub-projects to allow 1-step rebuild.
   * imgdata.makernotes...cameraspecific: removed the vendor name prefix from \ 
variables.
   * Bayer images: ensure that even margins have the same COLOR() for both the \ 
active sensor area and the full sensor area.
   * raw processing flag bit LIBRAW_PROCESSING_CHECK_DNG_ILLUMINANT inverted and \ 
renamed to
     LIBRAW_PROCESSING_DONT_CHECK_DNG_ILLUMINANT. If not set, DNG illuminant \ 
will be checked.
   * New libraw_decoder_t flags:
      LIBRAW_DECODER_FLATDATA - in-file data could be used as is (if byte order \ 
matches), e.g. via mmap()
      LIBRAW_DECODER_FLAT_BG2_SWAPPED - special flag for Sony ARQ: indicates \ 
R-G-G2-B channel
                                        order in 4-color data
   * Camera-recorded image crop data is parsed into imgdata.sizes.raw_inset_crop \ 
structure:
	ctop,cleft,cwidth,cheight - crop size.
	aspect - LibRawImageAspects enum (3to2, 4to3, etc)
   * New define LIBRAW_NO_WINSOCK2 to not include winsock2.h on compile

   * New processing flag LIBRAW_PROCESSING_PROVIDE_NONSTANDARD_WB
   If set (default is not), and when applicable, color.cam_mul[] and
   color.WB_Coeffs/WBCT_Coeffs will contain WB settings for a non-standard
   workflow.
   Right now only Sony DSC-F828 is affected: camera-recorded white balance
   can't be directly applied to raw data because WB is for RGB, while raw
   data is RGBE.

   * New processing flag: LIBRAW_PROCESSING_CAMERAWB_FALLBACK_TO_DAYLIGHT
   If set (default is not), LibRaw::dcraw_process() will fallback to
   daylight WB (excluding some very specific cases like Canon D30).
   This is how LibRaw 0.19 (and older) works.
   If not set: LibRaw::dcraw_process() will fallback to calculated auto WB if
   camera WB is requested, but appropriate white balance was not found in
   metadata.

  * Google changes cherry-picked (thanks to Jamie Pinheiro)
  * speedup: ppg interpolate: const loop invariant

  * Bugs fixed
   -Fixed several UBs found by OSS Fuzz
   -Fixed several problems found by other fuzzers.
   - Thumbnail size range check (CVE-2020-15503)
    Thanks to Jennifer Gehrke of Recurity Labs GmbH for problem report.
   - fixed possible overflows in canon and sigma makernotes parsers
   - fixed possible buffer overrun in crx (cr3) decoder
   - fixed memory leak in crx decoder (if compiled with LIBRAW_NO_CR3_MEMPOOL)
   - fixed possible overrun in Sony SRF and SR2 metadata parsers

  * Fixed typo in longitude (member of parsed GPS structure), update required \ 
for code that uses it.
   2021-09-19 20:46:50 by Nia Alarie | Files touched by this commit (4)
Log message:
libraw: apply patch for CVE-2020-15503
   2020-05-20 08:09:10 by Roland Illig | Files touched by this commit (52)
Log message:
mark packages that fail with -Werror=char-subscripts

These packages are susceptible to bugs when confronted with non-ASCII
characters.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94182.

It takes some time to analyze and fix these individually, therefore they
are only marked as "needs work".