Path to this page:
./
converters/py-charset-normalizer,
Universal Charset Detector
Branch: CURRENT,
Version: 3.4.0,
Package name: py312-charset-normalizer-3.4.0,
Maintainer: pkgsrc-usersA library that helps you read text from an unknown charset encoding.
Master sites:
Filesize: 104.121 KB
Version history: (Expand)
- (2024-10-10) Updated to version: py312-charset-normalizer-3.4.0
- (2023-11-01) Updated to version: py311-charset-normalizer-3.3.2
- (2023-10-23) Updated to version: py311-charset-normalizer-3.3.1
- (2023-09-30) Updated to version: py311-charset-normalizer-3.3.0
- (2023-07-08) Updated to version: py310-charset-normalizer-3.2.0
- (2023-04-24) Updated to version: py310-charset-normalizer-3.1.0
CVS history: (Expand)
2023-11-01 10:14:56 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 3.3.2
3.3.2
Fixed
- Unintentional memory usage regression when using large payload that match \
several encoding
- Regression on some detection case showcased in the documentation
|
2023-10-23 09:56:04 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 3.3.1
3.3.1
Changed
- Optional mypyc compilation upgraded to version 1.6.1 for Python >= 3.8
- Improved the general detection reliability based on reports from the community
|
2023-09-30 19:16:30 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message:
py-charset-normalizer: updated to 3.3.0
3.3.0
Added
- Allow to execute the CLI (e.g. normalizer) through `python -m \
charset_normalizer.cli` or `python -m charset_normalizer`
- Support for 9 forgotten encoding that are supported by Python but unlisted in \
`encoding.aliases` as they have no alias
Removed
- (internal) Redundant utils.is_ascii function and unused function \
is_private_use_only
- (internal) charset_normalizer.assets is moved inside charset_normalizer.constant
Changed
- (internal) Unicode code blocks in constants are updated using the latest \
v15.0.0 definition to improve detection
- Optional mypyc compilation upgraded to version 1.5.1 for Python >= 3.7
Fixed
- Unable to properly sort CharsetMatch when both chaos/noise and coherence were \
close due to an unreachable condition in \_\_lt\_\_
|
2023-07-08 06:35:31 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 3.2.0
3.2.0
Changed
- Typehint for function `from_path` no longer enforce `PathLike` as its first \
argument
- Minor improvement over the global detection reliability
Added
- Introduce function `is_binary` that relies on main capabilities, and optimized \
to detect binaries
- Propagate `enable_fallback` argument throughout `from_bytes`, `from_path`, and \
`from_fp` that allow a deeper control over the detection (default True)
- Explicit support for Python 3.12
Fixed
- Edge case detection failure where a file would contain 'very-long' camel cased word
|
2023-04-24 12:30:04 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 3.1.0
3.1.0
Added
- Argument `should_rename_legacy` for legacy function `detect` and disregard any \
new arguments without errors
Removed
- Support for Python 3.6
Changed
- Optional speedup provided by mypy/c 1.0.1
|
2022-11-18 19:50:29 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 3.0.1
3.0.1 (2022-11-18)
Fixed
Multi-bytes cutter/chunk generator did not always cut correctly
Changed
Speedup provided by mypy/c 0.990 on Python >= 3.7
3.0.0 (2022-10-20)
Added
Extend the capability of explain=True when cp_isolation contains at most two \
entries (min one), will log in details of the Mess-detector results
Support for alternative language frequency set in \
charset_normalizer.assets.FREQUENCIES
Add parameter language_threshold in from_bytes, from_path and from_fp to adjust \
the minimum expected coherence ratio
normalizer --version now specify if current version provide extra speedup \
(meaning mypyc compilation whl)
Changed
Build with static metadata using 'build' frontend
Make the language detection stricter
Optional: Module md.py can be compiled using Mypyc to provide an extra speedup \
up to 4x faster than v2.1
Fixed
CLI with opt --normalize fail when using full path for files
TooManyAccentuatedPlugin induce false positive on the mess detection when too \
few alpha character have been fed to it
Sphinx warnings when generating the documentation
Removed
Coherence detector no longer return 'Simple English' instead return 'English'
Coherence detector no longer return 'Classical Chinese' instead return 'Chinese'
Breaking: Method first() and best() from CharsetMatch
UTF-7 will no longer appear as "detected" without a recognized \
SIG/mark (is unreliable/conflict with ASCII)
Breaking: Class aliases CharsetDetector, CharsetDoctor, CharsetNormalizerMatch \
and CharsetNormalizerMatches
Breaking: Top-level function normalize
Breaking: Properties chaos_secondary_pass, coherence_non_latin and w_counter \
from CharsetMatch
Support for the backport unicodedata2
|
2022-09-14 13:10:00 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 2.1.1
2.1.1
Deprecated
- Function `normalize` scheduled for removal in 3.0
Changed
- Removed useless call to decode in fn is_unprintable
Fixed
- Third-party library (i18n xgettext) crashing not recognizing utf_8 (PEP 263) \
with underscore
|
2022-08-05 15:59:38 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-charset-normalizer: updated to 2.1.0
2.1.0 (2022-06-19)
Added
Output the Unicode table version when running the CLI with --version
Changed
Re-use decoded buffer for single byte character sets
Fixing some performance bottlenecks
Fixed
Workaround potential bug in cpython with Zero Width No-Break Space located in \
Arabic Presentation Forms-B, Unicode 1.1 not acknowledged as space
CLI default threshold aligned with the API threshold
Removed
Support for Python 3.5
Deprecated
Use of backport unicodedata from unicodedata2 as Python is quickly catching up, \
scheduled for removal in 3.0
|