Subject: CVS commit: pkgsrc/devel/py-cython
From: Thomas Klausner
Date: 2023-11-13 11:27:15
Message id: 20231113102715.BE8DCFA3D@cvs.NetBSD.org

Log Message:
py-cython: update to 3.0.5.

3.0.5 (2023-10-31)
==================

Features added
--------------

* Preliminary support for CPython 3.13a1 was added to allow early testing.
  (Github issue :issue:`5767`)

Bugs fixed
----------

* A compiler crash was fixed.
  (Github issue :issue:`5771`)

* A typo in the ``always_allow_keywords`` directive for Python code was fixed.
  Patch by lk-1984.  (Github issue :issue:`5772`)

* Some C compiler warnings were resolved.
  Patch by Pierre Jolivet.  (Github issue :issue:`5780`)

3.0.4 (2023-10-17)
==================

Features added
--------------

* A new compiler directive ``show_performance_hints`` was added to disable the
  newly added performance hint output.
  (Github issue :issue:`5748`)

Bugs fixed
----------

* ```cythonize` required ``distutils`` even for operations that did not build \ 
binaries.
  (Github issue :issue:`5751`)

* A regression in 3.0.3 was fixed that prevented calling inline functions
  from another inline function in ``.pxd`` files.
  (Github issue :issue:`5748`)

* Some C compiler warnings were resolved.
  Patch by Pierre Jolivet.  (Github issue :issue:`5756`)

3.0.3 (2023-10-05)
==================

Features added
--------------

* More warnings were added to help users migrate and avoid bugs.
  (Github issue :issue:`5650`)

* A warning-like category for performance hints was added that bypasses ``-Werror``.
  (Github issue :issue:`5673`)

* FastGIL now uses standard ``thread_local`` in C++.
  (Github issue :issue:`5640`)

* ``reference_wrapper`` was added to ``libcpp.functional``.
  Patch by Vyas Ramasubramani.  (Github issue :issue:`5671`)

* The ``cythonize`` command now supports the ``--cplus`` option known from the \ 
``cython`` command.
  (Github issue :issue:`5736`)

Bugs fixed
----------

* Performance regressions where the GIL was needlessly acquired were fixed.
  (Github issues :issue:`5670`, :issue:`5700`)

* A reference leak for exceptions in Python 3.12 was resolved.
  Patch by Eric Johnson.  (Github issue :issue:`5724`)

* ``fastcall`` calls with keyword arguments generated incorrect C code.
  (Github issue :issue:`5665`)

* Assigning the type converted result of a conditional (if-else) expression
  to ``int`` or ``bool`` variables could lead to incorrect C code.
  (Github issue :issue:`5731`)

* Early (unlikely) failures in Python function wrappers no longer set a
  traceback in order to simplify the C code flow.  Being mostly memory
  allocation errors, they probably would never have created a traceback anyway.
  (Github issue :issue:`5681`)

* Relative cimports from packages with ``__init__.py`` files could fail.
  (Github issue :issue:`5715`)

* Several issues with the Limited API support were resolved.
  (Github issues :issue:`5641`, :issue:`5648`, :issue:`5689`)

* The code generated for special-casing both Cython functions and PyCFunctions \ 
was cleaned up
  to avoid calling C-API functions that were not meant for the other type \ 
respectively.
  This could previously trigger assertions in CPython debug builds and now also \ 
plays better
  with the Limited API.
  (Github issues :issue:`4804`, :issue:`5739`)

* Fix some C compiler warnings.
  Patches by Ralf Gommers, Oleksandr Pavlyk, Sebastian Koslowski et al.
  (Github issues :issue:`5651`, :issue:`5663`, :issue:`5668`, :issue:`5717`, \ 
:issue:`5726`, :issue:`5734`)

* Generating gdb debugging information failed when using generator expressions.
  Patch by Oleksandr Pavlyk.  (Github issue :issue:`5552`)

* Passing a ``setuptools.Extension`` into ``cythonize()`` instead of a
  ``distutils.Extension`` could make it miss the matching extensions.

* ``cython -M`` needlessly required ``distutils``, which made it fail in Python 3.12.
  (Github issue :issue:`5681`)

Other changes
-------------

* The visible deprecation warning for ``DEF`` was removed again since it proved
  difficult for some users to migrate away from it.  The statement is still
  meant to be removed at some point (and thus, like ``IF``, should not be
  used in new code), but the time for sunset is probably not around the corner.
  (Github issue :issue:`4310`)

* The ``np_pythran`` option raise a ``DeprecationWarning`` if it receives other \ 
values
  than ``True`` and ``False``.  This will eventually be disallowed (in line with all
  other boolean options).

3.0.2 (2023-08-27)
==================

Bugs fixed
----------

* Using ``None`` as default value for arguments annotated as ``int`` could crash \ 
Cython.
  (Github issue :issue:`5643`)

* Default values of fused types that include ``complex`` could generate invalid \ 
C code
  with ``-DCYTHON_CCOMPLEX=0``.
  (Github issue :issue:`5644`)

* Using C++ enum class types in extension type method signatures could generate \ 
invalid C code.
  (Github issue :issue:`5637`)

3.0.1 (2023-08-25)
==================

Features added
--------------

* The error messages regarding exception declarations were improved in order to give
  better help about possible reasons and fixes.
  (Github issue :issue:`5547`)

Bugs fixed
----------

* Memory view types in Python argument annotations no longer accept ``None``.  \ 
They now
  require an explicit ``Optional[]`` or a ``None`` default value in order to \ 
allow ``None``
  to be passed.  This was an oversight in the 3.0.0 release and is a BACKWARDS \ 
INCOMPATIBLE
  change.  However, since it only applies to code using Python syntax, it \ 
probably only
  applies to newly written code that was written for Cython 3.0 and can easily \ 
be adapted.
  In most cases, we expect that this change will avoid bugs in user code rather than
  produce problems.
  (Github issue :issue:`5612`)

* ``nogil`` functions using parallel code could freeze when called with the GIL held.
  (Github issues :issue:`5564`, :issue:`5573`)

* Relative cimports could end up searching globally and find the same package \ 
installed
  elsewhere, potentially in another version.
  (Github issue :issue:`5511`)

* Attribute lookups on known standard library modules could accidentally search
  in the module namespace instead.
  (Github issue :issue:`5536`)

* Using constructed C++ default arguments could generate invalid C++ code.
  (Github issue :issue:`5553`)

* ``libcpp.memory.make_unique()`` was lacking C++ exception handling.
  (Github issue :issue:`5560`)

* Some non-public and deprecated CAPI usages were replaced by public
  (and thus more future proof) API code.

* Many issues with the Limited API support were resolved.
  Patches by Lisandro Dalcin et al.
  (Github issues :issue:`5549`, :issue:`5550`, :issue:`5556`, :issue:`5605`, \ 
:issue:`5617`)

* Some C compiler warnings were resolved.
  Patches by Matti Picus et al.  (Github issues :issue:`5557`, :issue:`5555`)

* Large Python integers are now stored in hex instead of decimal strings to work \ 
around
  security limits in Python and generally speed up their Python object creation.

* ``NULL`` could not be used as default for fused type pointer arguments.
  (Github issue :issue:`5554`)

* C functions that return pointer types now return ``NULL`` as default exception \ 
value.
  Previously, calling code wasn't aware of this and always tested for raised \ 
exceptions.
  (Github issue :issue:`5554`)

* Untyped literal default arguments in fused functions could generate invalid C code.
  (Github issue :issue:`5614`)

* C variables declared as ``const`` could generate invalid C code when used in \ 
closures,
  generator expressions, ctuples, etc.
  (Github issues :issue:`5558`,  :issue:`5333`)

* Enums could not refer to previously defined enums in their definition.
  (Github issue :issue:`5602`)

* The Python conversion code for anonymous C enums conflicted with regular int \ 
conversion.
  (Github issue :issue:`5623`)

* Using memory views for property methods (and other special methods) could lead to
  refcounting problems.
  (Github issue :issue:`5571`)

* Star-imports could generate code that tried to assign to constant C macros like
  ``PY_SSIZE_T_MAX`` and ``PY_SSIZE_T_MIN``.
  Patch by Philipp Wagner.  (Github issue :issue:`5562`)

* ``CYTHON_USE_TYPE_SPECS`` can now be (explicitly) enabled in PyPy.

* The template parameter "delimeters" in the Tempita ``Template`` \ 
class was corrected
  to "delimiters".  The old spelling is still available in the main \ 
template API but
  now issues a ``DeprecationWarning``.
  (Github issue :issue:`5608`)

* The ``cython --version`` output is now  less likely to reach both stdout and \ 
stderr.
  Patch by Eli Schwartz.  (Github issue :issue:`5504`)

* The sdist was missing the `Shadow.pyi` stub file.

3.0.0 unified release notes
===========================

Cython 3.0.0 has been a very large effort that cleaned up many old warts,
introduced many new features, and introduces a couple of intentional
behaviour changes, even though the goal remained to stay compatible as
much as possible with Cython 0.29.x. For details, see the `migration guide`_.

.. _`migration guide`: \ 
https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html

As the development was spread out over several years, a lot of things have
happened in the meantime. Many crucial bugfixes and some features were
backported to 0.29.x and are not strictly speaking "new" in Cython 3.0.0.

Major themes in 3.0.0
=====================

Compatibility with CPython and the Python C API
-----------------------------------------------

Since Cython 3.0.0 started development, CPython 3.8-3.11 were released.
All these are supported in Cython, including experimental support for the
in-development CPython 3.12. On the other end of the spectrum, support for
Python 2.6 was dropped.

Cython interacts very closely with the C-API of Python, which is where most
of the adaptation work happens.

Related changes
^^^^^^^^^^^^^^^

* The long deprecated include files ``python_*``, ``stdio``, ``stdlib`` and
  ``stl`` in ``Cython/Includes/Deprecated/`` were removed.  Use the ``libc.*``
  and ``cpython.*`` pxd modules instead.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2904`)

* The ``Py_hash_t`` type failed to accept arbitrary "index" values.
  (Github issue :issue:`2752`)

* ``@cython.trashcan(True)`` can be used on an extension type to enable the
  CPython :ref:`trashcan`. This allows deallocating deeply recursive objects
  without overflowing the stack. Patch by Jeroen Demeyer.  (Github issue \ 
:issue:`2842`)

* ``PyEval_InitThreads()`` is no longer used in Py3.7+ where it is a no-op.

* A low-level inline function ``total_seconds(timedelta)`` was added to
  ``cpython.datetime`` to bypass the Python method call.  Note that this function
  is not guaranteed to give exactly the same results for very large time intervals.
  Patch by Brock Mendel.  (Github issue :issue:`3616`)

* The internal CPython macro ``Py_ISSPACE()`` is no longer used.
  Original patch by Andrew Jones.  (Github issue :issue:`4111`)

* The value ``PyBUF_MAX_NDIM`` was added to the ``cpython.buffer`` module.
  Patch by John Kirkham.  (Github issue :issue:`3811`)

* A new module ``cpython.time`` was added with some low-level alternatives to
  Python's ``time`` module.
  Patch by Brock Mendel.  (Github issue :issue:`3767`)

* More C-API declarations for ``cpython.datetime``  were added.
  Patch by Bluenix2.  (Github issue :issue:`4128`)

* C-API declarations for context variables in Python 3.7 were added.
  Original patch by Zolisa Bleki.  (Github issue :issue:`2281`)

* C-API declarations for ``cpython.fileobject`` were added.
  Patch by Zackery Spytz.  (Github issue :issue:`3906`)

* The signature of ``PyFloat_FromString()`` in ``cpython.float`` was changed
  to match the signature in Py3.  It still has an automatic fallback for Py2.
  (Github issue :issue:`3909`)

* ``PyMem_[Raw]Calloc()`` was added to the ``cpython.mem`` declarations.
  Note that the ``Raw`` versions are no longer #defined by Cython.  The previous
  macros were not considered safe.
  Patch by William Schwartz and David Woods.  (Github issue :issue:`3047`)

* The runtime size check for imported ``PyVarObject`` types was improved
  to reduce false positives and adapt to Python 3.11.
  Patch by David Woods.  (Github issues :issue:`4827`, :issue:`4894`)

* The generated C code failed to compile in CPython 3.11a4 and later.
  (Github issue :issue:`4500`)

* ``pyximport`` no longer uses the deprecated ``imp`` module.
  Patch by Matúš Valo.  (Github issue :issue:`4560`)

* Improvements to ``PyTypeObject`` definitions in pxd wrapping of libpython.
  Patch by John Kirkham. (Github issue :issue:`4699`)

* Some old usages of the deprecated Python ``imp`` module were replaced with \ 
``importlib``.
  Patch by Matúš Valo.  (Github issue :issue:`4640`)

* ``cpdef`` enums no longer use ``OrderedDict`` but ``dict`` in Python 3.6 and later.
  Patch by GalaxySnail.  (Github issue :issue:`5180`)

* Several problems with CPython 3.12 were resolved.
  (Github issue :issue:`5238`)

* The exception handling code was adapted to CPython 3.12.
  (Github issue :issue:`5442`)

* The Python ``int`` handling code was adapted to make use of the new ``PyLong``
  internals in CPython 3.12.
  (Github issue :issue:`5353`)

* A compile error when using ``__debug__`` was resolved.

* The deprecated ``_PyGC_FINALIZED()`` C-API macro is no longer used.
  Patch by Thomas Caswell and Matúš Valo.  (Github issue :issue:`5481`)

* A crash in Python 2.7 was fixed when cleaning up extension type instances
  at program end.

Compatibility with other Python implementations
-----------------------------------------------

Cython tries to support other Python implementations, largely on a best-effort
basis. The most advanced support exists for PyPy, which is tested in our CI
and considered supported.

Related changes
^^^^^^^^^^^^^^^

* An unsupported C-API call in PyPy was fixed.
  Patch by Max Bachmann.  (Github issue :issue:`4055`)

* Support for the now unsupported Pyston V1 was removed in favour of Pyston V2.
  Patch by Marius Wachtler.  (Github issue :issue:`4211`)

* A C compiler warning in PyPy3 regarding ``PyEval_EvalCode()`` was resolved.

* Some compatibility issues with PyPy were resolved.
  Patches by Max Bachmann, Matti Picus.
  (Github issues :issue:`4454`, :issue:`4477`, :issue:`4478`, :issue:`4509`, \ 
:issue:`4517`)

* An initial set of adaptations for GraalVM Python was implemented.  Note that
  this does not imply any general support for this target or that your code
  will work at all in this environment.  But testing should be possible now.
  Patch by David Woods.  (Github issue :issue:`4328`)

* A work-around for StacklessPython < 3.8 was disabled in Py3.8 and later.
  (Github issue :issue:`4329`)

Initial support for Limited API
-------------------------------

CPython provides a stable, limited subset of its C-API as the so-called Limited API.
This C-API comes with the guarantee of a stable ABI, meaning that extensions modules
that were compiled for one version of CPython can also be imported in later versions
without recompilation.

There is initial support for this in Cython.  By defining the ``CYTHON_LIMITED_API``
macro, Cython cuts down its C-API usage and tries to adhere to the Limited C-API,
probably at the cost of a bit of performance.
In order to get full benefit from the limited API you will also need to define the
CPython macro ``Py_LIMITED_API`` to a specific CPython compatibility version,
which additionally restricts the C-API during the C compilation,
thus enforcing the forward compatibility of the extension module.

Note that "initial support" in Cython really means that setting the \ 
``Py_LIMITED_API``
macro will almost certainly not yet work for your specific code.
There are limitations in the Limited C-API
that are difficult for Cython to generate C code for, so some advanced Python \ 
features
(like async code) may not lead to C code that cannot adhere to the Limited C-API, or
where Cython simply does not know yet how to adhere to it.  Basically, if you \ 
get your
code to compile with both macros set, and it passes your test suite, then it \ 
should be
possible to import the extension module also in later CPython versions.

The experimental feature flags ``CYTHON_USE_MODULE_STATE`` and
``CYTHON_USE_TYPE_SPECS`` enable some individual aspects of the Limited API
implementation independently.

Related changes
^^^^^^^^^^^^^^^

* Preliminary support for the CPython's ``Py_LIMITED_API`` (stable ABI) is
  available by setting the  ``CYTHON_LIMITED_API`` C macro.  Note that the
  support is currently in an early stage and many features do not yet work.
  You currently still have to define ``Py_LIMITED_API`` externally in order
  to restrict the API usage.  This will change when the feature stabilises.
  Patches by Eddie Elizondo and David Woods.  (Github issues :issue:`3223`,
  :issue:`3311`, :issue:`3501`)

* Limited API support was improved.
  Patches by Matthias Braun.  (Github issues :issue:`3693`, :issue:`3707`)

* New C feature flags: ``CYTHON_USE_MODULE_STATE``, ``CYTHON_USE_TYPE_SPECS``
  Both are currently considered experimental.
  (Github issue :issue:`3611`)

* ``_Py_TPFLAGS_HAVE_VECTORCALL`` was always set on extension types when using \ 
the limited API.
  Patch by David Woods.  (Github issue :issue:`4453`)

* Limited API C preprocessor warning is compatible with MSVC. Patch by
  Victor Molina Garcia.  (Github issue :issue:`4826`)

* The embedding code no longer calls deprecated C-API functions but uses the new \ 
``PyConfig``
  API instead on CPython versions that support it (3.8+).
  Patch by Alexander Shadchin.  (Github issue :issue:`4895`)

* Some C code issue were resolved for the Limited API target.
  (Github issues :issue:`5264`, :issue:`5265`, :issue:`5266`)

* Conversion of Python ints to C ``int128`` is now always supported, although slow
  if dedicated C-API support is missing (``_PyLong_AsByteArray()``), specifically in
  the Limited C-API.
  (Github issue :issue:`5419`)

* Custom buffer slot methods are now supported in the Limited C-API of Python 3.9+.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5422`)

Improved fidelity to Python semantics
-------------------------------------

Implemented PEPs
^^^^^^^^^^^^^^^^

* `PEP-3131`_: Supporting Non-ASCII Identifiers (Github issue :issue:`2601`)
* `PEP-479`_: `generator_stop` (enabled by default for `language_level=3`) \ 
(Github issue :issue:`2580`)
* `PEP-487`_: Simpler customisation of class creation (Github issue :issue:`2781`)
* `PEP-563`_: Postponed Evaluation of Annotations (Github issue :issue:`3285`)
* `PEP-570`_: Positional-Only Parameters (Github issue :issue:`2915`)
* `PEP-572`_: Assignment Expressions (a.k.a. the walrus operator `:=`) (Github \ 
issue :issue:`2636`)
* `PEP-590`_: Vectorcall protocol (Github issue :issue:`2263`)
* `PEP-614`_: Relaxing Grammar Restrictions On Decorators (Github issue \ 
:issue:`4570`)

Typing support in the sense of `PEP-484`_ (Github issues :issue:`3949`, \ 
:issue:`4243`)
and `PEP-560`_ (Github issues :issue:`2753`, :issue:`3537`, :issue:`3764`) was \ 
also improved.

.. _`PEP-3131`: https://www.python.org/dev/peps/pep-3131
.. _`PEP-479`: https://www.python.org/dev/peps/pep-0479
.. _`PEP-484`: https://www.python.org/dev/peps/pep-0484
.. _`PEP-487`: https://www.python.org/dev/peps/pep-0487
.. _`PEP-560`: https://www.python.org/dev/peps/pep-0560
.. _`PEP-563`: https://www.python.org/dev/peps/pep-0563
.. _`PEP-570`: https://www.python.org/dev/peps/pep-0570
.. _`PEP-572`: https://www.python.org/dev/peps/pep-0572
.. _`PEP-590`: https://www.python.org/dev/peps/pep-0590
.. _`PEP-614`: https://www.python.org/dev/peps/pep-0614

The default language level was changed to ``3str``, i.e. Python 3 semantics,
but with ``str`` literals (also in Python 2.7).  This is a backwards incompatible
change from the previous default of Python 2 semantics.  The previous behaviour
is available through the directive ``language_level=2``.
(Github issue :issue:`2565`).  This covers changes such as using the
``print``-function instead of the ``print``-statement, and integer-integer
division giving a floating point answer. Most of these changes were available
in earlier versions of Cython but are now the default.

Cython 3.0.0 also aligns its own language semantics more closely with Python, in \ 
particular:

* the power operator has changed to give a result matching what Python does \ 
rather than
  keeping the same types as the input (as in C),
* operator overloading of ``cdef classes`` behaves much more like Python classes,
* Cython's behaviour when using type annotations aligns more closely with their
  standard use in Python.

Related changes
^^^^^^^^^^^^^^^

* Cython no longer generates ``__qualname__`` attributes for classes in Python
  2.x since they are problematic there and not correctly maintained for subclasses.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2772`)

* Binding staticmethods of Cython functions were not behaving like Python methods.
  Patch by Jeroen Demeyer.  (Github issue :issue:`3106`, :issue:`3102`)

* Compiling package ``__init__`` files could fail under Windows due to an
  undefined export symbol.  (Github issue :issue:`2968`)

* ``__init__.pyx`` files were not always considered as package indicators.
  (Github issue :issue:`2665`)

* Setting ``language_level=2`` in a file did not work if ``language_level=3``
  was enabled globally before.
  Patch by Jeroen Demeyer.  (Github issue :issue:`2791`)

* ``__doc__`` was not available inside of the class body during class creation.
  (Github issue :issue:`1635`)

* The first function line number of functions with decorators pointed to the
  signature line and not the first decorator line, as in Python.
  Patch by Felix Kohlgrüber.  (Github issue :issue:`2536`)

* Pickling unbound methods of Python classes failed.
  Patch by Pierre Glaser.  (Github issue :issue:`2972`)

* Item access (subscripting) with integer indices/keys always tried the
  Sequence protocol before the Mapping protocol, which diverged from Python
  semantics.  It now passes through the Mapping protocol first when supported.
  (Github issue :issue:`1807`)

* Nested dict literals in function call kwargs could incorrectly raise an
  error about duplicate keyword arguments, which are allowed when passing
  them from dict literals.
  (Github issue :issue:`2963`)

* Diverging from the usual behaviour, ``len(memoryview)``, ``len(char*)``
  and ``len(Py_UNICODE*)`` returned an unsigned ``size_t`` value.  They now
  return a signed ``Py_ssize_t``, like other usages of ``len()``.

* The unicode methods ``.upper()``, ``.lower()`` and ``.title()`` were
  incorrectly optimised for single character input values and only returned
  the first character if multiple characters should have been returned.
  They now use the original Python methods again.

* The ``cython.view.array`` type supports inheritance.
  Patch by David Woods.  (Github issue :issue:`3413`)

* The builtin ``abs()`` function can now be used on C numbers in nogil code.
  Patch by Elliott Sales de Andrade.  (Github issue :issue:`2748`)

* The attributes ``gen.gi_frame`` and ``coro.cr_frame`` of Cython compiled
  generators and coroutines now return an actual frame object for introspection.
  (Github issue :issue:`2306`)

* Inlined properties can be defined for external extension types.
  Patch by Matti Picus. (Github issue :issue:`2640`, redone later in :issue:`3571`)

* Unicode module names and imports are supported.
  Patch by David Woods.  (Github issue :issue:`3119`)

* ``__arg`` argument names in methods were not mangled with the class name.
  Patch by David Woods.  (Github issue :issue:`1382`)

* With ``language_level=3/3str``, Python classes without explicit base class
  are now new-style (type) classes also in Py2.  Previously, they were created
  as old-style (non-type) classes.
  (Github issue :issue:`3530`)

* Conditional blocks in Python code that depend on ``cython.compiled`` are
  eliminated at an earlier stage, which gives more freedom in writing
  replacement Python code.
  Patch by David Woods.  (Github issue :issue:`3507`)

* Python private name mangling now falls back to unmangled names for non-Python
  globals, since double-underscore names are not uncommon in C.  Unmangled Python
  names are also still found as a legacy fallback but produce a warning.
  Patch by David Woods.  (Github issue :issue:`3548`)

* The ``print`` statement (not the ``print()`` function) is allowed in
  ``nogil`` code without an explicit ``with gil`` section.

* ``repr()`` was assumed to return ``str`` instead of ``unicode`` with \ 
``language_level=3``.
  (Github issue :issue:`3736`)

* Type inference now understands that ``a, *b = x`` assigns a list to ``b``.

* No/single argument functions now accept keyword arguments by default in order
  to comply with Python semantics.  The marginally faster calling conventions
  ``METH_NOARGS`` and ``METH_O`` that reject keyword arguments are still available
  with the directive ``@cython.always_allow_keywords(False)``.
  (Github issue :issue:`3090`)

* Special methods for binary operators now follow Python semantics.
  Rather than e.g. a single ``__add__`` method for cdef classes, where
  "self" can be either the first or second argument, one can now define
  both ``__add__`` and ``__radd__`` as for standard Python classes.
  This behavior can be disabled with the ``c_api_binop_methods`` directive
  to return to the previous semantics in Cython code (available from Cython
  0.29.20), or the reversed method (``__radd__``) can be implemented in
  addition to an existing two-sided operator method (``__add__``) to get a
  backwards compatible implementation.
  (Github issue :issue:`2056`)

* Generator expressions in pxd-overridden ``cdef`` functions could
  fail to compile.
  Patch by Matúš Valo.  (Github issue :issue:`3477`)

* Calls to ``.__class__()`` of a known extension type failed.
  Patch by David Woods.  (Github issue :issue:`3954`)

* Structs could not be instantiated with positional arguments in
  pure Python mode.

* Annotations were not exposed on annotated (data-)classes.
  Patch by matsjoyce.  (Github issue :issue:`4151`)

* Docstrings of ``cpdef`` enums are now copied to the enum class.
  Patch by matham.  (Github issue :issue:`3805`)

* ``asyncio.iscoroutinefunction()`` now recognises coroutine functions
  also when compiled by Cython.
  Patch by Pedro Marques da Luz.  (Github issue :issue:`2273`)

* Self-documenting f-strings (``=``) were implemented.
  Patch by davfsa.  (Github issue :issue:`3796`)

* ``cython.array`` supports simple, non-strided views.
  (Github issue :issue:`3775`)

* Attribute annotations in Python classes are now ignored, because they are
  just Python objects in a dict (as opposed to the fields of extension types).
  Patch by David Woods.  (Github issues :issue:`4196`, :issue:`4198`)

* A warning was added when ``__defaults__`` or ``__kwdefaults__`` of Cython compiled
  functions were re-assigned, since this does not current have an effect.
  Patch by David Woods.  (Github issue :issue:`2650`)

* The ``self`` argument of static methods in .pxd files was incorrectly typed.
  Patch by David Woods.  (Github issue :issue:`3174`)

* Default values for memory views arguments were not properly supported.
  Patch by Corentin Cadiou.  (Github issue :issue:`4313`)

* Python object types were not allowed as ``->`` return type annotations.
  Patch by Matúš Valo.  (Github issue :issue:`4433`)

* The excess arguments in a for-in-range loop with more than 3 arguments to `range()`
  were silently ignored.
  Original patch by Max Bachmann. (Github issue :issue:`4550`)

* Unsupported decorators on cdef functions were not rejected in recent releases.
  Patch by David Woods.  (Github issue :issue:`4322`)

* Fused functions were binding unnecessarily, which prevented them from being \ 
pickled.
  Patch by David Woods.  (Github issue :issue:`4370`)

* Decorators on inner functions were not evaluated in the right scope.
  Patch by David Woods.  (Github issue :issue:`4367`)

* Cython did not type the ``self`` argument in special binary methods.
  Patch by David Woods.  (Github issue :issue:`4434`)

* Circular imports of compiled modules could fail needlessly even when the import
  could already be resolved from ``sys.modules``.
  Patch by Syam Gadde.  (Github issue :issue:`4390`)

* ``__del__(self)`` on extension types now maps to ``tp_finalize`` in Python 3.
  Original patch by ax487.  (Github issue :issue:`3612`)

* Reusing an extension type attribute name as a method name is now an error.
  Patch by 0dminnimda.  (Github issue :issue:`4661`)

* When using type annotations, ``func(x: list)`` or ``func(x: ExtType)`` (and other
  Python builtin or extension types) no longer allow ``None`` as input argument \ 
to ``x``.
  This is consistent with the normal typing semantics in Python, and was a \ 
common gotcha
  for users who did not expect ``None`` to be allowed as input.  To allow \ 
``None``, use
  ``typing.Optional`` as in ``func(x: Optional[list])``.  ``None`` is also \ 
automatically
  allowed when it is used as default argument, i.e. ``func(x: list = None)``.
  ``int`` and ``float`` are now also recognised in type annotations and restrict the
  value type at runtime.  They were previously ignored.
  Note that, for backwards compatibility reasons, the new behaviour does not \ 
apply when using
  Cython's C notation, as in ``func(list x)``.  Here, ``None`` is still allowed, \ 
as always.
  Also, the ``annotation_typing`` directive can now be enabled and disabled more \ 
finely
  within the module.
  (Github issues :issue:`2696`, :issue:`3883`, :issue:`4606`, :issue:`4669`, \ 
:issue:`4886`)

* The parser allowed some invalid spellings of ``...``.
  Patch by 0dminnimda.  (Github issue :issue:`4868`)

* The ``__self__`` attribute of fused functions reports its availability correctly
  with ``hasattr()``.  Patch by David Woods.
  (Github issue :issue:`4808`)

* Several optimised string methods failed to accept ``None`` as arguments to \ 
their options.
  Test patch by Kirill Smelkov.  (Github issue :issue:`4737`)

* Cython generators and coroutines now identify as ``CO_ASYNC_GENERATOR``,
  ``CO_COROUTINE`` and ``CO_GENERATOR`` accordingly.
  (Github issue :issue:`4902`)

* Memory views and the internal Cython array type now identify as \ 
``collections.abc.Sequence``.
  Patch by David Woods.  (Github issue :issue:`4817`)

* Context managers can be written in parentheses.
  Patch by David Woods.  (Github issue :issue:`4814`)

* Some parser issues were resolved.
  (Github issue :issue:`4992`)

* Unused ``**kwargs`` arguments did not show up in ``locals()``.
  (Github issue :issue:`4899`)

* Relative imports failed in compiled ``__init__.py`` package modules.
  Patch by Matúš Valo.  (Github issue :issue:`3442`)

* Extension types are now explicitly marked as immutable types to prevent them from
  being considered mutable.
  Patch by Max Bachmann.  (Github issue :issue:`5023`)

* ``int(Py_UCS4)`` returned the code point instead of the parsed digit value.
  (Github issue :issue:`5216`)

* Calling bound classmethods of builtin types could fail trying to call the \ 
unbound method.
  (Github issue :issue:`5051`)

* Generator expressions and comprehensions now look up their outer-most iterable
  on creation, as Python does, and not later on start, as they did previously.
  (Github issue :issue:`1159`)

* Bound C methods can now coerce to Python objects.
  (Github issues :issue:`4890`, :issue:`5062`)

* ``cpdef`` enums can now be pickled.
  (Github issue :issue:`5120`)

* The Python Enum of a ``cpdef enum`` now inherits from ``IntFlag`` to better match
  both Python and C semantics of enums.
  (Github issue :issue:`2732`)

* The special ``__*pow__`` methods now support the 2- and 3-argument variants.
  (Github issue :issue:`5160`)

* The ``**`` power operator now behaves more like in Python by returning the \ 
correct complex
  result if required by math.  A new ``cpow`` directive was added to turn on the \ 
previous
  C-like behaviour.
  (Github issue :issue:`4936`)

* With ``language_level=2``, imports of modules in packages could return the \ 
wrong module in Python 3.
  (Github issue :issue:`5308`)

* Function signatures containing a type like `tuple[()]` could not be printed.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5355`)

* ``__qualname__`` and ``__module__`` were not available inside of class bodies.
  (Github issue :issue:`4447`)

* A new directive ``embedsignature.format`` was added to select the format of the
  docstring embedded signatures between ``python``, ``c`` and argument ``clinic``.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5415`)

* ctuples can now be assigned from arbitrary sequences, not just Python tuples.

Improvements in Pure Python mode
--------------------------------

Cython strives to be able to
parse newer Python constructs for use with its `pure python`_ mode, which
has been a focus. In short, this allows to compile a wider range of Python
code into optimized C code.

.. _`pure python`: https://cython.readthedocs.io/en/latest/src/tutorial/pure.html

Pure python mode gained many new features and was generally overhauled to make
it as capable as the Cython syntax.  Except for using external C/C++ libraries,
it should now be possible to express all Cython code and use all features in
regular Python syntax.  The very few remaining exceptions or bugs are noted in
the documentation.

Additionally, the documentation has been substantially updated
(primarily by Matúš Valo and 0dminnimda) to show both the older Cython syntax
and pure Python syntax.

Related changes
^^^^^^^^^^^^^^^

* The ``cython.declare()`` and ``cython.cast()`` functions could fail in pure mode.
  Patch by Dmitry Shesterkin.  (Github issue :issue:`3244`)

* Fused argument types were not correctly handled in type annotations and
  ``cython.locals()``.
  Patch by David Woods.  (Github issues :issue:`3391`, :issue:`3142`)

* ``nogil`` functions now avoid acquiring the GIL on function exit if possible
  even if they contain ``with gil`` blocks.
  (Github issue :issue:`3554`)

* The ``@returns()`` decorator propagates exceptions by default for suitable C
  return types when no ``@exceptval()`` is defined.
  (Github issues :issue:`3625`, :issue:`3664`)

* Extension types inheriting from Python classes could not safely
  be exposed in ``.pxd``  files.
  (Github issue :issue:`4106`)

* Default arguments of methods were not exposed for introspection.
  Patch by Vladimir Matveev.  (Github issue :issue:`4061`)

* Literal list assignments to pointer variables declared in PEP-526
  notation failed to compile.

* The type ``cython.Py_hash_t`` is available in Python mode.

* A ``cimport`` is now supported in pure Python code by prefixing the
  imported module name with ``cython.cimports.``, e.g.
  ``from cython.cimports.libc.math import sin``.
  (GIthub issue :issue:`4190`)

* Directives starting with ``optimization.*`` in pure Python mode were \ 
incorrectly named.
  It should have been ``optimize.*``.
  Patch by David Woods.  (Github issue :issue:`4258`)

* Invalid and misspelled ``cython.*`` module names were not reported as errors.
  (Github issue :issue:`4947`)

* The ``annotation_typing`` directive was missing in pure Python mode.
  Patch by 0dminnimda.  (Github issue :issue:`5194`)

* Memoryviews with ``object`` item type were not supported in Python type \ 
declarations.
  (Github issue :issue:`4907`)

* Subscripted builtin types in type declarations (like ``list[float]``) are now
  better supported.
  (Github issue :issue:`5058`)

* Unknown type annotations (e.g. because of typos) now emit a warning at compile \ 
time.
  Patch by Matúš Valo.  (Github issue :issue:`5070`)

* ``typing.Optional`` could fail on tuple types.
  (Github issue :issue:`5263`)

* ``from cython cimport … as …`` could lead to imported names not being \ 
found in annotations.
  Patch by Chia-Hsiang Cheng.  (Github issue :issue:`5235`)

* Simple tuple types like ``(int, int)`` are no longer accepted in Python annotations
  and require the Python notation instead (e.g. ``tuple[cython.int, cython.int]``).
  (Github issue :issue:`5397`)

* The Python implementation of ``cimport cython.cimports…`` could raise an \ 
``ImportError``
  instead of an ``AttributeError`` when looking up package variable names.
  Patch by Matti Picus.  (Github issue :issue:`5411`)

* A new decorator ``@cython.with_gil`` is available in Python code to match the \ 
``with gil``
  function declaration in Cython syntax.

* ``with gil`` and ``with nogil(flag)`` now accept their flag argument also in \ 
Python code.
  Patch by Matúš Valo.  (Github issue :issue:`5113`)

Code generation changes
-----------------------

Cython has gained several major new features that speed up both the development
and the code. Dataclasses have gained an extension type equivalent that implements
the dataclass features in C code.  Similarly, the ``@functools.total_ordering``
decorator to an extension type will implement the comparison functions in C.

Finally, NumPy ufuncs can be generated from simple computation functions with the
new ``@cython.ufunc`` decorator.

Related changes
^^^^^^^^^^^^^^^

* ``with gil/nogil`` statements can be conditional based on compile-time
  constants, e.g. fused type checks.
  Patch by Noam Hershtig.  (Github issue :issue:`2579`)

* The names of Cython's internal types (functions, generator, coroutine, etc.)
  are now qualified with the module name of the internal Cython module that is
  used for sharing them across Cython implemented modules, for example
  ``_cython_3_0a5.coroutine``.  This was done to avoid making them look like
  homeless builtins, to help with debugging, and in order to avoid a CPython
  warning according to https://bugs.python.org/issue20204

* A ``@cython.total_ordering`` decorator has been added to automatically
  implement all comparison operators, similar to ``functools.total_ordering``.
  Patch by Spencer Brown.  (Github issue :issue:`2090`)

* A new decorator ``@cython.dataclasses.dataclass`` was implemented that provides
  compile time dataclass generation capabilities to ``cdef`` classes (extension \ 
types).
  Patch by David Woods.  (Github issue :issue:`2903`).  ``kw_only`` dataclasses
  added by Yury Sokov.  (Github issue :issue:`4794`)

* A new function decorator ``@cython.ufunc`` automatically generates a (NumPy) \ 
ufunc that
  applies the calculation function to an entire memoryview.
  (Github issue :issue:`4758`)

* Generated NumPy ufuncs could crash for large arrays due to incorrect GIL handling.
  (Github issue :issue:`5328`)

* Some invalid directive usages are now detected and rejected, e.g. using ``@ccall``
  together with ``@cfunc``, and applying ``@cfunc`` to a ``@ufunc``.  Cython also
  warns now when a directive is applied needlessly.
  (Github issue :issue:`5399` et al.)

* The normal ``@dataclasses.dataclass`` and ``@functools.total_ordering`` decorators
  can now be used on extension types.  Using the corresponding ``@cython.*`` \ 
decorator
  will automatically turn a Python class into an extension type (no need for \ 
``@cclass``).
  (Github issue :issue:`5292`)

Interaction with numpy
----------------------

The NumPy declarations (``cimport numpy``) were moved over to the NumPy project \ 
in order
to allow version specific changes on their side.

One effect is that Cython does not use deprecated NumPy C-APIs any more.  Thus, you
can define the respective NumPy C macro to get rid of the compatibility warning at
C compile time.

Related changes
^^^^^^^^^^^^^^^

* ``cython.inline()`` now sets the ``NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION``
  C macro automatically when ``numpy`` is imported in the code, to avoid C compiler
  warnings about deprecated NumPy C-API usage.

* Deprecated NumPy API usages were removed from ``numpy.pxd``.
  Patch by Matti Picus.  (Github issue :issue:`3365`)

* ``numpy.import_array()`` is automatically called if ``numpy`` has been cimported
  and it has not been called in the module code.  This is intended as a hidden
  fail-safe so user code should continue to call ``numpy.import_array``.
  Patch by David Woods.  (Github issue :issue:`3524`)

* The outdated getbuffer/releasebuffer implementations in the NumPy
  declarations were removed so that buffers declared as ``ndarray``
  now use the normal implementation in NumPy.

* Several macros/functions declared in the NumPy API are now usable without
  holding the GIL.

* The ``numpy`` declarations were updated.
  Patch by Brock Mendel.  (Github issue :issue:`3630`)

* ``ndarray.shape`` failed to compile with Pythran and recent NumPy.
  Patch by Serge Guelton.  (Github issue :issue:`3762`)

* A C-level compatibility issue with recent NumPy versions was resolved.
  Patch by David Woods.  (Github issue :issue:`4396`)

* The generated modules no longer import NumPy internally when using
  fused types but no memoryviews.
  Patch by David Woods.  (Github issue :issue:`4935`)

* ``np.long_t`` and ``np.ulong_t`` were removed from the NumPy declarations,
  syncing Cython with upstream NumPy v1.25.0.  The aliases were confusing
  since they could mean different things on different platforms.

Exception handling
------------------

Cython-implemented C functions now propagate exceptions by default, rather than
swallowing them in non-object returning function if the user forgot to add an
``except`` declaration to the signature.  This was a long-standing source of bugs,
but can require adding the ``noexcept`` declaration to existing functions if
exception propagation is really undesired.
(Github issue :issue:`4280`)

To ease the transition for this break in behaviour, it is possible to set
``legacy_implicit_noexcept=True``.

Related changes
^^^^^^^^^^^^^^^

* The ``assert`` statement is allowed in ``nogil`` sections.  Here, the GIL is
  only acquired if the ``AssertionError`` is really raised, which means that the
  evaluation of the asserted condition only allows C expressions.

* The exception handling annotation ``except +*`` was broken.
  Patch by David Woods.  (Github issues :issue:`3065`, :issue:`3066`)

* Improve conversion between function pointers with non-identical but
  compatible exception specifications.  Patches by David Woods.
  (Github issues :issue:`4770`, :issue:`4689`)

* Exceptions within for-loops that run over memoryviews could lead to a \ 
ref-counting error.
  Patch by David Woods.  (Github issue :issue:`4662`)

* To opt out of the new, safer exception handling behaviour, legacy code can set \ 
the new
  directive ``legacy_implicit_noexcept=True`` for a transition period to keep the
  previous, unsafe behaviour.  This directive will eventually be removed in a \ 
later release.
  Patch by Matúš Valo.  (Github issue :issue:`5094`)

* Cython implemented C functions now propagate exceptions by default, rather than
  swallowing them in non-object returning function if the user forgot to add an
  ``except`` declaration to the signature.  This was a long-standing source of bugs,
  but can require adding the ``noexcept`` declaration to existing functions if
  exception propagation is really undesired.
  (Github issue :issue:`4280`)

* The code ``except +nogil`` (declaring a C++ exception handler function called \ 
``nogil``)
  is now rejected because it is almost certainly a typo from ``except + nogil``.
  (Github issue :issue:`5430`)

* Handling freshly raised exceptions that didn't have a traceback yet could crash.
  (Github issue :issue:`5495`)

Optimizations
-------------

Generating efficient code has long been a goal of Cython, and 3.0 continues that.
Probably the most significant change is that Cython functions use the PEP-590 \ 
vectorcall
protocol on Python 3.7 and higher.

Related changes
^^^^^^^^^^^^^^^

* Name lookups in class bodies no longer go through an attribute lookup.
  Patch by Jeroen Demeyer.  (Github issue :issue:`3100`)

* Extension types that do not need their own ``tp_new`` implementation (because
  they have no object attributes etc.) directly inherit the implementation of
  their parent type if possible.
  (Github issue :issue:`1555`)

* Some list copying is avoided internally when a new list needs to be created
  but we already have a fresh one.
  (Github issue :issue:`3494`)

* Multiplication of Python numbers with small constant integers is faster.
  (Github issue :issue:`2808`)

* String concatenation can now happen in place if possible, by extending the
  existing string rather than always creating a new one.
  Patch by David Woods.  (Github issue :issue:`3453`)

* The ``str()`` builtin now calls ``PyObject_Str()`` instead of going
  through a Python call.
  Patch by William Ayd.  (Github issue :issue:`3279`)

* Reimports of already imported modules are substantially faster.
  (Github issue :issue:`2854`)

* The dispatch to fused functions is now linear in the number of arguments,
  which makes it much faster, often 2x or more, and several times faster for
  larger fused types with many specialisations.
  Patch by will-ca.  (Github issue :issue:`1385`)

* The fastcall/vectorcall protocols are used for several internal Python calls.
  (Github issue :issue:`3540`)

* ``nogil`` functions now avoid acquiring the GIL on function exit if possible
  even if they contain ``with gil`` blocks.
  (Github issue :issue:`3554`)

* Type inference now works for memory views and slices.
  Patch by David Woods.  (Github issue :issue:`2227`)

* For-in-loop iteration over ``bytearray`` and memory views is optimised.
  Patch by David Woods.  (Github issue :issue:`2227`)

* For-in-loop iteration over ``bytearray`` and memory views is optimised.
  Patch by David Woods.  (Github issue :issue:`2227`)

* ``float(…)`` is optimised for string arguments (str/bytes/bytearray).

* ``[...] * N`` is optimised for C integer multipliers ``N``.
  (Github issue :issue:`3922`)

* Some constant tuples containing strings were not deduplicated.
  Patch by David Woods.  (Github issue :issue:`4353`)

* Memory views can use atomic CPU instructions instead of locks in more cases.
  Patch by Sam Gross.  (Github issue :issue:`4912`)

* Cython avoids raising ``StopIteration`` in ``__next__`` methods when possible.
  Patch by David Woods.  (Github issue :issue:`3447`)

* Larger numbers of extension types with multiple subclasses could take very \ 
long to compile.
  Patch by Scott Wolchok.  (Github issue :issue:`5139`)

* Integer comparisons avoid Python coercions if possible.
  (Github issue :issue:`4821`)

* The call-time dispatch for fused memoryview types is less slow.
  (Github issue :issue:`5073`)

* Python's ``memoryview`` is now a known builtin type with optimised properties.
  (Github issue :issue:`3798`)

* Multiplying a sequence by a C integer avoids creating and intermediate Python \ 
integer.

* The reference counting of memory views involved useless overhead.
  (Github issue :issue:`5510`)

Compatibility with C
--------------------

The support for C features like ``const`` or ``volatile`` was substantially improved.

The generated code has been cleared up to reduce the number of C compiler \ 
warnings emitted.

Related changes
^^^^^^^^^^^^^^^

* A C compiler cast warning was resolved.
  Patch by Michael Buesch.  (Github issue :issue:`2775`)

* Constant integer expressions that used a negative exponent were evaluated
  as integer 0 instead of the expected float value.
  Patch by Kryštof Pilnáček.  (Github issue :issue:`2133`)

* Several declarations in ``cpython.*``, ``libc.*`` and ``libcpp.*`` were added.
  Patches by Jeroen Demeyer, Matthew Edwards, Chris Gyurgyik, Jerome Kieffer
  and Zackery Spytz.
  (Github issues :issue:`3468`, :issue:`3332`, :issue:`3202`, :issue:`3188`,
  :issue:`3179`, :issue:`2891`, :issue:`2826`, :issue:`2713`)

* The ``volatile`` C modifier is supported in Cython code.
  Patch by Jeroen Demeyer.  (Github issue :issue:`1667`)

* ``const`` can be used together with fused types.
  Patch by Thomas Vincent.  (Github issue :issue:`1772`)

* Temporary buffer indexing variables were not released and could show up in
  C compiler warnings, e.g. in generators.
  Patch by David Woods.  (Github issues :issue:`3430`, :issue:`3522`)

* The C property feature has been rewritten and now requires C property methods
  to be declared ``inline`` (:issue:`3571`).

* Cython generates C compiler branch hints for unlikely user defined if-clauses
  in more cases, when they end up raising exceptions unconditionally. This now
  includes exceptions being raised in ``nogil``/``with gil`` sections.

* Several issues with arithmetic overflow handling were resolved, including
  undefined behaviour in C.
  Patch by Sam Sneddon.  (Github issue :issue:`3588`)

* `libc.math` was extended to include all C99 function declarations.
  Patch by Dean Scarff.  (Github issue :issue:`3570`)

* Some C compiler warninge were resolved.
  Patches by Max Bachmann.  (Github issue :issue:`4053`, :issue:`4059`, \ 
:issue:`4054`, :issue:`4148`, :issue:`4162`)

* A C compiler warning about enum value casting was resolved in GCC.
  (Github issue :issue:`2749`)

* A C compiler warning about unused code was resolved.
  (Github issue :issue:`3763`)

* Some compiler problems and warnings were resolved.
  Patches by David Woods, 0dminnimda, Nicolas Pauss and others.
  (Github issues :issue:`4317`, :issue:`4324`, :issue:`4361`, :issue:`4357`)

* Some C compiler warnings were fixed.
  Patch by mwtian.  (Github issue :issue:`4831`)

* A case of undefined C behaviour was resolved in the list slicing code.
  Patch by Richard Barnes.  (Github issue :issue:`4734`)

* Typedefs for the ``bint`` type did not always behave like ``bint``.
  Patch by Nathan Manville and 0dminnimda.  (Github issue :issue:`4660`)

* Intel C compilers could complain about unsupported gcc pragmas.
  Patch by Ralf Gommers.  (Github issue :issue:`5052`)

* Structs that contained an array field resulted in incorrect C code.  Their \ 
initialisation
  now uses ``memcpy()``.
  Patch by Chia-Hsiang Cheng.  (Github issue :issue:`5178`)

* The module state struct was not initialised in correct C (before C23), leading to
  compile errors on Windows.
  Patch by yudonglin.  (Github issue :issue:`5169`)

* ``cdef public`` functions declared in .pxd files could use an incorrectly \ 
mangled C name.
  Patch by EpigeneMax.  (Github issue :issue:`2940`)

* ``const`` types could not be returned from functions.
  Patch by Mike Graham.  (Github issue :issue:`5135`)

* C11 ``complex.h`` is now properly detected.
  (Github issue :issue:`2513`)

* Standard C/C++ atomic operations are now used for memory views, if available.
  (Github issue :issue:`4925`)

* C arrays can be initialised inside of nogil functions.
  Patch by Matúš Valo.  (Github issue :issue:`1662`)

* Very long Python integer constants could exceed the maximum C name length of MSVC.
  Patch by 0dminnimda.  (Github issue :issue:`5290`)

* Some C compiler warnings were resolved.
  Patches by Matt Tyson, Lisandro Dalcin, Philipp Wagner, Matti Picus et al.
  (Github issues :issue:`5417`, :issue:`5418`, :issue:`5421`, :issue:`5437`, \ 
:issue:`5438`, :issue:`5443`)

* Some typedef declarations for libc function types were fixed.
  (Github issue :issue:`5498`)

* With MSVC, Cython no longer enables C-Complex support by accident (which is \ 
not supported there).
  (Github issue :issue:`5512`)

Compatibility with C++
----------------------

Many C++ features like forwarding references or ``std::move`` are now supported \ 
or even used
internally, if possible.

Cython's wrapping of the C++ standard library has been extended.

A new `cpp_locals`` directive enables C++ local variables to initialized when \ 
assigned to
rather than at the start of the function, making them behave more like Python \ 
variables,
and also removing the requirement for them to be default constructible.

Related changes
^^^^^^^^^^^^^^^

* C++ ``typeid()`` failed for fused types.
  Patch by David Woods.  (Github issue :issue:`3203`)

* ``std::move()`` is now used in C++ mode for internal temp variables to
  make them work without copying values.
  Patch by David Woods.  (Github issues :issue:`3253`, :issue:`1612`)

* The C++ ``typeid()`` function was allowed in C mode.
  Patch by Celelibi.  (Github issue :issue:`3637`)

* C++ references failed to compile when used as Python object indexes.
  Patch by David Woods.  (Github issue :issue:`3754`)

* The construct ``for x in cpp_function_call()`` failed to compile.
  Patch by David Woods.  (Github issue :issue:`3663`)

* Some C++ STL methods did not propagate exceptions.
  Patch by Max Bachmann.  (Github issue :issue:`4079`)

* A compile failure for C++ enums in Py3.4 / MSVC was resolved.
  Patch by Ashwin Srinath.  (Github issue :issue:`3782`)

* Cython compiled functions always provided a ``__self__`` attribute,
  regardless of being used as a method or not.
  Patch by David Woods.  (Github issue :issue:`4036`)

* Overloaded C++ static methods were lost.
  Patch by Ashwin Srinath.  (Github :issue:`1851`)

* Nested C++ types were not usable through ctypedefs.
  Patch by Vadim Pushtaev.  (Github issue :issue:`4039`)

* More declarations for C++ string methods were added.

* Converting C++ containers to Python lists uses less memory allocations.
  Patch by Max Bachmann.  (Github issue :issue:`4081`)

* ``std::move()`` is now also called for temps during ``yield``.
  Patch by Yu Feng.  (Github issue :issue:`4154`)

* The destructor is now called for fields in C++ structs.
  Patch by David Woods.  (Github issue :issue:`3226`)

* Conversion from Python dicts to ``std::map`` was broken.
  Patch by David Woods and Mikkel Skofelt.  (Github issues :issue:`4228`, \ 
:issue:`4231`)

* Code optimisations were not applied to methods of Cython implemented C++ classes.
  Patch by David Woods.  (Github issue :issue:`4212`)

* C++17 execution policies are supported in ``libcpp.algorithm``.
  Patch by Ashwin Srinath.  (Github issue :issue:`3790`)

* A new directive ``cpp_locals`` was added that allows local C++ variables to
  be lazily initialised (without default constructor), thus making them behave
  more like Python variables.
  Patch by David Woods.  (Github issue :issue:`4160`)

* Generated utility code for C++ conversions no longer depends on several user
  definable directives that may make it behave incorrectly.
  Patch by David Woods.  (Github issue :issue:`4206`)

* Several issues with the new ``cpp_locals`` directive were resolved and
  its test coverage improved.
  Patch by David Woods.  (Github issues :issue:`4265`, :issue:`4266`)

* Declarations for ``libcpp.algorithms``, ``libcpp.set`` and ``libcpp.unordered_set``
  were extended.
  Patch by David Woods.  (Github issues :issue:`4271`, :issue:`4273`)

* Several C++ library declarations were added and fixed.
  Patches by Dobatymo, account-login, Jonathan Helgert, Evgeny Yakimov, \ 
GalaxySnail, Max Bachmann.
  (Github issues :issue:`4408`, :issue:`4419`, :issue:`4410`, :issue:`4395`,
  :issue:`4423`, :issue:`4448`, :issue:`4462`, :issue:`3293`, :issue:`4522`,
  :issue:`2171`, :issue:`4531`)

* Templating C++ classes with memory view types lead to buggy code and is now \ 
rejected.
  Patch by David Woods.  (Github issue :issue:`3085`)

* ``prange`` loops generated incorrect code when ``cpp_locals`` is enabled.
  Patch by David Woods.  (Github issue :issue:`4354`)

* Direct assignments to C++ references are now allowed.
  Patch by David Woods.  (Github issue :issue:`1863`)

* Conversion from Python dict to C++ map now supports arbitrary Python mappings,
  not just dicts.

* Some C++ and CPython library declarations were extended and fixed.
  Patches by Max Bachmann, Till Hoffmann, Julien Jerphanion, Wenjun Si.
  (Github issues :issue:`4530`, :issue:`4528`, :issue:`4710`, :issue:`4746`,
  :issue:`4751`, :issue:`4818`, :issue:`4762`, :issue:`4910`)

* Some C/C++ warnings were resolved.
  Patches by Max Bachmann, Alexander Shadchin, at al.
  (Github issues :issue:`5004`, :issue:`5005`, :issue:`5019`, :issue:`5029`, \ 
:issue:`5096`)

* C++ references did not work on fused types.
  (Github issue :issue:`4717`)

* C++ iteration more safely stores the iterable in temporary variables.
  Patch by Xavier.  (Github issue :issue:`3828`)

* C++ post-increment/-decrement operators were not correctly looked up on \ 
declared C++
  classes, thus allowing Cython declarations to be missing for them and incorrect C++
  code to be generated.
  Patch by Max Bachmann.  (Github issue :issue:`4536`)

* ``cdef public`` functions used an incorrect linkage declaration in C++.
  Patch by Maximilien Colange.  (Github issue :issue:`1839`)

* Declarations were added for the C++ bit operations, some other parts of C++20 \ 
and CPython APIs.
  Patches by Jonathan Helgert, Dobatymo, William Ayd and Max Bachmann.
  (Github issues :issue:`4962`, :issue:`5101`, :issue:`5157`, :issue:`5163`, \ 
:issue:`5257`)

* ``cpp_locals`` no longer have to be "assignable".
  (Github issue :issue:`4558`)

* Nested ``cppclass`` definitions are supported.
  Patch by samaingw.  (Github issue :issue:`1218`)

* ``reversed()`` can now be used together with C++ iteration.
  Patch by Chia-Hsiang Cheng.  (Github issue :issue:`5002`)

* Some C++ warnings regarding ``const`` usage in internally generated utility \ 
code were resolved.
  Patch by Max Bachmann.  (Github issue :issue:`5301`)

* Cython generated C++ code accidentally used C++11 features in some cases.
  (Github issue :issue:`5316`)

* Fully qualified C++ names prefixed by a cimported module name could fail to \ 
compile.
  Patch by Chia-Hsiang Cheng.  (Github issue :issue:`5229`)

* C++ declarations for ``<cmath>``, ``<numbers>`` and ``std::any`` \ 
were added.
  Patches by Jonathan Helgert and Maximilien Colange.
  (Github issues :issue:`5262`, :issue:`5309`, :issue:`5314`)

* The ``extern "C"`` and ``extern "C++"`` markers that \ 
Cython generates for
  ``public`` functions can now be controlled by setting the C macro \ 
``CYTHON_EXTERN_C``.

* C++ containers of item type ``bint`` could conflict with those of item type \ 
``int``.
  (Github issue :issue:`5516`)

* Reverse iteration in C++ no longer removes the ``const`` qualifier from the \ 
item type.
  Patch by Isuru Fernando.  (Github issue :issue:`5478`)

Commandline Interface
---------------------

A number of new options were added to the ``cython`` and ``cythonize``
commands.

Related changes
^^^^^^^^^^^^^^^

* The command line parser was rewritten and modernised using ``argparse``.
  Patch by Egor Dranischnikow.  (Github issue :issue:`2952`, :issue:`3001`)

* ``cygdb`` gives better error messages when it fails to initialise the
  Python runtime support in gdb.
  Patch by Volker Weissmann.  (Github issue :issue:`3489`)

* ``--no-docstrings`` option added to ``cythonize`` script.
  Original patch by mo-han.  (Github issue :issue:`2889`)

* Code annotation accepts a new debugging argument ``--annotate-fullc`` that
  will include the complete syntax highlighted C file in the HTML output.
  (Github issue :issue:`2855`)

* ``cygdb`` has a new option ``--skip-interpreter`` that allows using a different
  Python runtime than the one used to generate the debugging information.
  Patch by Alessandro Molina.  (Github issue :issue:`4186`)

* ``cythonize()`` and the corresponding CLI command now regenerate the output files
  also when they already exist but were generated by a different Cython version.

* The ``cython`` and ``cythonize`` commands ignored non-existing input files \ 
without error.
  Patch by Matúš Valo.  (Github issue :issue:`4629`)

* ``cythonize --help`` now also prints information about the supported \ 
environment variables.
  Patch by Matúš Valo.  (Github issue :issue:`1711`)

* Using the ``--working`` option could lead to sources not being found.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5365`)

* Passing a language level and directives on the command line lost the language \ 
level setting.
  Patch by Matúš Valo.  (Github issue :issue:`5484`)

* ``cython --version`` now prints the version to both stdout and stderr (unless \ 
that is a TTY).
  (Github issue :issue:`5504`)

Build integration
-----------------

Cython has made a number of improvements both to how it compiles itself
and how it integrates with external build tools.  Most notably Cython
has been moving to use ``setuptools`` instead of the deprecated/removed
``distutils`` where possible.

The new ``--depfile`` option generates dependency files to help integrate
Cython with other build tools.

Related changes
^^^^^^^^^^^^^^^

* Binary Linux wheels now follow the manylinux2010 standard.
  Patch by Alexey Stepanov.  (Github issue :issue:`3355`)

* The search order for include files was changed. Previously it was
  ``include_directories``, ``Cython/Includes``, ``sys.path``. Now it is
  ``include_directories``, ``sys.path``, ``Cython/Includes``. This was done to
  allow third-party ``*.pxd`` files to override the ones in Cython.
  Patch by Matti Picus.  (Github issue :issue:`2905`)

* Source file fingerprinting now uses SHA-1 instead of MD5 since the latter
  tends to be slower and less widely supported these days.
  (Github issue :issue:`2790`)

* The Cython AST code serialiser class ``CodeWriter`` in ``Cython.CodeWriter``
  supports more syntax nodes.

* Parallel builds of Cython itself (``setup.py build_ext -j N``) failed on Windows.

* When importing the old Cython ``build_ext`` integration with distutils, the
  additional command line arguments leaked into the regular command.
  Patch by Kamekameha.  (Github issue :issue:`2209`)

* ``.pxd`` files can now be :ref:`versioned <versioning>` by adding an
  extension like "``.cython-30.pxd``" to prevent older Cython versions \ 
(than
  3.0 in this case) from picking them up.  (Github issue :issue:`3577`)

* The Cython ``CodeWriter`` can now handle more syntax constructs.
  Patch by Tao He.  (Github issue :issue:`3514`)

* The Cython ``CodeWriter`` mishandled no-argument ``return`` statements.
  Patch by Tao He.  (Github issue :issue:`3795`)

* Cython now detects when existing output files were not previously generated
  by itself and refuses to overwrite them.  It is a common mistake to name
  the module file of a wrapper after the library (source file) that it wraps,
  which can lead to surprising errors when the file gets overwritten.

* The ``Cython.Build.BuildExecutable`` tool no longer executes the program \ 
automatically.
  Use ``cythonrun`` for that.

* Python modules were not automatically recompiled when only their ``.pxd`` file \ 
changed.
  Patch by Golden Rockefeller.  (Github issue :issue:`1428`)

* An unnecessary slow-down at import time was removed from ``Cython.Distutils``.
  Original patch by Anthony Sottile.  (Github issue :issue:`4224`)

* A compiler crash when running Cython thread-parallel from distutils was resolved.
  (Github issue :issue:`4503`)

* An incompatibility with recent coverage.py versions was resolved.
  Patch by David Woods.  (Github issue :issue:`4440`)

* ``pyximport`` now uses ``cythonize()`` internally.
  Patch by Matúš Valo.  (Github issue :issue:`2304`)

* ``Cython.Distutils.build_ext`` now uses ``cythonize()`` internally (previously
  known as ``new_build_ext``), while still supporting the options that were
  available in the old implementation (``old_build_ext``).
  Patch by Matúš Valo.  (Github issue :issue:`3541`)

* Improve compatibility between classes pickled in Cython 3.0 and 0.29.x
  by accepting MD5, SHA-1 and SHA-256 checksums.
  (Github issue :issue:`4680`)

* ``pyximport`` failed for long filenames on Windows.
  Patch by Matti Picus.  (Github issue :issue:`4630`)

* A new Cython build option ``--cython-compile-minimal`` was added to compile only a
  smaller set of Cython's own modules, which can be used to reduce the package
  and install size.

* The environment variable ``CYTHON_FORCE_REGEN=1`` can be used to force \ 
``cythonize``
  to regenerate the output files regardless of modification times and changes.

* The ``cythonize`` and ``cython`` commands have a new option ``-M`` / ``--depfile``
  to generate ``.dep`` dependency files for the compilation unit.  This can be used
  by external build tools to track these dependencies.
  The ``cythonize`` option was already available in Cython :ref:`0.29.27`.
  Patches by Evgeni Burovski and Eli Schwartz.  (Github issue :issue:`1214`)

* Wheels now include a compiled parser again, which increases their size a little
  but gives about a 10% speed-up when running Cython.

* The wheel building process was migrated to use the ``cibuildwheel`` tool.
  Patch by Thomas Li.  (Github issue :issue:`4736`)

* ``setup.cfg`` was missing from the source distribution.
  (Github issue :issue:`5199`)

* Extended glob paths with ``/**/`` and ``\**\`` for finding source files failed \ 
on Windows.

* Coverage analysis failed in projects with a separate source subdirectory.
  Patch by Sviatoslav Sydorenko and Ruben Vorderman.  (Github issue :issue:`3636`)

* Cython could crash when finding import files with dots in their names.
  Patch by Matúš Valo.  (Github issue :issue:`5396`)

* A module loading problem with ``cython.inline()`` on Windows was resolved.

Deprecations
------------

Some older features of Cython have been deprecated. Most notable are the
compile time ``DEF`` and ``IF`` statements, although we emphasise that
they will remain until a good alternative exists for all their use-cases.

Related changes
^^^^^^^^^^^^^^^

* Dotted filenames for qualified module names (``pkg.mod.pyx``) are deprecated.
  Use the normal Python package directory layout instead.
  (Github issue :issue:`2686`)

* "Declaration after use" is now an error for variables.
  Patch by David Woods.  (Github issue :issue:`3976`)

* Variables can no longer be declared with ``cpdef``.
  Patch by David Woods.  (Github issue :issue:`887`)

* The compile-time ``DEF`` and ``IF`` statements are deprecated and generate a \ 
warning.
  They should be replaced with normal constants, code generation or C macros.
  (Github issue :issue:`4310`)

* C-style array declarations (``cdef int a[4]``) are now (silently) deprecated in
  favour of the Java-style ``cdef int[4] a`` form.  The latter was always available
  and the Python type declaration syntax already used it exclusively (``a: int[4]``).
  Patch by Matúš Valo.  (Github issue :issue:`5248`)

* The undocumented, untested and apparently useless syntax
  ``from somemodule cimport class/struct/union somename`` was removed.  The type
  modifier is not needed here and a plain ``cimport`` of the name will do.
  (Github issue :issue:`4904`)

Editor support
--------------

Related changes
^^^^^^^^^^^^^^^

* C compiler warnings and errors are now shown in Jupyter notebooks.
  Patch by Egor Dranischnikow.  (Github issue :issue:`3751`)

* An endless loop in ``cython-mode.el`` was resolved.
  Patch by Johannes Mueller.  (Github issue :issue:`3218`)

* The Emacs Cython mode file ``cython-mode.el`` is now maintained in a separate repo:
  https://github.com/cython/emacs-cython-mode

* The C code shown in the annotated HTML output could lack the last C code line(s).

Other changes
-------------

* Memoryviews failed to compile when the ``cache_builtins`` feature was disabled.
  Patch by David Woods.  (Github issue :issue:`3406`)

* Broadcast assignments to a multi-dimensional memory view slice could end
  up in the wrong places when the underlying memory view is known to be
  contiguous but the slice is not.
  (Github issue :issue:`2941`)

* The Pythran ``shape`` attribute is supported.
  Patch by Serge Guelton.  (Github issue :issue:`3307`)

* ``--no-capture`` added to ``runtests.py`` to prevent stdout/stderr capturing
  during srctree tests.
  Patch by Matti Picus.  (Github issue :issue:`2701`)

* Decoding an empty bytes/char* slice with large bounds could crash.
  Patch by Sam Sneddon.  (Github issue :issue:`3534`)

* Creating an empty unicode slice with large bounds could crash.
  Patch by Sam Sneddon.  (Github issue :issue:`3531`)

* Complex buffer item types of structs of arrays could fail to validate.
  Patch by Leo and smutch.  (Github issue :issue:`1407`)

* Error handling in ``cython.array`` creation was improved to avoid calling
  C-API functions with an error held.

* Error handling early in the module init code could lead to a crash.

* Exception position reporting could run into race conditions on threaded code.
  It now uses function-local variables again.

* A reference leak on import failures was resolved.
  Patch by Max Bachmann.  (Github issue :issue:`4056`)

* Casting to ctuples is now allowed.
  Patch by David Woods.  (Github issue :issue:`3808`)

* Some issues were resolved that could lead to duplicated C names.
  Patch by David Woods.  (Github issue :issue:`3716`, :issue:`3741`, :issue:`3734`)

* Inline functions and other code in ``.pxd`` files could accidentally
  inherit the compiler directives of the ``.pyx`` file that imported them.
  Patch by David Woods.  (Github issue :issue:`1071`)

* Parts of the documentation were (and are being) rewritten to show the
  Cython language syntax next to the equivalent Python syntax.
  Patches by 0dminnimda and Matúš Valo.  (Github issue :issue:`4187`)

* A name collision when including multiple generated API header files was resolved.
  Patch by David Woods.  (Github issue :issue:`4308`)

* Very early errors during module initialisation could lead to crashes.
  Patch by David Woods.  (Github issue :issue:`4377`)

* Type errors when passing memory view arguments could leak buffer references.
  Patch by David Woods.  (Github issue :issue:`4296`)

* The GIL can now safely be released inside of ``nogil`` functions (which may \ 
actually
  be called with the GIL held at runtime).
  Patch by David Woods.  (Github issue :issue:`4137`)

* The return type of a fused function is no longer ignored for function pointers,
  since it is relevant when passing them e.g. as argument into other fused functions.
  Patch by David Woods.  (Github issue :issue:`4644`)

* Using memoryview arguments in closures of inner functions could lead to \ 
ref-counting errors.
  Patch by David Woods.  (Github issue :issue:`4798`)

* Decorators like ``@cfunc`` and ``@ccall`` could leak into nested functions and \ 
classes.
  Patch by David Woods.  (Github issue :issue:`4092`)

* Cython now uses a ``.dev0`` version suffix for unreleased source installations.

* The ``Tempita`` module no longer contains HTML processing capabilities, which
  were found to be broken in Python 3.8 and later.
  Patch by Marcel Stimberg.  (Github issue :issue:`3309`)

* Nesting fused types in other fused types could fail to specialise the inner type.
  (Github issue :issue:`4725`)

* Iterating over memoryviews in generator expressions could leak a buffer reference.
  (Github issue :issue:`4968`)

* The C ``float`` type was not inferred on assignments.
  (Github issue :issue:`5234`)

* Type checks for Python's ``memoryview`` type generated incorrect C code.
  (Github issues :issue:`5268`, :issue:`5270`)

* Auto-generated utility code didn't always have all required user defined types \ 
available.
  (Github issue :issue:`5269`)

* ``cimport_from_pyx`` could miss some declarations.
  Patch by Chia-Hsiang Cheng.  (Github issue :issue:`5318`)

* For-loops now release the internal reference to their list/tuple iterable before
  instead of after the ``else:`` clause.  This probably has no practical impact.
  (Github issue :issue:`5347`)

* Extension type hierarchies were generated in the wrong order, thus leading to \ 
compile issues.
  Patch by Lisandro Dalcin.  (Github issue :issue:`5395`)

* The FAQ page was moved from the GitHub Wiki to the regular documentation
  to make it more visible.

Files:
RevisionActionfile
1.86modifypkgsrc/devel/py-cython/Makefile
1.29modifypkgsrc/devel/py-cython/PLIST
1.77modifypkgsrc/devel/py-cython/distinfo
1.4modifypkgsrc/devel/py-cython/patches/patch-runtests.py
1.1removepkgsrc/devel/py-cython/patches/patch-Cython_Distutils_old__build__ext.py