Path to this page:
Subject: CVS commit: pkgsrc/math/py-pandas
From: Adam Ciarcinski
Date: 2021-05-06 06:39:03
Message id: 20210506043903.862D8FA95@cvs.NetBSD.org
Log Message:
py-pandas: updated to 1.2.4
What's new in 1.2.4 (April 12, 2021)
Fixed regressions
- Fixed regression in :meth:`DataFrame.sum` when ``min_count`` greater than the \
:class:`DataFrame` shape was passed resulted in a ``ValueError`` \
(:issue:`39738`)
- Fixed regression in :meth:`DataFrame.to_json` raising ``AttributeError`` when \
run on PyPy (:issue:`39837`)
- Fixed regression in (in)equality comparison of ``pd.NaT`` with a \
non-datetimelike numpy array returning a scalar instead of an array \
(:issue:`40722`)
- Fixed regression in :meth:`DataFrame.where` not returning a copy in the case \
of an all True condition (:issue:`39595`)
- Fixed regression in :meth:`DataFrame.replace` raising ``IndexError`` when \
``regex`` was a multi-key dictionary (:issue:`39338`)
- Fixed regression in repr of floats in an ``object`` column not respecting \
``float_format`` when printed in the console or outputted through \
:meth:`DataFrame.to_string`, :meth:`DataFrame.to_html`, and \
:meth:`DataFrame.to_latex` (:issue:`40024`)
- Fixed regression in NumPy ufuncs such as ``np.add`` not passing through all \
arguments for :class:`DataFrame`
What's new in 1.2.3 (March 02, 2021)
Fixed regressions
- Fixed regression in :meth:`~DataFrame.to_excel` raising ``KeyError`` when \
giving duplicate columns with ``columns`` attribute (:issue:`39695`)
- Fixed regression in nullable integer unary ops propagating mask on assignment \
(:issue:`39943`)
- Fixed regression in :meth:`DataFrame.__setitem__` not aligning \
:class:`DataFrame` on right-hand side for boolean indexer (:issue:`39931`)
- Fixed regression in :meth:`~DataFrame.to_json` failing to use ``compression`` \
with URL-like paths that are internally opened in binary mode or with \
user-provided file objects that are opened in binary mode (:issue:`39985`)
- Fixed regression in :meth:`Series.sort_index` and \
:meth:`DataFrame.sort_index`, which exited with an ungraceful error when having \
kwarg ``ascending=None`` passed. Passing ``ascending=None`` is still considered \
invalid, and the improved error message suggests a proper usage (``ascending`` \
must be a boolean or a list-like of boolean) (:issue:`39434`)
- Fixed regression in :meth:`DataFrame.transform` and :meth:`Series.transform` \
giving incorrect column labels when passed a dictionary with a mix of list and \
non-list values (:issue:`40018`)
What's new in 1.2.2 (February 09, 2021)
---------------------------------------
These are the changes in pandas 1.2.2. See :ref:`release` for a full changelog
including other versions of pandas.
{{ header }}
.. ---------------------------------------------------------------------------
.. _whatsnew_122.regressions:
Fixed regressions
~~~~~~~~~~~~~~~~~
- Fixed regression in :func:`read_excel` that caused it to raise \
``AttributeError`` when checking version of older xlrd versions (:issue:`38955`)
- Fixed regression in :class:`DataFrame` constructor reordering element when \
construction from datetime ndarray with dtype not ``"datetime64[ns]"`` \
(:issue:`39422`)
- Fixed regression in :meth:`DataFrame.astype` and :meth:`Series.astype` not \
casting to bytes dtype (:issue:`39474`)
- Fixed regression in :meth:`~DataFrame.to_pickle` failing to create bz2/xz \
compressed pickle files with ``protocol=5`` (:issue:`39002`)
- Fixed regression in :func:`pandas.testing.assert_series_equal` and \
:func:`pandas.testing.assert_frame_equal` always raising ``AssertionError`` when \
comparing extension dtypes (:issue:`39410`)
- Fixed regression in :meth:`~DataFrame.to_csv` opening ``codecs.StreamWriter`` \
in binary mode instead of in text mode and ignoring user-provided ``mode`` \
(:issue:`39247`)
- Fixed regression in :meth:`Categorical.astype` casting to incorrect dtype when \
``np.int32`` is passed to dtype argument (:issue:`39402`)
- Fixed regression in :meth:`~DataFrame.to_excel` creating corrupt files when \
appending (``mode="a"``) to an existing file (:issue:`39576`)
- Fixed regression in :meth:`DataFrame.transform` failing in case of an empty \
DataFrame or Series (:issue:`39636`)
- Fixed regression in :meth:`~DataFrame.groupby` or :meth:`~DataFrame.resample` \
when aggregating an all-NaN or numeric object dtype column (:issue:`39329`)
- Fixed regression in :meth:`.Rolling.count` where the ``min_periods`` argument \
would be set to ``0`` after the operation (:issue:`39554`)
- Fixed regression in :func:`read_excel` that incorrectly raised when the \
argument ``io`` was a non-path and non-buffer and the ``engine`` argument was \
specified (:issue:`39528`)
.. ---------------------------------------------------------------------------
.. _whatsnew_122.bug_fixes:
Bug fixes
~~~~~~~~~
- :func:`pandas.read_excel` error message when a specified ``sheetname`` does \
not exist is now uniform across engines (:issue:`39250`)
- Fixed bug in :func:`pandas.read_excel` producing incorrect results when the \
engine ``openpyxl`` is used and the excel file is missing or has incorrect \
dimension information; the fix requires ``openpyxl`` >= 3.0.0, prior versions \
may still fail (:issue:`38956`, :issue:`39001`)
- Fixed bug in :func:`pandas.read_excel` sometimes producing a ``DataFrame`` \
with trailing rows of ``np.nan`` when the engine ``openpyxl`` is used \
(:issue:`39181`)
What's new in 1.2.1 (January 20, 2021)
--------------------------------------
These are the changes in pandas 1.2.1. See :ref:`release` for a full changelog
including other versions of pandas.
{{ header }}
.. ---------------------------------------------------------------------------
.. _whatsnew_121.regressions:
Fixed regressions
~~~~~~~~~~~~~~~~~
- Fixed regression in :meth:`~DataFrame.to_csv` that created corrupted zip files \
when there were more rows than ``chunksize`` (:issue:`38714`)
- Fixed regression in :meth:`~DataFrame.to_csv` opening \
``codecs.StreamReaderWriter`` in binary mode instead of in text mode \
(:issue:`39247`)
- Fixed regression in :meth:`read_csv` and other read functions were the \
encoding error policy (``errors``) did not default to ``"replace"`` \
when no encoding was specified (:issue:`38989`)
- Fixed regression in :func:`read_excel` with non-rawbyte file handles \
(:issue:`38788`)
- Fixed regression in :meth:`DataFrame.to_stata` not removing the created file \
when an error occured (:issue:`39202`)
- Fixed regression in ``DataFrame.__setitem__`` raising ``ValueError`` when \
expanding :class:`DataFrame` and new column is from type ``"0 - \
name"`` (:issue:`39010`)
- Fixed regression in setting with :meth:`DataFrame.loc` raising ``ValueError`` \
when :class:`DataFrame` has unsorted :class:`MultiIndex` columns and indexer is \
a scalar (:issue:`38601`)
- Fixed regression in setting with :meth:`DataFrame.loc` raising ``KeyError`` \
with :class:`MultiIndex` and list-like columns indexer enlarging \
:class:`DataFrame` (:issue:`39147`)
- Fixed regression in :meth:`~DataFrame.groupby()` with :class:`Categorical` \
grouping column not showing unused categories for ``grouped.indices`` \
(:issue:`38642`)
- Fixed regression in :meth:`.GroupBy.sem` where the presence of non-numeric \
columns would cause an error instead of being dropped (:issue:`38774`)
- Fixed regression in :meth:`.DataFrameGroupBy.diff` raising for ``int8`` and \
``int16`` columns (:issue:`39050`)
- Fixed regression in :meth:`DataFrame.groupby` when aggregating an \
``ExtensionDType`` that could fail for non-numeric values (:issue:`38980`)
- Fixed regression in :meth:`.Rolling.skew` and :meth:`.Rolling.kurt` modifying \
the object inplace (:issue:`38908`)
- Fixed regression in :meth:`DataFrame.any` and :meth:`DataFrame.all` not \
returning a result for tz-aware ``datetime64`` columns (:issue:`38723`)
- Fixed regression in :meth:`DataFrame.apply` with ``axis=1`` using str accessor \
in apply function (:issue:`38979`)
- Fixed regression in :meth:`DataFrame.replace` raising ``ValueError`` when \
:class:`DataFrame` has dtype ``bytes`` (:issue:`38900`)
- Fixed regression in :meth:`Series.fillna` that raised ``RecursionError`` with \
``datetime64[ns, UTC]`` dtype (:issue:`38851`)
- Fixed regression in comparisons between ``NaT`` and ``datetime.date`` objects \
incorrectly returning ``True`` (:issue:`39151`)
- Fixed regression in calling NumPy :func:`~numpy.ufunc.accumulate` ufuncs on \
DataFrames, e.g. ``np.maximum.accumulate(df)`` (:issue:`39259`)
- Fixed regression in repr of float-like strings of an ``object`` dtype having \
trailing 0's truncated after the decimal (:issue:`38708`)
- Fixed regression that raised ``AttributeError`` with PyArrow versions [0.16.0, \
1.0.0) (:issue:`38801`)
- Fixed regression in :func:`pandas.testing.assert_frame_equal` raising \
``TypeError`` with ``check_like=True`` when :class:`Index` or columns have mixed \
dtype (:issue:`39168`)
We have reverted a commit that resulted in several plotting related regressions \
in pandas 1.2.0 (:issue:`38969`, :issue:`38736`, :issue:`38865`, :issue:`38947` \
and :issue:`39126`).
As a result, bugs reported as fixed in pandas 1.2.0 related to inconsistent tick \
labeling in bar plots are again present (:issue:`26186` and :issue:`11465`)
What's new in 1.2.0 (December 26, 2020)
Performance improvements
- Performance improvements when creating DataFrame or Series with dtype ``str`` \
or :class:`StringDtype` from array with many string elements (:issue:`36304`, \
:issue:`36317`, :issue:`36325`, :issue:`36432`, :issue:`37371`)
- Performance improvement in :meth:`.GroupBy.agg` with the ``numba`` engine \
(:issue:`35759`)
- Performance improvements when creating :meth:`Series.map` from a huge \
dictionary (:issue:`34717`)
- Performance improvement in :meth:`.GroupBy.transform` with the ``numba`` \
engine (:issue:`36240`)
- :class:`.Styler` uuid method altered to compress data transmission over web \
whilst maintaining reasonably low table collision probability (:issue:`36345`)
- Performance improvement in :func:`to_datetime` with non-ns time unit for \
``float`` ``dtype`` columns (:issue:`20445`)
- Performance improvement in setting values on an :class:`IntervalArray` \
(:issue:`36310`)
- The internal index method :meth:`~Index._shallow_copy` now makes the new index \
and original index share cached attributes, avoiding creating these again, if \
created on either. This can speed up operations that depend on creating copies \
of existing indexes (:issue:`36840`)
- Performance improvement in :meth:`.RollingGroupby.count` (:issue:`35625`)
- Small performance decrease to :meth:`.Rolling.min` and :meth:`.Rolling.max` \
for fixed windows (:issue:`36567`)
- Reduced peak memory usage in :meth:`DataFrame.to_pickle` when using \
``protocol=5`` in python 3.8+ (:issue:`34244`)
- Faster ``dir`` calls when the object has many index labels, e.g. ``dir(ser)`` \
(:issue:`37450`)
- Performance improvement in :class:`ExpandingGroupby` (:issue:`37064`)
- Performance improvement in :meth:`Series.astype` and :meth:`DataFrame.astype` \
for :class:`Categorical` (:issue:`8628`)
- Performance improvement in :meth:`DataFrame.groupby` for ``float`` ``dtype`` \
(:issue:`28303`), changes of the underlying hash-function can lead to changes in \
float based indexes sort ordering for ties (e.g. :meth:`Index.value_counts`)
- Performance improvement in :meth:`pd.isin` for inputs with more than 1e6 \
elements (:issue:`36611`)
- Performance improvement for :meth:`DataFrame.__setitem__` with list-like \
indexers (:issue:`37954`)
- :meth:`read_json` now avoids reading entire file into memory when chunksize is \
specified (:issue:`34548`)
Bug fixes
Categorical
- :meth:`Categorical.fillna` will always return a copy, validate a passed fill \
value regardless of whether there are any NAs to fill, and disallow an ``NaT`` \
as a fill value for numeric categories (:issue:`36530`)
- Bug in :meth:`Categorical.__setitem__` that incorrectly raised when trying to \
set a tuple value (:issue:`20439`)
- Bug in :meth:`CategoricalIndex.equals` incorrectly casting non-category \
entries to ``np.nan`` (:issue:`37667`)
- Bug in :meth:`CategoricalIndex.where` incorrectly setting non-category entries \
to ``np.nan`` instead of raising ``TypeError`` (:issue:`37977`)
- Bug in :meth:`Categorical.to_numpy` and ``np.array(categorical)`` with \
tz-aware ``datetime64`` categories incorrectly dropping the time zone \
information instead of casting to object dtype (:issue:`38136`)
Datetime-like
- Bug in :meth:`DataFrame.combine_first` that would convert datetime-like column \
on other :class:`DataFrame` to integer when the column is not present in \
original :class:`DataFrame` (:issue:`28481`)
- Bug in :attr:`.DatetimeArray.date` where a ``ValueError`` would be raised with \
a read-only backing array (:issue:`33530`)
- Bug in ``NaT`` comparisons failing to raise ``TypeError`` on invalid \
inequality comparisons (:issue:`35046`)
- Bug in :class:`.DateOffset` where attributes reconstructed from pickle files \
differ from original objects when input values exceed normal ranges (e.g. \
months=12) (:issue:`34511`)
- Bug in :meth:`.DatetimeIndex.get_slice_bound` where ``datetime.date`` objects \
were not accepted or naive :class:`Timestamp` with a tz-aware \
:class:`.DatetimeIndex` (:issue:`35690`)
- Bug in :meth:`.DatetimeIndex.slice_locs` where ``datetime.date`` objects were \
not accepted (:issue:`34077`)
- Bug in :meth:`.DatetimeIndex.searchsorted`, \
:meth:`.TimedeltaIndex.searchsorted`, :meth:`PeriodIndex.searchsorted`, and \
:meth:`Series.searchsorted` with ``datetime64``, ``timedelta64`` or \
:class:`Period` dtype placement of ``NaT`` values being inconsistent with NumPy \
(:issue:`36176`, :issue:`36254`)
- Inconsistency in :class:`.DatetimeArray`, :class:`.TimedeltaArray`, and \
:class:`.PeriodArray` method ``__setitem__`` casting arrays of strings to \
datetime-like scalars but not scalar strings (:issue:`36261`)
- Bug in :meth:`.DatetimeArray.take` incorrectly allowing ``fill_value`` with a \
mismatched time zone (:issue:`37356`)
- Bug in :class:`.DatetimeIndex.shift` incorrectly raising when shifting empty \
indexes (:issue:`14811`)
- :class:`Timestamp` and :class:`.DatetimeIndex` comparisons between tz-aware \
and tz-naive objects now follow the standard library ``datetime`` behavior, \
returning ``True``/``False`` for ``!=``/``==`` and raising for inequality \
comparisons (:issue:`28507`)
- Bug in :meth:`.DatetimeIndex.equals` and :meth:`.TimedeltaIndex.equals` \
incorrectly considering ``int64`` indexes as equal (:issue:`36744`)
- :meth:`Series.to_json`, :meth:`DataFrame.to_json`, and :meth:`read_json` now \
implement time zone parsing when orient structure is ``table`` (:issue:`35973`)
- :meth:`astype` now attempts to convert to ``datetime64[ns, tz]`` directly from \
``object`` with inferred time zone from string (:issue:`35973`)
- Bug in :meth:`.TimedeltaIndex.sum` and :meth:`Series.sum` with ``timedelta64`` \
dtype on an empty index or series returning ``NaT`` instead of ``Timedelta(0)`` \
(:issue:`31751`)
- Bug in :meth:`.DatetimeArray.shift` incorrectly allowing ``fill_value`` with a \
mismatched time zone (:issue:`37299`)
- Bug in adding a :class:`.BusinessDay` with nonzero ``offset`` to a non-scalar \
other (:issue:`37457`)
- Bug in :func:`to_datetime` with a read-only array incorrectly raising \
(:issue:`34857`)
- Bug in :meth:`Series.isin` with ``datetime64[ns]`` dtype and \
:meth:`.DatetimeIndex.isin` incorrectly casting integers to datetimes \
(:issue:`36621`)
- Bug in :meth:`Series.isin` with ``datetime64[ns]`` dtype and \
:meth:`.DatetimeIndex.isin` failing to consider tz-aware and tz-naive datetimes \
as always different (:issue:`35728`)
- Bug in :meth:`Series.isin` with ``PeriodDtype`` dtype and \
:meth:`PeriodIndex.isin` failing to consider arguments with different \
``PeriodDtype`` as always different (:issue:`37528`)
- Bug in :class:`Period` constructor now correctly handles nanoseconds in the \
``value`` argument (:issue:`34621` and :issue:`17053`)
Timedelta
- Bug in :class:`.TimedeltaIndex`, :class:`Series`, and :class:`DataFrame` \
floor-division with ``timedelta64`` dtypes and ``NaT`` in the denominator \
(:issue:`35529`)
- Bug in parsing of ISO 8601 durations in :class:`Timedelta` and \
:func:`to_datetime` (:issue:`29773`, :issue:`36204`)
- Bug in :func:`to_timedelta` with a read-only array incorrectly raising \
(:issue:`34857`)
- Bug in :class:`Timedelta` incorrectly truncating to sub-second portion of a \
string input when it has precision higher than nanoseconds (:issue:`36738`)
Timezones
- Bug in :func:`date_range` was raising ``AmbiguousTimeError`` for valid input \
with ``ambiguous=False`` (:issue:`35297`)
- Bug in :meth:`Timestamp.replace` was losing fold information (:issue:`37610`)
Numeric
- Bug in :func:`to_numeric` where float precision was incorrect (:issue:`31364`)
- Bug in :meth:`DataFrame.any` with ``axis=1`` and ``bool_only=True`` ignoring \
the ``bool_only`` keyword (:issue:`32432`)
- Bug in :meth:`Series.equals` where a ``ValueError`` was raised when NumPy \
arrays were compared to scalars (:issue:`35267`)
- Bug in :class:`Series` where two Series each have a :class:`.DatetimeIndex` \
with different time zones having those indexes incorrectly changed when \
performing arithmetic operations (:issue:`33671`)
- Bug in :mod:`pandas.testing` module functions when used with \
``check_exact=False`` on complex numeric types (:issue:`28235`)
- Bug in :meth:`DataFrame.__rmatmul__` error handling reporting transposed \
shapes (:issue:`21581`)
- Bug in :class:`Series` flex arithmetic methods where the result when operating \
with a ``list``, ``tuple`` or ``np.ndarray`` would have an incorrect name \
(:issue:`36760`)
- Bug in :class:`.IntegerArray` multiplication with ``timedelta`` and \
``np.timedelta64`` objects (:issue:`36870`)
- Bug in :class:`MultiIndex` comparison with tuple incorrectly treating tuple as \
array-like (:issue:`21517`)
- Bug in :meth:`DataFrame.diff` with ``datetime64`` dtypes including ``NaT`` \
values failing to fill ``NaT`` results correctly (:issue:`32441`)
- Bug in :class:`DataFrame` arithmetic ops incorrectly accepting keyword \
arguments (:issue:`36843`)
- Bug in :class:`.IntervalArray` comparisons with :class:`Series` not returning \
Series (:issue:`36908`)
- Bug in :class:`DataFrame` allowing arithmetic operations with list of \
array-likes with undefined results. Behavior changed to raising ``ValueError`` \
(:issue:`36702`)
- Bug in :meth:`DataFrame.std` with ``timedelta64`` dtype and ``skipna=False`` \
(:issue:`37392`)
- Bug in :meth:`DataFrame.min` and :meth:`DataFrame.max` with ``datetime64`` \
dtype and ``skipna=False`` (:issue:`36907`)
- Bug in :meth:`DataFrame.idxmax` and :meth:`DataFrame.idxmin` with mixed dtypes \
incorrectly raising ``TypeError`` (:issue:`38195`)
Conversion
- Bug in :meth:`DataFrame.to_dict` with ``orient='records'`` now returns python \
native datetime objects for datetime-like columns (:issue:`21256`)
- Bug in :meth:`Series.astype` conversion from ``string`` to ``float`` raised in \
presence of ``pd.NA`` values (:issue:`37626`)
Strings
- Bug in :meth:`Series.to_string`, :meth:`DataFrame.to_string`, and \
:meth:`DataFrame.to_latex` adding a leading space when ``index=False`` \
(:issue:`24980`)
- Bug in :func:`to_numeric` raising a ``TypeError`` when attempting to convert a \
string dtype Series containing only numeric strings and ``NA`` (:issue:`37262`)
Interval
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` where \
:class:`Interval` dtypes would be converted to object dtypes (:issue:`34871`)
- Bug in :meth:`IntervalIndex.take` with negative indices and \
``fill_value=None`` (:issue:`37330`)
- Bug in :meth:`IntervalIndex.putmask` with datetime-like dtype incorrectly \
casting to object dtype (:issue:`37968`)
- Bug in :meth:`IntervalArray.astype` incorrectly dropping dtype information \
with a :class:`CategoricalDtype` object (:issue:`37984`)
Indexing
- Bug in :meth:`PeriodIndex.get_loc` incorrectly raising ``ValueError`` on \
non-datelike strings instead of ``KeyError``, causing similar errors in \
:meth:`Series.__getitem__`, :meth:`Series.__contains__`, and \
:meth:`Series.loc.__getitem__` (:issue:`34240`)
- Bug in :meth:`Index.sort_values` where, when empty values were passed, the \
method would break by trying to compare missing values instead of pushing them \
to the end of the sort order (:issue:`35584`)
- Bug in :meth:`Index.get_indexer` and :meth:`Index.get_indexer_non_unique` \
where ``int64`` arrays are returned instead of ``intp`` (:issue:`36359`)
- Bug in :meth:`DataFrame.sort_index` where parameter ascending passed as a list \
on a single level index gives wrong result (:issue:`32334`)
- Bug in :meth:`DataFrame.reset_index` was incorrectly raising a ``ValueError`` \
for input with a :class:`MultiIndex` with missing values in a level with \
``Categorical`` dtype (:issue:`24206`)
- Bug in indexing with boolean masks on datetime-like values sometimes returning \
a view instead of a copy (:issue:`36210`)
- Bug in :meth:`DataFrame.__getitem__` and :meth:`DataFrame.loc.__getitem__` \
with :class:`IntervalIndex` columns and a numeric indexer (:issue:`26490`)
- Bug in :meth:`Series.loc.__getitem__` with a non-unique :class:`MultiIndex` \
and an empty-list indexer (:issue:`13691`)
- Bug in indexing on a :class:`Series` or :class:`DataFrame` with a \
:class:`MultiIndex` and a level named ``"0"`` (:issue:`37194`)
- Bug in :meth:`Series.__getitem__` when using an unsigned integer array as an \
indexer giving incorrect results or segfaulting instead of raising ``KeyError`` \
(:issue:`37218`)
- Bug in :meth:`Index.where` incorrectly casting numeric values to strings \
(:issue:`37591`)
- Bug in :meth:`DataFrame.loc` returning empty result when indexer is a slice \
with negative step size (:issue:`38071`)
- Bug in :meth:`Series.loc` and :meth:`DataFrame.loc` raises when the index was \
of ``object`` dtype and the given numeric label was in the index \
(:issue:`26491`)
- Bug in :meth:`DataFrame.loc` returned requested key plus missing values when \
``loc`` was applied to single level from a :class:`MultiIndex` (:issue:`27104`)
- Bug in indexing on a :class:`Series` or :class:`DataFrame` with a \
:class:`CategoricalIndex` using a list-like indexer containing NA values \
(:issue:`37722`)
- Bug in :meth:`DataFrame.loc.__setitem__` expanding an empty :class:`DataFrame` \
with mixed dtypes (:issue:`37932`)
- Bug in :meth:`DataFrame.xs` ignored ``droplevel=False`` for columns \
(:issue:`19056`)
- Bug in :meth:`DataFrame.reindex` raising ``IndexingError`` wrongly for empty \
DataFrame with ``tolerance`` not ``None`` or ``method="nearest"`` \
(:issue:`27315`)
- Bug in indexing on a :class:`Series` or :class:`DataFrame` with a \
:class:`CategoricalIndex` using list-like indexer that contains elements that \
are in the index's ``categories`` but not in the index itself failing to raise \
``KeyError`` (:issue:`37901`)
- Bug on inserting a boolean label into a :class:`DataFrame` with a numeric \
:class:`Index` columns incorrectly casting to integer (:issue:`36319`)
- Bug in :meth:`DataFrame.iloc` and :meth:`Series.iloc` aligning objects in \
``__setitem__`` (:issue:`22046`)
- Bug in :meth:`MultiIndex.drop` does not raise if labels are partially found \
(:issue:`37820`)
- Bug in :meth:`DataFrame.loc` did not raise ``KeyError`` when missing \
combination was given with ``slice(None)`` for remaining levels (:issue:`19556`)
- Bug in :meth:`DataFrame.loc` raising ``TypeError`` when non-integer slice was \
given to select values from :class:`MultiIndex` (:issue:`25165`, :issue:`24263`)
- Bug in :meth:`Series.at` returning :class:`Series` with one element instead of \
scalar when index is a :class:`MultiIndex` with one level (:issue:`38053`)
- Bug in :meth:`DataFrame.loc` returning and assigning elements in wrong order \
when indexer is differently ordered than the :class:`MultiIndex` to filter \
(:issue:`31330`, :issue:`34603`)
- Bug in :meth:`DataFrame.loc` and :meth:`DataFrame.__getitem__` raising \
``KeyError`` when columns were :class:`MultiIndex` with only one level \
(:issue:`29749`)
- Bug in :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` raising \
blank ``KeyError`` without missing keys for :class:`IntervalIndex` \
(:issue:`27365`)
- Bug in setting a new label on a :class:`DataFrame` or :class:`Series` with a \
:class:`CategoricalIndex` incorrectly raising ``TypeError`` when the new label \
is not among the index's categories (:issue:`38098`)
- Bug in :meth:`Series.loc` and :meth:`Series.iloc` raising ``ValueError`` when \
inserting a list-like ``np.array``, ``list`` or ``tuple`` in an ``object`` \
Series of equal length (:issue:`37748`, :issue:`37486`)
- Bug in :meth:`Series.loc` and :meth:`Series.iloc` setting all the values of an \
``object`` Series with those of a list-like ``ExtensionArray`` instead of \
inserting it (:issue:`38271`)
Missing
- Bug in :meth:`.SeriesGroupBy.transform` now correctly handles missing values \
for ``dropna=False`` (:issue:`35014`)
- Bug in :meth:`Series.nunique` with ``dropna=True`` was returning incorrect \
results when both ``NA`` and ``None`` missing values were present \
(:issue:`37566`)
- Bug in :meth:`Series.interpolate` where kwarg ``limit_area`` and \
``limit_direction`` had no effect when using methods ``pad`` and ``backfill`` \
(:issue:`31048`)
MultiIndex
- Bug in :meth:`DataFrame.xs` when used with :class:`IndexSlice` raises \
``TypeError`` with message ``"Expected label or tuple of labels"`` \
(:issue:`35301`)
- Bug in :meth:`DataFrame.reset_index` with ``NaT`` values in index raises \
``ValueError`` with message ``"cannot convert float NaN to integer"`` \
(:issue:`36541`)
- Bug in :meth:`DataFrame.combine_first` when used with :class:`MultiIndex` \
containing string and ``NaN`` values raises ``TypeError`` (:issue:`36562`)
- Bug in :meth:`MultiIndex.drop` dropped ``NaN`` values when non existing key \
was given as input (:issue:`18853`)
- Bug in :meth:`MultiIndex.drop` dropping more values than expected when index \
has duplicates and is not sorted (:issue:`33494`)
I/O
- :func:`read_sas` no longer leaks resources on failure (:issue:`35566`)
- Bug in :meth:`DataFrame.to_csv` and :meth:`Series.to_csv` caused a \
``ValueError`` when it was called with a filename in combination with ``mode`` \
containing a ``b`` (:issue:`35058`)
- Bug in :meth:`read_csv` with ``float_precision='round_trip'`` did not handle \
``decimal`` and ``thousands`` parameters (:issue:`35365`)
- :meth:`to_pickle` and :meth:`read_pickle` were closing user-provided file \
objects (:issue:`35679`)
- :meth:`to_csv` passes compression arguments for ``'gzip'`` always to \
``gzip.GzipFile`` (:issue:`28103`)
- :meth:`to_csv` did not support zip compression for binary file object not \
having a filename (:issue:`35058`)
- :meth:`to_csv` and :meth:`read_csv` did not honor ``compression`` and \
``encoding`` for path-like objects that are internally converted to file-like \
objects (:issue:`35677`, :issue:`26124`, :issue:`32392`)
- :meth:`DataFrame.to_pickle`, :meth:`Series.to_pickle`, and :meth:`read_pickle` \
did not support compression for file-objects (:issue:`26237`, :issue:`29054`, \
:issue:`29570`)
- Bug in :func:`LongTableBuilder.middle_separator` was duplicating LaTeX \
longtable entries in the List of Tables of a LaTeX document (:issue:`34360`)
- Bug in :meth:`read_csv` with ``engine='python'`` truncating data if multiple \
items present in first row and first element started with BOM (:issue:`36343`)
- Removed ``private_key`` and ``verbose`` from :func:`read_gbq` as they are no \
longer supported in ``pandas-gbq`` (:issue:`34654`, :issue:`30200`)
- Bumped minimum pytables version to 3.5.1 to avoid a ``ValueError`` in \
:meth:`read_hdf` (:issue:`24839`)
- Bug in :func:`read_table` and :func:`read_csv` when ``delim_whitespace=True`` \
and ``sep=default`` (:issue:`36583`)
- Bug in :meth:`DataFrame.to_json` and :meth:`Series.to_json` when used with \
``lines=True`` and ``orient='records'`` the last line of the record is not \
appended with 'new line character' (:issue:`36888`)
- Bug in :meth:`read_parquet` with fixed offset time zones. String \
representation of time zones was not recognized (:issue:`35997`, :issue:`36004`)
- Bug in :meth:`DataFrame.to_html`, :meth:`DataFrame.to_string`, and \
:meth:`DataFrame.to_latex` ignoring the ``na_rep`` argument when \
``float_format`` was also specified (:issue:`9046`, :issue:`13828`)
- Bug in output rendering of complex numbers showing too many trailing zeros \
(:issue:`36799`)
- Bug in :class:`HDFStore` threw a ``TypeError`` when exporting an empty \
DataFrame with ``datetime64[ns, tz]`` dtypes with a fixed HDF5 store \
(:issue:`20594`)
- Bug in :class:`HDFStore` was dropping time zone information when exporting a \
Series with ``datetime64[ns, tz]`` dtypes with a fixed HDF5 store \
(:issue:`20594`)
- :func:`read_csv` was closing user-provided binary file handles when \
``engine="c"`` and an ``encoding`` was requested (:issue:`36980`)
- Bug in :meth:`DataFrame.to_hdf` was not dropping missing rows with \
``dropna=True`` (:issue:`35719`)
- Bug in :func:`read_html` was raising a ``TypeError`` when supplying a \
``pathlib.Path`` argument to the ``io`` parameter (:issue:`37705`)
- :meth:`DataFrame.to_excel`, :meth:`Series.to_excel`, \
:meth:`DataFrame.to_markdown`, and :meth:`Series.to_markdown` now support \
writing to fsspec URLs such as S3 and Google Cloud Storage (:issue:`33987`)
- Bug in :func:`read_fwf` with ``skip_blank_lines=True`` was not skipping blank \
lines (:issue:`37758`)
- Parse missing values using :func:`read_json` with ``dtype=False`` to ``NaN`` \
instead of ``None`` (:issue:`28501`)
- :meth:`read_fwf` was inferring compression with ``compression=None`` which was \
not consistent with the other ``read_*`` functions (:issue:`37909`)
- :meth:`DataFrame.to_html` was ignoring ``formatters`` argument for \
``ExtensionDtype`` columns (:issue:`36525`)
- Bumped minimum xarray version to 0.12.3 to avoid reference to the removed \
``Panel`` class (:issue:`27101`, :issue:`37983`)
- :meth:`DataFrame.to_csv` was re-opening file-like handles that also implement \
``os.PathLike`` (:issue:`38125`)
- Bug in the conversion of a sliced ``pyarrow.Table`` with missing values to a \
DataFrame (:issue:`38525`)
- Bug in :func:`read_sql_table` raising a ``sqlalchemy.exc.OperationalError`` \
when column names contained a percentage sign (:issue:`37517`)
Period
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` where \
:class:`Period` dtypes would be converted to object dtypes (:issue:`34871`)
Plotting
- Bug in :meth:`DataFrame.plot` was rotating xticklabels when ``subplots=True``, \
even if the x-axis wasn't an irregular time series (:issue:`29460`)
- Bug in :meth:`DataFrame.plot` where a marker letter in the ``style`` keyword \
sometimes caused a ``ValueError`` (:issue:`21003`)
- Bug in :meth:`DataFrame.plot.bar` and :meth:`Series.plot.bar` where ticks \
positions were assigned by value order instead of using the actual value for \
numeric or a smart ordering for string (:issue:`26186`, :issue:`11465`). This \
fix has been reverted in pandas 1.2.1, see :doc:`v1.2.1`
- Twinned axes were losing their tick labels which should only happen to all but \
the last row or column of 'externally' shared axes (:issue:`33819`)
- Bug in :meth:`Series.plot` and :meth:`DataFrame.plot` was throwing a \
:exc:`ValueError` when the Series or DataFrame was
indexed by a :class:`.TimedeltaIndex` with a fixed frequency and the x-axis \
lower limit was greater than the upper limit (:issue:`37454`)
- Bug in :meth:`.DataFrameGroupBy.boxplot` when ``subplots=False`` would raise a \
``KeyError`` (:issue:`16748`)
- Bug in :meth:`DataFrame.plot` and :meth:`Series.plot` was overwriting \
matplotlib's shared y axes behavior when no ``sharey`` parameter was passed \
(:issue:`37942`)
- Bug in :meth:`DataFrame.plot` was raising a ``TypeError`` with \
``ExtensionDtype`` columns (:issue:`32073`)
Styler
- Bug in :meth:`Styler.render` HTML was generated incorrectly because of \
formatting error in ``rowspan`` attribute, it now matches with w3 syntax \
(:issue:`38234`)
Groupby/resample/rolling
- Bug in :meth:`.DataFrameGroupBy.count` and :meth:`SeriesGroupBy.sum` returning \
``NaN`` for missing categories when grouped on multiple ``Categoricals``. Now \
returning ``0`` (:issue:`35028`)
- Bug in :meth:`.DataFrameGroupBy.apply` that would sometimes throw an erroneous \
``ValueError`` if the grouping axis had duplicate entries (:issue:`16646`)
- Bug in :meth:`DataFrame.resample` that would throw a ``ValueError`` when \
resampling from ``"D"`` to ``"24H"`` over a transition into \
daylight savings time (DST) (:issue:`35219`)
- Bug when combining methods :meth:`DataFrame.groupby` with \
:meth:`DataFrame.resample` and :meth:`DataFrame.interpolate` raising a \
``TypeError`` (:issue:`35325`)
- Bug in :meth:`.DataFrameGroupBy.apply` where a non-nuisance grouping column \
would be dropped from the output columns if another groupby method was called \
before ``.apply`` (:issue:`34656`)
- Bug when subsetting columns on a \
:class:`~pandas.core.groupby.DataFrameGroupBy` (e.g. \
``df.groupby('a')[['b']])``) would reset the attributes ``axis``, ``dropna``, \
``group_keys``, ``level``, ``mutated``, ``sort``, and ``squeeze`` to their \
default values (:issue:`9959`)
- Bug in :meth:`.DataFrameGroupBy.tshift` failing to raise ``ValueError`` when a \
frequency cannot be inferred for the index of a group (:issue:`35937`)
- Bug in :meth:`DataFrame.groupby` does not always maintain column index name \
for ``any``, ``all``, ``bfill``, ``ffill``, ``shift`` (:issue:`29764`)
- Bug in :meth:`.DataFrameGroupBy.apply` raising error with ``np.nan`` group(s) \
when ``dropna=False`` (:issue:`35889`)
- Bug in :meth:`.Rolling.sum` returned wrong values when dtypes where mixed \
between float and integer and ``axis=1`` (:issue:`20649`, :issue:`35596`)
- Bug in :meth:`.Rolling.count` returned ``np.nan`` with \
:class:`~pandas.api.indexers.FixedForwardWindowIndexer` as window, \
``min_periods=0`` and only missing values in the window (:issue:`35579`)
- Bug where :class:`pandas.core.window.Rolling` produces incorrect window sizes \
when using a ``PeriodIndex`` (:issue:`34225`)
- Bug in :meth:`.DataFrameGroupBy.ffill` and :meth:`.DataFrameGroupBy.bfill` \
where a ``NaN`` group would return filled values instead of ``NaN`` when \
``dropna=True`` (:issue:`34725`)
- Bug in :meth:`.RollingGroupby.count` where a ``ValueError`` was raised when \
specifying the ``closed`` parameter (:issue:`35869`)
- Bug in :meth:`.DataFrameGroupBy.rolling` returning wrong values with partial \
centered window (:issue:`36040`)
- Bug in :meth:`.DataFrameGroupBy.rolling` returned wrong values with time aware \
window containing ``NaN``. Raises ``ValueError`` because windows are not \
monotonic now (:issue:`34617`)
- Bug in :meth:`.Rolling.__iter__` where a ``ValueError`` was not raised when \
``min_periods`` was larger than ``window`` (:issue:`37156`)
- Using :meth:`.Rolling.var` instead of :meth:`.Rolling.std` avoids numerical \
issues for :meth:`.Rolling.corr` when :meth:`.Rolling.var` is still within \
floating point precision while :meth:`.Rolling.std` is not (:issue:`31286`)
- Bug in :meth:`.DataFrameGroupBy.quantile` and :meth:`.Resampler.quantile` \
raised ``TypeError`` when values were of type ``Timedelta`` (:issue:`29485`)
- Bug in :meth:`.Rolling.median` and :meth:`.Rolling.quantile` returned wrong \
values for :class:`.BaseIndexer` subclasses with non-monotonic starting or \
ending points for windows (:issue:`37153`)
- Bug in :meth:`DataFrame.groupby` dropped ``nan`` groups from result with \
``dropna=False`` when grouping over a single column (:issue:`35646`, \
:issue:`35542`)
- Bug in :meth:`.DataFrameGroupBy.head`, :meth:`DataFrameGroupBy.tail`, \
:meth:`SeriesGroupBy.head`, and :meth:`SeriesGroupBy.tail` would raise when used \
with ``axis=1`` (:issue:`9772`)
- Bug in :meth:`.DataFrameGroupBy.transform` would raise when used with \
``axis=1`` and a transformation kernel (e.g. "shift") (:issue:`36308`)
- Bug in :meth:`.DataFrameGroupBy.resample` using ``.agg`` with sum produced \
different result than just calling ``.sum`` (:issue:`33548`)
- Bug in :meth:`.DataFrameGroupBy.apply` dropped values on ``nan`` group when \
returning the same axes with the original frame (:issue:`38227`)
- Bug in :meth:`.DataFrameGroupBy.quantile` couldn't handle with arraylike ``q`` \
when grouping by columns (:issue:`33795`)
- Bug in :meth:`DataFrameGroupBy.rank` with ``datetime64tz`` or period dtype \
incorrectly casting results to those dtypes instead of returning ``float64`` \
dtype (:issue:`38187`)
Reshaping
- Bug in :meth:`DataFrame.crosstab` was returning incorrect results on inputs \
with duplicate row names, duplicate column names or duplicate names between row \
and column labels (:issue:`22529`)
- Bug in :meth:`DataFrame.pivot_table` with ``aggfunc='count'`` or \
``aggfunc='sum'`` returning ``NaN`` for missing categories when pivoted on a \
``Categorical``. Now returning ``0`` (:issue:`31422`)
- Bug in :func:`concat` and :class:`DataFrame` constructor where input index \
names are not preserved in some cases (:issue:`13475`)
- Bug in func :meth:`crosstab` when using multiple columns with ``margins=True`` \
and ``normalize=True`` (:issue:`35144`)
- Bug in :meth:`DataFrame.stack` where an empty DataFrame.stack would raise an \
error (:issue:`36113`). Now returning an empty Series with empty MultiIndex.
- Bug in :meth:`Series.unstack`. Now a Series with single level of Index trying \
to unstack would raise a ``ValueError`` (:issue:`36113`)
- Bug in :meth:`DataFrame.agg` with ``func={'name':<FUNC>}`` incorrectly \
raising ``TypeError`` when ``DataFrame.columns==['Name']`` (:issue:`36212`)
- Bug in :meth:`Series.transform` would give incorrect results or raise when the \
argument ``func`` was a dictionary (:issue:`35811`)
- Bug in :meth:`DataFrame.pivot` did not preserve :class:`MultiIndex` level \
names for columns when rows and columns are both multiindexed (:issue:`36360`)
- Bug in :meth:`DataFrame.pivot` modified ``index`` argument when ``columns`` \
was passed but ``values`` was not (:issue:`37635`)
- Bug in :meth:`DataFrame.join` returned a non deterministic level-order for the \
resulting :class:`MultiIndex` (:issue:`36910`)
- Bug in :meth:`DataFrame.combine_first` caused wrong alignment with dtype \
``string`` and one level of ``MultiIndex`` containing only ``NA`` \
(:issue:`37591`)
- Fixed regression in :func:`merge` on merging :class:`.DatetimeIndex` with \
empty DataFrame (:issue:`36895`)
- Bug in :meth:`DataFrame.apply` not setting index of return value when ``func`` \
return type is ``dict`` (:issue:`37544`)
- Bug in :meth:`DataFrame.merge` and :meth:`pandas.merge` returning inconsistent \
ordering in result for ``how=right`` and ``how=left`` (:issue:`35382`)
- Bug in :func:`merge_ordered` couldn't handle list-like ``left_by`` or \
``right_by`` (:issue:`35269`)
- Bug in :func:`merge_ordered` returned wrong join result when length of \
``left_by`` or ``right_by`` equals to the rows of ``left`` or ``right`` \
(:issue:`38166`)
- Bug in :func:`merge_ordered` didn't raise when elements in ``left_by`` or \
``right_by`` not exist in ``left`` columns or ``right`` columns (:issue:`38167`)
- Bug in :func:`DataFrame.drop_duplicates` not validating bool dtype for \
``ignore_index`` keyword (:issue:`38274`)
ExtensionArray
- Fixed bug where :class:`DataFrame` column set to scalar extension type via a \
dict instantiation was considered an object type rather than the extension type \
(:issue:`35965`)
- Fixed bug where ``astype()`` with equal dtype and ``copy=False`` would return \
a new object (:issue:`28488`)
- Fixed bug when applying a NumPy ufunc with multiple outputs to an \
:class:`.IntegerArray` returning ``None`` (:issue:`36913`)
- Fixed an inconsistency in :class:`.PeriodArray`'s ``__init__`` signature to \
those of :class:`.DatetimeArray` and :class:`.TimedeltaArray` (:issue:`37289`)
- Reductions for :class:`.BooleanArray`, :class:`.Categorical`, \
:class:`.DatetimeArray`, :class:`.FloatingArray`, :class:`.IntegerArray`, \
:class:`.PeriodArray`, :class:`.TimedeltaArray`, and :class:`.PandasArray` are \
now keyword-only methods (:issue:`37541`)
- Fixed a bug where a ``TypeError`` was wrongly raised if a membership check \
was made on an ``ExtensionArray`` containing nan-like values (:issue:`37867`)
Other
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` incorrectly \
raising an ``AssertionError`` instead of a ``ValueError`` when invalid parameter \
combinations are passed (:issue:`36045`)
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` with numeric \
values and string ``to_replace`` (:issue:`34789`)
- Fixed metadata propagation in :meth:`Series.abs` and ufuncs called on Series \
and DataFrames (:issue:`28283`)
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` incorrectly \
casting from ``PeriodDtype`` to object dtype (:issue:`34871`)
- Fixed bug in metadata propagation incorrectly copying DataFrame columns as \
metadata when the column name overlaps with the metadata name (:issue:`37037`)
- Fixed metadata propagation in the :class:`Series.dt`, :class:`Series.str` \
accessors, :class:`DataFrame.duplicated`, :class:`DataFrame.stack`, \
:class:`DataFrame.unstack`, :class:`DataFrame.pivot`, :class:`DataFrame.append`, \
:class:`DataFrame.diff`, :class:`DataFrame.applymap` and \
:class:`DataFrame.update` methods (:issue:`28283`, :issue:`37381`)
- Fixed metadata propagation when selecting columns with \
``DataFrame.__getitem__`` (:issue:`28283`)
- Bug in :meth:`Index.intersection` with non-:class:`Index` failing to set the \
correct name on the returned :class:`Index` (:issue:`38111`)
- Bug in :meth:`RangeIndex.intersection` failing to set the correct name on the \
returned :class:`Index` in some corner cases (:issue:`38197`)
- Bug in :meth:`Index.difference` failing to set the correct name on the \
returned :class:`Index` in some corner cases (:issue:`38268`)
- Bug in :meth:`Index.union` behaving differently depending on whether operand \
is an :class:`Index` or other list-like (:issue:`36384`)
- Bug in :meth:`Index.intersection` with non-matching numeric dtypes casting to \
``object`` dtype instead of minimal common dtype (:issue:`38122`)
- Bug in :meth:`IntervalIndex.union` returning an incorrectly-typed \
:class:`Index` when empty (:issue:`38282`)
- Passing an array with 2 or more dimensions to the :class:`Series` constructor \
now raises the more specific ``ValueError`` rather than a bare ``Exception`` \
(:issue:`35744`)
- Bug in ``dir`` where ``dir(obj)`` wouldn't show attributes defined on the \
instance for pandas objects (:issue:`37173`)
- Bug in :meth:`Index.drop` raising ``InvalidIndexError`` when index has \
duplicates (:issue:`38051`)
- Bug in :meth:`RangeIndex.difference` returning :class:`Int64Index` in some \
cases where it should return :class:`RangeIndex` (:issue:`38028`)
- Fixed bug in :func:`assert_series_equal` when comparing a datetime-like array \
with an equivalent non extension dtype array (:issue:`37609`)
- Bug in :func:`.is_bool_dtype` would raise when passed a valid string such as \
``"boolean"`` (:issue:`38386`)
- Fixed regression in logical operators raising ``ValueError`` when columns of \
:class:`DataFrame` are a :class:`CategoricalIndex` with unused categories \
(:issue:`38367`)
Files: