complex functional testing for applications and libraries.
2020-05-16 10:00:01 by Adam Ciarcinski | Files touched by this commit (9) | |
Log message:
py-test: updated to 5.4.2
pytest 5.4.2:
Bug Fixes
* Fix crash with captured output when using the capsysbinary fixture.
* Ensure a unittest.IsolatedAsyncioTestCase is actually awaited.
* Fix TerminalRepr instances to be hashable again.
* Fix regression where functions registered with TestCase.addCleanup were not \
being called on test failures.
* Allow users to still set the deprecated TerminalReporter.writer attribute.
* Revert “tmpdir: clean up indirection via config for factories” 6767 as it \
breaks pytest-xdist.
* Fixed regression: asyncbase.TestCase tests are executed correctly again.
* Fix File.from_constructor so it forwards extra keyword arguments to the \
constructor.
* Classes with broken __getattribute__ methods are displayed correctly during \
failures.
* Fix _is_setup_py for files encoded differently than locale.
pytest 5.4.1:
Bug Fixes
* Revert the change introduced by 6330, which required all arguments to \
@pytest.mark.parametrize to be explicitly defined in the function signature.
The intention of the original change was to remove what was expected to be an \
unintended/surprising behavior, but it turns out many people relied on it, so \
the restriction has been reverted.
* Fix crash when plugins return an unknown stats while using the --reportlog option.
pytest 5.4.0:
Breaking Changes
* Matching of -k EXPRESSION to test names is now case-insensitive.
* Plugins specified with -p are now loaded after internal plugins, which results \
in their hooks being called before the internal ones.
This makes the -p behavior consistent with PYTEST_PLUGINS.
* Removed the long-deprecated pytest_itemstart hook.
This hook has been marked as deprecated and not been even called by pytest for \
over 10 years now.
* Reversed / fix meaning of “+/-” in error diffs. “-” means that sth. \
expected is missing in the result and “+” means that there are unexpected \
extras in the result.
* The cached_result attribute of FixtureDef is now set to None when the result \
is unavailable, instead of being deleted.
If your plugin performs checks like hasattr(fixturedef, 'cached_result'), for \
example in a pytest_fixture_post_finalizer hook implementation, replace it with \
fixturedef.cached_result is not None. If you del the attribute, set it to None \
instead.
Deprecations
* Option --no-print-logs is deprecated and meant to be removed in a future \
release. If you use --no-print-logs, please try out --show-capture and provide \
feedback.
--show-capture command-line option was added in pytest 3.5.0 and allows to \
specify how to display captured output when tests fail: no, stdout, stderr, log \
or all (the default).
* Deprecate the unused/broken pytest_collect_directory hook. It was misaligned \
since the removal of the Directory collector in 2010 and incorrect/unusable as \
soon as collection was split from test execution.
* Deprecate using direct constructors for Nodes.
Instead they are now constructed via Node.from_parent.
This transitional mechanism enables us to untangle the very intensely entangled \
Node relationships by enforcing more controlled creation/configuration patterns.
As part of this change, session/config are already disallowed parameters and as \
we work on the details we might need disallow a few more as well.
Subclasses are expected to use super().from_parent if they intend to expand the \
creation of Nodes.
* The TerminalReporter.writer attribute has been deprecated and should no longer \
be used. This was inadvertently exposed as part of the public API of that plugin \
and ties it too much with py.io.TerminalWriter.
Features
* New –capture=tee-sys option to allow both live printing and capturing of \
test output.
* Now all arguments to @pytest.mark.parametrize need to be explicitly declared \
in the function signature or via indirect. Previously it was possible to omit an \
argument if a fixture with the same name existed, which was just an accident of \
implementation and was not meant to be a part of the API.
* Changed default for -r to fE, which displays failures and errors in the short \
test summary. -rN can be used to disable it (the old behavior).
* New options have been added to the junit_logging option: log, out-err, and all.
* Excess warning summaries are now collapsed per file to ensure readable display \
of warning summaries.
Improvements
* pytest.mark.parametrize accepts integers for ids again, converting it to strings.
* Use “yellow” main color with any XPASSED tests.
* Revert “A warning is now issued when assertions are made for None”.
The warning proved to be less useful than initially expected and had quite a few \
false positive cases.
* tmpdir_factory.mktemp now fails when given absolute and non-normalized paths.
* The pytest_warning_captured hook now receives a location parameter with the \
code location that generated the warning.
* pytester: the testdir fixture respects environment settings from the \
monkeypatch fixture for inner runs.
* --fulltrace is honored with collection errors.
* Make --showlocals work also with --tb=short.
* Add support for matching lines consecutively with LineMatcher’s \
fnmatch_lines() and re_match_lines().
* Code is now highlighted in tracebacks when pygments is installed.
Users are encouraged to install pygments into their environment and provide \
feedback, because the plan is to make pygments a regular dependency in the \
future.
* Import usage error message with invalid -o option.
* pytest.mark.parametrize supports iterators and generators for ids.
Bug Fixes
* Add support for calling pytest.xfail() and pytest.importorskip() with doctests.
* --trace now works with unittests.
* Fixed some warning reports produced by pytest to point to the correct location \
of the warning in the user’s code.
* Fix --last-failed to collect new tests from files with known failures.
* Report PytestUnknownMarkWarning at the level of the user’s code, not pytest’s.
* Fix interaction with --pdb and unittests: do not use unittest’s TestCase.debug().
* Fix summary entries appearing twice when f/F and s/S report chars were used at \
the same time in the -r command-line option (for example -rFf).
The upper case variants were never documented and the preferred form should be \
the lower case.
* Fallback to green (instead of yellow) for non-last items without previous \
passes with colored terminal progress indicator.
* --disable-warnings is honored with -ra and -rA.
* Fix bug in the comparison of request key with cached key in fixture.
A construct if key == cached_key: can fail either because == is explicitly \
disallowed, or for, e.g., NumPy arrays, where the result of a == b cannot \
generally be converted to bool. The implemented fix replaces == with is.
* Make capture output streams .write() method return the same return value from \
original streams.
* Fix EncodedFile.writelines to call the underlying buffer’s writelines method.
* Fix internal crash when faulthandler starts initialized (for example with \
PYTHONFAULTHANDLER=1 environment variable set) and faulthandler_timeout defined \
in the configuration file.
* Fix node ids which contain a parametrized empty-string variable.
* Assertion rewriting hooks are (re)stored for the current item, which fixes \
them being still used after e.g. pytester’s testdir.runpytest etc.
* pytest.exit() is handled when emitted from the pytest_sessionfinish hook. This \
includes quitting from a debugger.
* When pytest.raises() is used as a function (as opposed to a context manager), \
a match keyword argument is now passed through to the tested function. \
Previously it was swallowed and ignored (regression in pytest 5.1.0).
* Do not display empty lines inbetween traceback for unexpected exceptions with \
doctests.
* The testdir fixture works within doctests now.
Improved Documentation
* Add list of fixtures to start of fixture chapter.
* Expand first sentence on fixtures into a paragraph.
Trivial/Internal Changes
* Remove usage of parser module, deprecated in Python 3.9.
|
2020-01-30 20:44:13 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.3.5
pytest 5.3.5:
Bug Fixes
- Fix regression in pytest 5.3.4 causing an INTERNALERROR due to a wrong assertion.
|
2020-01-22 19:56:04 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.3.4
pytest 5.3.4:
Bug Fixes
Revert 6436: unfortunately this change has caused a number of regressions in \
many suites, so the team decided to revert this change and make a new release \
while we continue to look for a solution.
|
2020-01-20 12:56:13 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.3.3
pytest 5.3.3:
Bug Fixes
* Captured output during teardown is shown with -rP.
* Fix a pytest-xdist crash when dealing with exceptions raised in subprocesses \
created by the multiprocessing module.
* FixtureDef objects now properly register their finalizers with autouse and \
parameterized fixtures that execute before them in the fixture stack so they are \
torn down at the right times, and in the right order.
* Fix parsing of outcomes containing multiple errors with testdir results \
(regression in 5.3.0).
Trivial/Internal Changes
* Optimized automatic renaming of test parameter IDs.
|
2019-12-15 12:30:37 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message:
py-test5: updated to 5.3.2
pytest 5.3.2:
Improvements
* Revert “A warning is now issued when assertions are made for None”.
The warning proved to be less useful than initially expected and had quite a few \
false positive cases.
Bug Fixes
* junitxml: Logs for failed test are now passed to junit report in case the test \
fails during call phase.
* The supporting files in the .pytest_cache directory are kept with \
--cache-clear, which only clears cached values now.
* Fix assertion rewriting for egg-based distributions and editable installs (pip \
install --editable).
|
2019-11-28 07:28:25 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.3.1
pytest 5.3.1:
Improvements
* Improve check for misspelling of pytest.mark.parametrize.
* Handle exit.Exception raised in notify_exception (via pytest_internalerror), \
e.g. when quitting pdb from post mortem.
Bug Fixes
* pytester: fix no_fnmatch_line when used after positive matching.
* Fix line detection for doctest samples inside property docstrings, as a \
workaround to bpo-17446.
* Fix compatibility with pytest-parallel (regression in pytest 5.3.0).
* Clear the sys.last_traceback, sys.last_type and sys.last_value attributes by \
deleting them instead of setting them to None. This better matches the behaviour \
of the Python standard library.
|
2019-11-20 09:34:19 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.3.0
pytest 5.3.0:
Deprecations
* The default value of junit_family option will change to xunit2 in pytest 6.0, \
given that this is the version supported by default in modern tools that \
manipulate this type of file.
In order to smooth the transition, pytest will issue a warning in case the \
--junitxml option is given in the command line but junit_family is not \
explicitly configured in pytest.ini.
For more information, see the docs.
Features
* The pytest team has created the pytest-reportlog plugin, which provides a new \
--report-log=FILE option that writes report logs into a file as the test session \
executes.
Each line of the report log contains a self contained JSON object corresponding \
to a testing event, such as a collection or a test result report. The file is \
guaranteed to be flushed after writing each line, so systems can read and \
process events in real-time.
The plugin is meant to replace the --resultlog option, which is deprecated and \
meant to be removed in a future release. If you use --resultlog, please try out \
pytest-reportlog and provide feedback.
* When sys.pycache_prefix (Python 3.8+) is set, it will be used by pytest to \
cache test files changed by the assertion rewriting mechanism.
This makes it easier to benefit of cached .pyc files even on file systems \
without permissions.
* Allow selective auto-indentation of multiline log messages.
Adds command line option --log-auto-indent, config option log_auto_indent and \
support for per-entry configuration of indentation behavior on calls to \
logging.log().
Alters the default for auto-indention from on to off. This restores the older \
behavior that existed prior to v4.6.0. This reversion to earlier behavior was \
done because it is better to activate new features that may lead to broken tests \
explicitly rather than implicitly.
* pytester learned two new functions, no_fnmatch_line and no_re_match_line.
The functions are used to ensure the captured text does not match the given pattern.
The previous idiom was to use re.match:
assert re.match(pat, result.stdout.str()) is None
Or the in operator:
assert text in result.stdout.str()
But the new functions produce best output on failure.
* Added tolerances to complex values when printing pytest.approx.
For example, repr(pytest.approx(3+4j)) returns (3+4j) ± 5e-06 ∠ ±180°. This \
is polar notation indicating a circle around the expected value, with a radius \
of 5e-06. For approx comparisons to return True, the actual value should fall \
within this circle.
* Added the pluginmanager as an argument to pytest_addoption so that hooks can \
be invoked when setting up command line options. This is useful for having one \
plugin communicate things to another plugin, such as default values or which set \
of command line options to add.
Improvements
* Use multiple colors with terminal summary statistics.
* Quitting from debuggers is now properly handled in doctest items.
* Improved verbose diff output with sequences.
Before:
After:
* Display untruncated assertion message with -vv.
* Fixed plurality mismatch in test summary (e.g. display “1 error” instead \
of “1 errors”).
* Config.InvocationParams.args is now always a tuple to better convey that it \
should be immutable and avoid accidental modifications.
* pytest.main now returns a pytest.ExitCode instance now, except for when custom \
exit codes are used (where it returns int then still).
* Align prefixes in output of pytester’s LineMatcher.
* Collection errors are reported as errors (and not failures like before) in the \
terminal’s short test summary.
* pytester.spawn does not skip/xfail tests on FreeBSD anymore unconditionally.
* The “[XXX%]” indicator in the test summary is now colored according to the \
final (new) multi-colored line’s main color.
* Added --co as a synonym to --collect-only.
* atomicwrites is now only used on Windows, fixing a performance regression with \
assertion rewriting on Unix.
* Now parametrization will use the __name__ attribute of any object for the id, \
if present. Previously it would only use __name__ for functions and classes.
* Improved failure reporting with pytester’s Hookrecorder.assertoutcome.
* The reason for a stopped session, e.g. with --maxfail / -x, now gets reported \
in the test summary.
* Improved cache.set robustness and performance.
Bug Fixes
* Fixed --setup-plan showing inaccurate information about fixture lifetimes.
* Fixed line offset mismatch of skipped tests in terminal summary.
* The PytestDoctestRunner is now properly invalidated when unconfiguring the \
doctest plugin.
This is important when used with pytester’s runpytest_inprocess.
* BaseExceptions are now handled in saferepr, which includes \
pytest.fail.Exception etc.
* pytester: fixed order of arguments in rm_rf warning when cleaning up temporary \
directories, and do not emit warnings for errors with os.open.
* Fixed result of getmodpath method.
Trivial/Internal Changes
* RunResult from pytester now displays the mnemonic of the ret attribute when it \
is a valid pytest.ExitCode value.
|
2019-11-18 10:04:48 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message:
py-test5: updated to 5.2.4
pytest 5.2.4:
Bug Fixes
- Fix incorrect discovery of non-test ``__init__.py`` files.
- Revert "The first test in a package (``__init__.py``) marked with \
``@pytest.mark.skip`` is now correctly skipped.".
|