2020-10-03 08:03:18 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.17.0 Release 0.17.0 - Fix a spurious invalidation of `Memory.cache`'d functions called with `Parallel` under Jupyter or IPython. - Bump vendored loky to 2.9.0 and cloudpickle to 1.6.0. In particular this fixes a problem to add compat for Python 3.9. |
2020-08-10 16:41:37 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message: py-joblib: updated to 0.16.0 Release 0.16.0 -------------- - Fix a problem in the constructors of of Parallel backends classes that inherit from the `AutoBatchingMixin` that prevented the dask backend to properly batch short tasks. - Fix a problem in the way the joblib dask backend batches calls that would badly interact with the dask callable pickling cache and lead to wrong results or errors. - Prevent a dask.distributed bug from surfacing in joblib's dask backend during nested Parallel calls (due to joblib's auto-scattering feature) - Workaround for a race condition after Parallel calls with the dask backend that would cause low level warnings from asyncio coroutines: |
2020-05-28 14:06:28 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message: py-joblib: updated 0.15.1 Release 0.15.1 - Make joblib work on Python 3 installation that do not ship with the lzma package in their standard library. |
2020-05-16 19:20:21 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.15.0 Release 0.15.0 - Drop support for Python 2 and Python 3.5. All objects in ``joblib.my_exceptions`` and ``joblib.format_stack`` are now deprecated and will be removed in joblib 0.16. Note that no deprecation warning will be raised for these objects Python < 3.7. - Fix many bugs related to the temporary files and folder generated when automatically memory mapping large numpy arrays for efficient inter-process communication. In particular, this would cause `PermissionError` exceptions to be raised under Windows and large leaked files in `/dev/shm` under Linux in case of crash. - Make the dask backend collect results as soon as they complete leading to a performance improvement: - Fix the number of jobs reported by ``effective_n_jobs`` when ``n_jobs=None`` called in a parallel backend context. - Upgraded vendored cloupickle to 1.4.1 and loky to 2.8.0. This allows for Parallel calls of dynamically defined functions with type annotations in particular. |
2019-12-14 11:54:16 by Adam Ciarcinski | Files touched by this commit (2) | |
Log message: py-joblib: updated to 0.14.1 Release 0.14.1 Configure the loky workers' environment to mitigate oversubsription with nested \ multi-threaded code in the following case: allow for a suitable number of threads for numba (NUMBA_NUM_THREADS); enable Interprocess Communication for scheduler coordination when the nested \ code uses Threading Building Blocks (TBB) (ENABLE_IPC=1) https://github.com/joblib/joblib/pull/951 Fix a regression where the loky backend was not reusing previously spawned \ workers. https://github.com/joblib/joblib/pull/968 Revert https://github.com/joblib/joblib/pull/847 to avoid using pkg_resources \ that introduced a performance regression under Windows: \ https://github.com/joblib/joblib/issues/965 |
2019-10-31 11:24:52 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.14.0 Release 0.14.0 Improved the load balancing between workers to avoid stranglers caused by an \ excessively large batch size when the task duration is varying significantly \ (because of the combined use of joblib.Parallel and joblib.Memory with a \ partially warmed cache for instance). https://github.com/joblib/joblib/pull/899 Add official support for Python 3.8: fixed protocol number in Hasher and updated \ tests. Fix a deadlock when using the dask backend (when scattering large numpy arrays). \ https://github.com/joblib/joblib/pull/914 Warn users that they should never use joblib.load with files from untrusted \ sources. Fix security related API change introduced in numpy 1.6.3 that would \ prevent using joblib with recent numpy versions. \ https://github.com/joblib/joblib/pull/879 Upgrade to cloudpickle 1.1.1 that add supports for the upcoming Python 3.8 \ release among other things. https://github.com/joblib/joblib/pull/878 Fix semaphore availability checker to avoid spawning resource trackers on module \ import. https://github.com/joblib/joblib/pull/893 Fix the oversubscription protection to only protect against nested Parallel \ calls. This allows joblib to be run in background threads. \ https://github.com/joblib/joblib/pull/934 Fix ValueError (negative dimensions) when pickling large numpy arrays on \ Windows. https://github.com/joblib/joblib/pull/920 Upgrade to loky 2.6.0 that add supports for the setting environment variables in \ child before loading any module. https://github.com/joblib/joblib/pull/940 Fix the oversubscription protection for native libraries using threadpools \ (OpenBLAS, MKL, Blis and OpenMP runtimes). The maximal number of threads is can \ now be set in children using the inner_max_num_threads in parallel_backend. It \ defaults to cpu_count() // n_jobs. |
2019-02-14 10:01:45 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.13.2 0.13.2: Add a non-regression test, reporting that cloudpickle versions between 0.5.4 and \ 0.7 introduced a bug where global variables changes in a parent process between \ two calls to joblib.Parallel would not be propagated into the workers 0.13.1: Memory now accepts pathlib.Path objects as location parameter. Also, a warning \ is raised if the returned backend is None while location is not None. Make Parallel raise an informative RuntimeError when the active parallel backend \ has zero worker. Make the DaskDistributedBackend wait for workers before trying to schedule work. \ This is useful in particular when the workers are provisionned dynamically but \ provisionning is not immediate (for instance using Kubernetes, Yarn or an HPC \ job queue). |
2018-11-30 13:20:44 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.13.0 Release 0.13.0: Fix nested backend in SequentialBackend to avoid changing the default backend to Sequential. Fix nested_backend behavior to avoid setting the default number of workers to -1 when the backend is not dask. Release 0.12.5 Include loky 2.3.1 with better error reporting when a worker is abruptly terminated. Also fixes spurious debug output. Include cloudpickle 0.5.6. Fix a bug with the handling of global variables by locally defined functions. Release 0.12.4 Include loky 2.3.0 with many bugfixes, notably w.r.t. when setting non-default multiprocessing contexts. Also include improvement on memory management of long running worker processes and fixed issues when using the loky backend under PyPy. Raises a more explicit exception when a corrupted MemorizedResult is loaded. Loading a corrupted cached file with mmap mode enabled would recompute the results and return them without memmory mapping. Release 0.12.3 Fix joblib import setting the global start_method for multiprocessing. Fix MemorizedResult not picklable. Fix Memory, MemorizedFunc and MemorizedResult round-trip pickling + unpickling. Fixed a regression in Memory when positional arguments are called as kwargs several times with different values. Integration of loky 2.2.2 that fixes issues with the selection of the default start method and improve the reporting when calling functions with arguments that raise an exception when unpickling. Prevent MemorizedFunc.call_and_shelve from loading cached results to RAM when not necessary. Results in big performance improvements |
2018-08-14 19:39:32 by Adam Ciarcinski | Files touched by this commit (3) | |
Log message: py-joblib: updated to 0.12.2 Release 0.12.2: Integrate loky 2.2.0 to fix regression with unpicklable arguments and functions \ reported by users. Loky 2.2.0 also provides a protection against memory leaks long running \ applications when psutil is installed. Joblib now includes the code for the dask backend which has been updated to \ properly handle nested parallelism and data scattering at the same time. Restored some private API attribute and arguments (MemorizedResult.argument_hash \ and BatchedCalls.__init__'s pickle_cache) for backward compat. Fix a deprecation warning message (for Memory's cachedir). Release 0.12.1: Make sure that any exception triggered when serializing jobs in the queue will \ be wrapped as a PicklingError as in past versions of joblib. Fix kwonlydefaults key error in filter_args |
2018-07-06 05:13:36 by Min Sik Kim | Files touched by this commit (4) | |
Log message: devel/py-joblib: Import version 0.12.0 Joblib is a set of tools to provide lightweight pipelining in Python. In particular, joblib offers transparent disk-caching of the output values and lazy re-evaluation (memoize pattern), easy simple parallel computing, and logging and tracing of the execution. Joblib is optimized to be fast and robust in particular on large data and has specific optimizations for numpy arrays. Packaged by Kamel Ibn Aziz Derouiche for pkgsrc-wip and updated by me. |