./parallel/parallel, Build and execute shell commands in parallel

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 20210722, Package name: parallel-20210722, Maintainer: cheusov

GNU Parallel is a shell tool for executing jobs in parallel using one or
more machines. A job is typically a single command or a small script that
has to be run for each of the lines in the input. The typical input is a
list of files, a list of hosts, a list of users, or a list of tables.


Required to run:
[lang/perl5]

Required to build:
[pkgtools/cwrappers]

Master sites: (Expand)

Filesize: 2196.185 KB

Version history: (Expand)


CVS history: (Expand)


   2021-10-26 13:10:38 by Nia Alarie | Files touched by this commit (29)
Log message:
parallel: Replace RMD160 checksums with BLAKE2s checksums

All checksums have been double-checked against existing RMD160 and
SHA512 hashes
   2021-10-07 16:46:48 by Nia Alarie | Files touched by this commit (29)
Log message:
parallel: Remove SHA1 hashes for distfiles
   2021-08-13 14:46:09 by Nia Alarie | Files touched by this commit (2) | Package updated
Log message:
parallel: update to 20210722

New in this release:

* --results no longer prints the result to standard output (stdout) as voted in \ 
https://lists.gnu.org/archive/html/parallel/2020-12/msg00003.html
* parset supports associative arrays in bash, ksh, zsh.
* Online HTML is now generated by Sphinx.
* Bug fixes and man page updates.
   2021-05-24 21:56:06 by Thomas Klausner | Files touched by this commit (3575)
Log message:
*: recursive bump for perl 5.34
   2021-05-02 10:48:33 by Nia Alarie | Files touched by this commit (3) | Package updated
Log message:
parallel: update to 20210422

20210422

* Bug fixes and man page updates.

20210322

* Bug fixes and man page updates.
   2020-11-24 19:14:20 by Nia Alarie | Files touched by this commit (2) | Package updated
Log message:
parallel: Update to 20201122

20201122

* Bug fixes and man page updates.

News about GNU Parallel:

* https://bash-prompt.net/guides/gnu-parallel-multi-server/

20201022

* --termseq now works for jobs running remotely.

* parsort: Performance optimized for 64-core machines.

* Bug fixes and man page updates.

News about GNU Parallel:

* Juan Sierra Pons has made a testimonial:
  http://www.elsotanillo.net/wp-content/uploads/GnuParallel_JuanSierraPons.mp4

* 「翻译」在生物信息学中使用 GNU-Parallel
  https://cloud.tencent.com/developer/article/1705499

20200922

* New CPU detection for GNU/Linux.

* Bug fixes and man page updates.

News about GNU Parallel:

* Aug 21, 2020 - More Unix tools
  https://floki.blog/2020/08/more_unix_tools

20200822

New in this release:

* Max command line length is changed for MacOS - making this version
  beta quality for MacOS.

* Bug fixes and man page updates.

20200722

New in this release:

* No new functionality

* Bug fixes and man page updates.

News about GNU Parallel:

* Tips for Running Jobs on your VM
  \ 
https://support.ehelp.edu.au/support/solutions/articles/6000089713-tips-for-running-jobs-on-your-vm#parallel

* Introduction to GNU Parallel: Parallelizing Massive Individual Tasks
  https://www.youtube.com/watch?v=QSFLTEyCKvo

* Slides for same
  \ 
http://www.hpc.lsu.edu/training/weekly-materials/2020-Spring/GNUParallel_Spring_2020.pdf

* Linux-Fu: Parallel Universe
  https://hackaday.com/2020/06/29/linux-fu-parallel-universe/

* Resumable batch jobs with gnu parallel https://igor.io/parallel/

* Introducing Parallel into Shell
  https://www.petelawson.com/post/parallel-in-shell/

* AV1 encoding parallelization with GNU Parallel
  \ 
https://www.reddit.com/r/AV1/comments/fjvqp9/av1_encoding_parallelization_with_gnu_parallel/

* Beschleunigte Befehlsbearbeitung mit GNU Parallel
  \ 
https://www.linux-community.de/ausgaben/linuxuser/2013/06/beschleunigte-befehlsbearbeitung-mit-gnu-parallel/

20200622

New in this release:

* No new functionality

* Bug fixes and man page updates.

News about GNU Parallel:

* GNU Parallel is used at NERSC:
  https://www.nersc.gov/assets/Uploads/08-Workflows-20200616.pdf

* Resumable batch jobs with GNU Parallel https://igor.io/parallel/

* Lab Meeting technical talk
  https://www.slideshare.net/hoffmanlab/gnu-parallel-194030490

* Open Sky Software Website Credits
  https://www.openskysoftware.com/site-credits.htm

* Speed up your Python Scripts using Ubuntu and GNU Parallel
  \ 
https://ecce.esri.ca/wpecce/2019/12/24/speed-up-your-python-scripts-using-ubuntu-and-gnu-parallel/

* Indispensable command-line tools
  https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html

* GNU Parallel for simulations
  http://pdebuyl.be/blog/2020/gnu-parallel-for-simulations.html

* Introduction to GNU parallel
  https://bioinformaticsworkbook.org/Appendix/GNUparallel/GNU_parallel_examples.html

20200522

New in this release:

* While running a job $PARALLEL_JOBSLOT is the jobslot of the job. It
  is equal to {%} unless the job is being retried. See {%} for
  details.

* While running a job $PARALLEL_SSHLOGIN is the sshlogin line with
  number of cores removed. E.g. '4//usr/bin/specialssh user@host'
  becomes: '/usr/bin/specialssh user@host'

* While running a job $PARALLEL_SSHHOST is the host part of an
  sshlogin line. E.g. '4//usr/bin/specialssh user@host' becomes:
  'host'

* --plus activates the replacement strings {slot} = $PARALLEL_JOBSLOT,
  {sshlogin} = $PARALLEL_SSHLOGIN, {host} = $PARALLEL_SSHHOST

* Bug fixes and man page updates.

News about GNU Parallel:

* Portable Batch System (PBS) & GNU Parallel - Running a Program
  Multiple Times in Parallel
  https://www.youtube.com/watch?v=6ccbWu6Befo

* GNU Parallel przykład https://www.youtube.com/watch?v=gs_wG4Kt2G4

* Demo of LINUX APP - GNU PARALLEL - running multiple Gstreamer webcam
  .sh scripts with only 1 command
  https://www.youtube.com/watch?v=trQuA_wmWjg
   2020-08-31 20:13:29 by Thomas Klausner | Files touched by this commit (3631)
Log message:
*: bump PKGREVISION for perl-5.32.
   2020-05-19 16:35:25 by Nia Alarie | Files touched by this commit (3) | Package updated
Log message:
parallel: Update to 20200422
20200422

This release celebrates GNU Parallel's 10 years as a GNU tool:

> commit ed2dfb1043768154d4e7678e01e10287155fa834
> Author: Ole Tange <ole@tange.dk>
> Date:   Thu Apr 22 01:23:00 2010 +0200
>
>    Name change: Parallel is now GNU Parallel.
>    Basic structure for sshlogin and sshloginfile.

New in this release:

* parsort makes GNU sort run faster for files with more than 1M lines.

* Bug fixes and man page updates.

News about GNU Parallel:

* Several clusters have updated their documentation of GNU Parallel:
  https://www.grid5000.fr/w/GNU_Parallel
  https://research-it.berkeley.edu/virtual-training-running-jobs-parallel-savio
  http://www.hpc.lsu.edu/training/archive/tutorials.php

* add_pho uses GNU Parallel: https://github.com/SeedlingsBabylab/add_pho

* GNU parallel使用笔记https://zhuanlan.zhihu.com/p/37340011

* Virtual Workshop: Running Jobs in Parallel on Savio
  \ 
https://ais.berkeley.edu/events/virtual-workshop-running-jobs-parallel-savio/2020-04-20

* Free Concurrency with GNU Parallel
  https://phili.pe/posts/free-concurrency-with-gnu-parallel/

20200322

* Bug fixes and man page updates.

News about GNU Parallel:

* The GNU Parallel course in Copenhagen is cancelled
  https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-med-ole-tange/

* The GNU Parallel course in Århus is cancelled
  \ 
https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/

* The 10 year anniversary may be cancelled, see
  https://www.gnu.org/software/parallel/10-years-anniversary.html

* Running shell commands in parallel, via GNU Parallel
  \ 
https://dev.to/saveriomiroddi/running-shell-commands-in-parallel-via-gnu-parallel-3a72

* Faster nmap scanning with the help of GNU parallel
  https://badshah.io/experiment/faster-nmap-scanning-with-the-help-of-gnu-parallel/

20200222

* Bug fixes and man page updates.

20200122

* --blocktimeout dur - Time out for reading block when using
  --pipe. If it takes longer than dur to read a full block, use the
  partial block read so far.

* Bug fixes and man page updates.

News about GNU Parallel:

* GNU Parallel course in Copenhagen
  https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-med-ole-tange/

* GNU Parallel course in Århus
  \ 
https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/

* GNU Parallel pour accélérer vos process sous Linux
  https://www.yvonh.com/gnu-parallel-pour-accelerer-vos-process-sous-linux/

* How to copy a file to multiple directories in Linux
  https://net2.com/how-to-copy-a-file-to-multiple-directories-in-linux/

* Running linux commands in parallel \ 
https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8

20191222

* GNU Parallel course in Copenhagen
  https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-med-ole-tange/

* GNU Parallel course in Århus
  \ 
https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/

* GNU Parallel is used in
  https://github.com/JeffersonLab/rfw_tsf_extractor

* Bug fixes and man page updates.

20191122

* GNU Parallel was presented at Driving IT. Slides:
  cloud.prosa.dk/s/drivingit

* Restarting supervisord processes in parallel
  https://blog.naderman.de/2019/11/14/restarting-supervisord-processes-in-parallel/

* Using GNU Parallel to Create Archives Faster
  https://www.reddit.com/r/DataHoarder/comments/dsgmhc/parallel_archiving_techniques/

* Bug fixes and man page updates.

20191022

* --tee will use --output-error=warn-nopipe if supported by tee.

* GNU Parallel will be presented at Driving IT 2019:
  https://ida.dk/arrangementer-og-kurser/konferencer/driving-it/tools

* UMN Duluth: Job parallelization with task arrays and GNU parallel
  \ 
https://www.msi.umn.edu/tutorials/umn-duluth-job-parallelization-task-arrays-and-gnu-parallel

* Genome updater uses GNU Parallel
  https://github.com/pirovc/genome_updater

* Using GNU-Parallel for bioinformatics
  https://www.danielecook.com/using-gnu-parallel-for-bioinformatics/

* Speeding up PostgreSQL ETL pipeline with the help of GODS
  \ 
https://cfengine.com/company/blog-detail/speeding-up-postgresql-etl-pipeline-with-the-help-of-gods/

* Runing linux commands in parallel
  https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8

* Research Computing University of Colorado Boulder contains an intro
  to GNU Parallel
  https://readthedocs.org/projects/curc/downloads/pdf/latest/

* 如何使用Parallel在Shell中并行执行命令
  https://www.myfreax.com/gnu-parallel/

* 如何测试 Amazon Elastic File System
  https://aws.amazon.com/cn/blogs/china/how-to-test-drive-amazon-elastic-file-system/

* Bug fixes and man page updates.

20190922

* --nice is now inherited by the nice level that GNU Parallel is
  started at. So 'nice -n10 parallel' will also cause remote jobs to
  be run at nice level 10.

* --delay is now accurate to within 10 ms

* GNU Parallel will be presented at Driving IT 2019:
  https://ida.dk/arrangementer-og-kurser/konferencer/driving-it/tools

* Greenland Ice Sheet solid ice discharge from 1986 through 2017
  https://www.earth-syst-sci-data.net/11/769/2019/essd-11-769-2019.pdf

* App Center Cloud Build Machines > Installed Software > Utilities
  https://docs.microsoft.com/en-us/appcenter/build/software

* Curryfinger - SNI & Host header spoofing utility
  https://dualuse.io/blog/curryfinger/

* Hyak parallel-sql
  https://wiki.cac.washington.edu/display/hyakusers/Hyak+parallel-sql

* A meditation on analysing data with Perl in parallel using GNU Parallel
  \ 
https://www.reddit.com/r/perl/comments/cyo2qc/a_meditation_on_analysing_data_with_perl_in/

* Accélérer ses tâches avec GNU Parallel et les lots de tâches (UL)
  \ 
https://www.eventbrite.ca/e/inscription-accelerer-ses-taches-avec-gnu-parallel-et-les-lots-de-taches-ul-71412840729

* RMACC Summit GNU Parallel
  https://curc.readthedocs.io/en/latest/software/GNUParallel.html

* Introducing Parallel into Shell
  https://petelawson.com/post/parallel-in-shell/

* Linux Productivity Tools
  https://www.usenix.org/conference/lisa19/presentation/maheshwari

* Gnu/parallel + Nfs 实现共享文件网络的并发计算等操作
  \ 
http://atomicer.cn/2018/01/18/gnu-parallel-nfs-%E5%AE%9E%E7%8E%B0%E5%85%B1%E4%BA%AB%E6%96%87%E4%BB%B6%E7%BD%91%E7%BB%9C%E7%9A%84%E5%B9%B6%E5%8F%91%E8%AE%A1%E7%AE%97%E7%AD%89%E6%93%8D%E4%BD%9C/#more

* Bug fixes and man page updates.