Parallelise like a boss… with a single line of code (in python)
Ever found yourself waiting for hours for some iteration/calculation on a big dataset/list/series/dictionary?
As for myself: numerous times have I waited, many times have I tried to rewrite the function to be executed on multiple cores in concurrency, and in most cases I have given up the attempt because by the time I thought I got it right — a new mistake popped up and I was half into completion of my initial operation anyway, so I just kept on waiting.
Concurrency with python takes some time to master, due to the GIL that allows only one thread to hold the control of the Python interpreter.
Another problem is sometimes a mixed output from your iteration… Your data is sent in chunks into multiple different processes and there is no telling which process will finish execution first/last; so your results may come mixed up. And I know what you are going to say… executor.map() is not an answer to all the possible situations.
long-story-short…
$ pip install verstack
The idea behind this post is not to teach you how the concurrency/multiprocessing…
