Shared memory parallelism in Julia | Kiran Pamnany | Cambridge Julia Meetup (May 2018)
Why Hacker News recommends it
Recommended in Julia threading threads as an insightful, simple explanation of the new task scheduler.
What it is
Kiran Pamnany's Cambridge Julia Meetup talk on Julia's shared memory parallelism and its new task scheduler. It argues that fork-join and work-stealing models nest poorly and presents depth-first scheduling for composable parallelism.
Recommended by 4 HN commenters across 3 different months, 2018 to 2019.
What Hacker News said
Recommends jrevels · 2018-10 · on “Julia parallel task runtime (WIP)”
This work has been a monumental collaborative effort and I think the eventual outcome will be pretty spectacular. I gave a talk at an MIT Julia meetup a few months ago that the author of this PR also spoke at; I found his talk to be an insightful overview of this work: [this video]
Recommends boromi · 2018-10 · on “Julia parallel task runtime (WIP)”
checkout [this video] which explains the new features of the task scheduler in a simple fashion. Basically Julia is employing a new algorithm to schedule tasks, where traditional methods would get stuck/blocked. Essentially, if you run a job with parallel tasks and one of the threads finishes, it's extremely difficult to schedulate the free thread, since the data may still be used by the other threads that are busy; thus you need to have clever way to determine how to allocate computational resources. Julia is implementing results of a new paper on how to optimally do this.
Recommends StefanKarpinski · 2019-07 · on “Composable multi-threaded parallelism in Julia”
Consider a machine with N cores running a program which spawns N parallel tasks, each of which in turn spawns N parallel tasks. Which subtasks do the cores work on initially? If it's the first subtask of each top-level task, that's breadth-first scheduling. If it's all N subtasks of the first top-level task, that's depth-first scheduling. The latter is better for high-performance computing because it tends to be the lower level libraries that are really carefully tuned to make optimal use of the machine (think BLAS/FTT/parallel sum/prefix sum). Previously it was unknown how to implement this kind of scheduling efficiently. See this talk by Kiran Pamnany for more details: [this video]
Recommends falkaer · 2019-11 · on “Julia v1.3”
By far the most interesting part of this release is the new multi-threading features, I recommend reading [LINK] for an overview, and watching [this video] for a talk about some unique stuff that's being worked on with Julia's multi-threading (though the depth-first scheduling is not in 1.3 afaik).
Cites oconnor663 · 2019-11 · on “Julia v1.3”
Does the Rust Rayon library have the issue the speaker was describing, where parallel inner loops mostly get executed serially once an outer loop is parallelized? (I assume it does, from its Cilk heritage?) Are there workarounds for when that is a problem? Edit: Dumb mistake on my part. I was looking at the video linked in the top comment ( [this video] ) and then I came back to this thread and thought it was the topic.