Understanding the Python GIL

What Hacker News said

Strong rec famousactress · 2014-01 · on “Code is not Literature”

"The point of such a presentation is to take a piece of code that the presenter has understood deeply and for them to help the audience understand the core ideas..." I do get lots of value out of that. My favorite example is Beazley's GIL talk: [this video]

Recommends Luyt · 2013-06 · on “Python's Hardest Problem (2012)”

"I never actually got around to learning -- or even thinking about!-- what exactly the GIL is" David Beazley does a good job on explaining the GIL. There are multiple videos of his GIL talks on YouTube. This one, for instance: [this video] He has a page about this at [LINK]

Recommends devxpy · 2018-07 · on “Code with Mu: a simple Python editor for beginner programmers”

David probably isnt the best guy for that. But here's one of the famous ones on GIL. [this video] Beware though, his talks are known to be devastating to the programmer, resulting in serious head injuries, trauma, headaches and other fun stuff.

Recommends gshulegaard · 2022-08 · on “How to choose the right Python concurrency API”

I am not expert, and this might be dated information, but historically there is a "thrashing" introduced between the GIL and OS scheduled threads which has surprising performance implications. David Beazley does a much better illustrating the nuances than I ever could: [this video] Including a wild phenomenon where a threaded Python program's performance can degrade by increasing number of CPU cores available to a system. Coroutines avoid this by not requiring individual thread scheduling by the OS. So they effectively release the GIL like threads on IO but avoid OS level thread switching.

Cites zerkten · 2017-01 · on “Problems I Have with Python”

> If you want speed you go to any other scripting language (other than Ruby). Ruby has historically had the same issues. Most Pythonistas I know aren't so evangelical. It's mostly a question of how to go about integrating C/C++ code. Many people complaining about the GIL (and the like) have some naive microbenchmark, don't understand the trade-offs/limitations of their runtime etc. That doesn't mean critique isn't important and required, but it's going to be better when it's properly researched and improves on the body of work out there ( [this video] ). > Transpiling it to another language like Google did, shows that the underlying technology is not worth much. How is this any general indicator

Cites gshulegaard · 2017-02 · on “The multiprocessing module in Python”

I would recommend you use coroutines rather than multiprocessing.dummy for balancing blocking operations (like IO). I believe multiprocessing.dummy is just a wrapper for threading which has context switching overhead [1]. Python3 has asyncio as part of the stdlib but for Python2 there are a variety of coroutine libraries out there (I think the most widely recognized being gevent). [1] [this video]

Cites gshulegaard · 2020-06 · on “Async Python is not faster”

> > Python's multiprocessing library is needed to overcome the GIL > No it's not, just use threads. I just wanted to expand on this a little to describe some of the downsides to threads in Python. Multi-threaded logic can be (and often is) slower than single-threaded logic because threading introduces overhead of lock contention and context switching. David Beazley did a talk illustrating this in 2010: [this video] He also did a great talk about coroutines in 2015 where he explores threading and coroutines a bit more: [LINK] In workloads that are often "blocked" like network calls our I/O bound work loads, threads can provide similar benefits to coroutines but with overhead. Coroutines seek to pr

Cites rvanlaar · 2023-10 · on “Progress on No-GIL CPython”

There's a GIL presentation [1] from 2010 which shows Python 3.2 on a 1 CPU machine was faster than on a dual core. In all, there might be reason to expect GILless python to be faster single core in certain scenario's. [1] [this video]