Python, Linkers, and Virtual Memory
Why Stack Overflow recommends it
Stack Overflow users link it when answering questions about confusing VIRT versus RES memory numbers in top, for Python and other runtimes like Node.js.
Recommended by 3 people across 3 different months, 2013 to 2016.
What it is
A PyCon talk by Brandon Rhodes explaining why tools like top report large virtual memory figures against a small resident set size for Python processes. It covers how linkers, shared libraries and virtual memory work, and compares running several interpreters with forking workers from one.
What people said
Recommends Noah · 2013-05 · Stack Overflow · on “Node/Express memory increase on every request”
Keep profiling your server. You will probably find that memory usage levels off over time. Try 200,000 requests and see if things change Also if there is memory available the operating system will try to use it. 32mb of ram is not enough to be concerned about. This talk [this video] is worth watching. It is about python but the concepts are the same for node.js
Recommends Marek Kowalski · 2013-09 · Stack Overflow · on “multiprocessing memory usage and twisted/gevents”
There was a great talk on Pycon which explains the subject of memory usage in python [this video] . It definitely a half an hour well spent. The bottom line is that to really know how much memory is used you should not be looking at top output, but check how much memory you have free before and after running your 100 workers.
Recommends Antonio Petricca · 2016-04 · Stack Overflow · on “Linux process virtual memory column in TOP/HTOP”
Here is a good explanatio [this video] n that I'm going to see and learn... Edit (2016-04-07): I've just seen it, and it is brilliant! Please look at /proc/ /smaps to know how physical ram is used by your process. Edit (2016-04-08): I'm going deeper into the problem, and I discovered that each time I create a thread the process increases the used VIRT. I have seen also that all other linux processes with threads allocate much of VIRT memory size, so I think it is absolutely normal !