Clang MapReduce -- Automatic C++ Refactoring at Google Scale

LLVMProject · 36K views · 2011 · 43:07

Open on YouTube

Compilers, interpreters & program analysisC, C++, Rust, Go & systems languagesTesting, debugging & developer toolstalkdated

Why Stack Overflow recommends it

Stack Overflow users link it when asked about C++ refactoring tools, as the canonical example of Clang-based refactoring and an explanation of why it is hard without compiler access.

Recommended by 4 people across 4 different months, 2012 to 2016.

What it is

A 2011 LLVM Developers' Meeting talk on how Google uses Clang together with MapReduce to automatically refactor C++ APIs and their callers across a codebase of over 100 million lines. It covers why C++ refactoring is hard and how Clang's library design makes large-scale analysis and rewriting practical.

What people said

Strong rec David C. Bishop · 2012-12 · Stack Overflow · on “any C/C++ refactoring tool based on libclang? (even simplest "toy example" )”

Google have been working on a tooling library for Clang. In since the 3.2 release [LINK] . It includes a ASTMatchers library so you can just build up a query and don't have to walk the AST. There is a great video talk [LINK] on the subject that walks through a simple rename example. (This is from the same guy as the MapReduce talk [this video] posted above but is newer and more about a simple practical implementation rather than the internal design and enterprise scale stuff Google have going on). The source for that example that renames a method is available in the tooling branch [LINK] . It may be somewhere in the trunk but I can't find it. Also Rename the getDeclAs function to getNodesAs as th

Strong rec Dirk Bester · 2014-04 · Stack Overflow · on “Why is Visual C++ lacking refactor functionality?”

There is a lot of fud and confusion around this issue. This amazing youtube video should clear up why C++ refactoring is hard: [this video] tl;dr Google refactors their entire 100 million line C++ codebase by using a compiler (Clang + LLVM) that allows access to its intermediate format. Bottom line, third parties are screwed here, there is no realistic way for them to refactor VS C++ unless MS outputs intermediate results the same way. If you think of it from the programming problem perspective this is obvious: in order to refactor VS C++ you have to be able to compile C++ the exact same way VS does with the same bugs, limitations, flaws, hacks, shortcuts, workarounds, etc. The usual suspects lik

Recommends Nielk · 2014-07 · Stack Overflow · on “How to find in my program a "const char* + int" expression”

You can have a try at the Map-Reduce Clang plugin. The tool was developped at Google to do just this kind of refactoring, mixing strong type-checking and regexp. (see video presentation here [this video] ).

Recommends Nikolay Bobovnikov · 2016-10 · Stack Overflow · on “Is there a working C++ refactoring tool?”

I recommend to try rtags [LINK] if you use emacs and haven't tried it yet (there is also a package [LINK] for vim available). It is a clang based client/server application that indexes C/C++ code, with these features included: go to definition/declaration find all references, go to next/previous rename symbol integration with clang’s “fixits” I decided to give it a try after watching this talk [LINK] which introduced rtags (and emacs) for me. (I have to say that I went this far only after my QtCreator failed to rename some symbols properly, which is a show-stopper for my using this great IDE for now) Besides what is supported by rtags, I also need some additional neat features, including: cr

Cites Reuben Morais · 2012-01 · Stack Overflow · on “any C/C++ refactoring tool based on libclang? (even simplest "toy example" )”

Google made a Clang based refactoring tool for their C++ code base and plans to release it. I don't know the current state of the project, but you can see this demo presented on the 2011 LLVM Developers Meeting: [this video] . Also, XCode's (4+) built-in auto-completion and refactoring functions are based on libclang.

Cites HighCommander4 · 2012-04 · Stack Overflow · on “Is there a working C++ refactoring tool?”

I expect clang [LINK] will significantly change the landscape of C++ refactoring tools out there over the next couple of years. It's an open-source, modular compiler that exposes an API for parsing and semantically analyzing C++ code. IDEs and other tools will be able to use this API rather than doing the difficult work of writing their own parser and semantic analyzer. Google already made a large-scale refactoring tool using clang [this video] .

Cites bames53 · 2014-01 · Stack Overflow · on “Does g++ meets std::string C++11 requirements”

libstd++'s implementation is non-conformant to C++11, but that doesn't mean your code is correctly guaranteeing the results you expect. Doing anything to modify the values stored in the character array returned by c_str() results in undefined behavior. The standard explicitly says this: 21.4.7.1 basic_string accessors const charT* c_str() const noexcept; const charT* data() const noexcept; 1 Returns: A pointer p such that p + i == &operator[](i) for each i in [0, size() ]. 2 Complexity: constant time. 3 Requires: The program shall not alter any of the values stored in the character array. Although above I quote C++11 this was also true of C++03. Does anyone have an idea about how to go at it