bitcoin-bram

Developer Notes

Table of Contents

Coding Style (General)

Various coding styles have been used during the history of the codebase, and the result is not very consistent. However, we’re now trying to converge to a single style, which is specified below. When writing patches, favor the new style over attempting to mimic the surrounding style, except for move-only commits.

Do not submit patches solely to modify the style of existing code.

Coding Style (C++)

Block style example:

int g_count = 0;

namespace foo {
class Class
{
    std::string m_name;

public:
    bool Function(const std::string& s, int n)
    {
        // Comment summarising what this section of code does
        for (int i = 0; i < n; ++i) {
            int total_sum = 0;
            // When something fails, return early
            if (!Something()) return false;
            ...
            if (SomethingElse(i)) {
                total_sum += ComputeSomething(g_count);
            } else {
                DoSomething(m_name, total_sum);
            }
        }

        // Success return is usually at the end
        return true;
    }
}
} // namespace foo

Coding Style (Python)

Refer to /test/functional/README.md#style-guidelines.

Coding Style (Doxygen-compatible comments)

Bitcoin Core uses Doxygen to generate its official documentation.

Use Doxygen-compatible comment blocks for functions, methods, and fields.

For example, to describe a function use:

/**
 * ... Description ...
 *
 * @param[in]  arg1 input description...
 * @param[in]  arg2 input description...
 * @param[out] arg3 output description...
 * @return Return cases...
 * @throws Error type and cases...
 * @pre  Pre-condition for function...
 * @post Post-condition for function...
 */
bool function(int arg1, const char *arg2, std::string& arg3)

A complete list of @xxx commands can be found at http://www.doxygen.nl/manual/commands.html. As Doxygen recognizes the comments by the delimiters (/** and */ in this case), you don’t need to provide any commands for a comment to be valid; just a description text is fine.

To describe a class, use the same construct above the class definition:

/**
 * Alerts are for notifying old versions if they become too obsolete and
 * need to upgrade. The message is displayed in the status bar.
 * @see GetWarnings()
 */
class CAlert

To describe a member or variable use:

//! Description before the member
int var;

or

int var; //!< Description after the member

Also OK:

///
/// ... Description ...
///
bool function2(int arg1, const char *arg2)

Not picked up by Doxygen:

//
// ... Description ...
//

Also not picked up by Doxygen:

/*
 * ... Description ...
 */

A full list of comment syntaxes picked up by Doxygen can be found at http://www.doxygen.nl/manual/docblocks.html, but the above styles are favored.

Recommendations:

Generating Documentation

The documentation can be generated with make docs and cleaned up with make clean-docs. The resulting files are located in doc/doxygen/html; open index.html in that directory to view the homepage.

Before running make docs, you’ll need to install these dependencies:

Linux: sudo apt install doxygen graphviz

MacOS: brew install doxygen graphviz

Development tips and tricks

Compiling for debugging

Run configure with --enable-debug to add additional compiler flags that produce better debugging builds.

Compiling for gprof profiling

Run configure with the --enable-gprof option, then make.

debug.log

If the code is behaving strangely, take a look in the debug.log file in the data directory; error and debugging messages are written there.

The -debug=... command-line option controls debugging; running with just -debug or -debug=1 will turn on all categories (and give you a very large debug.log file).

The Qt code routes qDebug() output to debug.log under category “qt”: run with -debug=qt to see it.

Testnet and Regtest modes

Run with the -testnet option to run with “play bitcoins” on the test network, if you are testing multi-machine code that needs to operate across the internet.

If you are testing something that can run on one machine, run with the -regtest option. In regression test mode, blocks can be created on-demand; see test/functional/ for tests that run in -regtest mode.

DEBUG_LOCKORDER

Bitcoin Core is a multi-threaded application, and deadlocks or other multi-threading bugs can be very difficult to track down. The --enable-debug configure option adds -DDEBUG_LOCKORDER to the compiler flags. This inserts run-time checks to keep track of which locks are held and adds warnings to the debug.log file if inconsistencies are detected.

Valgrind suppressions file

Valgrind is a programming tool for memory debugging, memory leak detection, and profiling. The repo contains a Valgrind suppressions file (valgrind.supp) which includes known Valgrind warnings in our dependencies that cannot be fixed in-tree. Example use:

$ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
$ valgrind --suppressions=contrib/valgrind.supp --leak-check=full \
      --show-leak-kinds=all src/test/test_bitcoin --log_level=test_suite
$ valgrind -v --leak-check=full src/bitcoind -printtoconsole
$ ./test/functional/test_runner.py --valgrind

Compiling for test coverage

LCOV can be used to generate a test coverage report based upon make check execution. LCOV must be installed on your system (e.g. the lcov package on Debian/Ubuntu).

To enable LCOV report generation during test runs:

./configure --enable-lcov
make
make cov

# A coverage report will now be accessible at `./test_bitcoin.coverage/index.html`.

Performance profiling with perf

Profiling is a good way to get a precise idea of where time is being spent in code. One tool for doing profiling on Linux platforms is called perf, and has been integrated into the functional test framework. Perf can observe a running process and sample (at some frequency) where its execution is.

Perf installation is contingent on which kernel version you’re running; see this thread for specific instructions.

Certain kernel parameters may need to be set for perf to be able to inspect the running process’s stack.

$ sudo sysctl -w kernel.perf_event_paranoid=-1
$ sudo sysctl -w kernel.kptr_restrict=0

Make sure you understand the security trade-offs of setting these kernel parameters.

To profile a running bitcoind process for 60 seconds, you could use an invocation of perf record like this:

$ perf record \
    -g --call-graph dwarf --per-thread -F 140 \
    -p `pgrep bitcoind` -- sleep 60

You could then analyze the results by running:

perf report --stdio | c++filt | less

or using a graphical tool like Hotspot.

See the functional test documentation for how to invoke perf within tests.

Sanitizers

Bitcoin Core can be compiled with various “sanitizers” enabled, which add instrumentation for issues regarding things like memory safety, thread race conditions, or undefined behavior. This is controlled with the --with-sanitizers configure flag, which should be a comma separated list of sanitizers to enable. The sanitizer list should correspond to supported -fsanitize= options in your compiler. These sanitizers have runtime overhead, so they are most useful when testing changes or producing debugging builds.

Some examples:

# Enable both the address sanitizer and the undefined behavior sanitizer
./configure --with-sanitizers=address,undefined

# Enable the thread sanitizer
./configure --with-sanitizers=thread

If you are compiling with GCC you will typically need to install corresponding “san” libraries to actually compile with these flags, e.g. libasan for the address sanitizer, libtsan for the thread sanitizer, and libubsan for the undefined sanitizer. If you are missing required libraries, the configure script will fail with a linker error when testing the sanitizer flags.

The test suite should pass cleanly with the thread and undefined sanitizers, but there are a number of known problems when using the address sanitizer. The address sanitizer is known to fail in sha256_sse4::Transform which makes it unusable unless you also use --disable-asm when running configure. We would like to fix sanitizer issues, so please send pull requests if you can fix any errors found by the address sanitizer (or any other sanitizer).

Not all sanitizer options can be enabled at the same time, e.g. trying to build with --with-sanitizers=address,thread will fail in the configure script as these sanitizers are mutually incompatible. Refer to your compiler manual to learn more about these options and which sanitizers are supported by your compiler.

Additional resources:

Locking/mutex usage notes

The code is multi-threaded and uses mutexes and the LOCK and TRY_LOCK macros to protect data structures.

Deadlocks due to inconsistent lock ordering (thread 1 locks cs_main and then cs_wallet, while thread 2 locks them in the opposite order: result, deadlock as each waits for the other to release its lock) are a problem. Compile with -DDEBUG_LOCKORDER (or use --enable-debug) to get lock order inconsistencies reported in the debug.log file.

Re-architecting the core code so there are better-defined interfaces between the various components is a goal, with any necessary locking done by the components (e.g. see the self-contained FillableSigningProvider class and its cs_KeyStore lock for example).

Threads

Ignoring IDE/editor files

In closed-source environments in which everyone uses the same IDE, it is common to add temporary files it produces to the project-wide .gitignore file.

However, in open source software such as Bitcoin Core, where everyone uses their own editors/IDE/tools, it is less common. Only you know what files your editor produces and this may change from version to version. The canonical way to do this is thus to create your local gitignore. Add this to ~/.gitconfig:

[core]
        excludesfile = /home/.../.gitignore_global

(alternatively, type the command git config --global core.excludesfile ~/.gitignore_global on a terminal)

Then put your favourite tool’s temporary filenames in that file, e.g.

# NetBeans
nbproject/

Another option is to create a per-repository excludes file .git/info/exclude. These are not committed but apply only to one repository.

If a set of tools is used by the build system or scripts the repository (for example, lcov) it is perfectly acceptable to add its files to .gitignore and commit them.

Development guidelines

A few non-style-related recommendations for developers, as well as points to pay attention to for reviewers of Bitcoin Core code.

General Bitcoin Core

Wallet

General C++

For general C++ guidelines, you may refer to the C++ Core Guidelines.

Common misconceptions are clarified in those sections:

C++ data structures

class A
{
    uint32_t m_count{0};
}
void Foo(Span<const int> data);

std::vector<int> vec{1,2,3};
Foo(vec);
enum class Tabs {
    INFO,
    CONSOLE,
    GRAPH,
    PEERS
};

int GetInt(Tabs tab)
{
    switch (tab) {
    case Tabs::INFO: return 0;
    case Tabs::CONSOLE: return 1;
    case Tabs::GRAPH: return 2;
    case Tabs::PEERS: return 3;
    } // no default case, so the compiler can warn about missing cases
    assert(false);
}

Rationale: The comment documents skipping default: label, and it complies with clang-format rules. The assertion prevents firing of -Wreturn-type warning on some compilers.

Strings and formatting

Shadowing

Although the shadowing warning (-Wshadow) is not enabled by default (it prevents issues arising from using a different variable with the same name), please name variables so that their names do not shadow variables defined in the source code.

When using nested cycles, do not name the inner cycle variable the same as in the upper cycle, etc.

Threads and synchronization

Scripts

Shebang

Source code organization

namespace mynamespace {
...
} // namespace mynamespace

namespace {
...
} // namespace
#ifndef BITCOIN_FOO_BAR_H
#define BITCOIN_FOO_BAR_H
...
#endif // BITCOIN_FOO_BAR_H

GUI

Subtrees

Several parts of the repository are subtrees of software maintained elsewhere.

Some of these are maintained by active developers of Bitcoin Core, in which case changes should probably go directly upstream without being PRed directly against the project. They will be merged back in the next subtree merge.

Others are external projects without a tight relationship with our project. Changes to these should also be sent upstream, but bugfixes may also be prudent to PR against Bitcoin Core so that they can be integrated quickly. Cosmetic changes should be purely taken upstream.

There is a tool in test/lint/git-subtree-check.sh (instructions) to check a subtree directory for consistency with its upstream repository.

Current subtrees include:

Upgrading LevelDB

Extra care must be taken when upgrading LevelDB. This section explains issues you must be aware of.

File Descriptor Counts

In most configurations, we use the default LevelDB value for max_open_files, which is 1000 at the time of this writing. If LevelDB actually uses this many file descriptors, it will cause problems with Bitcoin’s select() loop, because it may cause new sockets to be created where the fd value is >= 1024. For this reason, on 64-bit Unix systems, we rely on an internal LevelDB optimization that uses mmap() + close() to open table files without actually retaining references to the table file descriptors. If you are upgrading LevelDB, you must sanity check the changes to make sure that this assumption remains valid.

In addition to reviewing the upstream changes in env_posix.cc, you can use lsof to check this. For example, on Linux this command will show open .ldb file counts:

$ lsof -p $(pidof bitcoind) |\
    awk 'BEGIN { fd=0; mem=0; } /ldb$/ { if ($4 == "mem") mem++; else fd++ } END { printf "mem = %s, fd = %s\n", mem, fd}'
mem = 119, fd = 0

The mem value shows how many files are mmap’ed, and the fd value shows you many file descriptors these files are using. You should check that fd is a small number (usually 0 on 64-bit hosts).

See the notes in the SetMaxOpenFiles() function in dbwrapper.cc for more details.

Consensus Compatibility

It is possible for LevelDB changes to inadvertently change consensus compatibility between nodes. This happened in Bitcoin 0.8 (when LevelDB was first introduced). When upgrading LevelDB, you should review the upstream changes to check for issues affecting consensus compatibility.

For example, if LevelDB had a bug that accidentally prevented a key from being returned in an edge case, and that bug was fixed upstream, the bug “fix” would be an incompatible consensus change. In this situation, the correct behavior would be to revert the upstream fix before applying the updates to Bitcoin’s copy of LevelDB. In general, you should be wary of any upstream changes affecting what data is returned from LevelDB queries.

Scripted diffs

For reformatting and refactoring commits where the changes can be easily automated using a bash script, we use scripted-diff commits. The bash script is included in the commit message and our Travis CI job checks that the result of the script is identical to the commit. This aids reviewers since they can verify that the script does exactly what it is supposed to do. It is also helpful for rebasing (since the same script can just be re-run on the new master commit).

To create a scripted-diff:

The scripted-diff is verified by the tool test/lint/commit-script-check.sh. The tool’s default behavior, when supplied with a commit is to verify all scripted-diffs from the beginning of time up to said commit. Internally, the tool passes the first supplied argument to git rev-list --reverse to determine which commits to verify script-diffs for, ignoring commits that don’t conform to the commit message format described above.

For development, it might be more convenient to verify all scripted-diffs in a range A..B, for example:

test/lint/commit-script-check.sh origin/master..HEAD

Suggestions and examples

If you need to replace in multiple files, prefer git ls-files to find or globbing, and git grep to grep, to avoid changing files that are not under version control.

For efficient replacement scripts, reduce the selection to the files that potentially need to be modified, so for example, instead of a blanket git ls-files src | xargs sed -i s/apple/orange/, use git grep -l apple src | xargs sed -i s/apple/orange/.

Also, it is good to keep the selection of files as specific as possible — for example, replace only in directories where you expect replacements — because it reduces the risk that a rebase of your commit by re-running the script will introduce accidental changes.

Some good examples of scripted-diff:

To find all previous uses of scripted diffs in the repository, do:

git log --grep="-BEGIN VERIFY SCRIPT-"

Release notes

Release notes should be written for any PR that:

Release notes should be added to a PR-specific release note file at /doc/release-notes-<PR number>.md to avoid conflicts between multiple PRs. All release-notes* files are merged into a single /doc/release-notes.md file prior to the release.

RPC interface guidelines

A few guidelines for introducing and reviewing new RPC interfaces:

Internal interface guidelines

Internal interfaces between parts of the codebase that are meant to be independent (node, wallet, GUI), are defined in src/interfaces/. The main interface classes defined there are interfaces::Chain, used by wallet to access the node’s latest chain state, interfaces::Node, used by the GUI to control the node, and interfaces::Wallet, used by the GUI to control an individual wallet. There are also more specialized interface types like interfaces::Handler interfaces::ChainClient passed to and from various interface methods.

Interface classes are written in a particular style so node, wallet, and GUI code doesn’t need to run in the same process, and so the class declarations work more easily with tools and libraries supporting interprocess communication: