What Does __read_mostly In The Linux Kernel Do?

If you have spent any time looking at Linux kernel code you have undoubtedly come across the__read_mostly macro,
in fact you have probably seen it within a structure definition reference or even at the end of a simple integer declaration. I have stumbled across it many times in the past and have just skimmed it and never really paid a lot of attention to it, much like
some dust in the corner of a room. Today was different, I thought I should probably investigate this reference and see what its purpose is and how to take advantage or avoid its use in the future.

A few examples of the __read_mostly usage:

1 2 3 4 5 6

static struct nf_hook_ops ebt_ops_filter[] __read_mostly = { { .hook = ebt_hook, .owner = THIS_MODULE,   static int brnf_call_arptables __read_mostly = 1;

The __read_mostly macro is meant to inform the compiler that this piece of data will not be modified very often and mostly just read, hence the read mostly. This is akin to placing the kernel likely or unlikey macro (read
more about that here) pieces into code in which you know the variable is usually a certain value and thus do not setup an expensive branch. In the __read_mostly case the macro comes into play on SMP (or multiple core) systems. The compiler will group these
__read_mostly variables together and each core will have a cached copy of these variables (to speed up reads). If and when this variable changes or is written to, one of the CPUs or cores will take ownership and do the write, then the next CPU or core will
do the same until all cached variables are updated to the actual variables value. In stating that a variable is mostly read more often than not the CPU can just refer to the cached variable for quick reads, rather than grabbing the actual variable. The update
of the cached variable is costly and is known as cacheline bouncing, something that can be optimized by knowing how the variables or structures you declare are to be used.

The declaration of this macro is interesting:

#define __read_mostly __attribute__((__section__(".data.read_mostly")))

This informs the compiler to group all the __read_mostly data together in the final executable, rather than having it all over the place.

The subtlety of this macro brings up the question of…what about write_mostly variables? Is there a way to define variables as __write_mostly, and how is that handled? As far as I can tell, a variable not defined as __read_mostly falls into the write mostly
category. There you have it, a brief look at the __read_mostly macro.
http://www.lainoox.com/linux-kernel-__read_mostly/