Regions, Atomic Sectors, and DAX

One of the few reasons to allow multiple BLK namespaces per REGION is so that each BLK-namespace can be configured with a Block Translation Table (BTT) with unique atomic sector sizes. While a PMEM device can host a BTT, the LABEL specification does not provide for a sector size to be specified for a PMEM namespace. This is due to the expectation that the primary usage model for PMEM is via DAX, and the BTT is incompatible with DAX. However, for the cases where an application or filesystem still needs atomic sector update guarantees, it can register a BTT on a PMEM device or partition.

Example NVDIMM Platform

Figure 1 shows an example platform with four NVDIMMs, two integrated memory controllers (IMC), and a single CPU .

                             (a)               (b)           DIMM   BLK-REGION
          +-------------------+--------+--------+--------+
+------+  |       pm0.0       | blk2.0 | pm1.0  | blk2.1 |    0      region2
| imc0 +--+- - - region0- - - +--------+        +--------+
+--+---+  |       pm0.0       | blk3.0 | pm1.0  | blk3.1 |    1      region3
   |      +-------------------+--------v        v--------+
+--+---+                               |                 |
| cpu0 |                                     region1
+--+---+                               |                 |
   |      +----------------------------^        ^--------+
+--+---+  |           blk4.0           | pm1.0  | blk4.0 |    2      region4
| imc1 +--+----------------------------|        +--------+
+------+  |           blk5.0           | pm1.0  | blk5.0 |    3      region5
          +----------------------------+--------+--------+

Figure 1: Example sysfs layout

Each unique interface (BLK or PMEM) to DPA space is identified by a region device with a dynamically assigned id (REGION0 - REGION5).

  1. The first portion of DIMM0 and DIMM1 are interleaved as REGION0. A

    single PMEM namespace is created in the REGION0-SPA-range that spans most

    of DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that

    interleaved system-physical-address range is reclaimed as BLK-aperture

    accessed space starting at DPA-offset (a) into each DIMM. In that

    reclaimed space we create two BLK-aperture "namespaces" from REGION2 and

    REGION3 where "blk2.0" and "blk3.0" are just human readable names that

    could be set to any user-desired name in the LABEL.

  2. In the last portion of DIMM0 and DIMM1 we have an interleaved

    system-physical-address range, REGION1, that spans those two DIMMs as

    well as DIMM2 and DIMM3. Some of REGION1 is allocated to a PMEM namespace

    named "pm1.0", the rest is reclaimed in 4 BLK-aperture namespaces (for

    each DIMM in the interleave set), "blk2.1", "blk3.1", "blk4.0", and

    "blk5.0".

  3. The portion of DIMM2 and DIMM3 that do not participate in the REGION1

    interleaved system-physical-address range (i.e. the DPA address past

    offset (b) are also included in the "blk4.0" and "blk5.0" namespaces.

    Note, that this example shows that BLK-aperture namespaces don't need to

    be contiguous in DPA-space.

    This bus is provided by the kernel under the device

    /sys/devices/platform/nfit_test.0 when CONFIG_NFIT_TEST is enabled and

    the nfit_test.ko module is loaded. This not only test LIBNVDIMM but the

    acpi_nfit.ko driver as well.

Last updated