Managing Namespaces
Last updated
Was this helpful?
Last updated
Was this helpful?
Regions can be partitioned into one or more namespaces. Namespace operations include listing, creating, destroying, enabling, disabling, and checking (validating) using the following commands:
- dump the platform nvdimm device topology and attributes in json format
- provision or reconfigure a namespace
- destroy the given namespace(s)
- disable the given namespace(s)
- enable the given namespace(s)
- check namespace metadata consistency
The ndctl list -N command is used to display namespaces, for example:
The output can be filtered using any combination of the following options:
Examples:
List all active namespaces
List all active namespaces and print human readable values (MB, MiB, GB, GiB, etc)
List a single namespace (namespace0.0)
List all active 'fsdax' mode namespaces
By default, ndctl list -N lists only active/enabled namespaces. In the following example no active/enabled namespaces exist so no output is displayed:
Adding the -i flag includes both active/enabled and inactive/destroyed/disabled namespaces. The following example shows two previously destroyed namespaces. The size and uuid are clear indications that no information exists about the namespace. The ndctl destroy command simply NULLs the existing namespace entry and does not completely erase it from the metadata. For this reason, it may be possible to see more than one namespace using ndctl list -Ni
The -i option can be used in coordination with any other valid ndctl option.
Examples:
List all enabled and disabled namespaces
List a single disabled namespace (namespace0.0)
List all enabled and disabled 'fsdax' mode namespaces
The create-namespace command has a lot of options summarized below. The ndctl-create-namespace man page contains further details of each option.
-t, --type: Set the type of the namespace to either "pmem" or "blk"
-m, mode: Define the namespace mode - fsdax, devdax, sector, and raw
-s, --size: Specify the capacity of the namespace
-a, --align: Specify the alignments size, eg 4K or 2M
-e, --reconfigure: Reconfigure existing namespace configuration
-u, --uuid: Used for recovery purposes only
-n, --name: Specify a friendly name for namespaces that support labels
-l, --sector-size: Override the default sector size for block based namespaces
-M, --map: For 'fsdax' or 'devdax' namespaces, define whether metadata is stored in volatile memory (mem) or persistent storage (dev)
-c, --continue: Do not stop after creating one namespace. Instead, create as many namespaces as possible within the given --bus and --region filter restrictions.
-f, --force: Allow the operation to continue on enabled namespaces
-L, --autolabel, --no-autolabel: Manage labels for legacy NVDIMMs that do not support labels
-v, --verbose: Emit verbose messages during the creation process
-r, --region: Limit the operation to a specific region
-b, --bus: Limit the operation to a specific bus
The mode is the most important feature to get correct. The four modes available are defined as:
fsdax: Filesystem-DAX mode is the default mode of a namespace when specifying ndctl create-namespace with no options. It creates a block device (/dev/pmemX[.Y]) that supports the DAX capabilities of Linux filesystems (xfs and ext4 to date). DAX removes the page cache from the I/O path and allows mmap(2) to establish direct mappings to persistent memory media. The DAX capability enables workloads / working-sets that would exceed the capacity of the page cache to scale up to the capacity of persistent memory. Workloads that fit in page cache or perform bulk data transfers may not see benefit from DAX. When in doubt, pick this mode.
devdax: Device-DAX mode enables similar mmap(2) DAX mapping capabilities as Filesystem-DAX. However, instead of a block-device that can support a DAX-enabled filesystem, this mode emits a single character device file (/dev/daxX.Y). Use this mode to assign persistent memory to a virtual-machine, register persistent memory for RDMA, or when gigantic mappings are needed.
sector: Use this mode to host legacy filesystems that do not checksum metadata or applications that are not prepared for torn sectors after a crash. Expected usage for this mode is for small boot volumes. This mode is compatible with other operating systems.
raw: Raw mode is effectively just a memory disk that does not support DAX. Typically this indicates a namespace that was created by tooling or another operating system that did not know how to create a Linux fsdax or devdax mode namespace. This mode is compatible with other operating systems, but again, does not support DAX operation.
In addition to the namespace label metadata, both fsdax and devdax modes require space to store the Page Frame Number (PFN) metadata (also known as “struct page” metadata) for each page within the namespace. A PFN is simply in index within physical memory, or on the NVDIMM, that is counted in page-sized units. For modes with PFN metadata, shown in Table 1 below, the requirement is 64 bytes per 4 KiB of persistent memory. To calculate the estimated space for metadata, use:
Example 1: A 256GiB namespace requires ~3.93GiB of metadata
(263182090240/4096)*64 = 4112220160 bytes == 3.93GB
Example 2: A 1TiB namespace requires ~17.18GiB of metadata
(1099511627776/4096)*64 = 17179869184 bytes == 17.18GiB
The location of PFN metadata can either be in volatile DDR from the NVDIMM. The default is to keep metadata on the NVDIMM. The location is defined per namespace by the -M, --map configuration option. The following provides guidance for PFN metadata location:
map=mem: PFN metadata is stored in regular system memory (DDR)
Adequate for small persistent memory capacities
Frequent or high performance PFN access is required. For example, when time sensitive, or frequent, registration and deregistration of RDMA memory regions is important. Normal RDMA memory [de]registration can be achieved with 'map=dev', but there are situations where [de]registration needs to be very fast. This assumes the metadata fits in DDR memory.
map=dev: PFN metadata is stored on persistent memory (NVDIMM)
Intended for large persistent memory capacities. There might not be enough regular memory in the system if the ratio of persistent memory to DDR is high (>8:1)
The persistence of the PFN metadata is not important as it's re-mapped at system boot time or when new namespaces are created/destroyed. It is convenient to store the metadata on the NVDIMM because it scales with the persistent memory capacity. The space for metadata is calculated and reserved at namespace creation time, meaning the usable capacity is lower than the physical capacity depending on the mode and map values.
Table 1: Features for each namespace mode
Mode
Description
Device Path
Label Metadata
Atomicity
Filesystems
DAX
PFN Metadata
raw
raw
/dev/pmemN
no
no
yes
no
no
sector
sector atomic
/dev/pmemNs
yes
yes
yes
no
no
fsdax
filesystem DAX
/dev/pmemN
yes
no
yes
yes
yes
devdax
device DAX
/dev/daxN.M
yes
no
no
yes
yes
PFN Metadata Usage Example
The following demonstrates the difference between --map=dev and --map=mem. The example uses a ~250GiB region to create an fsdax namespace. A devdax namespace has identical behavior.
When a namespace using the default --map=dev is created, the usable 'size' is ~4GiB smaller than the 249GiB region size due to the space reserved for PFN metdata.
When using --map=mem, the usable capacity on the namespace increases, but the amount of 'used' memory as shown by free has increased by ~4GiB.
This example section has been split in to four, one for each mode described above:
Example 1: Create an 'fsdax' mode namespace using all available capacity (default behavior).
The ndctl create-namespace command shown below is the equivalent of ndctl create-namespace -m fsdax. A new /dev/pmem{X[.Y]} device is created. This can be used to create a new DAX enabled filesystem such as XFS or EXT4. The 'X' value represents the region on which the namespace is created. This defaults to zero (0). If more than one namespace is created within a region, the naming convention of pmem{X.Y} is used, where Y represents a sequentially increasing integer value for the new namespace. If multiple namespaces within a region are created, the first namespace is always created as 'pmem0', it does not get re-enumerated to 'pmem0.0' like subsequent namespaces.
Example 2: Create a 50GiB 'fsdax' mode namespace.
The 'size' provided to ndctl includes space required for metadata so the resulting available capacity for a filesystem will be smaller as the example below shows:
Note: If the remaining capacity needs to be assigned to another namespace using the same or different mode, the remaining capacity can be assigned to the new namespace without specifying the '-s <size>' option. For example, executing ndctl create-namespace after the command above creates a new namespace in the same region using all remaining capacity (~78GiB):
Example 3: Create a namespace with a friendly name (tag)
Tagging the namespace with a friendly name or description using the '-n, --name' option can be useful to know what a namespace is used for. This is particularly useful when provisioning multiple end-users, applications, or to identify a namespace with production or test/dev environments. Creating a namespace with a tag/name/description can be done using:
Example 4: Automatically create as many namespaces as possible
By default, the create-namespace operation creates a single namespace. On systems with more than one region (usually defined by the number of CPU sockets), this requires executing create-namespace for each region. On high CPU socket systems, this can be time consuming. Version 67 of ndctl introduced the -c, --continue option that will automatically create as many namespaces as it can. This is similar to using a FOR loop around create-namespace. This operation will abort if any creation attempt results in an error unless --force is also supplied.
The following example shows how to automatically create two fsdax namespaces on empty regions. The size of the namespaces will use the full capacity of each region.
Create a pmem namespace with devdax mode using all available capacity
Create a 50GiB 'devdax' namespace
Create a sector mode namespace using all available capacity and a default 4K sector size
Create a 'raw' mode namespace using all available capacity
Add or Change a namespace name (also referred to as a tag or description):
WARNING: Data will be lost or corrupted if the new size is smaller than the existing size or extending the namespace and growing the filesystem(s). Backup all data before resizing namespaces.
The following example resizes an existing 'fsdax' namespace (namespace0.0) to 20GiB:
If there is an existing EXT4 or XFS filesystem on this device, the partition table may need to be manually changed to accommodate the new device capacity before the filesystem can be resized. Due to the many configuration options available when using fsdax devices, such as whether it's being managed by the Linux device-mapper, LVM (Linux Volume Manager), or just an XFS/EXT4 filesystem on top, refer to the relevant Linux distribution documentation for specific procedures to handle your configuration.
Changing the namespace mode will destroy all current data. Backup all data before proceeding.
If a filesystem using an fsdax or sector namespace is mounted, unmount it before changing the mode.
If a devdax or raw namespace is currently being used by a running application, stop the application before changing modes.
Switching modes of an enabled/active namespace can be done using ndctl create-namespace -fe <namespace> --mode=<new_mode>. For example, to switch an enabled fsdax namespace to devdax mode, use:
If the namespace is currently disabled, the -f flag is not required.
Namespaces should be inactive/disabled prior to deletion otherwise a warning message is displayed. The following shows an attempt to destroy an active namespace
First disable the namespace, then destroy it
Alternatively using the -f flag to the command forcibly destroys the namespace without doing any validation checks and does not require that the namespace be disabled first, eg:
Warning: Disabling namespaces while the namespace is mounted or in-use will result in undefined behavior by the application(s) using the namespace. Always stop the application and unmount filesystems (fsdax mode) before disabling namespaces.
1) Disable the namespace (namespace0.0) using:
2) Verify the namespace is disabled. The -i flag is required to display inactive/disabled namespaces otherwise only active/enabled namespaces are shown. The example below filters the output to a specific namespace (namespace0.0) to avoid potentially large volumes of output being shown:
1) Enable the namespace (namespace0.0) using:
2) Verify the namespace is enabled. The state is only displayed for disabled namespaces. If the state field is not listed, the namespace is assumed to be enabled.
The check-namespace command only works on namespaces in sector mode. Attempting to check namespaces in any other mode will yield an error similar to the following which attempts to check an fsdax mode namespace:
A namespace in the sector mode will have metadata on it to describe the Kernel BTT (Block Translation Table). The check-namespace command can be used to check the consistency of this metadata, and optionally, also attempt to repair it, if it has enough information to do so.
The command usage is:
Where the available options include:
Examples
The namespace being checked has to be disabled before initiating a check on it as a precautionary measure.
The -f, --force option can override the need to manually disable the namespace first, although it is not recommended to do so:
Adding the -v, --verbose option yields more information, eg:
Check a namespace and perform repairs if possible or required can be done using:
The namespace configuration (label metadata) is stored at the beginning of the persistent memory address range. See the and sections for more information.
Changing properties of existing namespaces can be done online using the ndctl create-namespace -fe <namespace> <option=value> command. The -e, --reconfigure flag edits existing namespaces. Using the -f flag does not require that the namespace be manually disabled. The command may still fail if the namespace is currently being used. Supported options are listed above in the section, or review the ndctl-create-namespace . Some namespace properties are read-only and cannot be changed using the ndctl utility.
or can be achieved by changing existing namespace properties. They are discussed below under their own headings to provide additional detail.
The namespace mode can be changed on any enabled or disabled namespace. Valid modes are raw, sector, fsdax, and devdax. Refer to the ndctl-create-namespace man page or chapter for more information on each mode.