[PATCH v3] libnvdimm/region: Update nvdimm_has_flush() to handle explicit 'flush' callbacks
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided.
Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'.
Fix this by updating nvdimm_has_flush() adding a condition to nvdimm_has_flush() to test if a 'region->flush' callback is assigned. Also remove explicit test for 'nd_region->ndr_mapping' since regions may need 'flush' without any explicit mappings as in case of virtio-pmem.
References: [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stg...
Cc: stable@vger.kernel.org Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") Reported-by: Shivaprasad G Bhat sbhat@linux.ibm.com Signed-off-by: Vaibhav Jain vaibhav@linux.ibm.com --- Changelog:
v3: * Removed the test for ND_REGION_SYNC to handle case where a synchronous region still wants to expose a deep-flush function. [ Aneesh ] * Updated patch title and description from previous patch https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@li...
v2: * Added the fixes tag and addressed the patch to stable tree [ Aneesh ] * Updated patch description to address the virtio-pmem case. * Removed test for 'nd_region->ndr_mappings' from beginning of nvdimm_has_flush() to handle the virtio-pmem case. --- drivers/nvdimm/region_devs.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index ef23119db574..c4b17bdd527f 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) { int i;
- /* no nvdimm or pmem api == flushing capability unknown */ - if (nd_region->ndr_mappings == 0 - || !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) + /* no pmem api == flushing capability unknown */ + if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) return -ENXIO;
+ /* Test if an explicit flush function is defined */ + if (nd_region->flush) + return 1; + + /* Test if any flush hints for the region are available */ for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; struct nvdimm *nvdimm = nd_mapping->nvdimm; @@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) }
/* - * The platform defines dimm devices without hints, assume - * platform persistence mechanism like ADR + * The platform defines dimm devices without hints nor explicit flush, + * assume platform persistence mechanism like ADR */ return 0; }
Vaibhav Jain vaibhav@linux.ibm.com writes:
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided.
Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'.
Fix this by updating nvdimm_has_flush() adding a condition to nvdimm_has_flush() to test if a 'region->flush' callback is assigned. Also remove explicit test for 'nd_region->ndr_mapping' since regions may need 'flush' without any explicit mappings as in case of virtio-pmem.
References: [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stg...
Reviewed-by: Aneesh Kumar K.V aneesh.kumar@linux.ibm.com
Cc: stable@vger.kernel.org Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") Reported-by: Shivaprasad G Bhat sbhat@linux.ibm.com Signed-off-by: Vaibhav Jain vaibhav@linux.ibm.com
Changelog:
v3:
- Removed the test for ND_REGION_SYNC to handle case where a synchronous region still wants to expose a deep-flush function. [ Aneesh ]
- Updated patch title and description from previous patch https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@li...
v2:
- Added the fixes tag and addressed the patch to stable tree [ Aneesh ]
- Updated patch description to address the virtio-pmem case.
- Removed test for 'nd_region->ndr_mappings' from beginning of nvdimm_has_flush() to handle the virtio-pmem case.
drivers/nvdimm/region_devs.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index ef23119db574..c4b17bdd527f 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) { int i;
- /* no nvdimm or pmem api == flushing capability unknown */
- if (nd_region->ndr_mappings == 0
|| !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
- /* no pmem api == flushing capability unknown */
- if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) return -ENXIO;
- /* Test if an explicit flush function is defined */
- if (nd_region->flush)
return 1;- /* Test if any flush hints for the region are available */ for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; struct nvdimm *nvdimm = nd_mapping->nvdimm;
@@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) } /*
* The platform defines dimm devices without hints, assume* platform persistence mechanism like ADR
* The platform defines dimm devices without hints nor explicit flush,* assume platform persistence mechanism like ADR}
2.30.2 _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Hi Vaibhav,
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided.
Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'.
In case of async flush callback, do we still need "deep_flush" ?
Thanks, Pankaj
Fix this by updating nvdimm_has_flush() adding a condition to nvdimm_has_flush() to test if a 'region->flush' callback is assigned. Also remove explicit test for 'nd_region->ndr_mapping' since regions may need 'flush' without any explicit mappings as in case of virtio-pmem.
References: [1] "powerpc/papr_scm: Implement support for H_SCM_FLUSH hcall" https://lore.kernel.org/linux-nvdimm/161703936121.36.7260632399582101498.stg...
Cc: stable@vger.kernel.org Fixes: c5d4355d10d4 ("libnvdimm: nd_region flush callback support") Reported-by: Shivaprasad G Bhat sbhat@linux.ibm.com Signed-off-by: Vaibhav Jain vaibhav@linux.ibm.com
Changelog:
v3:
- Removed the test for ND_REGION_SYNC to handle case where a synchronous region still wants to expose a deep-flush function. [ Aneesh ]
- Updated patch title and description from previous patch https://lore.kernel.org/linux-nvdimm/5e64778d-bf48-9f10-7d3d-5e530e5db590@li...
v2:
- Added the fixes tag and addressed the patch to stable tree [ Aneesh ]
- Updated patch description to address the virtio-pmem case.
- Removed test for 'nd_region->ndr_mappings' from beginning of nvdimm_has_flush() to handle the virtio-pmem case.
drivers/nvdimm/region_devs.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index ef23119db574..c4b17bdd527f 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -1234,11 +1234,15 @@ int nvdimm_has_flush(struct nd_region *nd_region) { int i;
/* no nvdimm or pmem api == flushing capability unknown */if (nd_region->ndr_mappings == 0|| !IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API))
/* no pmem api == flushing capability unknown */if (!IS_ENABLED(CONFIG_ARCH_HAS_PMEM_API)) return -ENXIO;/* Test if an explicit flush function is defined */if (nd_region->flush)return 1;/* Test if any flush hints for the region are available */ for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; struct nvdimm *nvdimm = nd_mapping->nvdimm;@@ -1249,8 +1253,8 @@ int nvdimm_has_flush(struct nd_region *nd_region) }
/*
* The platform defines dimm devices without hints, assume* platform persistence mechanism like ADR
* The platform defines dimm devices without hints nor explicit flush,* assume platform persistence mechanism like ADR */ return 0;}
2.30.2 _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
Thanks Pankaj for looking into this patch,
Pankaj Gupta pankaj.gupta.linux@gmail.com writes:
Hi Vaibhav,
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided.
Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'.
In case of async flush callback, do we still need "deep_flush" ?
'deep_flush' in libnvdimm (specifically 'deep_flush_store()') anyways resorts to calling 'async_flush' callback if its defined. Which makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would ensure that writes to pmem are now durable even if there is a sudden power loss before cpu caches are flushed.
On non-nfit architectures the 'async_flush' callback should provide such a guarantee, which can be triggered by user-space writing to the 'deep_flush' sysfs attr.
In absence of 'deep_flush' sysfs attr not sure how else can user-space forcibly trigger async_flush callback for dev-dax char devices.
<snip>
In case a platform doesn't provide explicit flush-hints but provides an explicit flush callback, then nvdimm_has_flush() still returns '0' indicating that writes do not require flushing. This happens on PPC64 with patch at [1] applied, where 'deep_flush' of a region was denied even though an explicit flush function was provided.
Similar problem is also seen with virtio-pmem where the 'deep_flush' sysfs attribute is not visible as in absence of any registered nvdimm, 'nd_region->ndr_mappings == 0'.
In case of async flush callback, do we still need "deep_flush" ?
'deep_flush' in libnvdimm (specifically 'deep_flush_store()') anyways resorts to calling 'async_flush' callback if its defined. Which makes sense to me since in absence of eADR, 'echo 1 > deep_flush' would ensure that writes to pmem are now durable even if there is a sudden power loss before cpu caches are flushed.
On non-nfit architectures the 'async_flush' callback should provide such a guarantee, which can be triggered by user-space writing to the 'deep_flush' sysfs attr.
In absence of 'deep_flush' sysfs attr not sure how else can user-space forcibly trigger async_flush callback for dev-dax char devices.
O.k. that means for filesystem DAX deep_flush is alternative to fsync/msync call.
I still have to dig deeper to understand more about "QUEUE_FLAG_FUA" flag & why I was seeing REQ_FUA with virtio-pmem when doing fsync if its not enabled in function "blk_queue_write_cache". But this is for my understanding.
Overall patch looks good to me and it looks to solve (not tested though) the warning for virtio-pmem as well.
Reviewed-by: Pankaj Gupta pankaj.gupta@ionos.com
Thanks, Pankaj
linux-stable-mirror@lists.linaro.org
-
Aneesh Kumar K.V -
Pankaj Gupta -
Vaibhav Jain