*Commit f21916ec4826 ("s390/vfio-ap: clean up vfio_ap resources when KVM pointer invalidated") introduced a change that results in a circular lockdep when a Secure Execution guest that is configured with crypto devices is started. The problem resulted due to the fact that the patch moved the setting of the guest's AP masks within the protection of the matrix_dev->lock when the vfio_ap driver is notified that the KVM pointer has been set. Since it is not critical that setting/clearing of the guest's AP masks be done under the matrix_dev->lock when the driver is notified, the masks will not be updated under the matrix_dev->lock. The lock is necessary for the setting/unsetting of the KVM pointer, however, so that will remain in place.
The dependency chain for the circular lockdep resolved by this patch is (in reverse order):
2: vfio_ap_mdev_group_notifier: kvm->lock matrix_dev->lock
1: handle_pqap: matrix_dev->lock kvm_vcpu_ioctl: vcpu->mutex
0: kvm_s390_cpus_to_pv: vcpu->mutex kvm_vm_ioctl: kvm->lock
Please note: ----------- * If checkpatch is run against this patch series, you may get a "WARNING: Unknown commit id 'f21916ec4826', maybe rebased or not pulled?" message. The commit 'f21916ec4826', however, is definitely in the master branch on top of which this patch series was built, so I'm not sure why this message is being output by checkpatch. * All acks granted from previous review of this patch have been removed due to the fact that this patch introduces non-trivial changes (see change log below).
Change log v3=> v4: ------------------ * In vfio_ap_mdev_set_kvm() function, moved the setting of matrix_mdev->kvm_busy just prior to unlocking matrix_dev->lock.
* Reset queues regardless of regardless of the value of matrix_mdev->kvm in response to the VFIO_DEVICE_RESET ioctl.
Change log v2=> v3: ------------------ * Added two fields - 'bool kvm_busy' and 'wait_queue_head_t wait_for_kvm' to struct ap_matrix_mdev. The former indicates that the KVM pointer is in the process of being updated and the second allows a function that needs access to the KVM pointer to wait until it is no longer being updated. Resolves problem of synchronization between the functions that change the KVM pointer value and the functions that required access to it.
Change log v1=> v2: ------------------ * No longer holding the matrix_dev->lock prior to setting/clearing the masks supplying the AP configuration to a KVM guest. * Make all updates to the data in the matrix mdev that is used to manage AP resources used by the KVM guest in the vfio_ap_mdev_set_kvm() function instead of the group notifier callback. * Check for the matrix mdev's KVM pointer in the vfio_ap_mdev_unset_kvm() function instead of the vfio_ap_mdev_release() function.
Tony Krowiak (1): s390/vfio-ap: fix circular lockdep when setting/clearing crypto masks
drivers/s390/crypto/vfio_ap_ops.c | 309 ++++++++++++++++++-------- drivers/s390/crypto/vfio_ap_private.h | 2 + 2 files changed, 215 insertions(+), 96 deletions(-)