Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn --- v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem --- drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); }
- if (cpu < nr_cpu_ids) + if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
gentle ping?
Huacai
On Thu, Jun 13, 2024 at 3:43 PM Hongchen Zhang zhanghongchen@loongson.cn wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); }
if (cpu < nr_cpu_ids)
if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);-- 2.33.0
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); }
- if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
- if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); } - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
+ if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
On 7/24/2024 9:58 AM, Hongchen Zhang wrote:
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); } - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
Hmm..... housekeeping_cpumask() should never return offline CPU, so I guess you didn't hit issue with the CPU isolation, but the following code seems not good.
...
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; } else {
....
perhaps you could change the logic there and fix it ?
Thanks Ethan
+ if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
Hi Ethan,
On 2024/7/24 上午10:47, Ethan Zhao wrote:
On 7/24/2024 9:58 AM, Hongchen Zhang wrote:
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); } - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
Hmm..... housekeeping_cpumask() should never return offline CPU, so I guess you didn't hit issue with the CPU isolation, but the following code seems not good.
The issue is the dev node is online but the best selected cpu is offline, so it seems that there is no better way to directly set the cpu to nr_cpu_ids.
...
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; } else {
....
perhaps you could change the logic there and fix it ?
Thanks Ethan
+ if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
On 7/24/2024 11:09 AM, Hongchen Zhang wrote:
Hi Ethan,
On 2024/7/24 上午10:47, Ethan Zhao wrote:
On 7/24/2024 9:58 AM, Hongchen Zhang wrote:
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); } - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
Hmm..... housekeeping_cpumask() should never return offline CPU, so I guess you didn't hit issue with the CPU isolation, but the following code seems not good.
The issue is the dev node is online but the best selected cpu is offline, so it seems that there is no better way to directly set the cpu to nr_cpu_ids.
I mean where the bug is ? you should debug more about that. just add one cpu_online(cpu) check there might suggest there is bug in the cpu selection stage.
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; // <----- if you hit here, then local_pci_probe() should be called.
} else { cpumask_var_t wq_domain_mask;
if (!zalloc_cpumask_var(&wq_domain_mask, GFP_KERNEL)) { error = -ENOMEM; goto out; } cpumask_and(wq_domain_mask, housekeeping_cpumask(HK_TYPE_WQ), housekeeping_cpumask(HK_TYPE_DOMAIN));
cpu = cpumask_any_and(cpumask_of_node(node), wq_domain_mask); free_cpumask_var(wq_domain_mask); // <----- if you hit here, then work_on_cpu(cpu, local_pci_probe, &ddi) should be called. // do you mean there one offline cpu is selected ?
}
if (cpu < nr_cpu_ids) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
Thanks, Ethan
...
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; } else {
....
perhaps you could change the logic there and fix it ?
Thanks Ethan
+ if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
On 2024/7/24 下午2:40, Ethan Zhao wrote:
On 7/24/2024 11:09 AM, Hongchen Zhang wrote:
Hi Ethan,
On 2024/7/24 上午10:47, Ethan Zhao wrote:
On 7/24/2024 9:58 AM, Hongchen Zhang wrote:
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote:
Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument @cpu is a offline cpu would cause system stuck forever.
This can be happen if a node is online while all its CPUs are offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it).
So, in the above case, let pci_call_probe() call local_pci_probe() instead of work_on_cpu() when the best selected cpu is offline.
Fixes: 69a18b18699b ("PCI: Restrict probe functions to housekeeping CPUs") Cc: stable@vger.kernel.org Signed-off-by: Huacai Chen chenhuacai@loongson.cn Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn
v2 -> v3: Modify commit message according to Markus's suggestion v1 -> v2: Add a method to reproduce the problem
drivers/pci/pci-driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..32a99828e6a3 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver *drv, struct pci_dev *dev, free_cpumask_var(wq_domain_mask); } - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
Hmm..... housekeeping_cpumask() should never return offline CPU, so I guess you didn't hit issue with the CPU isolation, but the following code seems not good.
The issue is the dev node is online but the best selected cpu is offline, so it seems that there is no better way to directly set the cpu to nr_cpu_ids.
I mean where the bug is ? you should debug more about that. just add one cpu_online(cpu) check there might suggest there is bug in the cpu selection stage.
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; // <----- if you hit here, then local_pci_probe() should be called. } else { cpumask_var_t wq_domain_mask;
if (!zalloc_cpumask_var(&wq_domain_mask, GFP_KERNEL)) { error = -ENOMEM; goto out; } cpumask_and(wq_domain_mask, housekeeping_cpumask(HK_TYPE_WQ), housekeeping_cpumask(HK_TYPE_DOMAIN));
cpu = cpumask_any_and(cpumask_of_node(node), wq_domain_mask); free_cpumask_var(wq_domain_mask); // <----- if you hit here, then work_on_cpu(cpu, local_pci_probe, &ddi) should be called.
Yes, but if the offline cpu is selected, local_pci_probe should be called.
// do you mean there one offline cpu is selected ?
Yes, the offline cpu is selected.
}
if (cpu < nr_cpu_ids) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
Thanks, Ethan
...
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; } else {
....
perhaps you could change the logic there and fix it ?
Thanks Ethan
+ if ((cpu < nr_cpu_ids) && cpu_online(cpu)) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);
Ping?
On Wed, Jul 24, 2024 at 3:05 PM Hongchen Zhang zhanghongchen@loongson.cn wrote:
On 2024/7/24 下午2:40, Ethan Zhao wrote:
On 7/24/2024 11:09 AM, Hongchen Zhang wrote:
Hi Ethan,
On 2024/7/24 上午10:47, Ethan Zhao wrote:
On 7/24/2024 9:58 AM, Hongchen Zhang wrote:
Hi Ethan, On 2024/7/22 PM 3:39, Ethan Zhao wrote:
On 6/13/2024 3:42 PM, Hongchen Zhang wrote: > Call work_on_cpu(cpu, fn, arg) in pci_call_probe() while the argument > @cpu is a offline cpu would cause system stuck forever. > > This can be happen if a node is online while all its CPUs are > offline (We can use "maxcpus=1" without "nr_cpus=1" to reproduce it). > > So, in the above case, let pci_call_probe() call local_pci_probe() > instead of work_on_cpu() when the best selected cpu is offline. > > Fixes: 69a18b18699b ("PCI: Restrict probe functions to > housekeeping CPUs") > Cc: stable@vger.kernel.org > Signed-off-by: Huacai Chen chenhuacai@loongson.cn > Signed-off-by: Hongchen Zhang zhanghongchen@loongson.cn > --- > v2 -> v3: Modify commit message according to Markus's suggestion > v1 -> v2: Add a method to reproduce the problem > --- > drivers/pci/pci-driver.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c > index af2996d0d17f..32a99828e6a3 100644 > --- a/drivers/pci/pci-driver.c > +++ b/drivers/pci/pci-driver.c > @@ -386,7 +386,7 @@ static int pci_call_probe(struct pci_driver > *drv, struct pci_dev *dev, > free_cpumask_var(wq_domain_mask); > } > - if (cpu < nr_cpu_ids)
Why not choose the right cpu to callwork_on_cpu() ? the one that is online. Thanks, Ethan
Yes, let housekeeping_cpumask() return online cpu is a good idea, but it may be changed by command line. so the simplest way is to call local_pci_probe when the best selected cpu is offline.
Hmm..... housekeeping_cpumask() should never return offline CPU, so I guess you didn't hit issue with the CPU isolation, but the following code seems not good.
The issue is the dev node is online but the best selected cpu is offline, so it seems that there is no better way to directly set the cpu to nr_cpu_ids.
I mean where the bug is ? you should debug more about that. just add one cpu_online(cpu) check there might suggest there is bug in the cpu selection stage.
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; // <----- if you hit here, then local_pci_probe() should be called. } else { cpumask_var_t wq_domain_mask;
if (!zalloc_cpumask_var(&wq_domain_mask, GFP_KERNEL)) { error = -ENOMEM; goto out; } cpumask_and(wq_domain_mask, housekeeping_cpumask(HK_TYPE_WQ), housekeeping_cpumask(HK_TYPE_DOMAIN)); cpu = cpumask_any_and(cpumask_of_node(node), wq_domain_mask); free_cpumask_var(wq_domain_mask); // <----- if you hit here, then work_on_cpu(cpu,local_pci_probe, &ddi) should be called.
Yes, but if the offline cpu is selected, local_pci_probe should be called.
// do you mean there one offline cpu is selected ?Yes, the offline cpu is selected.
} if (cpu < nr_cpu_ids) error = work_on_cpu(cpu, local_pci_probe, &ddi); else error = local_pci_probe(&ddi);Thanks, Ethan
...
if (node < 0 || node >= MAX_NUMNODES || !node_online(node) || pci_physfn_is_probed(dev)) { cpu = nr_cpu_ids; } else {
....
perhaps you could change the logic there and fix it ?
Thanks Ethan
> + if ((cpu < nr_cpu_ids) && cpu_online(cpu)) > error = work_on_cpu(cpu, local_pci_probe, &ddi); > else > error = local_pci_probe(&ddi);
-- Best Regards Hongchen Zhang
linux-stable-mirror@lists.linaro.org
-
Ethan Zhao -
Hongchen Zhang -
Huacai Chen