On 06/25/14 01:42, Viresh Kumar wrote:

cpufreq-cpu0 driver supports only platforms which have single clock line shared among all CPUs.

We already have platforms where this limitation doesn't hold true.For example on Qualcomm's KRAIT all CPUs have separate clock line and so have separate policies.

Krait is not all capitalized.

Instead of adding another driver for this (Stephen just tried that: https://lkml.org/lkml/2014/6/24/918), we must reuse cpufreq-cpu0 driver.

cpufreq-cpu0 must be updated to break the assumption on which it is based (all cores sharing clock line) and this patch tries to do exactly that.

As we don't have standard DT bindings to mention CPUs sharing clock-line, i.e. affected_cpus, this patch also have few limitations. Though easy to fix once we have proper bindings.

It should be easy enough to read the clocks property from DT for all the CPU nodes and check to see if they're the same? I don't think we need an affected-cpus property. Also I'd like to see current DTs specifying the same data (clocks, regulators, opps) for each CPU node even if they all share the same clock, regulator, and opp. It seems that the cpu0 binding just wanted to save some space in the DT by consolidating them all under one node, but that seems misguided. Case in point, now that we have configurations that need more than one clock per cluster the code is complicated and the binding is not uniform.

Limitation: We only supports two types of platforms:

• All CPUs sharing same clock line, existing user platforms
• All CPUs have separate clock lines, KRAIT

And so platforms which have multiple clusters with multiple CPUs per cluster aren't supported yet. We need proper bindings for that first.

Signed-off-by: Viresh Kumar viresh.kumar@linaro.org

.../devicetree/bindings/cpufreq/cpufreq-cpu0.txt | 8 +- drivers/cpufreq/Kconfig | 5 +- drivers/cpufreq/cpufreq-cpu0.c | 280 +++++++++++++-------- 3 files changed, 190 insertions(+), 103 deletions(-)

diff --git a/Documentation/devicetree/bindings/cpufreq/cpufreq-cpu0.txt b/Documentation/devicetree/bindings/cpufreq/cpufreq-cpu0.txt index f055515..fa8861e 100644 --- a/Documentation/devicetree/bindings/cpufreq/cpufreq-cpu0.txt +++ b/Documentation/devicetree/bindings/cpufreq/cpufreq-cpu0.txt @@ -1,15 +1,17 @@ Generic CPU0 cpufreq driver -It is a generic cpufreq driver for CPU0 frequency management. It +It is a generic cpufreq driver for CPU frequency management. It supports both uniprocessor (UP) and symmetric multiprocessor (SMP) -systems which share clock and voltage across all CPUs. +systems which share clock and voltage across all CPUs OR have separate +clock and voltage for all CPUs. Both required and optional properties listed below must be defined under node /cpus/cpu@0. Required properties:

• operating-points: Refer to Documentation/devicetree/bindings/power/opp.txt
• for details

• for details. This should either be present only in cpu@0, if CPUs share clock
• line OR should be present in all cpu nodes, if they have separate clock lines.

Optional properties:

• clock-latency: Specify the possible maximum transition latency for clock,

diff --git a/drivers/cpufreq/cpufreq-cpu0.c b/drivers/cpufreq/cpufreq-cpu0.c index ee1ae30..d29f6a0 100644 --- a/drivers/cpufreq/cpufreq-cpu0.c +++ b/drivers/cpufreq/cpufreq-cpu0.c @@ -25,34 +25,43 @@ #include <linux/slab.h> #include <linux/thermal.h> -static unsigned int transition_latency; -static unsigned int voltage_tolerance; /* in percentage */ +struct private_data {

• struct device *cpu_dev;
• struct regulator *cpu_reg;
• struct thermal_cooling_device *cdev;
• unsigned int voltage_tolerance; /* in percentage */

+}; -static struct device *cpu_dev; -static struct clk *cpu_clk; -static struct regulator *cpu_reg; -static struct cpufreq_frequency_table *freq_table; -static struct thermal_cooling_device *cdev; +/*

• • Currently this driver supports only two types of platforms:
• • • All CPUs sharing same clock line, clock_per_cpu == false
• • • All CPUs having separate clock lines, clock_per_cpu == true
• • TODO: Scan affected/related CPUs from proper DT bindings
• */

+bool clock_per_cpu = false; static int cpu0_set_target(struct cpufreq_policy *policy, unsigned int index) { struct dev_pm_opp *opp;

• struct private_data *priv = policy->driver_data; unsigned long volt = 0, volt_old = 0, tol = 0; unsigned int old_freq, new_freq; long freq_Hz, freq_exact; int ret;

• freq_Hz = clk_round_rate(cpu_clk, freq_table[index].frequency * 1000);

• freq_Hz = clk_round_rate(policy->clk,

• 		 policy->freq_table[index].frequency * 1000);
  

  if (freq_Hz <= 0)

• freq_Hz = freq_table[index].frequency * 1000;
  

• freq_Hz = policy->freq_table[index].frequency * 1000;
  

freq_exact = freq_Hz; new_freq = freq_Hz / 1000;

• old_freq = clk_get_rate(cpu_clk) / 1000;

• old_freq = clk_get_rate(policy->clk) / 1000;

• if (!IS_ERR(cpu_reg)) {

• if (!IS_ERR(priv->cpu_reg)) { rcu_read_lock();

• opp = dev_pm_opp_find_freq_ceil(cpu_dev, &freq_Hz);
  

• opp = dev_pm_opp_find_freq_ceil(priv->cpu_dev, &freq_Hz);
  

  if (IS_ERR(opp)) { rcu_read_unlock(); pr_err("failed to find OPP for %ld\n", freq_Hz);

@@ -60,8 +69,8 @@ static int cpu0_set_target(struct cpufreq_policy *policy, unsigned int index) } volt = dev_pm_opp_get_voltage(opp); rcu_read_unlock();

• tol = volt * voltage_tolerance / 100;
  

• volt_old = regulator_get_voltage(cpu_reg);
  

• tol = volt * priv->voltage_tolerance / 100;
  

• volt_old = regulator_get_voltage(priv->cpu_reg);
  

  }

pr_debug("%u MHz, %ld mV --> %u MHz, %ld mV\n",

This patch would be less noisy and easier to review if we used local variables in this function for cpu_req, voltage_tolerance, and cpu_clk. Can we do that please?

@@ -99,77 +108,70 @@ static int cpu0_set_target(struct cpufreq_policy *policy, unsigned int index) static int cpu0_cpufreq_init(struct cpufreq_policy *policy) {

• policy->clk = cpu_clk;
• return cpufreq_generic_init(policy, freq_table, transition_latency);

-}

-static struct cpufreq_driver cpu0_cpufreq_driver = {

• .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
• .verify = cpufreq_generic_frequency_table_verify,
• .target_index = cpu0_set_target,
• .get = cpufreq_generic_get,
• .init = cpu0_cpufreq_init,
• .name = "generic_cpu0",
• .attr = cpufreq_generic_attr,

-};

-static int cpu0_cpufreq_probe(struct platform_device *pdev) -{

• struct cpufreq_frequency_table *freq_table;
• struct private_data *priv; struct device_node *np;

• int ret;

• char name[] = "cpuX";
• int ret, cpu = policy->cpu;

• cpu_dev = get_cpu_device(0);
• if (!cpu_dev) {

• pr_err("failed to get cpu0 device\n");
  

• return -ENODEV;
  

• priv = kzalloc(sizeof(*priv), GFP_KERNEL);
• if (!priv) {

• pr_err("%s: Memory allocation failed\n", __func__);
  

Useless allocation error message.

• return -ENOMEM;
  

  }

• np = of_node_get(cpu_dev->of_node);
• if (!np) {

• pr_err("failed to find cpu0 node\n");
  

• return -ENOENT;
  

• }

• /* Below calls can't fail from here */
• priv->cpu_dev = get_cpu_device(cpu);
• np = of_node_get(priv->cpu_dev->of_node);

• cpu_reg = regulator_get_optional(cpu_dev, "cpu0");
• if (IS_ERR(cpu_reg)) {

• /*
  

•  * If cpu0 regulator supply node is present, but regulator is
  

•  * not yet registered, we should try defering probe.
  

•  */
  

• if (PTR_ERR(cpu_reg) == -EPROBE_DEFER) {
  

• 	dev_err(cpu_dev, "cpu0 regulator not ready, retry\n");
  

• 	ret = -EPROBE_DEFER;
  

• 	goto out_put_node;
  

• }
  

• pr_warn("failed to get cpu0 regulator: %ld\n",
  

• 	PTR_ERR(cpu_reg));
  

• policy->clk = clk_get(priv->cpu_dev, NULL);
• if (IS_ERR(policy->clk)) {

• ret = PTR_ERR(policy->clk);
  

• pr_err("%s: failed to get cpu%d clock: %d\n", __func__, cpu,
  

•        ret);
  

• goto out_put_node;
  

  }

This won't work all the time. We need probe defer to work for the clocks because sometimes CPU clocks are registered after this driver probes. In my case this is always true unless I play games with initcall levels. The same is true for regulators.

• cpu_clk = clk_get(cpu_dev, NULL);
• if (IS_ERR(cpu_clk)) {

• ret = PTR_ERR(cpu_clk);
  

• pr_err("failed to get cpu0 clock: %d\n", ret);
  

• goto out_put_reg;
  

• ret = of_init_opp_table(priv->cpu_dev);
• if (ret) {

• pr_err("%s: failed to init OPP table for cpu%d: %d\n", __func__,
  

•        cpu, ret);
  

• goto out_put_clk;
  

  }

• ret = of_init_opp_table(cpu_dev);

• ret = dev_pm_opp_init_cpufreq_table(priv->cpu_dev, &freq_table); if (ret) {

• pr_err("failed to init OPP table: %d\n", ret);
  

• pr_err("%s: failed to init cpufreq table for cpu%d: %d\n",
  

•        __func__, cpu, ret);
  

  goto out_put_clk; }

• ret = dev_pm_opp_init_cpufreq_table(cpu_dev, &freq_table);

• ret = cpufreq_table_validate_and_show(policy, freq_table); if (ret) {

• pr_err("failed to init cpufreq table: %d\n", ret);
  

• goto out_put_clk;
  

• pr_err("%s: invalid frequency table for cpu%d: %d\n", __func__,
  

•        cpu, ret);
  

• goto out_put_reg;
  

  }

• of_property_read_u32(np, "voltage-tolerance", &voltage_tolerance);

• /* Set mask of CPUs sharing clock line with policy->cpu */
• if (!clock_per_cpu)

• cpumask_setall(policy->cpus);
  

• if (of_property_read_u32(np, "clock-latency", &transition_latency))

• transition_latency = CPUFREQ_ETERNAL;
  

• of_property_read_u32(np, "voltage-tolerance", &priv->voltage_tolerance);

• if (!IS_ERR(cpu_reg)) {

• if (of_property_read_u32(np, "clock-latency",

• 		 &policy->cpuinfo.transition_latency))
  

• policy->cpuinfo.transition_latency = CPUFREQ_ETERNAL;
  

• /* NOTE: Supports only 10 CPUs: 0-9 */
• WARN_ONCE(cpu > 9, "cpu: %d", cpu);

Why would the supply name be called cpuN-supply? Why not use cpu-supply in each CPU node? That way there is no limit on the number of CPUs. We can always check for cpu0-supply first to be compatible with older DTs but going forward we should be able to just use cpu-supply.

• name[3] = '0' + cpu;
• priv->cpu_reg = regulator_get_optional(priv->cpu_dev, name);
• if (IS_ERR(priv->cpu_reg)) {

• pr_warn("%s: failed to get cpu%d's regulator: %ld\n", __func__,
  

• 	cpu, PTR_ERR(priv->cpu_reg));
  

• } else { struct dev_pm_opp *opp; unsigned long min_uV, max_uV; int i;

[...]

+static int cpu0_cpufreq_probe(struct platform_device *pdev) +{

• const struct property *prop;
• struct device *cpu_dev;
• struct regulator *cpu_reg;
• int ret, cpu, count = 0;
• /* Optimization for regulator's -EPROBE_DEFER case */
• if (unlikely(clock_per_cpu))

• goto scan_regulator;
  

Why not look for the regulator first then?

• /*

• * Scan all CPU nodes to set clock_per_cpu
  

• *
  

• * FIXME: Scan affected-CPUs from DT and remove this loop.
  

• */
  

• for_each_possible_cpu(cpu) {

• cpu_dev = get_cpu_device(cpu);
  

• if (!cpu_dev) {
  

• 	pr_err("%s: failed to get cpu%d device\n", __func__,
  

• 	       cpu);
  

• 	return -ENODEV;
  

• }
  

• /* Count how many CPU nodes have defined operating-points */
  

• prop = of_find_property(cpu_dev->of_node, "operating-points",
  

• 			NULL);
  

• if (prop) {
  

• 	count++;
  

• } else if (cpu == 0) {
  

• 	pr_err("%s: operating-points missing: cpu%d\n",
  

• 	       __func__, cpu);
  

• 	return -ENOENT;
  

• }
  

• }
• /* Either one or all CPUs must have operating-points property */
• if (count == cpu + 1) {

Is this supposed to be count == cpu? At this point in the code count equals the number of CPUs present.

• clock_per_cpu = true;
  

• } else if (count != 1) {

• pr_err("%s: operating-points missing for few CPUs\n", __func__);
  

• return -ENOENT;
  

• }

+scan_regulator:

• /* Check only CPU0's regulator for EPROBE_DEFER */
• cpu_dev = get_cpu_device(0); /* Can't fail now */
• cpu_reg = regulator_get_optional(cpu_dev, "cpu0");
• if (!IS_ERR(cpu_reg)) {

• regulator_put(cpu_reg);
  

• } else if (PTR_ERR(cpu_reg) == -EPROBE_DEFER) {

• /*
  

•  * If cpu0 regulator supply node is present, but regulator is
  

•  * not yet registered, we should try defering probe.
  

•  */
  

• dev_err(cpu_dev, "cpu0 regulator not ready, retry\n");
  

• return -EPROBE_DEFER;
  

• }

I don't think this driver should be using regulator_get_optional() (Mark B. please correct me if I'm wrong). I doubt a supply is actually optional for CPUs, just some DTs aren't specifying them. In those cases, the regulator core will insert a dummy supply and the code will work without having to check for probe defer and error pointers.

On 06/26/14 03:52, Viresh Kumar wrote:

On 26 June 2014 00:32, Stephen Boyd sboyd@codeaurora.org wrote:

...

I don't think this driver should be using regulator_get_optional() (Mark B. please correct me if I'm wrong). I doubt a supply is actually optional for CPUs, just some DTs aren't specifying them. In those cases, the regulator core will insert a dummy supply and the code will work without having to check for probe defer and error pointers.

Hi Stephen,

Thanks for your comments.

Leaving the above one, I have tried to fix all you mentioned. And it surely looks much better now.

I would like to wait for a day or two before sending V2, as people might be reviewing it and the above issue is still wide open..

But in case you wanna test it (completely changed I must say, but for good), its here:

git://git.linaro.org/people/viresh.kumar/linux.git cpufreq/cpu0-krait-v2

I gave it a spin. It looks mostly good except for the infinite loop:

diff --git a/drivers/cpufreq/cpufreq-cpu0.c b/drivers/cpufreq/cpufreq-cpu0.c index b7ee67c4d1c0..6744321ae33d 100644 --- a/drivers/cpufreq/cpufreq-cpu0.c +++ b/drivers/cpufreq/cpufreq-cpu0.c @@ -138,8 +138,10 @@ try_again: }

/* Try with "cpu-supply" */ - if (reg == reg_cpu0) + if (reg == reg_cpu0) { + reg = reg_cpu; goto try_again; + }

dev_warn(cpu_dev, "failed to get cpu%d regulator: %ld\n", cpu, PTR_ERR(cpu_reg));

and I think we just want reg_cpu to be "cpu", not "cpu-supply" because I think the regulator core adds in the "-supply" part already.

After fixing that I can get cpufreq going. I'm currently working on populating the OPPs at runtime without relying on DT. So eventually I'll need this patch:

diff --git a/drivers/cpufreq/cpufreq-cpu0.c b/drivers/cpufreq/cpufreq-cpu0.c index b7ee67c4d1c0..6744321ae33d 100644 --- a/drivers/cpufreq/cpufreq-cpu0.c +++ b/drivers/cpufreq/cpufreq-cpu0.c @@ -239,11 +241,6 @@ static int cpu0_cpufreq_init(struct cpufreq_policy *policy) }

ret = of_init_opp_table(cpu_dev); - if (ret) { - dev_err(cpu_dev, "failed to init OPP table: %d\n", ret); - goto out_put_node; - } - ret = dev_pm_opp_init_cpufreq_table(cpu_dev, &freq_table); if (ret) { dev_err(cpu_dev, "failed to init cpufreq table: %d\n", ret);

which I hope is ok.

Finally, checking for equivalent pointers from clk_get() will work now, but it isn't future-proof if/when the clock framework starts returning dynamically allocated clock pointers for each clk_get() invocation. Maybe we need a function in the common clock framework that tells us if the clocks are the same either via DT or by taking two clock pointers?

On 27 June 2014 05:36, Stephen Boyd sboyd@codeaurora.org wrote:

I gave it a spin. It looks mostly good except for the infinite loop:

Great !!

diff --git a/drivers/cpufreq/cpufreq-cpu0.c b/drivers/cpufreq/cpufreq-cpu0.c index b7ee67c4d1c0..6744321ae33d 100644 --- a/drivers/cpufreq/cpufreq-cpu0.c +++ b/drivers/cpufreq/cpufreq-cpu0.c @@ -138,8 +138,10 @@ try_again: }

            /* Try with "cpu-supply" */

•           if (reg == reg_cpu0)
  

•           if (reg == reg_cpu0) {
  

•                   reg = reg_cpu;
                    goto try_again;
  

•           }
  
            dev_warn(cpu_dev, "failed to get cpu%d regulator: %ld\n",
                     cpu, PTR_ERR(cpu_reg));
  

Awesome code, I wrote it. Thanks for fixing it.

and I think we just want reg_cpu to be "cpu", not "cpu-supply" because I think the regulator core adds in the "-supply" part already.

Okay.

After fixing that I can get cpufreq going. I'm currently working on populating the OPPs at runtime without relying on DT. So eventually I'll need this patch:

I see..

diff --git a/drivers/cpufreq/cpufreq-cpu0.c b/drivers/cpufreq/cpufreq-cpu0.c index b7ee67c4d1c0..6744321ae33d 100644 --- a/drivers/cpufreq/cpufreq-cpu0.c +++ b/drivers/cpufreq/cpufreq-cpu0.c @@ -239,11 +241,6 @@ static int cpu0_cpufreq_init(struct cpufreq_policy *policy) }

    ret = of_init_opp_table(cpu_dev);

•   if (ret) {
  

•           dev_err(cpu_dev, "failed to init OPP table: %d\n", ret);
  

•           goto out_put_node;
  

•   }
  

•   ret = dev_pm_opp_init_cpufreq_table(cpu_dev, &freq_table);
    if (ret) {
            dev_err(cpu_dev, "failed to init cpufreq table: %d\n", ret);
  
  

which I hope is ok.

I need to see details of these routines once again, but will surely make it work for you.