LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Guenter Roeck <linux@roeck-us.net>
To: Nathan Rossi <nathan@nathanrossi.com>
Cc: linux-hwmon@vger.kernel.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Nathan Rossi <nathan.rossi@digi.com>,
	Jean Delvare <jdelvare@suse.com>,
	Jonathan Corbet <corbet@lwn.net>
Subject: Re: [PATCH v3 3/3] hwmon: Driver for Texas Instruments INA238
Date: Mon, 1 Nov 2021 07:57:31 -0700	[thread overview]
Message-ID: <5569b2b8-ac67-757b-f7cb-302c9f663e80@roeck-us.net> (raw)
In-Reply-To: <CA+aJhH0EBUjQhjQhxj-kfJzmpqGN6ZMNn_M-pAae_V9yPQhs3A@mail.gmail.com>

On 10/31/21 10:55 PM, Nathan Rossi wrote:
> On Mon, 1 Nov 2021 at 13:48, Guenter Roeck <linux@roeck-us.net> wrote:
>>
>> On 10/31/21 7:20 PM, Nathan Rossi wrote:
>> [ ... ]
>>>>> +
>>>>> +     if (attr != hwmon_in_max && attr != hwmon_in_min)
>>>>> +             return -EOPNOTSUPP;
>>>>> +
>>>>> +     /* convert decimal to register value */
>>>>> +     switch (channel) {
>>>>> +     case 0:
>>>>> +             /* signed value, clamp to max range +/-163 mV */
>>>>> +             regval = clamp_val(val, -163, 163);
>>>>> +             regval = (regval * 1000L * (4 - (int)data->gain + 1)) /
>>>>
>>>> nit: The typecast "(int)" is not needed here.
>>>
>>> Due to the unsigned type of gain, it causes promotion of regval (and
>>> the rest of the numerator) to unsigned long which causes issues with
>>> negative numbers on the divide. It makes more sense for gain to be an
>>> int to begin with, I will change it to int to avoid the need for type
>>> casting.
>>>
>>
>> Are you sure ? I initially thought that as well and wrote a little test
>> program with that expression, but it didn't do the promotion to unsigned.
>>
> 
> It definitely calculates incorrectly at run time (on an arm 32-bit
> platform), looking at the gcc output from -fdump-tree-original reveals
> some more insight. Which is that the promotion to long overrides the
> unsigned (from the 1000L) on long=64 but not on long=32.
> 
> Where regval is int, and gain is unsigned int (u32).
> 
> regval = (regval * 1000L * (4 - gain + 1)) / 5;
> -> armv7-a (invalid)
> regval = (int) ((((long unsigned int) regval * (long unsigned int) (5
> - gain)) * 1000) / 5);
> -> x86-64 (valid result)
> regval = (int) ((unsigned int) (gain * 4294967096 + 1000) * (unsigned
> int) regval);
> 
> note: 4294967096 is -800, 1000 * (4 - gain + 1) => (-800 * gain) + 1000
> 
> Slight variation without the 1000 being long.
> 
> regval = (regval * 1000 * (4 - gain + 1)) / 5;
> -> armv7-a (invalid)
> regval = (int) ((((unsigned int) regval * (5 - gain)) * 1000) / 5);
> -> x86-64 (invalid)
> regval = (int) ((((unsigned int) regval * (5 - gain)) * 1000) / 5);
> 
> regval = (regval * 1000LL * (4 - gain + 1)) / 5;
> -> armv7-a (valid)
> regval = (int) ((unsigned int) (gain * 4294967096 + 1000) * (unsigned
> int) regval);
> -> x86-64 (valid)
> regval = (int) ((unsigned int) (gain * 4294967096 + 1000) * (unsigned
> int) regval);
> 
> I think it still makes sense to change gain to be int, and avoid the
> unsigned type issues.
> 

Thanks for the details. I agree, changing gain to int makes sense.

Thanks,
Guenter

      reply	other threads:[~2021-11-01 14:57 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-28  8:10 [PATCH v3 0/3] Driver for TI INA238 I2C Power Monitor Nathan Rossi
2021-10-28  8:10 ` [PATCH v3 2/3] dt-bindings: hwmon: ti,ina2xx: Add ti,shunt-gain property Nathan Rossi
2021-10-28  8:10 ` [PATCH v3 1/3] dt-bindings: hwmon: ti,ina2xx: Document ti,ina238 compatible string Nathan Rossi
2021-10-28  8:10 ` [PATCH v3 3/3] hwmon: Driver for Texas Instruments INA238 Nathan Rossi
2021-10-29 10:11   ` Guenter Roeck
2021-11-01  2:20     ` Nathan Rossi
2021-11-01  3:48       ` Guenter Roeck
2021-11-01  5:55         ` Nathan Rossi
2021-11-01 14:57           ` Guenter Roeck [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5569b2b8-ac67-757b-f7cb-302c9f663e80@roeck-us.net \
    --to=linux@roeck-us.net \
    --cc=corbet@lwn.net \
    --cc=jdelvare@suse.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-hwmon@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nathan.rossi@digi.com \
    --cc=nathan@nathanrossi.com \
    --subject='Re: [PATCH v3 3/3] hwmon: Driver for Texas Instruments INA238' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).