LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: John Garry <john.garry@huawei.com>
To: Ian Rogers <irogers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>,
	"Namhyung Kim" <namhyung@kernel.org>,
	Kajol Jain <kjain@linux.ibm.com>,
	linux-perf-users <linux-perf-users@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Stephane Eranian <eranian@google.com>
Subject: Re: [PATCH] perf test: Make metric testing more robust.
Date: Wed, 4 Aug 2021 18:18:56 +0100	[thread overview]
Message-ID: <3e853906-f758-8ad6-2297-27e4f097591a@huawei.com> (raw)
In-Reply-To: <CAP-5=fWWvGi8gGfwGORF-4y7V6G_gCxY7=SX2zo_cyxKxTEf+A@mail.gmail.com>

On 04/08/2021 15:55, Ian Rogers wrote:
> 
> 
> On Wed, Aug 4, 2021, 2:11 AM John Garry <john.garry@huawei.com 
> <mailto:john.garry@huawei.com>> wrote:
> 
>     On 04/08/2021 08:25, Ian Rogers wrote:
>      > When testing metric expressions we fake counter values from 1 going
>      > upward. For some metrics this can yield negative values that are
>     clipped
>      > to zero, and then cause divide by zero failures. A workaround for
>     this
>      > case is to try a second time with counter values going in the
>     opposite
>      > direction.
>      >
>      > This case was seen in a metric like:
>      >    event1 / max(event2 - event3, 0)
> 
>     is this the standard method to make the metric evaluation fail when
>     results are not as expected? In this example, event2 should be greater
>     than event3 always. Dividing by max(x, 0) would seem a bit silly.
> 
> 
> I wouldn't say it was standard but it is in a metric a third party gave 
> us.

I agree that making it more robust is a good thing. But masking bogus 
expressions isn't great. After all, we're here to find them :)

 > It would be possible to get the same test failure on more standard
 > expressions, so it would be nice if these tests were more robust.

so something like this would fail also:
event1 / (event2 + event3 - 1 - event4)

assuming we have ascending values from 1 for event1. And this would seem 
a valid expression.

Anyway, it would be nice if we could reject max(0, x) and any divide by 
negative numbers, apart from your change.

Cheers,
john


>      >
>      > Signed-off-by: Ian Rogers <irogers@google.com
>     <mailto:irogers@google.com>>
>      > ---
>      >   tools/perf/tests/pmu-events.c | 32 ++++++++++++++++++++++++++------
>      >   1 file changed, 26 insertions(+), 6 deletions(-)
>      >
>      > diff --git a/tools/perf/tests/pmu-events.c
>     b/tools/perf/tests/pmu-events.c
>      > index b8aff8fb50d8..6c1cd58605c1 100644
>      > --- a/tools/perf/tests/pmu-events.c
>      > +++ b/tools/perf/tests/pmu-events.c
>      > @@ -600,8 +600,18 @@ static int test_parsing(void)
>      >                       }
>      >
>      >                       if (expr__parse(&result, &ctx,
>     pe->metric_expr, 0)) {
>      > -                             expr_failure("Parse failed", map, pe);
>      > -                             ret++;
>      > +                             /*
>      > +                              * Parsing failed, make numbers go
>     from large to
>      > +                              * small which can resolve divide
>     by zero
>      > +                              * issues.
>      > +                              */
>      > +                             k = 1024;
>      > +                             hashmap__for_each_entry((&ctx.ids),
>     cur, bkt)
>      > +                                     expr__add_id_val(&ctx,
>     strdup(cur->key), k--);
>      > +                             if (expr__parse(&result, &ctx,
>     pe->metric_expr, 0)) {
>      > +                                     expr_failure("Parse
>     failed", map, pe);
>      > +                                     ret++;
>      > +                             }
>      >                       }
>      >                       expr__ctx_clear(&ctx);
>      >               }
>      > @@ -656,10 +666,20 @@ static int metric_parse_fake(const char *str)
>      >               }
>      >       }
>      >
>      > -     if (expr__parse(&result, &ctx, str, 0))
>      > -             pr_err("expr__parse failed\n");
>      > -     else
>      > -             ret = 0;
>      > +     ret = 0;
>      > +     if (expr__parse(&result, &ctx, str, 0)) {
>      > +             /*
>      > +              * Parsing failed, make numbers go from large to
>     small which can
>      > +              * resolve divide by zero issues.
>      > +              */
>      > +             i = 1024;
>      > +             hashmap__for_each_entry((&ctx.ids), cur, bkt)
>      > +                     expr__add_id_val(&ctx, strdup(cur->key), i--);
>      > +             if (expr__parse(&result, &ctx, str, 0)) {
>      > +                     pr_err("expr__parse failed\n");
>      > +                     ret = -1;
>      > +             }
>      > +     }
>      >
>      >   out:
>      >       expr__ctx_clear(&ctx);
>      >
> 


  parent reply	other threads:[~2021-08-04 17:19 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-04  7:25 Ian Rogers
2021-08-04  9:11 ` John Garry
     [not found]   ` <CAP-5=fWWvGi8gGfwGORF-4y7V6G_gCxY7=SX2zo_cyxKxTEf+A@mail.gmail.com>
2021-08-04 17:18     ` John Garry [this message]
2021-09-16  0:14       ` Ian Rogers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3e853906-f758-8ad6-2297-27e4f097591a@huawei.com \
    --to=john.garry@huawei.com \
    --cc=acme@kernel.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=eranian@google.com \
    --cc=irogers@google.com \
    --cc=jolsa@redhat.com \
    --cc=kjain@linux.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --subject='Re: [PATCH] perf test: Make metric testing more robust.' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).