LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: Andi Kleen <andi@firstfloor.org>
Cc: Ingo Molnar <mingo@elte.hu>, linux-kernel@vger.kernel.org, akpm@osdl.org
Subject: Re: [PATCH REPOST for 2.6.25] Use an own random generator for pageattr-test.c
Date: Tue, 11 Mar 2008 22:59:45 +0100 (CET)	[thread overview]
Message-ID: <alpine.LFD.1.00.0803112248550.3781@apollo.tec.linutronix.de> (raw)
In-Reply-To: <20080311214925.GA428@one.firstfloor.org>

On Tue, 11 Mar 2008, Andi Kleen wrote:
> > It's a question of how you write such test code to achieve
> > reproducability. It's not rocket science to track the variables of a
> > test run and print them along with the printk, when a wrong state is
> > detected. 
> 
> If you wanted to do that you would need some variant of my patch too --
> to do it with random32() you would first need to print all NR_CPUS
> state (and implement "kernel less" first for NR_CPUS > 128 kernels :),
> then keep track of all CPUs the test thread ran on and print that
> out too and also disable the regular timer reseeder to avoid races.
> Clearly doesn't make sense.
> 
> random32() in lib/ is simply unusable as a deterministic RND,
> it's more like super weak strange /dev/random variant which
> probably should be never put into lib/ anyways because it's unlikely
> to be generally useful.
> 
> So for a random but repeatable sequence like you describe you could keep the 
> patch, replace the static int next = 1; with static int next and add a 
> get_random_bytes(&next, sizeof(next)); and then print out the next value

Oh well. Randomized test code is there to catch bugs by statistical
spreading. Having a pseudo randomized scenario which is repeatable per
machine is defeating the randomized approach. Repeating a test with a
stale pattern is pretty useless unless you catch a bug right in the
first run.

The only bug that code ever caught aside of tons of false positives
was when we increased the runtime length and added the thread which
repeated the test.

Finding a bug, when it was exposed by a static pattern, is trivial,
but the challenge is to make such tests useful enough with random
patterns. And there are ways to do that, e.g. by making the debug
output informative enough to provide information about the problem in
detail instead of printing some useless info "a != b".

Thanks,

	tglx

  reply	other threads:[~2008-03-11 22:00 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-03-11  1:30 Andi Kleen
2008-03-11  2:39 ` Andrew Morton
2008-03-11  8:25 ` Thomas Gleixner
2008-03-11 10:45   ` Andi Kleen
2008-03-11 11:41     ` Thomas Gleixner
2008-03-11 11:48       ` Andi Kleen
2008-03-11 21:08         ` Thomas Gleixner
2008-03-11 21:49           ` Andi Kleen
2008-03-11 21:59             ` Thomas Gleixner [this message]
2008-03-11 22:11               ` Andi Kleen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.LFD.1.00.0803112248550.3781@apollo.tec.linutronix.de \
    --to=tglx@linutronix.de \
    --cc=akpm@osdl.org \
    --cc=andi@firstfloor.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --subject='Re: [PATCH REPOST for 2.6.25] Use an own random generator for pageattr-test.c' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).