LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: "Du, Changbin" <changbin.du@intel.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: changbin.du@intel.com, yamada.masahiro@socionext.com,
	michal.lkml@markovi.net, tglx@linutronix.de, mingo@redhat.com,
	akpm@linux-foundation.org, x86@kernel.org, lgirdwood@gmail.com,
	broonie@kernel.org, arnd@arndb.de, linux-kbuild@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org
Subject: Re: [PATCH 0/5] kernel hacking: GCC optimization for debug experience (-Og)
Date: Wed, 2 May 2018 17:07:57 +0800	[thread overview]
Message-ID: <20180502090756.tztulppgfefccd7q@intel.com> (raw)
In-Reply-To: <20180502073315.sso3aaak45aeuyst@gmail.com>

On Wed, May 02, 2018 at 09:33:15AM +0200, Ingo Molnar wrote:
> 
> * changbin.du@intel.com <changbin.du@intel.com> wrote:
> 
> > Comparison of system performance: a bit drop.
> > 
> >     w/o CONFIG_DEBUG_EXPERIENCE
> >     $ time make -j4
> >     real    6m43.619s
> >     user    19m5.160s
> >     sys     2m20.287s
> > 
> >     w/ CONFIG_DEBUG_EXPERIENCE
> >     $ time make -j4
> >     real    6m55.054s
> >     user    19m11.129s
> >     sys     2m36.345s
> 
> Sorry, that's not a proper kbuild performance measurement - there's no noise 
> estimation at all.
> 
> Below is a description that should produce more reliable numbers.
> 
> Thanks,
> 
> 	Ingo
>
Thanks for your suggestion, I will try your tips to eliminate noise. Since it is
tested in KVM guest, so I just reboot the guest before testing. But in host side
I still need to consider these noises.

> 
> =========================>
> 
> So here's a pretty reliable way to measure kernel build time, which tries to avoid 
> the various pitfalls of caching.
> 
> First I make sure that cpufreq is set to 'performance':
> 
>   for ((cpu=0; cpu<120; cpu++)); do
>     G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
>     [ -f $G ] && echo performance > $G
>   done
> 
> [ ... because it can be *really* annoying to discover that an ostensible 
>   performance regression was a cpufreq artifact ... again. ;-) ]
> 
> Then I copy a kernel tree to /tmp (ramfs) as root:
> 
> 	cd /tmp
> 	rm -rf linux
> 	git clone ~/linux linux
> 	cd linux
> 	make defconfig >/dev/null
> 	
> ... and then we can build the kernel in such a loop (as root again):
> 
>   perf stat --repeat 10 --null --pre			'\
> 	cp -a kernel ../kernel.copy.$(date +%s);	 \
> 	rm -rf *;					 \
> 	git checkout .;					 \
> 	echo 1 > /proc/sys/vm/drop_caches;		 \
> 	find ../kernel* -type f | xargs cat >/dev/null;  \
> 	make -j kernel >/dev/null;			 \
> 	make clean >/dev/null 2>&1;			 \
> 	sync						'\
> 							 \
> 	make -j16 >/dev/null
> 
> ( I have tested these by pasting them into a terminal. Adjust the ~/linux source 
>   git tree and the '-j16' to your system. )
> 
> Notes:
> 
>  - the 'pre' script portion is not timed by 'perf stat', only the raw build times
> 
>  - we flush all caches via drop_caches and re-establish everything again, but:
> 
>  - we also introduce an intentional memory leak by slowly filling up ramfs with 
>    copies of 'kernel/', thus continously changing the layout of free memory, 
>    cached data such as compiler binaries and the source code hierarchy. (Note 
>    that the leak is about 8MB per iteration, so it isn't massive.)
> 
> With 10 iterations this is the statistical stability I get this on a big box:
> 
>  Performance counter stats for 'make -j128 kernel' (10 runs):
> 
>       26.346436425 seconds time elapsed    (+- 0.19%)
> 
> ... which, despite a high iteration count of 10, is still surprisingly noisy, 
> right?
> 
> A 0.2% stddev is probably not enough to call a 0.7% regression with good 
> confidence, so I had to use *30* iterations to make measurement noise to be about 
> an order of magnitude lower than the effect I'm trying to measure:
> 
>  Performance counter stats for 'make -j128' (30 runs):
> 
>       26.334767571 seconds time elapsed    (+- 0.09% )
> 
> i.e. "26.334 +- 0.023" seconds is a number we can have pretty high confidence in, 
> on this system.
> 
> And just to demonstrate that it's all real, I repeated the whole 30-iteration 
> measurement again:
> 
>  Performance counter stats for 'make -j128' (30 runs):
> 
>       26.311166142 seconds time elapsed    (+- 0.07%)
> 

-- 
Thanks,
Changbin Du

  reply	other threads:[~2018-05-02  9:18 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-01 13:00 changbin.du
2018-05-01 13:00 ` [PATCH 1/5] x86/mm: surround level4_kernel_pgt with #ifdef CONFIG_X86_5LEVEL...#endif changbin.du
2018-05-01 13:00 ` [PATCH 2/5] regulator: add dummy of_find_regulator_by_node changbin.du
2018-05-01 20:40   ` Mark Brown
2018-05-02  9:24     ` Du, Changbin
2018-05-05  1:44   ` Applied "regulator: add dummy function of_find_regulator_by_node" to the regulator tree Mark Brown
2018-05-01 13:00 ` [PATCH 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler atuo-inline optimizations changbin.du
2018-05-01 14:54   ` Steven Rostedt
2018-05-02  9:27     ` Du, Changbin
2018-05-01 13:00 ` [PATCH 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization changbin.du
2018-05-01 15:25   ` Randy Dunlap
2018-05-02  9:24     ` Du, Changbin
2018-05-02 20:30   ` Arnd Bergmann
2018-05-01 13:00 ` [PATCH 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE changbin.du
2018-05-02  7:33 ` [PATCH 0/5] kernel hacking: GCC optimization for debug experience (-Og) Ingo Molnar
2018-05-02  9:07   ` Du, Changbin [this message]
2018-05-02 11:09     ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180502090756.tztulppgfefccd7q@intel.com \
    --to=changbin.du@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=broonie@kernel.org \
    --cc=lgirdwood@gmail.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=mingo@kernel.org \
    --cc=mingo@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    --cc=yamada.masahiro@socionext.com \
    --subject='Re: [PATCH 0/5] kernel hacking: GCC optimization for debug experience (-Og)' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).