LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
@ 2011-01-17 21:56 Vivek Goyal
  2011-01-17 22:00 ` Jeff Moyer
  2011-01-26 18:57 ` Vivek Goyal
  0 siblings, 2 replies; 7+ messages in thread
From: Vivek Goyal @ 2011-01-17 21:56 UTC (permalink / raw)
  To: linux kernel mailing list, Jens Axboe; +Cc: Moyer Jeff Moyer

o Jeff Moyer was doing some testing on a RAM backed disk and
  blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
  somebody else reported that blkiocg_lookup_group() is eating 6% extra
  cpu. Though looking at the code I can't think why the overhead of
  this function is so high. One thing is that it is called with very high
  frequency (once for every IO).

o For lot of folks blkio controller will be compiled in but they might
  not have actually created cgroups. Hence optimize the case of root
  cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
  in root group (common case).

Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
---
 block/blk-throttle.c |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

Index: linux-2.6/block/blk-throttle.c
===================================================================
--- linux-2.6.orig/block/blk-throttle.c	2011-01-17 16:23:37.041280712 -0500
+++ linux-2.6/block/blk-throttle.c	2011-01-17 16:36:09.362940975 -0500
@@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a
 	 * tree of blkg (instead of traversing through hash list all
 	 * the time.
 	 */
-	tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
+
+	/*
+	 * This is the common case when there are no blkio cgroups.
+ 	 * Avoid lookup in this case
+ 	 */
+	if (blkcg == &blkio_root_cgroup)
+		tg = &td->root_tg;
+	else
+		tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
 
 	/* Fill in device details for root group */
 	if (tg && !tg->blkg.dev && bdi->dev && dev_name(bdi->dev)) {


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-17 21:56 [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group Vivek Goyal
@ 2011-01-17 22:00 ` Jeff Moyer
  2011-01-26 18:57 ` Vivek Goyal
  1 sibling, 0 replies; 7+ messages in thread
From: Jeff Moyer @ 2011-01-17 22:00 UTC (permalink / raw)
  To: Vivek Goyal; +Cc: linux kernel mailing list, Jens Axboe

Vivek Goyal <vgoyal@redhat.com> writes:

> o Jeff Moyer was doing some testing on a RAM backed disk and
>   blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
>   somebody else reported that blkiocg_lookup_group() is eating 6% extra
>   cpu. Though looking at the code I can't think why the overhead of
>   this function is so high. One thing is that it is called with very high
>   frequency (once for every IO).
>
> o For lot of folks blkio controller will be compiled in but they might
>   not have actually created cgroups. Hence optimize the case of root
>   cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
>   in root group (common case).
>
> Reported-by: Jeff Moyer <jmoyer@redhat.com>
> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>

Acked-by: Jeff Moyer <jmoyer@redhat.com>

In case it wasn't obvious, this patch was tested and confirmed to fix
the performance degradation.

Cheers,
Jeff

> ---
>  block/blk-throttle.c |   10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
>
> Index: linux-2.6/block/blk-throttle.c
> ===================================================================
> --- linux-2.6.orig/block/blk-throttle.c	2011-01-17 16:23:37.041280712 -0500
> +++ linux-2.6/block/blk-throttle.c	2011-01-17 16:36:09.362940975 -0500
> @@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a
>  	 * tree of blkg (instead of traversing through hash list all
>  	 * the time.
>  	 */
> -	tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
> +
> +	/*
> +	 * This is the common case when there are no blkio cgroups.
> + 	 * Avoid lookup in this case
> + 	 */
> +	if (blkcg == &blkio_root_cgroup)
> +		tg = &td->root_tg;
> +	else
> +		tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
>  
>  	/* Fill in device details for root group */
>  	if (tg && !tg->blkg.dev && bdi->dev && dev_name(bdi->dev)) {

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-17 21:56 [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group Vivek Goyal
  2011-01-17 22:00 ` Jeff Moyer
@ 2011-01-26 18:57 ` Vivek Goyal
  2011-01-31 19:20   ` Jeff Moyer
  1 sibling, 1 reply; 7+ messages in thread
From: Vivek Goyal @ 2011-01-26 18:57 UTC (permalink / raw)
  To: linux kernel mailing list, Jens Axboe; +Cc: Moyer Jeff Moyer

On Mon, Jan 17, 2011 at 04:56:06PM -0500, Vivek Goyal wrote:
> o Jeff Moyer was doing some testing on a RAM backed disk and
>   blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
>   somebody else reported that blkiocg_lookup_group() is eating 6% extra
>   cpu. Though looking at the code I can't think why the overhead of
>   this function is so high. One thing is that it is called with very high
>   frequency (once for every IO).
> 
> o For lot of folks blkio controller will be compiled in but they might
>   not have actually created cgroups. Hence optimize the case of root
>   cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
>   in root group (common case).
> 
> Reported-by: Jeff Moyer <jmoyer@redhat.com>
> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>

Hi Jens,

Do you have any concerns regarding this patch?

Thanks
Vivek

> ---
>  block/blk-throttle.c |   10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> Index: linux-2.6/block/blk-throttle.c
> ===================================================================
> --- linux-2.6.orig/block/blk-throttle.c	2011-01-17 16:23:37.041280712 -0500
> +++ linux-2.6/block/blk-throttle.c	2011-01-17 16:36:09.362940975 -0500
> @@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a
>  	 * tree of blkg (instead of traversing through hash list all
>  	 * the time.
>  	 */
> -	tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
> +
> +	/*
> +	 * This is the common case when there are no blkio cgroups.
> + 	 * Avoid lookup in this case
> + 	 */
> +	if (blkcg == &blkio_root_cgroup)
> +		tg = &td->root_tg;
> +	else
> +		tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
>  
>  	/* Fill in device details for root group */
>  	if (tg && !tg->blkg.dev && bdi->dev && dev_name(bdi->dev)) {
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-26 18:57 ` Vivek Goyal
@ 2011-01-31 19:20   ` Jeff Moyer
  2011-01-31 19:24     ` Jens Axboe
  0 siblings, 1 reply; 7+ messages in thread
From: Jeff Moyer @ 2011-01-31 19:20 UTC (permalink / raw)
  To: Vivek Goyal; +Cc: linux kernel mailing list, Jens Axboe

Vivek Goyal <vgoyal@redhat.com> writes:

> On Mon, Jan 17, 2011 at 04:56:06PM -0500, Vivek Goyal wrote:
>> o Jeff Moyer was doing some testing on a RAM backed disk and
>>   blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
>>   somebody else reported that blkiocg_lookup_group() is eating 6% extra
>>   cpu. Though looking at the code I can't think why the overhead of
>>   this function is so high. One thing is that it is called with very high
>>   frequency (once for every IO).
>> 
>> o For lot of folks blkio controller will be compiled in but they might
>>   not have actually created cgroups. Hence optimize the case of root
>>   cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
>>   in root group (common case).
>> 
>> Reported-by: Jeff Moyer <jmoyer@redhat.com>
>> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
>
> Hi Jens,
>
> Do you have any concerns regarding this patch?

Acked-by: Jeff Moyer <jmoyer@redhat.com>

Jens, this is a pretty easy performance regression fixup.  I think it
should be pulled in sooner rather than later.  We've also witnessed this
slowdown on big performance testing rigs, so it's not just a ramdisk
issue.

Cheers,
Jeff

>> ---
>>  block/blk-throttle.c |   10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> Index: linux-2.6/block/blk-throttle.c
>> ===================================================================
>> --- linux-2.6.orig/block/blk-throttle.c	2011-01-17 16:23:37.041280712 -0500
>> +++ linux-2.6/block/blk-throttle.c	2011-01-17 16:36:09.362940975 -0500
>> @@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a
>>  	 * tree of blkg (instead of traversing through hash list all
>>  	 * the time.
>>  	 */
>> -	tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
>> +
>> +	/*
>> +	 * This is the common case when there are no blkio cgroups.
>> + 	 * Avoid lookup in this case
>> + 	 */
>> +	if (blkcg == &blkio_root_cgroup)
>> +		tg = &td->root_tg;
>> +	else
>> +		tg = tg_of_blkg(blkiocg_lookup_group(blkcg, key));
>>  
>>  	/* Fill in device details for root group */
>>  	if (tg && !tg->blkg.dev && bdi->dev && dev_name(bdi->dev)) {
>> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-31 19:20   ` Jeff Moyer
@ 2011-01-31 19:24     ` Jens Axboe
  2011-01-31 19:25       ` Jens Axboe
  0 siblings, 1 reply; 7+ messages in thread
From: Jens Axboe @ 2011-01-31 19:24 UTC (permalink / raw)
  To: Jeff Moyer; +Cc: Vivek Goyal, linux kernel mailing list

On 2011-01-31 20:20, Jeff Moyer wrote:
> Vivek Goyal <vgoyal@redhat.com> writes:
> 
>> On Mon, Jan 17, 2011 at 04:56:06PM -0500, Vivek Goyal wrote:
>>> o Jeff Moyer was doing some testing on a RAM backed disk and
>>>   blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
>>>   somebody else reported that blkiocg_lookup_group() is eating 6% extra
>>>   cpu. Though looking at the code I can't think why the overhead of
>>>   this function is so high. One thing is that it is called with very high
>>>   frequency (once for every IO).
>>>
>>> o For lot of folks blkio controller will be compiled in but they might
>>>   not have actually created cgroups. Hence optimize the case of root
>>>   cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
>>>   in root group (common case).
>>>
>>> Reported-by: Jeff Moyer <jmoyer@redhat.com>
>>> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
>>
>> Hi Jens,
>>
>> Do you have any concerns regarding this patch?
> 
> Acked-by: Jeff Moyer <jmoyer@redhat.com>
> 
> Jens, this is a pretty easy performance regression fixup.  I think it
> should be pulled in sooner rather than later.  We've also witnessed this
> slowdown on big performance testing rigs, so it's not just a ramdisk
> issue.

Yep, will fold it into 2.6.38, thanks!


-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-31 19:24     ` Jens Axboe
@ 2011-01-31 19:25       ` Jens Axboe
  2011-01-31 19:30         ` Jeff Moyer
  0 siblings, 1 reply; 7+ messages in thread
From: Jens Axboe @ 2011-01-31 19:25 UTC (permalink / raw)
  To: Jeff Moyer; +Cc: Vivek Goyal, linux kernel mailing list

On 2011-01-31 20:24, Jens Axboe wrote:
> On 2011-01-31 20:20, Jeff Moyer wrote:
>> Vivek Goyal <vgoyal@redhat.com> writes:
>>
>>> On Mon, Jan 17, 2011 at 04:56:06PM -0500, Vivek Goyal wrote:
>>>> o Jeff Moyer was doing some testing on a RAM backed disk and
>>>>   blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly
>>>>   somebody else reported that blkiocg_lookup_group() is eating 6% extra
>>>>   cpu. Though looking at the code I can't think why the overhead of
>>>>   this function is so high. One thing is that it is called with very high
>>>>   frequency (once for every IO).
>>>>
>>>> o For lot of folks blkio controller will be compiled in but they might
>>>>   not have actually created cgroups. Hence optimize the case of root
>>>>   cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening
>>>>   in root group (common case).
>>>>
>>>> Reported-by: Jeff Moyer <jmoyer@redhat.com>
>>>> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
>>>
>>> Hi Jens,
>>>
>>> Do you have any concerns regarding this patch?
>>
>> Acked-by: Jeff Moyer <jmoyer@redhat.com>
>>
>> Jens, this is a pretty easy performance regression fixup.  I think it
>> should be pulled in sooner rather than later.  We've also witnessed this
>> slowdown on big performance testing rigs, so it's not just a ramdisk
>> issue.
> 
> Yep, will fold it into 2.6.38, thanks!

Just to be clear, it's queued up in for-linus so it will go to 2.6.38 on
the next push. It was updated last week, this is what it has so far:

Sergey Senozhatsky (1):
      loop: queue_lock NULL pointer derefence in blk_throtl_exit

Stephen M. Cameron (1):
      cciss: make cciss_revalidate not loop through CISS_MAX_LUNS volumes unnecessarily.

Tao Ma (1):
      blktrace: Don't output messages if NOTIFY isn't set.

Tracey Dent (2):
      drivers/block/Makefile: replace the use of <module>-objs with <module>-y
      drivers/block/aoe/Makefile: replace the use of <module>-objs with <module>-y

Vivek Goyal (2):
      cfq: rename a function to give it more appropriate name
      blkio-throttle: Avoid calling blkiocg_lookup_group() for root group

 block/blk-throttle.c       |   10 +++++++++-
 block/cfq-iosched.c        |    6 +++---
 drivers/block/Makefile     |    2 +-
 drivers/block/aoe/Makefile |    2 +-
 drivers/block/cciss.c      |    2 +-
 drivers/block/loop.c       |    3 +++
 kernel/trace/blktrace.c    |    7 +++++++
 7 files changed, 25 insertions(+), 7 deletions(-)

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
  2011-01-31 19:25       ` Jens Axboe
@ 2011-01-31 19:30         ` Jeff Moyer
  0 siblings, 0 replies; 7+ messages in thread
From: Jeff Moyer @ 2011-01-31 19:30 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Vivek Goyal, linux kernel mailing list

Jens Axboe <jaxboe@fusionio.com> writes:

>> Yep, will fold it into 2.6.38, thanks!

Awesome, thanks Jens!

-Jeff

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2011-01-31 19:30 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-17 21:56 [PATCH] blkio-throttle: Avoid calling blkiocg_lookup_group() for root group Vivek Goyal
2011-01-17 22:00 ` Jeff Moyer
2011-01-26 18:57 ` Vivek Goyal
2011-01-31 19:20   ` Jeff Moyer
2011-01-31 19:24     ` Jens Axboe
2011-01-31 19:25       ` Jens Axboe
2011-01-31 19:30         ` Jeff Moyer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).