LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: "Ouyang Zhaowei (Charles)" <ouyangzhaowei@huawei.com>
To: "Roger Pau Monné" <roger.pau@citrix.com>
Cc: <linux-kernel@vger.kernel.org>, <weiping.ding@huawei.com>,
	xen-devel <xen-devel@lists.xenproject.org>,
	David Vrabel <david.vrabel@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>
Subject: Re: xen-blkfront: weird behavior of "iostat" after VM live-migrate which xen-blkfront module has indirect descriptors
Date: Fri, 30 Jan 2015 16:37:19 +0800	[thread overview]
Message-ID: <54CB42BF.5050003@huawei.com> (raw)
In-Reply-To: <54C5A6A8.1080402@huawei.com>



On 2015.1.26 10:30, Ouyang Zhaowei (Charles) wrote:
> 
> On 2015.1.23 19:15, Roger Pau Monné wrote:
>> Hello,
>>
>> El 23/01/15 a les 8.59, Ouyang Zhaowei (Charles) ha escrit:
>>> Hi Roger,
>>>
>>> We are testing the indirect feature of xen-blkfront module these days.
>>> And we found that, after VM live-migrate a couple of times, the "%util" of iostat keeps being 100%, and there are several requests stock in "avgqu-sz".
>>> We have checked some later version of Linux, and it happens on Ubuntu 14.04, Ubuntu 14.10 and RHEL 7.0.
>>>
>>> The iostat shows like below:
>>>
>>> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>>>            0.00    0.00    0.00    0.00    0.00  100.00
>>>
>>> Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
>>> xvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00     4.00    0.00    0.00    0.00   0.00 100.00
>>> dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
>>> dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
>>>
>>> Could you tell us that why is this happening, is this a bug?
>>
>> It is a bug indeed, thanks for reporting it. The problem seems to be 
>> that blk_put_request (which is used to discard the old requests before 
>> requeuing them) doesn't update the queue statistics. The following 
>> patch solves the problem for me, could you try it and report back?

Hi Roger,

After near 1000 times migrate test, the "%util" of iostat did not become 100% anymore, seems like the patch fix this bug

Thanks

Ouyang Zhaowei

>>
>> ---
>> commit bb4317c051ca81a2906edb7ccc505cbd6d1d80c7
>> Author: Roger Pau Monne <roger.pau@citrix.com>
>> Date:   Fri Jan 23 12:10:51 2015 +0100
>>
>>     xen-blkfront: fix accounting of reqs when migrating
>>     
>>     Current migration code uses blk_put_request in order to finish a request
>>     before requeuing it. This function doesn't update the statistics of the
>>     queue, which completely screws accounting. Use blk_end_request_all instead
>>     which properly updates the statistics of the queue.
>>     
>>     Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
>> index 5ac312f..aac41c1 100644
>> --- a/drivers/block/xen-blkfront.c
>> +++ b/drivers/block/xen-blkfront.c
>> @@ -1493,7 +1493,7 @@ static int blkif_recover(struct blkfront_info *info)
>>  		merge_bio.tail = copy[i].request->biotail;
>>  		bio_list_merge(&bio_list, &merge_bio);
>>  		copy[i].request->bio = NULL;
>> -		blk_put_request(copy[i].request);
>> +		blk_end_request_all(copy[i].request, 0);
>>  	}
>>  
>>  	kfree(copy);
>> @@ -1516,7 +1516,7 @@ static int blkif_recover(struct blkfront_info *info)
>>  		req->bio = NULL;
>>  		if (req->cmd_flags & (REQ_FLUSH | REQ_FUA))
>>  			pr_alert("diskcache flush request found!\n");
>> -		__blk_put_request(info->rq, req);
>> +		__blk_end_request_all(req, 0);
>>  	}
>>  	spin_unlock_irq(&info->io_lock);
>>  
>>
> 
> Hi Roger,
> 
> Thanks for answering this question. Sure, I'll try this patch and test VM migrating, so far it seems this patch has solved this bug (after 10 times migrate).
> I'll keep testing it for more times and will let you know if it's OK.
> 
> Regards,
> Ouyang Zhaowei
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> .
> 


      reply	other threads:[~2015-01-30  8:37 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-23  7:59 Ouyang Zhaowei (Charles)
2015-01-23 11:15 ` Roger Pau Monné
2015-01-26  2:30   ` Ouyang Zhaowei (Charles)
2015-01-30  8:37     ` Ouyang Zhaowei (Charles) [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54CB42BF.5050003@huawei.com \
    --to=ouyangzhaowei@huawei.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=david.vrabel@citrix.com \
    --cc=konrad.wilk@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=roger.pau@citrix.com \
    --cc=weiping.ding@huawei.com \
    --cc=xen-devel@lists.xenproject.org \
    --subject='Re: xen-blkfront: weird behavior of "iostat" after VM live-migrate which xen-blkfront module has indirect descriptors' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).