[PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[Yu Kuai]

We run a test that delete and recover devcies frequently(two devices on
the same host), and we found that 'active_queues' is super big after a
period of time.

If device a and device b share a tag set, and a is deleted, then
blk_mq_exit_queue() will clear BLK_MQ_F_TAG_QUEUE_SHARED because there
is only one queue that are using the tag set. However, if b is still
active, the active_queues of b might never be cleared even if b is
deleted.

Thus clear active_queues before BLK_MQ_F_TAG_QUEUE_SHARED is cleared.

Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
 block/blk-mq.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 2c4ac51e54eb..2fe396385a4a 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2994,10 +2994,12 @@ static void queue_set_hctx_shared(struct request_queue *q, bool shared)
 	int i;
 
 	queue_for_each_hw_ctx(q, hctx, i) {
-		if (shared)
+		if (shared) {
 			hctx->flags |= BLK_MQ_F_TAG_QUEUE_SHARED;
-		else
+		} else {
+			blk_mq_tag_idle(hctx);
 			hctx->flags &= ~BLK_MQ_F_TAG_QUEUE_SHARED;
+		}
 	}
 }
 
-- 
2.31.1

Re: [PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[yukuai (C)]

On 2021/07/31 14:21, Yu Kuai wrote:
> We run a test that delete and recover devcies frequently(two devices on
> the same host), and we found that 'active_queues' is super big after a
> period of time.
> 
> If device a and device b share a tag set, and a is deleted, then
> blk_mq_exit_queue() will clear BLK_MQ_F_TAG_QUEUE_SHARED because there
> is only one queue that are using the tag set. However, if b is still
> active, the active_queues of b might never be cleared even if b is
> deleted.
> 
> Thus clear active_queues before BLK_MQ_F_TAG_QUEUE_SHARED is cleared.
> 
> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
> ---
>   block/blk-mq.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 2c4ac51e54eb..2fe396385a4a 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2994,10 +2994,12 @@ static void queue_set_hctx_shared(struct request_queue *q, bool shared)
>   	int i;
>   
>   	queue_for_each_hw_ctx(q, hctx, i) {
> -		if (shared)
> +		if (shared) {
>   			hctx->flags |= BLK_MQ_F_TAG_QUEUE_SHARED;
> -		else
> +		} else {
> +			blk_mq_tag_idle(hctx);
>   			hctx->flags &= ~BLK_MQ_F_TAG_QUEUE_SHARED;
> +		}
>   	}
>   }
>   
> 

ping...

Re: [PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[Ming Lei]

On Sat, Jul 31, 2021 at 02:21:30PM +0800, Yu Kuai wrote:
> We run a test that delete and recover devcies frequently(two devices on
> the same host), and we found that 'active_queues' is super big after a
> period of time.
> 
> If device a and device b share a tag set, and a is deleted, then
> blk_mq_exit_queue() will clear BLK_MQ_F_TAG_QUEUE_SHARED because there
> is only one queue that are using the tag set. However, if b is still
> active, the active_queues of b might never be cleared even if b is
> deleted.
> 
> Thus clear active_queues before BLK_MQ_F_TAG_QUEUE_SHARED is cleared.
> 
> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
> ---
>  block/blk-mq.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 2c4ac51e54eb..2fe396385a4a 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2994,10 +2994,12 @@ static void queue_set_hctx_shared(struct request_queue *q, bool shared)
>  	int i;
>  
>  	queue_for_each_hw_ctx(q, hctx, i) {
> -		if (shared)
> +		if (shared) {
>  			hctx->flags |= BLK_MQ_F_TAG_QUEUE_SHARED;
> -		else
> +		} else {
> +			blk_mq_tag_idle(hctx);
>  			hctx->flags &= ~BLK_MQ_F_TAG_QUEUE_SHARED;
> +		}
>  	}
>  }

Looks correct, the only remained queue in tagset has to be idled before
clearing BLK_MQ_F_TAG_QUEUE_SHARED:

Reviewed-by: Ming Lei <ming.lei@redhat.com>



Thanks,
Ming

Re: [PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[yukuai (C)]

Hi, jens

Can you please consider to apply this patch.

By the way, sorry about the wrong email I replied.

Thanks
Kuai

On 2021/08/06 10:25, Ming Lei wrote:
> On Sat, Jul 31, 2021 at 02:21:30PM +0800, Yu Kuai wrote:
>> We run a test that delete and recover devcies frequently(two devices on
>> the same host), and we found that 'active_queues' is super big after a
>> period of time.
>>
>> If device a and device b share a tag set, and a is deleted, then
>> blk_mq_exit_queue() will clear BLK_MQ_F_TAG_QUEUE_SHARED because there
>> is only one queue that are using the tag set. However, if b is still
>> active, the active_queues of b might never be cleared even if b is
>> deleted.
>>
>> Thus clear active_queues before BLK_MQ_F_TAG_QUEUE_SHARED is cleared.
>>
>> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
>> ---
>>   block/blk-mq.c | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>> index 2c4ac51e54eb..2fe396385a4a 100644
>> --- a/block/blk-mq.c
>> +++ b/block/blk-mq.c
>> @@ -2994,10 +2994,12 @@ static void queue_set_hctx_shared(struct request_queue *q, bool shared)
>>   	int i;
>>   
>>   	queue_for_each_hw_ctx(q, hctx, i) {
>> -		if (shared)
>> +		if (shared) {
>>   			hctx->flags |= BLK_MQ_F_TAG_QUEUE_SHARED;
>> -		else
>> +		} else {
>> +			blk_mq_tag_idle(hctx);
>>   			hctx->flags &= ~BLK_MQ_F_TAG_QUEUE_SHARED;
>> +		}
>>   	}
>>   }
> 
> Looks correct, the only remained queue in tagset has to be idled before
> clearing BLK_MQ_F_TAG_QUEUE_SHARED:
> 
> Reviewed-by: Ming Lei <ming.lei@redhat.com>
> 
> 
> 
> Thanks,
> Ming
> 
> .
> 

Re: [PATCH] blk-mq: clear active_queues before clearing BLK_MQ_F_TAG_QUEUE_SHARED

[Jens Axboe]

On 7/31/21 12:21 AM, Yu Kuai wrote:
> We run a test that delete and recover devcies frequently(two devices on
> the same host), and we found that 'active_queues' is super big after a
> period of time.
> 
> If device a and device b share a tag set, and a is deleted, then
> blk_mq_exit_queue() will clear BLK_MQ_F_TAG_QUEUE_SHARED because there
> is only one queue that are using the tag set. However, if b is still
> active, the active_queues of b might never be cleared even if b is
> deleted.
> 
> Thus clear active_queues before BLK_MQ_F_TAG_QUEUE_SHARED is cleared.

Applied, thanks.

-- 
Jens Axboe