LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: Sagi Grimberg <sagi@grimberg.me>,
	linux-block@vger.kernel.org, linux-nvme@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 14/16] nvme: utilize two queue maps, one for reads and one for writes
Date: Wed, 31 Oct 2018 08:32:36 -0600	[thread overview]
Message-ID: <29cd36b5-623a-1143-3aee-a81265ec70eb@kernel.dk> (raw)
In-Reply-To: <193a50fe-0c02-9f01-f110-5cfa1cec9283@grimberg.me>

On 10/30/18 7:57 PM, Sagi Grimberg wrote:
> 
>> +static int queue_irq_offset(struct nvme_dev *dev)
>> +{
>> +	/* if we have more than 1 vec, admin queue offsets us 1 */
> 
> offsets us by 1?

Fixed

>> @@ -1934,13 +2048,48 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
>>   	 * setting up the full range we need.
>>   	 */
>>   	pci_free_irq_vectors(pdev);
>> -	result = pci_alloc_irq_vectors_affinity(pdev, 1, nr_io_queues + 1,
>> -			PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd);
>> -	if (result <= 0)
>> -		return -EIO;
>> +
>> +	/*
>> +	 * For irq sets, we have to ask for minvec == maxvec. This passes
>> +	 * any reduction back to us, so we can adjust our queue counts and
>> +	 * IRQ vector needs.
>> +	 */
>> +	do {
>> +		nvme_calc_io_queues(dev, nr_io_queues);
>> +		irq_sets[0] = dev->io_queues[NVMEQ_TYPE_READ];
>> +		irq_sets[1] = dev->io_queues[NVMEQ_TYPE_WRITE];
>> +		if (!irq_sets[1])
>> +			affd.nr_sets = 1;
>> +
>> +		/*
>> +		 * Need IRQs for read+write queues, and one for the admin queue
>> +		 */
>> +		nr_io_queues = irq_sets[0] + irq_sets[1] + 1;
>> +
>> +		result = pci_alloc_irq_vectors_affinity(pdev, nr_io_queues,
>> +				nr_io_queues,
>> +				PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd);
>> +
>> +		/*
>> +		 * Need to reduce our vec counts
>> +		 */
>> +		if (result == -ENOSPC) {
>> +			nr_io_queues--;
>> +			if (!nr_io_queues)
>> +				return result;
>> +			continue;
>> +		} else if (result <= 0)
>> +			return -EIO;
>> +		break;
>> +	} while (1);
>> +
>>   	dev->num_vecs = result;
>>   	dev->max_qid = max(result - 1, 1);
>>   
>> +	dev_info(dev->ctrl.device, "%d/%d/%d read/write queues\n",
>> +					dev->io_queues[NVMEQ_TYPE_READ],
>> +					dev->io_queues[NVMEQ_TYPE_WRITE]);
>> +
> 
> Perhaps it would be better if we move this code into a function.

Agree, I've done that now.

-- 
Jens Axboe


  reply	other threads:[~2018-10-31 14:32 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-30 18:32 [PATCHSET v3 0/16] blk-mq: Add support for multiple queue maps Jens Axboe
2018-10-30 18:32 ` [PATCH 01/16] blk-mq: kill q->mq_map Jens Axboe
2018-10-31  0:28   ` Sagi Grimberg
2018-10-30 18:32 ` [PATCH 02/16] blk-mq: abstract out queue map Jens Axboe
2018-10-31  0:31   ` Sagi Grimberg
2018-10-31 14:17     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 03/16] blk-mq: provide dummy blk_mq_map_queue_type() helper Jens Axboe
2018-10-31  0:32   ` Sagi Grimberg
2018-10-30 18:32 ` [PATCH 04/16] blk-mq: pass in request/bio flags to queue mapping Jens Axboe
2018-10-31  0:37   ` Sagi Grimberg
2018-10-31 14:18     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 05/16] blk-mq: allow software queue to map to multiple hardware queues Jens Axboe
2018-10-31  0:49   ` Sagi Grimberg
2018-10-31 14:19     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 06/16] blk-mq: add 'type' attribute to the sysfs hctx directory Jens Axboe
2018-10-31  0:53   ` Sagi Grimberg
2018-10-31 14:21     ` Jens Axboe
2018-11-01 21:59   ` Omar Sandoval
2018-11-01 22:50     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 07/16] blk-mq: support multiple hctx maps Jens Axboe
2018-10-31  0:59   ` Sagi Grimberg
2018-10-31 14:23     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 08/16] blk-mq: separate number of hardware queues from nr_cpu_ids Jens Axboe
2018-10-31  1:00   ` Sagi Grimberg
2018-10-30 18:32 ` [PATCH 09/16] blk-mq: cache request hardware queue mapping Jens Axboe
2018-10-31  1:01   ` Sagi Grimberg
2018-11-01  9:27   ` Hannes Reinecke
2018-11-01 12:22     ` Jens Axboe
2018-10-30 18:32 ` [PATCH 10/16] blk-mq: cleanup and improve list insertion Jens Axboe
2018-10-31  1:03   ` Sagi Grimberg
2018-11-01  9:28   ` Hannes Reinecke
2018-10-30 18:32 ` [PATCH 11/16] blk-mq: improve plug list sorting Jens Axboe
2018-10-31  1:04   ` Sagi Grimberg
2018-11-01  9:30   ` Hannes Reinecke
2018-10-30 18:32 ` [PATCH 12/16] blk-mq: initial support for multiple queue maps Jens Axboe
2018-10-31  1:14   ` Sagi Grimberg
2018-10-30 18:32 ` [PATCH 13/16] irq: add support for allocating (and affinitizing) sets of IRQs Jens Axboe
2018-10-31  1:17   ` Sagi Grimberg
2018-11-02 14:37   ` Ming Lei
2018-11-02 15:09     ` Keith Busch
2018-11-03  2:22       ` Ming Lei
2018-10-30 18:32 ` [PATCH 14/16] nvme: utilize two queue maps, one for reads and one for writes Jens Axboe
2018-10-31  1:57   ` Sagi Grimberg
2018-10-31 14:32     ` Jens Axboe [this message]
2018-10-30 18:32 ` [PATCH 15/16] block: add REQ_HIPRI and inherit it from IOCB_HIPRI Jens Axboe
2018-10-31  1:58   ` Sagi Grimberg
2018-10-30 18:32 ` [PATCH 16/16] nvme: add separate poll queue map Jens Axboe
2018-10-30 18:35 ` [PATCHSET v3 0/16] blk-mq: Add support for multiple queue maps Keith Busch

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=29cd36b5-623a-1143-3aee-a81265ec70eb@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=sagi@grimberg.me \
    --subject='Re: [PATCH 14/16] nvme: utilize two queue maps, one for reads and one for writes' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).