LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096
@ 2018-04-19 12:30 Wanpeng Li
  2018-04-19 13:06 ` Cornelia Huck
  0 siblings, 1 reply; 4+ messages in thread
From: Wanpeng Li @ 2018-04-19 12:30 UTC (permalink / raw)
  To: linux-kernel, kvm; +Cc: Paolo Bonzini, Radim Krčmář, Tonny Lu

From: Wanpeng Li <wanpengli@tencent.com>

Our virtual machines make use of device assignment by configuring
12 NVMe disks for high I/O performance. Each NVMe device has 129 
MSI-X Table entries:
Capabilities: [50] MSI-X: Enable+ Count=129 Masked-Vector table: BAR=0 offset=00002000
The windows virtual machines fail to boot since they will map the number of 
MSI-table entries that the NVMe hardware reported to the bus to msi routing 
table, this will exceed the 1024. This patch extends MAX_IRQ_ROUTES to 4096,
In the future this might be extended if needed.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Tonny Lu <tonnylu@tencent.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Tonny Lu <tonnylu@tencent.com>
---
 include/linux/kvm_host.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6930c63..815ae66 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1050,7 +1050,7 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
 #elif defined(CONFIG_ARM64)
 #define KVM_MAX_IRQ_ROUTES 4096
 #else
-#define KVM_MAX_IRQ_ROUTES 1024
+#define KVM_MAX_IRQ_ROUTES 4096
 #endif
 
 bool kvm_arch_can_set_irq_routing(struct kvm *kvm);
-- 
2.7.4

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096
  2018-04-19 12:30 [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096 Wanpeng Li
@ 2018-04-19 13:06 ` Cornelia Huck
       [not found]   ` <KL1PR02MB1301D3C3A62E6CED2530496180B50@KL1PR02MB1301.apcprd02.prod.outlook.com>
  0 siblings, 1 reply; 4+ messages in thread
From: Cornelia Huck @ 2018-04-19 13:06 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: linux-kernel, kvm, Paolo Bonzini, Radim Krčmář, Tonny Lu

On Thu, 19 Apr 2018 05:30:40 -0700
Wanpeng Li <kernellwp@gmail.com> wrote:

> From: Wanpeng Li <wanpengli@tencent.com>
> 
> Our virtual machines make use of device assignment by configuring
> 12 NVMe disks for high I/O performance. Each NVMe device has 129 
> MSI-X Table entries:
> Capabilities: [50] MSI-X: Enable+ Count=129 Masked-Vector table: BAR=0 offset=00002000
> The windows virtual machines fail to boot since they will map the number of 
> MSI-table entries that the NVMe hardware reported to the bus to msi routing 
> table, this will exceed the 1024. This patch extends MAX_IRQ_ROUTES to 4096,
> In the future this might be extended if needed.
> 
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Tonny Lu <tonnylu@tencent.com>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> Signed-off-by: Tonny Lu <tonnylu@tencent.com>
> ---
>  include/linux/kvm_host.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 6930c63..815ae66 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -1050,7 +1050,7 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
>  #elif defined(CONFIG_ARM64)
>  #define KVM_MAX_IRQ_ROUTES 4096
>  #else
> -#define KVM_MAX_IRQ_ROUTES 1024
> +#define KVM_MAX_IRQ_ROUTES 4096
>  #endif
>  
>  bool kvm_arch_can_set_irq_routing(struct kvm *kvm);

So, this basically means we have 4096 everywhere, no?

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096
       [not found]   ` <KL1PR02MB1301D3C3A62E6CED2530496180B50@KL1PR02MB1301.apcprd02.prod.outlook.com>
@ 2018-04-19 14:09     ` Cornelia Huck
  2018-04-20  0:55       ` Wanpeng Li
  0 siblings, 1 reply; 4+ messages in thread
From: Cornelia Huck @ 2018-04-19 14:09 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Wanpeng Li, linux-kernel, kvm, Paolo Bonzini,
	Radim Krčmář,
	Tonny Lu

On Thu, 19 Apr 2018 13:42:55 +0000
Wanpeng Li <wanpeng.li@hotmail.com> wrote:

> On Thu, 19 Apr 2018 05:30:40 -0700
> 
> Wanpeng Li <kernellwp@gmail.com> wrote:
> 
> > From: Wanpeng Li <wanpengli@tencent.com>
> >
> > Our virtual machines make use of device assignment by configuring
> > 12 NVMe disks for high I/O performance. Each NVMe device has 129
> > MSI-X Table entries:
> > Capabilities: [50] MSI-X: Enable+ Count=129 Masked-Vector table: BAR=0 offset=00002000
> > The windows virtual machines fail to boot since they will map the number of
> > MSI-table entries that the NVMe hardware reported to the bus to msi routing
> > table, this will exceed the 1024. This patch extends MAX_IRQ_ROUTES to 4096,
> > In the future this might be extended if needed.
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Krčmář <rkrcmar@redhat.com>
> > Cc: Tonny Lu <tonnylu@tencent.com>
> > Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > Signed-off-by: Tonny Lu <tonnylu@tencent.com>
> > ---
> >  include/linux/kvm_host.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> > index 6930c63..815ae66 100644
> > --- a/include/linux/kvm_host.h
> > +++ b/include/linux/kvm_host.h
> > @@ -1050,7 +1050,7 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
> >  #elif defined(CONFIG_ARM64)
> >  #define KVM_MAX_IRQ_ROUTES 4096
> >  #else
> > -#define KVM_MAX_IRQ_ROUTES 1024
> > +#define KVM_MAX_IRQ_ROUTES 4096
> >  #endif
> >
> >  bool kvm_arch_can_set_irq_routing(struct kvm *kvm);  
> 
> So, this basically means we have 4096 everywhere, no?
> 
> I suspect different architectures maybe extend to different limits again according to their requirements.

Yes, but for now, we have the same everywhere (as you also bumped the
limit on power and 32-bit arm, implicitly). If that's ok, we might as
well get rid of the ifdeffery.

Also, my additional remark in f3f710bc64e12 still holds:

"We need to find a more general solution, though, as we can't just grow
the routing table indefinitly."

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096
  2018-04-19 14:09     ` Cornelia Huck
@ 2018-04-20  0:55       ` Wanpeng Li
  0 siblings, 0 replies; 4+ messages in thread
From: Wanpeng Li @ 2018-04-20  0:55 UTC (permalink / raw)
  To: Cornelia Huck
  Cc: Wanpeng Li, linux-kernel, kvm, Paolo Bonzini,
	Radim Krčmář,
	Tonny Lu

2018-04-19 22:09 GMT+08:00 Cornelia Huck <cohuck@redhat.com>:
> On Thu, 19 Apr 2018 13:42:55 +0000
> Wanpeng Li <wanpeng.li@hotmail.com> wrote:
>
>> On Thu, 19 Apr 2018 05:30:40 -0700
>>
>> Wanpeng Li <kernellwp@gmail.com> wrote:
>>
>> > From: Wanpeng Li <wanpengli@tencent.com>
>> >
>> > Our virtual machines make use of device assignment by configuring
>> > 12 NVMe disks for high I/O performance. Each NVMe device has 129
>> > MSI-X Table entries:
>> > Capabilities: [50] MSI-X: Enable+ Count=129 Masked-Vector table: BAR=0 offset=00002000
>> > The windows virtual machines fail to boot since they will map the number of
>> > MSI-table entries that the NVMe hardware reported to the bus to msi routing
>> > table, this will exceed the 1024. This patch extends MAX_IRQ_ROUTES to 4096,
>> > In the future this might be extended if needed.
>> >
>> > Cc: Paolo Bonzini <pbonzini@redhat.com>
>> > Cc: Radim Krčmář <rkrcmar@redhat.com>
>> > Cc: Tonny Lu <tonnylu@tencent.com>
>> > Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
>> > Signed-off-by: Tonny Lu <tonnylu@tencent.com>
>> > ---
>> >  include/linux/kvm_host.h | 2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> > index 6930c63..815ae66 100644
>> > --- a/include/linux/kvm_host.h
>> > +++ b/include/linux/kvm_host.h
>> > @@ -1050,7 +1050,7 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
>> >  #elif defined(CONFIG_ARM64)
>> >  #define KVM_MAX_IRQ_ROUTES 4096
>> >  #else
>> > -#define KVM_MAX_IRQ_ROUTES 1024
>> > +#define KVM_MAX_IRQ_ROUTES 4096
>> >  #endif
>> >
>> >  bool kvm_arch_can_set_irq_routing(struct kvm *kvm);
>>
>> So, this basically means we have 4096 everywhere, no?
>>
>> I suspect different architectures maybe extend to different limits again according to their requirements.
>
> Yes, but for now, we have the same everywhere (as you also bumped the
> limit on power and 32-bit arm, implicitly). If that's ok, we might as

I suspect they will have the same issue when configured as our
production environment, so v2 gets rid of the ifdeffey.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2018-04-20  0:55 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-19 12:30 [PATCH] KVM: X86: Extend MAX_IRQ_ROUTES to 4096 Wanpeng Li
2018-04-19 13:06 ` Cornelia Huck
     [not found]   ` <KL1PR02MB1301D3C3A62E6CED2530496180B50@KL1PR02MB1301.apcprd02.prod.outlook.com>
2018-04-19 14:09     ` Cornelia Huck
2018-04-20  0:55       ` Wanpeng Li

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).