Re: [PATCH 2/6] PCI/MSI: add hooks to populate the msi_domain field

[Jiang Liu <jiang.liu@linux.intel.com>]

On 2014/12/9 22:03, Marc Zyngier wrote:
> Hi Gerry,
> 
> On 09/12/14 12:47, Jiang Liu wrote:
>> On 2014/12/9 20:12, Marc Zyngier wrote:
>>> Yijing,
>>>
>>> On 09/12/14 11:57, Yijing Wang wrote:
>>>>>>> +void __weak pcibios_set_phb_msi_domain(struct pci_bus *bus)
>>>>>>> +{
>>>>>>> +}
>>>>>>> +
>>>>>>> +static void pci_set_bus_msi_domain(struct pci_bus *bus)
>>>>>>> +{
>>>>>>> +	struct pci_dev *bridge = bus->self;
>>>>>>> +
>>>>>>> +	if (!bridge)
>>>>>>> +		pcibios_set_phb_msi_domain(bus);
>>>>>>> +	else
>>>>>>> +		dev_set_msi_domain(&bus->dev, dev_get_msi_domain(&bridge->dev));
>>>>>>> +}
>>>>>>
>>>>>>
>>>>>> Hi Marc, we can not assume pci devices under same phb share the same msi irq domain,
>>>>>> now in x86, pci devices under the same phb may associate different msi irq domain.
>>>>
>>>> Hi Marc,
>>>>
>>>>>
>>>>> Well, this is not supposed to be a perfect solution yet, but instead a
>>>>> basis for discussion. What I'd like to find out is:
>>>>>
>>>>> - What is the minimum granularity for associating a device with its MSI
>>>>> domain in existing platforms?
>>>>
>>>> PCI device, after Gerry's msi irq domain patchset which now in linux-next,
>>>> in x86, we will find msi irq domain by pci_dev.
>>>
>>> Are you *really* associating the MSI domain on a per pci-device basis?
>>> That is, you have devices on the same PCI bus talking to different MSI hw?
>> Hi Marc,
>> 	This is a little wild:(
>> 	On x86 platform with Intel VT-d(not the case for AMD-v),
>> interrupt remapping is tight to DMA remapping (IOMMU) unit.
>> For most common cases, IOMMU unit manages PCI bus and its sub-hierarchy.
>> But it may also manage a specific PCI device. This is typically used to
>> provide QoS for audio device by using dedicated IOMMU unit to avoid
>> resource contention on DMA remapping tables. BIOS uses ACPI table to
>> report PCI bus/device to IOMMU unit mapping relationship. (To be honest,
>> I have no really experience with such a hardware platform yet, just for
>> theoretical analysis)
>> 	On the other hand, we now support hierarchy irqdomain. So to
>> support per-PCI IOMMU unit case, we need maintain irqdomain at PCI
>> device level.
>> 	This piece of code from your [4/6] is flexible enough, which
>> retrieves msi_domain from PCI device, then fallback to PCI bus,
>> then fallback to platform specific method.
>> 	domain = dev_get_msi_domain(&dev->dev);
>> 	if (!domain && dev->bus->msi)
>>  		domain = dev->bus->msi->domain;
>>  	if (!domain)
>>  		domain = arch_get_pci_msi_domain(dev);
> 
> OK. But what I'd really like to see is a way to setup the
> device<->domain binding as early as possible, without having to use more
> conditional code in pci_msi_get_domain.
> 
> IOW, can we do something similar to what pci_set_bus_msi_domain and
> pci_set_msi_domain do in this patch?
Hi Marc,
	I have checked x86 code, we could set pci_dev->msi_domain
when creating PCI devices, just need to find some hook points
into PCI core next step. If arch code doesn't set pci_dev->msi_domain,
PCI MSI core may provide a default way to set pci_dev->msi_domain.
This may make the implementation simpler, I guess:)
Thanks!
Gerry

> 
> Thanks,
> 
> 	M.
>