[PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Robert Hancock]

This fixes some problems with ATAPI devices on nForce4 controllers in ADMA mode
on systems with memory located above 4GB. We need to delay setting the 64-bit
DMA mask until the PRD table and padding buffer are allocated so that they don't
get allocated above 4GB and break legacy mode (which is needed for ATAPI
devices).

Signed-off-by: Robert Hancock <hancockr@shaw.ca>

--- linux-2.6.24-rc1-git10edit/drivers/ata/sata_nv.c.before	2007-11-13 19:04:18.000000000 -0600
+++ linux-2.6.24-rc1-git10edit/drivers/ata/sata_nv.c	2007-11-13 19:02:34.000000000 -0600
@@ -247,6 +247,7 @@
 	void __iomem		*ctl_block;
 	void __iomem		*gen_block;
 	void __iomem		*notifier_clear_block;
+	u64			adma_dma_mask;
 	u8			flags;
 	int			last_issue_ncq;
 };
@@ -748,7 +749,7 @@
 		adma_enable = 0;
 		nv_adma_register_mode(ap);
 	} else {
-		bounce_limit = *ap->dev->dma_mask;
+		bounce_limit = pp->adma_dma_mask;
 		segment_boundary = NV_ADMA_DMA_BOUNDARY;
 		sg_tablesize = NV_ADMA_SGTBL_TOTAL_LEN;
 		adma_enable = 1;
@@ -763,6 +764,11 @@
 		config_mask = NV_MCP_SATA_CFG_20_PORT0_EN |
 			      NV_MCP_SATA_CFG_20_PORT0_PWB_EN;
 
+	/* Set appropriate DMA mask. */
+	rc = pci_set_dma_mask(pdev, bounce_limit);
+	if (rc)
+		return rc;
+
 	if (adma_enable) {
 		new_reg = current_reg | config_mask;
 		pp->flags &= ~NV_ADMA_ATAPI_SETUP_COMPLETE;
@@ -1134,6 +1140,7 @@
 	void *mem;
 	dma_addr_t mem_dma;
 	void __iomem *mmio;
+	struct pci_dev *pdev = to_pci_dev(dev);
 	u16 tmp;
 
 	VPRINTK("ENTER\n");
@@ -1153,6 +1160,14 @@
 	pp->notifier_clear_block = pp->gen_block +
 	       NV_ADMA_NOTIFIER_CLEAR + (4 * ap->port_no);
 
+	/* Now that the legacy PRD and padding buffer are allocated we can
+	   safely raise the DMA mask to allocate the CPB/APRD table. */
+	pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+	pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+	/* Store the mask that was actually used so we can restore it later
+	   if needed */
+	pp->adma_dma_mask = *dev->dma_mask;
+
 	mem = dmam_alloc_coherent(dev, NV_ADMA_PORT_PRIV_DMA_SZ,
 				  &mem_dma, GFP_KERNEL);
 	if (!mem)
@@ -2408,12 +2423,6 @@
 	hpriv->type = type;
 	host->private_data = hpriv;
 
-	/* set 64bit dma masks, may fail */
-	if (type == ADMA) {
-		if (pci_set_dma_mask(pdev, DMA_64BIT_MASK) == 0)
-			pci_set_consistent_dma_mask(pdev, DMA_64BIT_MASK);
-	}
-
 	/* request and iomap NV_MMIO_BAR */
 	rc = pcim_iomap_regions(pdev, 1 << NV_MMIO_BAR, DRV_NAME);
 	if (rc)

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Alan Cox]

On Tue, 13 Nov 2007 22:15:51 -0600
Robert Hancock <hancockr@shaw.ca> wrote:

> This fixes some problems with ATAPI devices on nForce4 controllers in ADMA mode
> on systems with memory located above 4GB. We need to delay setting the 64-bit
> DMA mask until the PRD table and padding buffer are allocated so that they don't
> get allocated above 4GB and break legacy mode (which is needed for ATAPI
> devices).
> 
> Signed-off-by: Robert Hancock <hancockr@shaw.ca>

Acked-by: Alan Cox <alan@redhat.com>

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Tejun Heo]

Robert Hancock wrote:
> This fixes some problems with ATAPI devices on nForce4 controllers in ADMA mode
> on systems with memory located above 4GB. We need to delay setting the 64-bit
> DMA mask until the PRD table and padding buffer are allocated so that they don't
> get allocated above 4GB and break legacy mode (which is needed for ATAPI
> devices).
> 
> Signed-off-by: Robert Hancock <hancockr@shaw.ca>

applied to #tj-upstream-fixes.

-- 
tejun

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Robert Hancock]

Tejun Heo wrote:
> Robert Hancock wrote:
>> This fixes some problems with ATAPI devices on nForce4 controllers in ADMA mode
>> on systems with memory located above 4GB. We need to delay setting the 64-bit
>> DMA mask until the PRD table and padding buffer are allocated so that they don't
>> get allocated above 4GB and break legacy mode (which is needed for ATAPI
>> devices).
>>
>> Signed-off-by: Robert Hancock <hancockr@shaw.ca>
> 
> applied to #tj-upstream-fixes.
> 

I have a report that these patches crashed but the previous patch worked:

https://bugzilla.redhat.com/show_bug.cgi?id=351451

So there may still be a problem here.

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Tejun Heo]

Robert Hancock wrote:
> Tejun Heo wrote:
>> Robert Hancock wrote:
>>> This fixes some problems with ATAPI devices on nForce4 controllers in
>>> ADMA mode
>>> on systems with memory located above 4GB. We need to delay setting
>>> the 64-bit
>>> DMA mask until the PRD table and padding buffer are allocated so that
>>> they don't
>>> get allocated above 4GB and break legacy mode (which is needed for ATAPI
>>> devices).
>>>
>>> Signed-off-by: Robert Hancock <hancockr@shaw.ca>
>>
>> applied to #tj-upstream-fixes.
>>
> 
> I have a report that these patches crashed but the previous patch worked:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=351451
> 
> So there may still be a problem here.

Hmmm... The change seemed safe to me.  Anyways, dropping the patch for
now.  Please re-send later.  Also, please format patch description such
that it fits in 80c.

Thanks.

-- 
tejun

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Tejun Heo]

Robert Hancock wrote:
> Tejun Heo wrote:
>> Robert Hancock wrote:
>>> This fixes some problems with ATAPI devices on nForce4 controllers in
>>> ADMA mode
>>> on systems with memory located above 4GB. We need to delay setting
>>> the 64-bit
>>> DMA mask until the PRD table and padding buffer are allocated so that
>>> they don't
>>> get allocated above 4GB and break legacy mode (which is needed for ATAPI
>>> devices).
>>>
>>> Signed-off-by: Robert Hancock <hancockr@shaw.ca>
>>
>> applied to #tj-upstream-fixes.
>>
> 
> I have a report that these patches crashed but the previous patch worked:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=351451
> 
> So there may still be a problem here.

Any progress?

-- 
tejun

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Robert Hancock]

Tejun Heo wrote:
> Robert Hancock wrote:
>> Tejun Heo wrote:
>>> Robert Hancock wrote:
>>>> This fixes some problems with ATAPI devices on nForce4 controllers in
>>>> ADMA mode
>>>> on systems with memory located above 4GB. We need to delay setting
>>>> the 64-bit
>>>> DMA mask until the PRD table and padding buffer are allocated so that
>>>> they don't
>>>> get allocated above 4GB and break legacy mode (which is needed for ATAPI
>>>> devices).
>>>>
>>>> Signed-off-by: Robert Hancock <hancockr@shaw.ca>
>>> applied to #tj-upstream-fixes.
>>>
>> I have a report that these patches crashed but the previous patch worked:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=351451
>>
>> So there may still be a problem here.
> 
> Any progress?

It looks like the problem is that even though we set the DMA mask after 
we allocate the PRD and pad buffers, when the other port is set up, the 
DMA mask is already over 64-bit and so it allocates its buffers over 4GB 
and fails. I think we just need to explicitly set to 32-bit first, 
getting the reporter to try that one now.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: [PATCH 2/2] sata_nv: fix ATAPI issues with memory over 4GB (v3)

[Tejun Heo]

Robert Hancock wrote:
> It looks like the problem is that even though we set the DMA mask after
> we allocate the PRD and pad buffers, when the other port is set up, the
> DMA mask is already over 64-bit and so it allocates its buffers over 4GB
> and fails. I think we just need to explicitly set to 32-bit first,
> getting the reporter to try that one now.

Ah.. right.  That makes sense.  Thanks.

-- 
tejun