LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* DMA descriptor alignment
@ 2007-11-12  6:15 Larry Finger
  2007-11-13 17:00 ` Andi Kleen
  0 siblings, 1 reply; 3+ messages in thread
From: Larry Finger @ 2007-11-12  6:15 UTC (permalink / raw)
  To: LKML

For those variants of BCM43xx cards that use 64-bit DMA, there is a requirement that all descriptor
rings must be aligned on an 8K boundary and must fit within an 8K page. On the x86_64 architecture
where the page size is 4K, I was getting addresses like 0x67AF000 when using dma_alloc_coherent
calls. From the description of the dma_pool_create and dma_pool_allocate routines, I thought they
would fix my problems; however, even with a dma_pool_create(name, dev, 8192, 8192, 8192) call, I'm
still getting 4K rather than 8K alignment, which results in DMA errors.

Is there a bug in these routines, am I using them incorrectly, or do I have a misunderstanding of
what it takes to get this kind of alignment?

Thanks,

Larry

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: DMA descriptor alignment
  2007-11-12  6:15 DMA descriptor alignment Larry Finger
@ 2007-11-13 17:00 ` Andi Kleen
  2007-11-14 14:41   ` Larry Finger
  0 siblings, 1 reply; 3+ messages in thread
From: Andi Kleen @ 2007-11-13 17:00 UTC (permalink / raw)
  To: Larry Finger; +Cc: LKML

Larry Finger <Larry.Finger@lwfinger.net> writes:

> For those variants of BCM43xx cards that use 64-bit DMA, there is a requirement that all descriptor
> rings must be aligned on an 8K boundary and must fit within an 8K page. On the x86_64 architecture
> where the page size is 4K, I was getting addresses like 0x67AF000 when using dma_alloc_coherent
> calls.

Normally x86-64 dma_alloc_coherent calls the buddy allocator which gives
you always naturally aligned blocks. But there is a fallback calling
into swiotlb and swiotlb uses best fit allocation which only guarantees
single page alignment. That is probably what you're seeing.

My dma zone rework would remove that fallback case and should make it work.

> From the description of the dma_pool_create and dma_pool_allocate routines, I thought they
> would fix my problems; however, even with a dma_pool_create(name, dev, 8192, 8192, 8192) call, I'm
> still getting 4K rather than 8K alignment, which results in DMA errors.

They cannot give you more alignment than the underlying allocator.

> Is there a bug in these routines, am I using them incorrectly, or do I have a misunderstanding of
> what it takes to get this kind of alignment?

My suggestion as a short term workaround would be to first allocate 
8K using dma_alloc_coherent and if that has the wrong alignment get 16K 
and align yourself. When the driver is loaded later that might be unreliable,
but near boot or with enough free memory using order 2 should usually work.
 
With the dma zone rework that could be removed later.

-Andi

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: DMA descriptor alignment
  2007-11-13 17:00 ` Andi Kleen
@ 2007-11-14 14:41   ` Larry Finger
  0 siblings, 0 replies; 3+ messages in thread
From: Larry Finger @ 2007-11-14 14:41 UTC (permalink / raw)
  To: Andi Kleen; +Cc: LKML

Andi Kleen wrote:
> Larry Finger <Larry.Finger@lwfinger.net> writes:
> 
>> For those variants of BCM43xx cards that use 64-bit DMA, there is a requirement that all descriptor
>> rings must be aligned on an 8K boundary and must fit within an 8K page. On the x86_64 architecture
>> where the page size is 4K, I was getting addresses like 0x67AF000 when using dma_alloc_coherent
>> calls.
> 
> Normally x86-64 dma_alloc_coherent calls the buddy allocator which gives
> you always naturally aligned blocks. But there is a fallback calling
> into swiotlb and swiotlb uses best fit allocation which only guarantees
> single page alignment. That is probably what you're seeing.
> 
> My dma zone rework would remove that fallback case and should make it work.
> 
>> From the description of the dma_pool_create and dma_pool_allocate routines, I thought they
>> would fix my problems; however, even with a dma_pool_create(name, dev, 8192, 8192, 8192) call, I'm
>> still getting 4K rather than 8K alignment, which results in DMA errors.
> 
> They cannot give you more alignment than the underlying allocator.
> 
>> Is there a bug in these routines, am I using them incorrectly, or do I have a misunderstanding of
>> what it takes to get this kind of alignment?
> 
> My suggestion as a short term workaround would be to first allocate 
> 8K using dma_alloc_coherent and if that has the wrong alignment get 16K 
> and align yourself. When the driver is loaded later that might be unreliable,
> but near boot or with enough free memory using order 2 should usually work.
>  
> With the dma zone rework that could be removed later.

I will be most interested in the dma zone rework; however, I now have the descriptors properly
allocated merely by asking for 8K, just as I had thought it should work. I'm not quite sure what was
wrong before, but I added a test to make certain that the alignment is OK just in case the problem
comes back.

Thanks for your response,

Larry

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2007-11-14 14:42 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-11-12  6:15 DMA descriptor alignment Larry Finger
2007-11-13 17:00 ` Andi Kleen
2007-11-14 14:41   ` Larry Finger

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).