LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Chip Coldwell <coldwell@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: Andi Kleen <ak@suse.de>, Andy Currid <Acurrid@nvidia.com>,
	Mark Langsdorf <mark.langsdorf@amd.com>,
	Joe Moriarty <Joe.Moriarty@Sun.COM>, Russ Doty <rdoty@redhat.com>,
	Lonni Friedman <lfriedman@nvidia.com>,
	Thorsten Kellermann <Thorsten.Kellermann@Sun.COM>
Subject: Re: data corruption with nvidia chipsets and IDE/SATA drives (k8 cpu errata needed?)
Date: Wed, 21 Feb 2007 12:03:27 -0500 (EST)	[thread overview]
Message-ID: <Pine.LNX.4.64.0702211144430.10805@bogart.boston.redhat.com> (raw)
In-Reply-To: <200701170829.54540.ak@suse.de>

On Wed, 17 Jan 2007, Andi Kleen wrote:

> On Wednesday 17 January 2007 07:31, Chris Wedgwood wrote:
> > On Tue, Jan 16, 2007 at 08:52:32PM +0100, Christoph Anton Mitterer wrote:
> > > I agree,... it seems drastic, but this is the only really secure
> > > solution.
> >
> > I'd like to here from Andi how he feels about this?  It seems like a
> > somewhat drastic solution in some ways given a lot of hardware doesn't
> > seem to be affected (or maybe in those cases it's just really hard to
> > hit, I don't know).
> 
> AMD is looking at the issue. Only Nvidia chipsets seem to be affected,
> although there were similar problems on VIA in the past too.
> Unless a good workaround comes around soon I'll probably default
> to iommu=soft on Nvidia.

We (Sun, AMD, Nvidia and Red Hat) have been testing a patch that seems
to solve the problem.  AMD and Nvidia analyzed an HDT trace that
seemed to indicate that CPU updates of the GATT were still in cache
when a subsequent table walk caused by a device load used a stale GATT
PTE.  That analysis inspired this patch, submitted to this list as an
RFC.  It is not obvious (to me, at least) why this problem has only
shown up on Nvidia SATA controllers.

We are continuing to investigate.

diff --git a/arch/x86_64/kernel/pci-gart.c b/arch/x86_64/kernel/pci-gart.c
index 030eb37..1dd461a 100644
--- a/arch/x86_64/kernel/pci-gart.c
+++ b/arch/x86_64/kernel/pci-gart.c
@@ -69,6 +69,8 @@ static u32 gart_unmapped_entry;
 #define AGPEXTERN
 #endif
 
+#define GATT_CLFLUSH(i) asm volatile ("clflush (%0)" :: "r" (iommu_gatt_base + (i)))
+
 /* backdoor interface to AGP driver */
 AGPEXTERN int agp_memory_reserved;
 AGPEXTERN __u32 *agp_gatt_table;
@@ -221,6 +223,7 @@ static dma_addr_t dma_map_area(struct device *dev, dma_addr_t phys_mem,
 	for (i = 0; i < npages; i++) {
 		iommu_gatt_base[iommu_page + i] = GPTE_ENCODE(phys_mem);
 		SET_LEAK(iommu_page + i);
+		GATT_CLFLUSH(iommu_page + i);
 		phys_mem += PAGE_SIZE;
 	}
 	return iommu_bus_base + iommu_page*PAGE_SIZE + (phys_mem & ~PAGE_MASK);
@@ -348,6 +351,7 @@ static int __dma_map_cont(struct scatterlist *sg, int start, int stopat,
 		while (pages--) { 
 			iommu_gatt_base[iommu_page] = GPTE_ENCODE(addr); 
 			SET_LEAK(iommu_page);
+			GATT_CLFLUSH(iommu_page);
 			addr += PAGE_SIZE;
 			iommu_page++;
 		}


Chip

-- 
Charles M. "Chip" Coldwell
Senior Software Engineer
Red Hat, Inc
978-392-2426


  parent reply	other threads:[~2007-02-21 17:03 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <fa.E9jVXDLMKzMZNCbslzUxjMhsInE@ifi.uio.no>
2007-01-03 23:41 ` data corruption with nvidia chipsets and IDE/SATA drives // memory hole mapping related bug?! Robert Hancock
2007-01-15 22:56   ` Christoph Anton Mitterer
2007-01-15 23:05     ` Christoph Anton Mitterer
2007-01-16  0:23       ` Robert Hancock
2007-01-16 13:54         ` Christoph Anton Mitterer
2007-01-16 14:26           ` Robert Hancock
2007-01-16 18:01             ` data corruption with nvidia chipsets and IDE/SATA drives (k8 cpu errata needed?) Chris Wedgwood
2007-01-16 19:52               ` Christoph Anton Mitterer
2007-01-16 20:31                 ` Chris Wedgwood
2007-01-16 21:29                   ` Andi Kleen
2007-01-17  1:17                     ` Christoph Anton Mitterer
2007-01-17 14:48                     ` Chip Coldwell
2007-01-17 19:46                       ` Chip Coldwell
2007-01-17 22:15                       ` Andi Kleen
2007-01-18 21:57                         ` Chip Coldwell
2007-01-18 22:49                           ` Andi Kleen
2007-01-18  9:29                     ` joachim
2007-01-18 14:34                       ` Christoph Anton Mitterer
2007-01-18 16:42                       ` Chris Wedgwood
2007-01-18 11:00                     ` Erik Andersen
2007-01-18 14:43                       ` Christoph Anton Mitterer
2007-01-18 16:36                       ` Chris Wedgwood
2007-01-18 23:23                       ` Andi Kleen
2007-02-21 17:03                     ` Chip Coldwell [this message]
2007-01-16 21:54                   ` Allen Martin
2007-01-17  1:12                   ` Christoph Anton Mitterer
2007-01-16 20:16               ` Arkadiusz Miskiewicz
2007-01-16 20:21                 ` Christoph Anton Mitterer
2007-01-16 20:31               ` Krzysztof Halasa
2007-01-16 20:35                 ` Chris Wedgwood
2007-03-22 12:32     ` data corruption with nvidia chipsets and IDE/SATA drives // memory hole mapping related bug?! Christoph Anton Mitterer
     [not found] <fa.Tb2r8Il/2H8l9dhmsGAAQB2WEZg@ifi.uio.no>
     [not found] ` <fa.MydhQJqKsehLS9lIc2JSZC8Q77A@ifi.uio.no>
     [not found]   ` <fa.M9SCVAz1qZ6vrR9HTrRb95KdBSY@ifi.uio.no>
     [not found]     ` <fa.5WKg4jbSxjbUXedux14VStNyV+8@ifi.uio.no>
     [not found]       ` <fa.lphwLNvMksoBFaqfqCzMG1UVhsA@ifi.uio.no>
2007-03-05  6:25         ` data corruption with nvidia chipsets and IDE/SATA drives (k8 cpu errata needed?) Robert Hancock
2007-03-12 13:06           ` Andi Kleen
2007-03-12 14:56             ` Jeff Garzik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Pine.LNX.4.64.0702211144430.10805@bogart.boston.redhat.com \
    --to=coldwell@redhat.com \
    --cc=Acurrid@nvidia.com \
    --cc=Joe.Moriarty@Sun.COM \
    --cc=Thorsten.Kellermann@Sun.COM \
    --cc=ak@suse.de \
    --cc=lfriedman@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mark.langsdorf@amd.com \
    --cc=rdoty@redhat.com \
    --subject='Re: data corruption with nvidia chipsets and IDE/SATA drives (k8 cpu errata needed?)' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).