From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Cyrus-Session-Id: sloti22d1t05-1732904-1525111268-2-6819733371026355373 X-Sieve: CMU Sieve 3.0 X-Spam-known-sender: no X-Spam-score: 0.0 X-Spam-hits: BAYES_00 -1.9, HEADER_FROM_DIFFERENT_DOMAINS 0.25, MAILING_LIST_MULTI -1, ME_NOAUTH 0.01, RCVD_IN_DNSWL_HI -5, LANGUAGES en, BAYES_USED global, SA_VERSION 3.4.0 X-Spam-source: IP='209.132.180.67', Host='vger.kernel.org', Country='US', FromHeader='com', MailFrom='org' X-Spam-charsets: X-Resolved-to: greg@kroah.com X-Delivered-to: greg@kroah.com X-Mail-from: stable-owner@vger.kernel.org ARC-Seal: i=1; a=rsa-sha256; cv=none; d=messagingengine.com; s=fm2; t= 1525111267; b=NKpFupFKzKRJjOerploFyjIACn7J9CHtwBSpQvSrg2N6wXxr6x qEvKUJc1CScNMtbiRUTcwAFx/2VBVZKQg+Cxxo8QeSiniM8arD6/sR3qEGFgIMvZ Z9iPWehx9CbXpQhLq0v5HS/woZg8+mWakLmg9ao1i2PpMiUwt+o32bwZcMHgnVMS 2dGgujo7WOa8gvOvQz5Q/W+VtgplrO9VbVdow62cDCmrZ+OmZ7HKayZ0YwMl9Wtf BFWPodkryduScLsiaAXN7GCuJsAhIRU1wAx8hXkS72ixOktAgRjWQSPh+Y2f1UTR 0C50zHN4Q7m2v9xf3lkULGCl49HGxn5YgcDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=from:to:cc:subject:date:message-id :in-reply-to:references:sender:list-id; s=fm2; t=1525111267; bh= eFRWTzR9zuTtzStOfD+nrQZMjhxcfV9CVR28R5Wduik=; b=bOrdzy7k7pub7Q+N x5jI9yuctnB0MZaHDQTQOjYVVW2uj3b1aZ0CyG9KDnxP5TeLsxDcdlkbuEPRTGW2 1amCoLNfEsCODQ5XqUDpc/4hvuhxwrVIdUpSHr8OctdMWQAyKNPr64RYCSbb1aId wQ8zjtovPJF2QUZNxY+BrIhhK5mrTLhYVqwLHZhMS7l/f6WahYlu9dqkzvCnDWEX XtV62mNM3o290Amysf5EuxRc7Zhcwb8X4nCi5AxmrtCuKCNELHMyURd67JVjDH4a idMMqWeSKjLeIPJpqTC+73JGMjB1lO0/d+zmx7+x0R9/1OzVOttLPXIzTY+VrFJr lZfu9g== ARC-Authentication-Results: i=1; mx3.messagingengine.com; arc=none (no signatures found); dkim=none (no signatures found); dmarc=none (p=none,has-list-id=yes,d=none) header.from=hpe.com; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=stable-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-cm=none score=0; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=hpe.com header.result=pass header_is_org_domain=yes; x-vs=clean score=-100 state=0 Authentication-Results: mx3.messagingengine.com; arc=none (no signatures found); dkim=none (no signatures found); dmarc=none (p=none,has-list-id=yes,d=none) header.from=hpe.com; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=stable-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-cm=none score=0; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=hpe.com header.result=pass header_is_org_domain=yes; x-vs=clean score=-100 state=0 X-ME-VSCategory: clean X-CM-Envelope: MS4wfAWwGFQx278zuSnOHcqRUqr5xUrctF+EZVROMwlvTIZ9YT5y0Dg0h22Vm39R/ddkXBkl8FYggO/nVILiZM9pCDts4Ljuh1puP+QURNj4U8BbpMtklLYJ fGOhB3tIXxQMLbQGdquPG8q5dnFjbQyRX1UlQdVeXHZ2Vyrz6W5wV0Tq7H3CwMJZwWm2s1aEOPiOH8uIRriGQCAwul/yU1ITtowSEobKpt0dOtvsxXq4kso2 X-CM-Analysis: v=2.3 cv=Tq3Iegfh c=1 sm=1 tr=0 a=UK1r566ZdBxH71SXbqIOeA==:117 a=UK1r566ZdBxH71SXbqIOeA==:17 a=Kd1tUaAdevIA:10 a=MvuuwTCpAAAA:8 a=Z4Rwk6OoAAAA:8 a=iox4zFpeAAAA:8 a=20KFwNOVAAAA:8 a=oGMlB6cnAAAA:8 a=tA7aZXjiAAAA:8 a=VwQbUJbxAAAA:8 a=ICxiNaJX4HUcj9YNticA:9 a=lN0riLXZGKYI8htB:21 a=tBvKmnuPread9wYJ:21 a=dVHiktpip_riXrfdqayU:22 a=HkZW87K1Qel5hWWM3VKY:22 a=WzC6qhA0u3u7Ye7llzcV:22 a=NdAtdrkLVvyUPsUoGJp4:22 a=kIIFJ0VLUOy1gFZzwZHL:22 a=AjGcO6oz07-iQ99wixmX:22 X-ME-CMScore: 0 X-ME-CMCategory: none Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754506AbeD3SAy (ORCPT ); Mon, 30 Apr 2018 14:00:54 -0400 Received: from g9t5009.houston.hpe.com ([15.241.48.73]:56060 "EHLO g9t5009.houston.hpe.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754104AbeD3SA0 (ORCPT ); Mon, 30 Apr 2018 14:00:26 -0400 From: Toshi Kani To: mhocko@suse.com, akpm@linux-foundation.org, tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com Cc: cpandya@codeaurora.org, linux-mm@kvack.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Toshi Kani , Joerg Roedel , stable@vger.kernel.org Subject: [PATCH 2/3] x86/mm: add TLB purge to free pmd/pte page interfaces Date: Mon, 30 Apr 2018 11:59:24 -0600 Message-Id: <20180430175925.2657-3-toshi.kani@hpe.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180430175925.2657-1-toshi.kani@hpe.com> References: <20180430175925.2657-1-toshi.kani@hpe.com> Sender: stable-owner@vger.kernel.org X-Mailing-List: stable@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-Mailing-List: linux-kernel@vger.kernel.org List-ID: ioremap() calls pud_free_pmd_page() / pmd_free_pte_page() when it creates a pud / pmd map. The following preconditions are met at their entry. - All pte entries for a target pud/pmd address range have been cleared. - System-wide TLB purges have been peformed for a target pud/pmd address range. The preconditions assure that there is no stale TLB entry for the range. Speculation may not cache TLB entries since it requires all levels of page entries, including ptes, to have P & A-bits set for an associated address. However, speculation may cache pud/pmd entries (paging-structure caches) when they have P-bit set. Add a system-wide TLB purge (INVLPG) to a single page after clearing pud/pmd entry's P-bit. SDM 4.10.4.1, Operation that Invalidate TLBs and Paging-Structure Caches, states that: INVLPG invalidates all paging-structure caches associated with the current PCID regardless of the liner addresses to which they correspond. Fixes: 28ee90fe6048 ("x86/mm: implement free pmd/pte page interfaces") Signed-off-by: Toshi Kani Cc: Andrew Morton Cc: Michal Hocko Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Joerg Roedel Cc: --- arch/x86/mm/pgtable.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 37e3cbac59b9..816fd41ee854 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -720,24 +720,40 @@ int pmd_clear_huge(pmd_t *pmd) * @pud: Pointer to a PUD. * @addr: Virtual address associated with pud. * - * Context: The pud range has been unmaped and TLB purged. + * Context: The pud range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ int pud_free_pmd_page(pud_t *pud, unsigned long addr) { - pmd_t *pmd; + pmd_t *pmd, *pmd_sv; + pte_t *pte; int i; if (pud_none(*pud)) return 1; pmd = (pmd_t *)pud_page_vaddr(*pud); + pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); - for (i = 0; i < PTRS_PER_PMD; i++) - if (!pmd_free_pte_page(&pmd[i], addr + (i * PMD_SIZE))) - return 0; + for (i = 0; i < PTRS_PER_PMD; i++) { + pmd_sv[i] = pmd[i]; + if (!pmd_none(pmd[i])) + pmd_clear(&pmd[i]); + } pud_clear(pud); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (!pmd_none(pmd_sv[i])) { + pte = (pte_t *)pmd_page_vaddr(pmd_sv[i]); + free_page((unsigned long)pte); + } + } + + free_page((unsigned long)pmd_sv); free_page((unsigned long)pmd); return 1; @@ -748,7 +764,7 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) * @pmd: Pointer to a PMD. * @addr: Virtual address associated with pmd. * - * Context: The pmd range has been unmaped and TLB purged. + * Context: The pmd range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) @@ -760,6 +776,10 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + free_page((unsigned long)pte); return 1;