LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@suse.de>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: [PATCH 4.9 22/43] x86/fpu: Make init_fpstate correct with optimized XSAVE
Date: Tue, 24 Aug 2021 13:05:53 -0400	[thread overview]
Message-ID: <20210824170614.710813-23-sashal@kernel.org> (raw)
In-Reply-To: <20210824170614.710813-1-sashal@kernel.org>

From: Thomas Gleixner <tglx@linutronix.de>

commit f9dfb5e390fab2df9f7944bb91e7705aba14cd26 upstream.

The XSAVE init code initializes all enabled and supported components with
XRSTOR(S) to init state. Then it XSAVEs the state of the components back
into init_fpstate which is used in several places to fill in the init state
of components.

This works correctly with XSAVE, but not with XSAVEOPT and XSAVES because
those use the init optimization and skip writing state of components which
are in init state. So init_fpstate.xsave still contains all zeroes after
this operation.

There are two ways to solve that:

   1) Use XSAVE unconditionally, but that requires to reshuffle the buffer when
      XSAVES is enabled because XSAVES uses compacted format.

   2) Save the components which are known to have a non-zero init state by other
      means.

Looking deeper, #2 is the right thing to do because all components the
kernel supports have all-zeroes init state except the legacy features (FP,
SSE). Those cannot be hard coded because the states are not identical on all
CPUs, but they can be saved with FXSAVE which avoids all conditionals.

Use FXSAVE to save the legacy FP/SSE components in init_fpstate along with
a BUILD_BUG_ON() which reminds developers to validate that a newly added
component has all zeroes init state. As a bonus remove the now unused
copy_xregs_to_kernel_booting() crutch.

The XSAVE and reshuffle method can still be implemented in the unlikely
case that components are added which have a non-zero init state and no
other means to save them. For now, FXSAVE is just simple and good enough.

  [ bp: Fix a typo or two in the text. ]

Fixes: 6bad06b76892 ("x86, xsave: Use xsaveopt in context-switch path when supported")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20210618143444.587311343@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 arch/x86/include/asm/fpu/internal.h | 30 ++++++-----------------
 arch/x86/kernel/fpu/xstate.c        | 38 ++++++++++++++++++++++++++---
 2 files changed, 43 insertions(+), 25 deletions(-)

diff --git a/arch/x86/include/asm/fpu/internal.h b/arch/x86/include/asm/fpu/internal.h
index ebda4718eb8f..793c04cba0de 100644
--- a/arch/x86/include/asm/fpu/internal.h
+++ b/arch/x86/include/asm/fpu/internal.h
@@ -221,6 +221,14 @@ static inline void copy_fxregs_to_kernel(struct fpu *fpu)
 	}
 }
 
+static inline void fxsave(struct fxregs_state *fx)
+{
+	if (IS_ENABLED(CONFIG_X86_32))
+		asm volatile( "fxsave %[fx]" : [fx] "=m" (*fx));
+	else
+		asm volatile("fxsaveq %[fx]" : [fx] "=m" (*fx));
+}
+
 /* These macros all use (%edi)/(%rdi) as the single memory argument. */
 #define XSAVE		".byte " REX_PREFIX "0x0f,0xae,0x27"
 #define XSAVEOPT	".byte " REX_PREFIX "0x0f,0xae,0x37"
@@ -294,28 +302,6 @@ static inline void copy_fxregs_to_kernel(struct fpu *fpu)
 		     : "D" (st), "m" (*st), "a" (lmask), "d" (hmask)	\
 		     : "memory")
 
-/*
- * This function is called only during boot time when x86 caps are not set
- * up and alternative can not be used yet.
- */
-static inline void copy_xregs_to_kernel_booting(struct xregs_state *xstate)
-{
-	u64 mask = -1;
-	u32 lmask = mask;
-	u32 hmask = mask >> 32;
-	int err;
-
-	WARN_ON(system_state != SYSTEM_BOOTING);
-
-	if (static_cpu_has(X86_FEATURE_XSAVES))
-		XSTATE_OP(XSAVES, xstate, lmask, hmask, err);
-	else
-		XSTATE_OP(XSAVE, xstate, lmask, hmask, err);
-
-	/* We should never fault when copying to a kernel buffer: */
-	WARN_ON_FPU(err);
-}
-
 /*
  * This function is called only during boot time when x86 caps are not set
  * up and alternative can not be used yet.
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index dbd396c91348..02ad98ec5149 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -407,6 +407,24 @@ static void __init print_xstate_offset_size(void)
 	}
 }
 
+/*
+ * All supported features have either init state all zeros or are
+ * handled in setup_init_fpu() individually. This is an explicit
+ * feature list and does not use XFEATURE_MASK*SUPPORTED to catch
+ * newly added supported features at build time and make people
+ * actually look at the init state for the new feature.
+ */
+#define XFEATURES_INIT_FPSTATE_HANDLED		\
+	(XFEATURE_MASK_FP |			\
+	 XFEATURE_MASK_SSE |			\
+	 XFEATURE_MASK_YMM |			\
+	 XFEATURE_MASK_OPMASK |			\
+	 XFEATURE_MASK_ZMM_Hi256 |		\
+	 XFEATURE_MASK_Hi16_ZMM	 |		\
+	 XFEATURE_MASK_PKRU |			\
+	 XFEATURE_MASK_BNDREGS |		\
+	 XFEATURE_MASK_BNDCSR)
+
 /*
  * setup the xstate image representing the init state
  */
@@ -414,6 +432,8 @@ static void __init setup_init_fpu_buf(void)
 {
 	static int on_boot_cpu __initdata = 1;
 
+	BUILD_BUG_ON(XCNTXT_MASK != XFEATURES_INIT_FPSTATE_HANDLED);
+
 	WARN_ON_FPU(!on_boot_cpu);
 	on_boot_cpu = 0;
 
@@ -432,10 +452,22 @@ static void __init setup_init_fpu_buf(void)
 	copy_kernel_to_xregs_booting(&init_fpstate.xsave);
 
 	/*
-	 * Dump the init state again. This is to identify the init state
-	 * of any feature which is not represented by all zero's.
+	 * All components are now in init state. Read the state back so
+	 * that init_fpstate contains all non-zero init state. This only
+	 * works with XSAVE, but not with XSAVEOPT and XSAVES because
+	 * those use the init optimization which skips writing data for
+	 * components in init state.
+	 *
+	 * XSAVE could be used, but that would require to reshuffle the
+	 * data when XSAVES is available because XSAVES uses xstate
+	 * compaction. But doing so is a pointless exercise because most
+	 * components have an all zeros init state except for the legacy
+	 * ones (FP and SSE). Those can be saved with FXSAVE into the
+	 * legacy area. Adding new features requires to ensure that init
+	 * state is all zeroes or if not to add the necessary handling
+	 * here.
 	 */
-	copy_xregs_to_kernel_booting(&init_fpstate.xsave);
+	fxsave(&init_fpstate.fxsave);
 }
 
 static int xfeature_uncompacted_offset(int xfeature_nr)
-- 
2.30.2


  parent reply	other threads:[~2021-08-24 17:37 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-24 17:05 [PATCH 4.9 00/43] 4.9.281-rc1 review Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 01/43] iio: adc: Fix incorrect exit of for-loop Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 02/43] ASoC: intel: atom: Fix reference to PCM buffer address Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 03/43] i2c: dev: zero out array used for i2c reads from userspace Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 04/43] ACPI: NFIT: Fix support for virtual SPA ranges Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 05/43] ppp: Fix generating ifname when empty IFLA_IFNAME is specified Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 06/43] net: Fix memory leak in ieee802154_raw_deliver Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 07/43] net: bridge: fix memleak in br_add_if() Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 08/43] tcp_bbr: fix u32 wrap bug in round logic if bbr_init() called after 2B packets Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 09/43] xen/events: Fix race in set_evtchn_to_irq Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 10/43] x86/tools: Fix objdump version check again Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 11/43] PCI/MSI: Enable and mask MSI-X early Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 12/43] PCI/MSI: Do not set invalid bits in MSI mask Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 13/43] PCI/MSI: Correct misleading comments Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 14/43] PCI/MSI: Use msi_mask_irq() in pci_msi_shutdown() Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 15/43] PCI/MSI: Protect msi_desc::masked for multi-MSI Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 16/43] PCI/MSI: Mask all unused MSI-X entries Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 17/43] PCI/MSI: Enforce that MSI-X table entry is masked for update Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 18/43] PCI/MSI: Enforce MSI[X] entry updates to be visible Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 19/43] vmlinux.lds.h: Handle clang's module.{c,d}tor sections Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 20/43] mac80211: drop data frames without key on encrypted links Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 21/43] KVM: nSVM: avoid picking up unsupported bits from L2 in int_ctl (CVE-2021-3653) Sasha Levin
2021-08-24 17:05 ` Sasha Levin [this message]
2021-08-24 17:05 ` [PATCH 4.9 23/43] dmaengine: usb-dmac: Fix PM reference leak in usb_dmac_probe() Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 24/43] ARM: dts: am43x-epos-evm: Reduce i2c0 bus speed for tps65218 Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 25/43] dmaengine: of-dma: router_xlate to return -EPROBE_DEFER if controller is not yet available Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 26/43] scsi: megaraid_mm: Fix end of loop tests for list_for_each_entry() Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 27/43] scsi: scsi_dh_rdac: Avoid crash during rdac_bus_attach() Sasha Levin
2021-08-24 17:05 ` [PATCH 4.9 28/43] scsi: core: Avoid printing an error if target_alloc() returns -ENXIO Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 29/43] ARM: dts: nomadik: Fix up interrupt controller node names Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 30/43] Bluetooth: hidp: use correct wait queue when removing ctrl_wait Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 31/43] dccp: add do-while-0 stubs for dccp_pr_debug macros Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 32/43] vhost: Fix the calculation in vhost_overflow() Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 33/43] net: 6pack: fix slab-out-of-bounds in decode_data Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 34/43] net: qlcnic: add missed unlock in qlcnic_83xx_flash_read32 Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 35/43] mmc: dw_mmc: call the dw_mci_prep_stop_abort() by default Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 36/43] mmc: dw_mmc: Fix hang on data CRC error Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 37/43] ALSA: hda - fix the 'Capture Switch' value change notifications Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 38/43] ipack: tpci200: fix many double free issues in tpci200_pci_probe Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 39/43] btrfs: prevent rename2 from exchanging a subvol with a directory from different parents Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 40/43] ASoC: intel: atom: Fix breakage for PCM buffer address setup Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 41/43] locks: print a warning when mount fails due to lack of "mand" support Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 42/43] fs: warn about impending deprecation of mandatory locks Sasha Levin
2021-08-24 17:06 ` [PATCH 4.9 43/43] Linux 4.9.281-rc1 Sasha Levin
2021-08-25 20:26 ` [PATCH 4.9 00/43] 4.9.281-rc1 review Guenter Roeck
2021-08-25 21:17 ` Daniel Díaz
2021-08-25 22:37 ` Shuah Khan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210824170614.710813-23-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=bp@suse.de \
    --cc=gregkh@linuxfoundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --subject='Re: [PATCH 4.9 22/43] x86/fpu: Make init_fpstate correct with optimized XSAVE' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).