LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Atish Patra <atishp@atishpatra.org>
To: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Cc: "linux-kernel@vger.kernel.org List"
	<linux-kernel@vger.kernel.org>,
	Thadeu Lima de Souza Cascardo <cascardo@canonical.com>,
	Pierce Andjelkovic <pierceandjelkovic@gmail.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	linux-riscv <linux-riscv@lists.infradead.org>
Subject: Re: [PATCH] RISC-V: prevent sbi_send_cpumask_ipi race with ftrace
Date: Thu, 12 Aug 2021 08:53:44 -0700	[thread overview]
Message-ID: <CAOnJCULTxFmRbpw4wp7SYN8EJxFhSN5J04QDE=cfxLAB01ZOFA@mail.gmail.com> (raw)
In-Reply-To: <20210812123433.27871-1-dimitri.ledkov@canonical.com>

On Thu, Aug 12, 2021 at 5:36 AM Dimitri John Ledkov
<dimitri.ledkov@canonical.com> wrote:
>
> From: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
>
> ftrace will patch instructions in sbi_send_cpumask_ipi, which is going to
> be used by flush_icache_range, leading to potential races and crashes like
> this:
>
> [    0.000000] ftrace: allocating 38893 entries in 152 pages
> [    0.000000] Oops - illegal instruction [#1]
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.11.0-1014-generic #14-Ubuntu
> [    0.000000] epc: ffffffe00000920e ra : ffffffe000009384 sp : ffffffe001803d30
> [    0.000000]  gp : ffffffe001a14240 tp : ffffffe00180f440 t0 : ffffffe07fe38000
> [    0.000000]  t1 : ffffffe0019cd338 t2 : 0000000000000000 s0 : ffffffe001803d70
> [    0.000000]  s1 : 0000000000000000 a0 : ffffffe0000095aa a1 : 0000000000000001
> [    0.000000]  a2 : 0000000000000002 a3 : 0000000000000000 a4 : 0000000000000000
> [    0.000000]  a5 : 0000000000000000 a6 : 0000000000000004 a7 : 0000000052464e43
> [    0.000000]  s2 : 0000000000000002 s3 : 0000000000000001 s4 : 0000000000000000
> [    0.000000]  s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000
> [    0.000000]  s8 : ffffffe001a170c0 s9 : 0000000000000001 s10: 0000000000000001
> [    0.000000]  s11: 00000000fffcc5d0 t3 : 0000000000000068 t4 : 000000000000000b
> [    0.000000]  t5 : ffffffe0019cd3e0 t6 : ffffffe001803cd8
> [    0.000000] status: 0000000200000100 badaddr: 000000000513f187 cause: 0000000000000002
> [    0.000000] ---[ end trace f67eb9af4d8d492b ]---
> [    0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
> [    0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
>
> Where ffffffe00000920e lies in the middle of sbi_send_cpumask_ipi.
>
> Reproduced on Unmatched board using Ubuntu kernels. See
> https://people.canonical.com/~xnox/lp1934548/ for sample images,
> kernels, debug symbols.
>
> BugLink: https://bugs.launchpad.net/bugs/1934548
> Reported-by: Pierce Andjelkovic <pierceandjelkovic@gmail.com>
> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
> Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
> cc: Paul Walmsley <paul.walmsley@sifive.com>
> cc: linux-riscv@lists.infradead.org
> cc: stable@vger.kernel.org
> ---
>  arch/riscv/kernel/sbi.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> index 7402a417f38e..158199865c68 100644
> --- a/arch/riscv/kernel/sbi.c
> +++ b/arch/riscv/kernel/sbi.c
> @@ -562,7 +562,7 @@ long sbi_get_mimpid(void)
>         return __sbi_base_ecall(SBI_EXT_BASE_GET_MIMPID);
>  }
>
> -static void sbi_send_cpumask_ipi(const struct cpumask *target)
> +static void notrace sbi_send_cpumask_ipi(const struct cpumask *target)
>  {
>         struct cpumask hartid_mask;
>

flush_icache_range doesn't invoke sbi_send_cpumask_ipi.
flush_icache_range->flush_icache_all->sbi_remote_fence_i->__sbi_rfence->sbi_ecall

Moreover, sbi.c should be removed from ftrace path as it is compiled
with notrace flag after the patch [1]

CFLAGS_REMOVE_sbi.o   = $(CC_FLAGS_FTRACE)

This solution was proposed as a result of earlier discussion [2] last year.

[1] https://patchwork.kernel.org/project/linux-riscv/patch/1608220905-1962-5-git-send-email-guoren@kernel.org/
[2] https://lkml.org/lkml/2020/11/3/735

The proposed fix probably hiding the root cause somehow.

Do you have the patch[1] in your kernel ?


> --
> 2.30.2
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv



-- 
Regards,
Atish

  reply	other threads:[~2021-08-12 15:53 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-12 12:34 Dimitri John Ledkov
2021-08-12 15:53 ` Atish Patra [this message]
2021-08-13 17:09   ` Dimitri John Ledkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAOnJCULTxFmRbpw4wp7SYN8EJxFhSN5J04QDE=cfxLAB01ZOFA@mail.gmail.com' \
    --to=atishp@atishpatra.org \
    --cc=cascardo@canonical.com \
    --cc=dimitri.ledkov@canonical.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=paul.walmsley@sifive.com \
    --cc=pierceandjelkovic@gmail.com \
    --subject='Re: [PATCH] RISC-V: prevent sbi_send_cpumask_ipi race with ftrace' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).