LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* question on lazy tlb flush
@ 2019-05-29  4:54 Zhenzhong Duan
  2019-05-29 14:17 ` Rik van Riel
  0 siblings, 1 reply; 3+ messages in thread
From: Zhenzhong Duan @ 2019-05-29  4:54 UTC (permalink / raw)
  To: LKML; +Cc: riel

Hi Maintainers,

A question raised when I learned below code.  Appreciate any help me 
understand the code.

void native_flush_tlb_others(const struct cpumask *cpumask,
                              const struct flush_tlb_info *info)

{

...

         /*
          * If no page tables were freed, we can skip sending IPIs to
          * CPUs in lazy TLB mode. They will flush the CPU themselves
          * at the next context switch.
          *
          * However, if page tables are getting freed, we need to send the
          * IPI everywhere, to prevent CPUs in lazy TLB mode from tripping
          * up on the new contents of what used to be page tables, while
          * doing a speculative memory access.
          */
         if (info->freed_tables)
                 smp_call_function_many(cpumask, flush_tlb_func_remote,
                                (void *)info, 1);
         else
                 on_each_cpu_cond_mask(tlb_is_not_lazy, 
flush_tlb_func_remote,
                                 (void *)info, 1, GFP_ATOMIC, cpumask);

}

I just didn't understand how a kernel thread could trip up on the new 
contents of what used to be page tables. I presume the freed page tables 
are user mapping?

But kernel thread only access kernel address space, is kernel space also 
freed?


thanks

Zhenzhong


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: question on lazy tlb flush
  2019-05-29  4:54 question on lazy tlb flush Zhenzhong Duan
@ 2019-05-29 14:17 ` Rik van Riel
  2019-05-30  2:09   ` Zhenzhong Duan
  0 siblings, 1 reply; 3+ messages in thread
From: Rik van Riel @ 2019-05-29 14:17 UTC (permalink / raw)
  To: Zhenzhong Duan, LKML

[-- Attachment #1: Type: text/plain, Size: 2198 bytes --]

On Wed, 2019-05-29 at 12:54 +0800, Zhenzhong Duan wrote:
> Hi Maintainers,
> 
> A question raised when I learned below code.  Appreciate any help me 
> understand the code.
> 
> void native_flush_tlb_others(const struct cpumask *cpumask,
>                               const struct flush_tlb_info *info)
> 
> {
> 
> ...
> 
>          /*
>           * If no page tables were freed, we can skip sending IPIs to
>           * CPUs in lazy TLB mode. They will flush the CPU themselves
>           * at the next context switch.
>           *
>           * However, if page tables are getting freed, we need to
> send the
>           * IPI everywhere, to prevent CPUs in lazy TLB mode from
> tripping
>           * up on the new contents of what used to be page tables,
> while
>           * doing a speculative memory access.
>           */
>          if (info->freed_tables)
>                  smp_call_function_many(cpumask,
> flush_tlb_func_remote,
>                                 (void *)info, 1);
>          else
>                  on_each_cpu_cond_mask(tlb_is_not_lazy, 
> flush_tlb_func_remote,
>                                  (void *)info, 1, GFP_ATOMIC,
> cpumask);
> 
> }
> 
> I just didn't understand how a kernel thread could trip up on the
> new 
> contents of what used to be page tables. I presume the freed page
> tables 
> are user mapping?

That is correct, and you ask a very good question.

The software does indeed not access userspace memory
addresses from kernel threads.

However, the CPU itself can do speculative loads from
userspace memory addresses, even when the software thread
is running exclusively in kernel mode.

Add to that the fact that the page table pages can get
reused for something else after they have been freed, and
that the CPU can cache intermediate translations (eg. PUD
and PMD level things get cached in the TLB), and you might
end up with a speculative load poking at a PCI register,
or something else that trips up the machine.

For that reason, when page table pages get freed, we need
to flush the TLBs of all users, inluding lazy TLB kernel
threads.

-- 
All Rights Reversed.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: question on lazy tlb flush
  2019-05-29 14:17 ` Rik van Riel
@ 2019-05-30  2:09   ` Zhenzhong Duan
  0 siblings, 0 replies; 3+ messages in thread
From: Zhenzhong Duan @ 2019-05-30  2:09 UTC (permalink / raw)
  To: Rik van Riel, LKML

On 2019/5/29 22:17, Rik van Riel wrote:
> On Wed, 2019-05-29 at 12:54 +0800, Zhenzhong Duan wrote:
>> Hi Maintainers,
>>
>> A question raised when I learned below code.  Appreciate any help me
>> understand the code.
>>
>> void native_flush_tlb_others(const struct cpumask *cpumask,
>>                                const struct flush_tlb_info *info)
>>
>> {
>>
>> ...
>>
>>           /*
>>            * If no page tables were freed, we can skip sending IPIs to
>>            * CPUs in lazy TLB mode. They will flush the CPU themselves
>>            * at the next context switch.
>>            *
>>            * However, if page tables are getting freed, we need to
>> send the
>>            * IPI everywhere, to prevent CPUs in lazy TLB mode from
>> tripping
>>            * up on the new contents of what used to be page tables,
>> while
>>            * doing a speculative memory access.
>>            */
>>           if (info->freed_tables)
>>                   smp_call_function_many(cpumask,
>> flush_tlb_func_remote,
>>                                  (void *)info, 1);
>>           else
>>                   on_each_cpu_cond_mask(tlb_is_not_lazy,
>> flush_tlb_func_remote,
>>                                   (void *)info, 1, GFP_ATOMIC,
>> cpumask);
>>
>> }
>>
>> I just didn't understand how a kernel thread could trip up on the
>> new
>> contents of what used to be page tables. I presume the freed page
>> tables
>> are user mapping?
> That is correct, and you ask a very good question.
>
> The software does indeed not access userspace memory
> addresses from kernel threads.
>
> However, the CPU itself can do speculative loads from
> userspace memory addresses, even when the software thread
> is running exclusively in kernel mode.
>
> Add to that the fact that the page table pages can get
> reused for something else after they have been freed, and
> that the CPU can cache intermediate translations (eg. PUD
> and PMD level things get cached in the TLB), and you might
> end up with a speculative load poking at a PCI register,
> or something else that trips up the machine.
>
> For that reason, when page table pages get freed, we need
> to flush the TLBs of all users, inluding lazy TLB kernel
> threads.

Understood. Thanks for your detailed explanation :)

Zhenzhong


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2019-05-30  2:10 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-29  4:54 question on lazy tlb flush Zhenzhong Duan
2019-05-29 14:17 ` Rik van Riel
2019-05-30  2:09   ` Zhenzhong Duan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).