LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* Bug in VM accounting code, probably exploitable
@ 2004-05-11 19:50 Stas Sergeev
  2004-05-11 20:45 ` Hugh Dickins
  2004-05-20 19:43 ` Marcelo Tosatti
  0 siblings, 2 replies; 4+ messages in thread
From: Stas Sergeev @ 2004-05-11 19:50 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 2016 bytes --]


Hello.

As far as I know, if overcommit is
disabled, the OOM kill should never
happen.
It seems to be the bug in the linux
kernel though (any version I think,
probably also including 2.4.x), which
makes it possible to overcommit almost
arbitrary and provoke an OOM kill
afterwards.
Attached is a program that demonstrates
the bug. Don't forget to "swapoff -a"
before starting it, or touching pages
will take eternity. And the amount of
RAM must be <1Gb, or the prog will not
work:)

On 2.4.25 I get:
---
May 11 22:28:18 lin kernel: __alloc_pages: 0-order allocation failed 
(gfp=0x1d2/0)
May 11 22:28:20 lin syslogd: /var/log/debug: Cannot allocate memory
May 11 22:28:18 lin kernel: VM: killing process mozilla-bin
May 11 22:28:18 lin kernel: __alloc_pages: 0-order allocation failed 
(gfp=0x1f0/0)
May 11 22:28:20 lin kernel: __alloc_pages: 0-order allocation failed 
(gfp=0x1d2/0)
May 11 22:28:21 lin kernel: __alloc_pages: 0-order allocation failed 
(gfp=0x1d2/0)
May 11 22:28:21 lin kernel: VM: killing process X
May 11 22:28:21 lin gnome-name-server[1254]: input condition is: 0x11, 
exiting
May 11 22:29:00 lin kernel: __alloc_pages: 0-order allocation failed 
(gfp=0x1d2/0)
May 11 22:29:00 lin kernel: VM: killing process overc_test
---
As you can see, the program caused many
other processes to be killed, before it
died itself.

2.6.6 seems to kill only the test
program itself, but there is one
nasty side-effect here, that mprotect()
fails to merge VMAs because one VMA can
end up with VM_ACCOUNT flag set, and
another - without that flag. That makes
several apps of mine to malfuncate.
Also it might be possible to bypass the
security_vm_enough_memory() checks, which
may probably be exploitable in some other
ways.

I am also attaching the fix for 2.6
kernels.

Would be nice if someone can confirm
the bug and verify the fix. Perhaps
I am missing something obvious here,
at least the fact that both 2.6 and 2.4
behaves similar, confuses me. And the
fix looks also very strange, but it
seems to work.

[-- Attachment #2: overc_test.c --]
[-- Type: text/plain, Size: 964 bytes --]


#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <asm/page.h>
#include <errno.h>

#define SIZE (1024 * 1024 * 1024)

int main(int argc, char *argv[])
{
  char *addr;
  int i;

  if ((addr = mmap(0, SIZE, PROT_READ | PROT_WRITE,
      MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)) != MAP_FAILED) {
    printf("You have more than %i memory, increase SIZE\n", SIZE);
    return 1;
  }
  perror("mmap()");
  printf("OK, now trying uncommited...\n");
  if ((addr = mmap(0, SIZE, PROT_NONE,
      MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)) == MAP_FAILED) {
    perror("mmap()");
    return 1;
  }
  if (mprotect(addr, SIZE, PROT_READ | PROT_WRITE) == -1) {
    perror("mprotect()");
    return 1;
  }

  printf("Memory of size %i successfully committed!\n", SIZE);
  printf("overcommit_memory: ");
  fflush(stdout);
  system("cat /proc/sys/vm/overcommit_memory");

  for (i = 0; i < SIZE; i += PAGE_SIZE) {
    addr[i] = 1;
  }

  return 0;
}

[-- Attachment #3: vm_acct.diff --]
[-- Type: text/plain, Size: 517 bytes --]


--- linux/include/linux/mm.h	2004-04-14 09:41:36.000000000 +0400
+++ linux/include/linux/mm.h	2004-05-11 15:29:00.447968384 +0400
@@ -126,7 +126,7 @@
 #define VM_NONLINEAR	0x00800000	/* Is non-linear (remap_file_pages) */
 
 /* It makes sense to apply VM_ACCOUNT to this vma. */
-#define VM_MAYACCT(vma) (!!((vma)->vm_flags & VM_HUGETLB))
+#define VM_MAYACCT(vma) (!((vma)->vm_flags & VM_HUGETLB))
 
 #ifndef VM_STACK_DEFAULT_FLAGS		/* arch can override this */
 #define VM_STACK_DEFAULT_FLAGS VM_DATA_DEFAULT_FLAGS

[-- Attachment #4: Type: text/plain, Size: 82 bytes --]


Scanned by evaluation version of Dr.Web antivirus Daemon 
http://drweb.ru/unix/


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Bug in VM accounting code, probably exploitable
  2004-05-11 19:50 Bug in VM accounting code, probably exploitable Stas Sergeev
@ 2004-05-11 20:45 ` Hugh Dickins
  2004-05-20 19:43 ` Marcelo Tosatti
  1 sibling, 0 replies; 4+ messages in thread
From: Hugh Dickins @ 2004-05-11 20:45 UTC (permalink / raw)
  To: Stas Sergeev; +Cc: Andrew Morton, linux-kernel

On Tue, 11 May 2004, Stas Sergeev wrote:
> mprotect() fails to merge VMAs because one VMA can end up with
> VM_ACCOUNT flag set, and another without that flag. That makes
> several apps of mine to malfuncate.
> And the fix looks also very strange, but it seems to work.

Great find!  Someone has got their test the wrong way round.
Your patch is good, but since that VM_MAYACCT macro is being
used in one place only, and just hiding what it's actually about,
I'd prefer the patch below.  Against 2.6.6: Andrew, please apply.

--- 2.6.6/include/linux/mm.h	2004-05-10 03:33:36.000000000 +0100
+++ linux/include/linux/mm.h	2004-05-11 21:26:12.296881936 +0100
@@ -112,9 +112,6 @@ struct vm_area_struct {
 #define VM_HUGETLB	0x00400000	/* Huge TLB Page VM */
 #define VM_NONLINEAR	0x00800000	/* Is non-linear (remap_file_pages) */
 
-/* It makes sense to apply VM_ACCOUNT to this vma. */
-#define VM_MAYACCT(vma) (!!((vma)->vm_flags & VM_HUGETLB))
-
 #ifndef VM_STACK_DEFAULT_FLAGS		/* arch can override this */
 #define VM_STACK_DEFAULT_FLAGS VM_DATA_DEFAULT_FLAGS
 #endif
--- 2.6.6/mm/mprotect.c	2004-05-10 03:33:48.000000000 +0100
+++ linux/mm/mprotect.c	2004-05-11 21:27:17.079033552 +0100
@@ -174,8 +174,7 @@ mprotect_fixup(struct vm_area_struct *vm
 	 * a MAP_NORESERVE private mapping to writable will now reserve.
 	 */
 	if (newflags & VM_WRITE) {
-		if (!(vma->vm_flags & (VM_ACCOUNT|VM_WRITE|VM_SHARED))
-				&& VM_MAYACCT(vma)) {
+		if (!(vma->vm_flags & (VM_ACCOUNT|VM_WRITE|VM_SHARED|VM_HUGETLB))) {
 			charged = (end - start) >> PAGE_SHIFT;
 			if (security_vm_enough_memory(charged))
 				return -ENOMEM;


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Bug in VM accounting code, probably exploitable
  2004-05-11 19:50 Bug in VM accounting code, probably exploitable Stas Sergeev
  2004-05-11 20:45 ` Hugh Dickins
@ 2004-05-20 19:43 ` Marcelo Tosatti
  2004-05-22 12:46   ` Stas Sergeev
  1 sibling, 1 reply; 4+ messages in thread
From: Marcelo Tosatti @ 2004-05-20 19:43 UTC (permalink / raw)
  To: Stas Sergeev; +Cc: linux-kernel

On Tue, May 11, 2004 at 11:50:27PM +0400, Stas Sergeev wrote:
> 
> Hello.
> 
> As far as I know, if overcommit is
> disabled, the OOM kill should never
> happen.
> It seems to be the bug in the linux
> kernel though (any version I think,
> probably also including 2.4.x), which
> makes it possible to overcommit almost
> arbitrary and provoke an OOM kill
> afterwards.
> Attached is a program that demonstrates
> the bug. Don't forget to "swapoff -a"
> before starting it, or touching pages
> will take eternity. And the amount of
> RAM must be <1Gb, or the prog will not
> work:)
> 
> On 2.4.25 I get:
> ---
> May 11 22:28:18 lin kernel: __alloc_pages: 0-order allocation failed 
> (gfp=0x1d2/0)
> May 11 22:28:20 lin syslogd: /var/log/debug: Cannot allocate memory
> May 11 22:28:18 lin kernel: VM: killing process mozilla-bin
> May 11 22:28:18 lin kernel: __alloc_pages: 0-order allocation failed 
> (gfp=0x1f0/0)
> May 11 22:28:20 lin kernel: __alloc_pages: 0-order allocation failed 
> (gfp=0x1d2/0)
> May 11 22:28:21 lin kernel: __alloc_pages: 0-order allocation failed 
> (gfp=0x1d2/0)
> May 11 22:28:21 lin kernel: VM: killing process X
> May 11 22:28:21 lin gnome-name-server[1254]: input condition is: 0x11, 
> exiting
> May 11 22:29:00 lin kernel: __alloc_pages: 0-order allocation failed 
> (gfp=0x1d2/0)
> May 11 22:29:00 lin kernel: VM: killing process overc_test
> ---
> As you can see, the program caused many
> other processes to be killed, before it
> died itself.

About v2.4, can you try v2.4.26 with CONFIG_OOM_KILLER=y ? 

As for the overcommit, I think it has always been "broken"? (its always 
possible to overcommit).

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Bug in VM accounting code, probably exploitable
  2004-05-20 19:43 ` Marcelo Tosatti
@ 2004-05-22 12:46   ` Stas Sergeev
  0 siblings, 0 replies; 4+ messages in thread
From: Stas Sergeev @ 2004-05-22 12:46 UTC (permalink / raw)
  To: Marcelo Tosatti; +Cc: linux-kernel

Hi Marcelo.

Marcelo Tosatti wrote:
>>As you can see, the program caused many
>>other processes to be killed, before it
>>died itself.
> About v2.4, can you try v2.4.26 with CONFIG_OOM_KILLER=y ? 
OK, with CONFIG_OOM_KILLER=y (2.4.26) I get this:
---
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1514 
(overc_test).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1320 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1406 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1407 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1409 
(mozilla-bin).
---
Better than without OOM killer, although
still not perfect - for some reasons it also
decided to kill mozilla *after* killing the
test program. Strange.

> As for the overcommit, I think it has always been "broken"? (its always 
> possible to overcommit).
At least I received a couple of messages that
stated that my test program doesn't OOM the
RedHat Fedora 2.4 kernels - mprotect() fails
with ENOMEM. That resembles the correct
behaveour of the fixed 2.6 so I think at least
RedHat got rid of that problem somehow.
Hope that helps.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2004-05-22 12:46 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-05-11 19:50 Bug in VM accounting code, probably exploitable Stas Sergeev
2004-05-11 20:45 ` Hugh Dickins
2004-05-20 19:43 ` Marcelo Tosatti
2004-05-22 12:46   ` Stas Sergeev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).