LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* mlock() confusing 1 half of system RAM limit
@ 2018-06-01  9:53 Alex Richman
  2018-06-01 11:44 ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Alex Richman @ 2018-06-01  9:53 UTC (permalink / raw)
  To: linux-man, linux-mm, linux-kernel

Hi,

I'm seeing a 1/2 of system RAM limit when calling mlock().  mlock(2) says:
 > ENOMEM (Linux 2.4 and earlier) the calling process tried to lock more 
than half of RAM.
Which implies to me that the 1 half of system RAM limit should be only 
Linux 2.4 and earlier, but I'm running linux 4.4.127 and still seeing 
the limit.

This is on a system with 66007168 KB total memory, locking 33794621440 
bytes works, locking 33795653632 bytes does not work.

Anyone know if this is the man pages or the kernel or something unrelated?

More details here: 
https://stackoverflow.com/questions/50639709/mlock-of-large-shm-address-range-fails-with-enomem-even-with-lots-of-free-memo

Thanks,
- Alex.

-- 
Alex Richman
alex.r@gblabs.co.uk
Engineering
GB Labs
2 Orpheus House,
Calleva park,
Reading
RG7 8TA
Tel:+44 (0)118 455 5000
www.gblabs.com



The information contained in this message and any attachment may be proprietary, confidential and privileged.
If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering
this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you received this communication in error, please contact me immediately,
and delete the communication (including attachments, if applicable) from any computer or network system.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mlock() confusing 1 half of system RAM limit
  2018-06-01  9:53 mlock() confusing 1 half of system RAM limit Alex Richman
@ 2018-06-01 11:44 ` Michal Hocko
  2018-06-01 12:26   ` Alex Richman
  0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2018-06-01 11:44 UTC (permalink / raw)
  To: Alex Richman; +Cc: linux-man, linux-mm, linux-kernel

On Fri 01-06-18 10:53:57, Alex Richman wrote:
> Hi,
> 
> I'm seeing a 1/2 of system RAM limit when calling mlock().  mlock(2) says:
> > ENOMEM (Linux 2.4 and earlier) the calling process tried to lock more than
> half of RAM.
> Which implies to me that the 1 half of system RAM limit should be only Linux
> 2.4 and earlier, but I'm running linux 4.4.127 and still seeing the limit.
> 
> This is on a system with 66007168 KB total memory, locking 33794621440 bytes
> works, locking 33795653632 bytes does not work.

I assume you are doing mlock on MAP_SHARED resp. on mmap of shmem
file. You are likely to hit the shmem limit. This is half of the
available memory by default.
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mlock() confusing 1 half of system RAM limit
  2018-06-01 11:44 ` Michal Hocko
@ 2018-06-01 12:26   ` Alex Richman
  2018-06-01 13:05     ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Alex Richman @ 2018-06-01 12:26 UTC (permalink / raw)
  To: Michal Hocko; +Cc: linux-man, linux-kernel

I am using a shm MAP_SHARED, along these lines:
 > shm_fd = shm_open(handle, (O_RDWR | O_CREAT), (S_IRWXU | S_IRWXG | 
S_IRWXO));
 > ftruncate(shm_fd, channel->sled_size)
 > channel->sled = mmap(NULL, channel->sled_size, (PROT_READ | 
PROT_WRITE), (MAP_SHARED | MAP_NORESERVE), shm_fd, 0);
 > mlock(channel->sled, channel->sled_size) /* Fails with ENOMEM. */

But shmmax is unlimited on my box:
# sysctl -a | grep shm
kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096

Any ideas?

Many thanks,
- Alex.


On 01/06/18 12:44, Michal Hocko wrote:
> On Fri 01-06-18 10:53:57, Alex Richman wrote:
>> Hi,
>>
>> I'm seeing a 1/2 of system RAM limit when calling mlock().  mlock(2) says:
>>> ENOMEM (Linux 2.4 and earlier) the calling process tried to lock more than
>> half of RAM.
>> Which implies to me that the 1 half of system RAM limit should be only Linux
>> 2.4 and earlier, but I'm running linux 4.4.127 and still seeing the limit.
>>
>> This is on a system with 66007168 KB total memory, locking 33794621440 bytes
>> works, locking 33795653632 bytes does not work.
> I assume you are doing mlock on MAP_SHARED resp. on mmap of shmem
> file. You are likely to hit the shmem limit. This is half of the
> available memory by default.

-- 
Alex Richman
alex.r@gblabs.co.uk
Engineering
GB Labs
2 Orpheus House,
Calleva park,
Reading
RG7 8TA
Tel:+44 (0)118 455 5000
www.gblabs.com



The information contained in this message and any attachment may be proprietary, confidential and privileged.
If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering
this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you received this communication in error, please contact me immediately,
and delete the communication (including attachments, if applicable) from any computer or network system.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mlock() confusing 1 half of system RAM limit
  2018-06-01 12:26   ` Alex Richman
@ 2018-06-01 13:05     ` Michal Hocko
  2018-06-01 13:24       ` Alex Richman
  0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2018-06-01 13:05 UTC (permalink / raw)
  To: Alex Richman; +Cc: linux-man, linux-kernel

On Fri 01-06-18 13:26:59, Alex Richman wrote:
> I am using a shm MAP_SHARED, along these lines:
> > shm_fd = shm_open(handle, (O_RDWR | O_CREAT), (S_IRWXU | S_IRWXG |
> S_IRWXO));
> > ftruncate(shm_fd, channel->sled_size)
> > channel->sled = mmap(NULL, channel->sled_size, (PROT_READ | PROT_WRITE),
> > (MAP_SHARED | MAP_NORESERVE), shm_fd, 0);
> > mlock(channel->sled, channel->sled_size) /* Fails with ENOMEM. */
> 
> But shmmax is unlimited on my box:
> # sysctl -a | grep shm
> kernel.shm_next_id = -1
> kernel.shm_rmid_forced = 0
> kernel.shmall = 18446744073692774399
> kernel.shmmax = 18446744073692774399
> kernel.shmmni = 4096
> 
> Any ideas?

shm_open uses tmpfs/shmem under the cover and that has the internal
limit as explained above.
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mlock() confusing 1 half of system RAM limit
  2018-06-01 13:05     ` Michal Hocko
@ 2018-06-01 13:24       ` Alex Richman
  2018-06-01 13:41         ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Alex Richman @ 2018-06-01 13:24 UTC (permalink / raw)
  To: Michal Hocko; +Cc: linux-man, linux-kernel

Ah, that's it.  Increased the limit on the mount and it works fine now.

Thanks!
- Alex.

On 01/06/18 14:05, Michal Hocko wrote:
> On Fri 01-06-18 13:26:59, Alex Richman wrote:
>> I am using a shm MAP_SHARED, along these lines:
>>> shm_fd = shm_open(handle, (O_RDWR | O_CREAT), (S_IRWXU | S_IRWXG |
>> S_IRWXO));
>>> ftruncate(shm_fd, channel->sled_size)
>>> channel->sled = mmap(NULL, channel->sled_size, (PROT_READ | PROT_WRITE),
>>> (MAP_SHARED | MAP_NORESERVE), shm_fd, 0);
>>> mlock(channel->sled, channel->sled_size) /* Fails with ENOMEM. */
>> But shmmax is unlimited on my box:
>> # sysctl -a | grep shm
>> kernel.shm_next_id = -1
>> kernel.shm_rmid_forced = 0
>> kernel.shmall = 18446744073692774399
>> kernel.shmmax = 18446744073692774399
>> kernel.shmmni = 4096
>>
>> Any ideas?
> shm_open uses tmpfs/shmem under the cover and that has the internal
> limit as explained above.

-- 
Alex Richman
alex.r@gblabs.co.uk
Engineering
GB Labs
2 Orpheus House,
Calleva park,
Reading
RG7 8TA
Tel:+44 (0)118 455 5000
www.gblabs.com



The information contained in this message and any attachment may be proprietary, confidential and privileged.
If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering
this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you received this communication in error, please contact me immediately,
and delete the communication (including attachments, if applicable) from any computer or network system.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mlock() confusing 1 half of system RAM limit
  2018-06-01 13:24       ` Alex Richman
@ 2018-06-01 13:41         ` Michal Hocko
  0 siblings, 0 replies; 6+ messages in thread
From: Michal Hocko @ 2018-06-01 13:41 UTC (permalink / raw)
  To: Alex Richman; +Cc: linux-man, linux-kernel

On Fri 01-06-18 14:24:23, Alex Richman wrote:
> Ah, that's it.  Increased the limit on the mount and it works fine now.

Well, it is hidden very well. It is not until the page fault time (or
get_user_pages aka __mm_populate) when the operation fails and remasks
ENOSPC -> VM_FAULT_SIGBUS -> EFAULT -> ENOMEM on the way.
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2018-06-01 13:41 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-01  9:53 mlock() confusing 1 half of system RAM limit Alex Richman
2018-06-01 11:44 ` Michal Hocko
2018-06-01 12:26   ` Alex Richman
2018-06-01 13:05     ` Michal Hocko
2018-06-01 13:24       ` Alex Richman
2018-06-01 13:41         ` Michal Hocko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).