[PATCH v3 0/4] mm: cma: add some debug information for CMA

[PATCH v3 0/4] mm: cma: add some debug information for CMA

[Stefan Strogin]

Hi all.

Here is the third version of a patch set that adds some debugging facility for
CMA.

This patch set is based on v4.0-rc1 and Sasha Levin's patch set
"mm: cma: debugfs access to CMA" v5 [1].
It is also available on git:
git://github.com/stefanstrogin/linux -b cmainfo-v3

We want an interface to see a list of currently allocated CMA buffers and some
useful information about them (like /proc/vmallocinfo but for physically
contiguous buffers allocated with CMA).

For example. We want a big (megabytes) CMA buffer to be allocated in runtime
in default CMA region. If someone already uses CMA then the big allocation
could fail. If it happened then with such an interface we could find who used
CMA at the moment of failure, who caused fragmentation and so on. Ftrace also
would be helpful here, but with ftrace we can see the whole history of
allocations and releases, whereas with this patch set we can see a snapshot of
CMA region with actual information about its allocations.

These patches add some files to debugfs when CONFIG_CMA_DEBUGFS is enbled.

/sys/kernel/debug/cma/cma-<N>/used and cma/cma-<N>/maxchunk are added to show
used size in pages and the biggest free chunk in each CMA region.

When CONFIG_CMA_BUFFER_LIST is enabled (depended on CONFIG_CMA_DEBUGFS)
/sys/kernel/debug/cma/cma-<N>/buffers is added. It contains a list of
currently allocated contiguous buffers for each CMA region (N stands for
region number). Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)

Another added option is CONFIG_CMA_ALLOC_STACKTRACE. When it's enabled then
stack traces are also saved when the allocations are made. The stack traces
are added to each entry in cma/cma-<N>/buffers (see an example [3]). This
can be used to see who and whence allocated each buffer.

Also added trace events for cma_alloc() and cma_release().

Changes from v2 (https://lkml.org/lkml/2015/2/12/647):
- Rebased on v4.0-rc1 and Sasha Levin's patch set v5 [1].
- Changed kmalloc() to vmalloc() in cma_buffer_list_read().
- Added CONFIG_CMA_BUFFER_LIST and CONFIG_CMA_ALLOC_STACKTRACE.
- Considered order_per_bit for returning page number in cma_get_used() and
  cma_get_maxchunk().
- Reordered the patches to make the one with trace events indepentent of
  others.
- Moved the prototypes of cma_buffer_list_add() and cma_buffer_list_del()
  from include/linux/cma.h to mm/cma.h.
- Various small changes.

Changes from v1 (aka "mm: cma: add /proc/cmainfo")
(https://lkml.org/lkml/2014/12/26/95):
- Rebased on v3.19 and Sasha Levin's patch set v4 [2].
- Moved debug code from cma.c to cma_debug.c.
- Moved cmainfo to debugfs and splited it by CMA region.
- Splited 'cmainfo' into 'buffers', 'used' and 'maxchunk'.
- Used CONFIG_CMA_DEBUGFS instead of CONFIG_CMA_DEBUG.
- Added trace events for cma_alloc() and cma_release().
- Don't use seq_files.
- A small change of debug output in cma_release().
- cma_buffer_list_del() now supports releasing chunks which ranges don't match
  allocations. E.g. we have buffer1: [0x0, 0x1], buffer2: [0x2, 0x3], then
  cma_buffer_list_del(cma, 0x1 /*or 0x0*/, 1 /*(or 2 or 3)*/) should work.
- Various small changes.

[1] https://lkml.org/lkml/2015/2/12/657

[2] https://lkml.org/lkml/2015/1/28/755

[3] root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
 [<c1142c4b>] cma_alloc+0x1bb/0x200
 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
 [<c1487312>] ahci_port_start+0xe2/0x210
 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
 [<c1473650>] ata_host_activate+0xd0/0x110
 [<c14881bf>] ahci_host_activate+0x3f/0x170
 [<c14854e4>] ahci_init_one+0x764/0xab0
 [<c12e415f>] pci_device_probe+0x6f/0xd0
 [<c14378a8>] driver_probe_device+0x68/0x210
 [<c1437b09>] __driver_attach+0x79/0x80
 [<c1435eef>] bus_for_each_dev+0x4f/0x80
 [<c143749e>] driver_attach+0x1e/0x20
 [<c1437197>] bus_add_driver+0x157/0x200
 [<c14381bd>] driver_register+0x5d/0xf0
<...> 
0x2f41b000 - 0x2f41c000 (4 kB), allocated by pid 1264 (NetworkManager)
 [<c1142c4b>] cma_alloc+0x1bb/0x200
 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
 [<c14c5d13>] e1000_setup_all_tx_resources+0x93/0x540
 [<c14c8021>] e1000_open+0x31/0x120
 [<c16264cf>] __dev_open+0x9f/0x130
 [<c16267ce>] __dev_change_flags+0x8e/0x150
 [<c16268b8>] dev_change_flags+0x28/0x60
 [<c1633ee0>] do_setlink+0x2a0/0x760
 [<c1634acb>] rtnl_newlink+0x60b/0x7b0
 [<c16314f4>] rtnetlink_rcv_msg+0x84/0x1f0
 [<c164b58e>] netlink_rcv_skb+0x8e/0xb0
 [<c1631461>] rtnetlink_rcv+0x21/0x30
 [<c164af7a>] netlink_unicast+0x13a/0x1d0
 [<c164b250>] netlink_sendmsg+0x240/0x3e0
 [<c160cbfd>] do_sock_sendmsg+0xbd/0xe0
<...>


Dmitry Safonov (1):
  mm: cma: add functions to get region pages counters

Stefan Strogin (3):
  mm: cma: add trace events to debug physically-contiguous memory
    allocations
  mm: cma: add number of pages to debug message in cma_release()
  mm: cma: add list of currently allocated CMA buffers to debugfs

 include/linux/cma.h        |   2 +
 include/trace/events/cma.h |  57 ++++++++++++++
 mm/Kconfig                 |  17 +++++
 mm/cma.c                   |  45 +++++++++++-
 mm/cma.h                   |  26 +++++++
 mm/cma_debug.c             | 180 +++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 326 insertions(+), 1 deletion(-)
 create mode 100644 include/trace/events/cma.h

-- 
2.1.0

[PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations

[Stefan Strogin]

Add trace events for cma_alloc() and cma_release().

Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>
---
 include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++
 mm/cma.c                   |  6 +++++
 2 files changed, 63 insertions(+)
 create mode 100644 include/trace/events/cma.h

diff --git a/include/trace/events/cma.h b/include/trace/events/cma.h
new file mode 100644
index 0000000..3fe7a56
--- /dev/null
+++ b/include/trace/events/cma.h
@@ -0,0 +1,57 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM cma
+
+#if !defined(_TRACE_CMA_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_CMA_H
+
+#include <linux/types.h>
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(cma_alloc,
+
+	TP_PROTO(struct cma *cma, unsigned long pfn, int count),
+
+	TP_ARGS(cma, pfn, count),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, pfn)
+		__field(unsigned long, count)
+	),
+
+	TP_fast_assign(
+		__entry->pfn = pfn;
+		__entry->count = count;
+	),
+
+	TP_printk("pfn=%lu page=%p count=%lu\n",
+		  __entry->pfn,
+		  pfn_to_page(__entry->pfn),
+		  __entry->count)
+);
+
+TRACE_EVENT(cma_release,
+
+	TP_PROTO(struct cma *cma, unsigned long pfn, int count),
+
+	TP_ARGS(cma, pfn, count),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, pfn)
+		__field(unsigned long, count)
+	),
+
+	TP_fast_assign(
+		__entry->pfn = pfn;
+		__entry->count = count;
+	),
+
+	TP_printk("pfn=%lu page=%p count=%lu\n",
+		  __entry->pfn,
+		  pfn_to_page(__entry->pfn),
+		  __entry->count)
+);
+
+#endif /* _TRACE_CMA_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/mm/cma.c b/mm/cma.c
index 9e3d44a..3a63c96 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -23,6 +23,7 @@
 #  define DEBUG
 #endif
 #endif
+#define CREATE_TRACE_POINTS
 
 #include <linux/memblock.h>
 #include <linux/err.h>
@@ -34,6 +35,7 @@
 #include <linux/cma.h>
 #include <linux/highmem.h>
 #include <linux/io.h>
+#include <trace/events/cma.h>
 
 #include "cma.h"
 
@@ -408,6 +410,9 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
 		start = bitmap_no + mask + 1;
 	}
 
+	if (page)
+		trace_cma_alloc(cma, pfn, count);
+
 	pr_debug("%s(): returned %p\n", __func__, page);
 	return page;
 }
@@ -440,6 +445,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)
 
 	free_contig_range(pfn, count);
 	cma_clear_bitmap(cma, pfn, count);
+	trace_cma_release(cma, pfn, count);
 
 	return true;
 }
-- 
2.1.0

Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations

[Michal Nazarewicz]

On Tue, Feb 24 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
> Add trace events for cma_alloc() and cma_release().
>
> Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>

Looks good to me but than again I don’t know much about trace points so
perhaps someone else should ack it as well.

> ---
>  include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++
>  mm/cma.c                   |  6 +++++
>  2 files changed, 63 insertions(+)
>  create mode 100644 include/trace/events/cma.h

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations

[Aneesh Kumar K.V]

Stefan Strogin <s.strogin@partner.samsung.com> writes:

> Add trace events for cma_alloc() and cma_release().
>
> Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>
> ---
>  include/trace/events/cma.h | 57 ++++++++++++++++++++++++++++++++++++++++++++++
>  mm/cma.c                   |  6 +++++
>  2 files changed, 63 insertions(+)
>  create mode 100644 include/trace/events/cma.h
>
> diff --git a/include/trace/events/cma.h b/include/trace/events/cma.h
> new file mode 100644
> index 0000000..3fe7a56
> --- /dev/null
> +++ b/include/trace/events/cma.h
> @@ -0,0 +1,57 @@
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM cma
> +
> +#if !defined(_TRACE_CMA_H) || defined(TRACE_HEADER_MULTI_READ)
> +#define _TRACE_CMA_H
> +
> +#include <linux/types.h>
> +#include <linux/tracepoint.h>
> +
> +TRACE_EVENT(cma_alloc,
> +
> +	TP_PROTO(struct cma *cma, unsigned long pfn, int count),
> +
> +	TP_ARGS(cma, pfn, count),
> +
> +	TP_STRUCT__entry(
> +		__field(unsigned long, pfn)
> +		__field(unsigned long, count)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->pfn = pfn;
> +		__entry->count = count;
> +	),
> +
> +	TP_printk("pfn=%lu page=%p count=%lu\n",
> +		  __entry->pfn,
> +		  pfn_to_page(__entry->pfn),
> +		  __entry->count)
> +);
> +
> +TRACE_EVENT(cma_release,
> +
> +	TP_PROTO(struct cma *cma, unsigned long pfn, int count),
> +
> +	TP_ARGS(cma, pfn, count),
> +
> +	TP_STRUCT__entry(
> +		__field(unsigned long, pfn)
> +		__field(unsigned long, count)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->pfn = pfn;
> +		__entry->count = count;
> +	),
> +
> +	TP_printk("pfn=%lu page=%p count=%lu\n",
> +		  __entry->pfn,
> +		  pfn_to_page(__entry->pfn),
> +		  __entry->count)
> +);
> +
> +#endif /* _TRACE_CMA_H */
> +
> +/* This part must be outside protection */
> +#include <trace/define_trace.h>
> diff --git a/mm/cma.c b/mm/cma.c
> index 9e3d44a..3a63c96 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -23,6 +23,7 @@
>  #  define DEBUG
>  #endif
>  #endif
> +#define CREATE_TRACE_POINTS
>
>  #include <linux/memblock.h>
>  #include <linux/err.h>
> @@ -34,6 +35,7 @@
>  #include <linux/cma.h>
>  #include <linux/highmem.h>
>  #include <linux/io.h>
> +#include <trace/events/cma.h>
>
>  #include "cma.h"
>
> @@ -408,6 +410,9 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
>  		start = bitmap_no + mask + 1;
>  	}
>
> +	if (page)
> +		trace_cma_alloc(cma, pfn, count);
> +
>  	pr_debug("%s(): returned %p\n", __func__, page);
>  	return page;
>  }
> @@ -440,6 +445,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)
>
>  	free_contig_range(pfn, count);
>  	cma_clear_bitmap(cma, pfn, count);
> +	trace_cma_release(cma, pfn, count);
>
>  	return true;

Are we interested only in successful allocation and release ? Should we also
have the trace point carry information regarding failure ?

-aneesh

Re: [PATCH v3 1/4] mm: cma: add trace events to debug physically-contiguous memory allocations

[Stefan Strogin]

Hi Aneesh,

On 03/03/15 12:13, Aneesh Kumar K.V wrote:
> 
> Are we interested only in successful allocation and release ? Should we also
> have the trace point carry information regarding failure ?
> 
> -aneesh
> 

I think we actually can be interested in tracing allocation failures
too. Thanks for the remark.

Should it be smth like that?
@@ -408,6 +410,8 @@ struct page *cma_alloc(struct cma *cma, int count,
unsigned int align)
 		start = bitmap_no + mask + 1;
 	}

+	trace_cma_alloc(cma, page, count);
+
 	pr_debug("%s(): returned %p\n", __func__, page);
 	return page;
 }

and in include/trace/events/cma.h:
+TRACE_EVENT(cma_alloc,
<...>
+	TP_fast_assign(
+		__entry->page = page;
+		__entry->count = count;
+	),
+
+	TP_printk("page=%p pfn=%lu count=%lu\n",
+		  __entry->page,
+		  __entry->page ? page_to_pfn(__entry->page) : 0,
+		  __entry->count)

[PATCH v3 2/4] mm: cma: add number of pages to debug message in cma_release()

[Stefan Strogin]

It's more useful to print address and number of pages which are being released,
not only address.

Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>
---
 mm/cma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/cma.c b/mm/cma.c
index 3a63c96..111bf62 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -434,7 +434,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)
 	if (!cma || !pages)
 		return false;
 
-	pr_debug("%s(page %p)\n", __func__, (void *)pages);
+	pr_debug("%s(page %p, count %d)\n", __func__, (void *)pages, count);
 
 	pfn = page_to_pfn(pages);
 
-- 
2.1.0

Re: [PATCH v3 2/4] mm: cma: add number of pages to debug message in cma_release()

[Michal Nazarewicz]

On Tue, Feb 24 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
> It's more useful to print address and number of pages which are being released,
> not only address.
>
> Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>

Acked-by: Michal Nazarewicz <mina86@mina86.com>

> ---
>  mm/cma.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/cma.c b/mm/cma.c
> index 3a63c96..111bf62 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -434,7 +434,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)
>  	if (!cma || !pages)
>  		return false;
>  
> -	pr_debug("%s(page %p)\n", __func__, (void *)pages);
> +	pr_debug("%s(page %p, count %d)\n", __func__, (void *)pages, count);
>  
>  	pfn = page_to_pfn(pages);
>  
> -- 
> 2.1.0
>

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

[PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Stefan Strogin]

When CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
/sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated
CMA buffers for each CMA region (N stands for number of CMA region).

Format is:
<base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)

When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when
the allocations are made. The stack traces are added to cma/cma-<N>/buffers
for each buffer list entry.

Example:

root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
 [<c1142c4b>] cma_alloc+0x1bb/0x200
 [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
 [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
 [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
 [<c1487312>] ahci_port_start+0xe2/0x210
 [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
 [<c1473650>] ata_host_activate+0xd0/0x110
 [<c14881bf>] ahci_host_activate+0x3f/0x170
 [<c14854e4>] ahci_init_one+0x764/0xab0
 [<c12e415f>] pci_device_probe+0x6f/0xd0
 [<c14378a8>] driver_probe_device+0x68/0x210
 [<c1437b09>] __driver_attach+0x79/0x80
 [<c1435eef>] bus_for_each_dev+0x4f/0x80
 [<c143749e>] driver_attach+0x1e/0x20
 [<c1437197>] bus_add_driver+0x157/0x200
 [<c14381bd>] driver_register+0x5d/0xf0
<...>

Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>
---
 mm/Kconfig     |  17 +++++++
 mm/cma.c       |   9 +++-
 mm/cma.h       |  26 ++++++++++
 mm/cma_debug.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 207 insertions(+), 1 deletion(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index 390214d..5ee2388 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -523,6 +523,23 @@ config CMA_DEBUGFS
 	help
 	  Turns on the DebugFS interface for CMA.
 
+config CMA_BUFFER_LIST
+	bool "List of currently allocated CMA buffers in debugfs"
+	depends on CMA_DEBUGFS
+	help
+	  /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently
+	  allocated CMA buffers for each CMA region (N stands for number of
+	  CMA region).
+	  Format is:
+	  <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
+
+config CMA_ALLOC_STACKTRACE
+	bool "Add stack trace to CMA buffer list"
+	depends on CMA_BUFFER_LIST && STACKTRACE
+	help
+	  Add stack traces saved at the moment of allocation for each buffer
+	  listed in /sys/kernel/debug/cma/cma-<N>/buffers.
+
 config CMA_AREAS
 	int "Maximum count of the CMA areas"
 	depends on CMA
diff --git a/mm/cma.c b/mm/cma.c
index 111bf62..e97c0ad 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -128,6 +128,10 @@ static int __init cma_activate_area(struct cma *cma)
 	INIT_HLIST_HEAD(&cma->mem_head);
 	spin_lock_init(&cma->mem_head_lock);
 #endif
+#ifdef CONFIG_CMA_BUFFER_LIST
+	INIT_LIST_HEAD(&cma->buffer_list);
+	mutex_init(&cma->list_lock);
+#endif
 
 	return 0;
 
@@ -410,8 +414,10 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
 		start = bitmap_no + mask + 1;
 	}
 
-	if (page)
+	if (page) {
 		trace_cma_alloc(cma, pfn, count);
+		cma_buffer_list_add(cma, pfn, count);
+	}
 
 	pr_debug("%s(): returned %p\n", __func__, page);
 	return page;
@@ -446,6 +452,7 @@ bool cma_release(struct cma *cma, struct page *pages, int count)
 	free_contig_range(pfn, count);
 	cma_clear_bitmap(cma, pfn, count);
 	trace_cma_release(cma, pfn, count);
+	cma_buffer_list_del(cma, pfn, count);
 
 	return true;
 }
diff --git a/mm/cma.h b/mm/cma.h
index 1132d73..6e7a488 100644
--- a/mm/cma.h
+++ b/mm/cma.h
@@ -1,6 +1,8 @@
 #ifndef __MM_CMA_H__
 #define __MM_CMA_H__
 
+#include <linux/sched.h>
+
 struct cma {
 	unsigned long   base_pfn;
 	unsigned long   count;
@@ -11,8 +13,32 @@ struct cma {
 	struct hlist_head mem_head;
 	spinlock_t mem_head_lock;
 #endif
+#ifdef CONFIG_CMA_BUFFER_LIST
+	struct list_head buffer_list;
+	struct mutex	list_lock;
+#endif
 };
 
+#ifdef CONFIG_CMA_BUFFER_LIST
+struct cma_buffer {
+	unsigned long pfn;
+	unsigned long count;
+	pid_t pid;
+	char comm[TASK_COMM_LEN];
+#ifdef CONFIG_CMA_ALLOC_STACKTRACE
+	unsigned long trace_entries[16];
+	unsigned int nr_entries;
+#endif
+	struct list_head list;
+};
+
+extern int cma_buffer_list_add(struct cma *cma, unsigned long pfn, int count);
+extern void cma_buffer_list_del(struct cma *cma, unsigned long pfn, int count);
+#else
+#define cma_buffer_list_add(cma, pfn, count) { }
+#define cma_buffer_list_del(cma, pfn, count) { }
+#endif /* CONFIG_CMA_BUFFER_LIST */
+
 extern struct cma cma_areas[MAX_CMA_AREAS];
 extern unsigned cma_area_count;
 
diff --git a/mm/cma_debug.c b/mm/cma_debug.c
index 6f0b976..cb74a0c 100644
--- a/mm/cma_debug.c
+++ b/mm/cma_debug.c
@@ -2,6 +2,7 @@
  * CMA DebugFS Interface
  *
  * Copyright (c) 2015 Sasha Levin <sasha.levin@oracle.com>
+ * Copyright (c) 2015 Stefan Strogin <stefan.strogin@gmail.com>
  */
  
 
@@ -10,6 +11,9 @@
 #include <linux/list.h>
 #include <linux/kernel.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/stacktrace.h>
+#include <linux/vmalloc.h>
 
 #include "cma.h"
 
@@ -21,6 +25,103 @@ struct cma_mem {
 
 static struct dentry *cma_debugfs_root;
 
+#ifdef CONFIG_CMA_BUFFER_LIST
+/* Must be called under cma->list_lock */
+static int __cma_buffer_list_add(struct cma *cma, unsigned long pfn, int count)
+{
+	struct cma_buffer *cmabuf;
+#ifdef CONFIG_CMA_ALLOC_STACKTRACE
+	struct stack_trace trace;
+#endif
+
+	cmabuf = kmalloc(sizeof(*cmabuf), GFP_KERNEL);
+	if (!cmabuf) {
+		pr_warn("%s(page %p, count %d): failed to allocate buffer list entry\n",
+			__func__, pfn_to_page(pfn), count);
+		return -ENOMEM;
+	}
+
+#ifdef CONFIG_CMA_ALLOC_STACKTRACE
+	trace.nr_entries = 0;
+	trace.max_entries = ARRAY_SIZE(cmabuf->trace_entries);
+	trace.entries = &cmabuf->trace_entries[0];
+	trace.skip = 2;
+	save_stack_trace(&trace);
+	cmabuf->nr_entries = trace.nr_entries;
+#endif
+	cmabuf->pfn = pfn;
+	cmabuf->count = count;
+	cmabuf->pid = task_pid_nr(current);
+	get_task_comm(cmabuf->comm, current);
+
+	list_add_tail(&cmabuf->list, &cma->buffer_list);
+
+	return 0;
+}
+
+/**
+ * cma_buffer_list_add() - add a new entry to a list of allocated buffers
+ * @cma:     Contiguous memory region for which the allocation is performed.
+ * @pfn:     Base PFN of the allocated buffer.
+ * @count:   Number of allocated pages.
+ *
+ * This function adds a new entry to the list of allocated contiguous memory
+ * buffers in a CMA region.
+ */
+int cma_buffer_list_add(struct cma *cma, unsigned long pfn, int count)
+{
+	int ret;
+
+	mutex_lock(&cma->list_lock);
+	ret = __cma_buffer_list_add(cma, pfn, count);
+	mutex_unlock(&cma->list_lock);
+
+	return ret;
+}
+
+/**
+ * cma_buffer_list_del() - delete an entry from a list of allocated buffers
+ * @cma:   Contiguous memory region for which the allocation was performed.
+ * @pfn:   Base PFN of the released buffer.
+ * @count: Number of pages.
+ *
+ * This function deletes a list entry added by cma_buffer_list_add().
+ */
+void cma_buffer_list_del(struct cma *cma, unsigned long pfn, int count)
+{
+	struct cma_buffer *cmabuf, *tmp;
+	int found = 0;
+	unsigned long buf_end_pfn, free_end_pfn = pfn + count;
+
+	mutex_lock(&cma->list_lock);
+	list_for_each_entry_safe(cmabuf, tmp, &cma->buffer_list, list) {
+
+		buf_end_pfn = cmabuf->pfn + cmabuf->count;
+		if (pfn <= cmabuf->pfn && free_end_pfn >= buf_end_pfn) {
+			list_del(&cmabuf->list);
+			kfree(cmabuf);
+			found = 1;
+		} else if (pfn <= cmabuf->pfn && free_end_pfn < buf_end_pfn) {
+			cmabuf->count -= free_end_pfn - cmabuf->pfn;
+			cmabuf->pfn = free_end_pfn;
+			found = 1;
+		} else if (pfn > cmabuf->pfn && pfn < buf_end_pfn) {
+			if (free_end_pfn < buf_end_pfn)
+				__cma_buffer_list_add(cma, free_end_pfn,
+						buf_end_pfn - free_end_pfn);
+			cmabuf->count = pfn - cmabuf->pfn;
+			found = 1;
+		}
+	}
+	mutex_unlock(&cma->list_lock);
+
+	if (!found)
+		pr_err("%s(page %p, count %d): couldn't find buffer list entry\n",
+		       __func__, pfn_to_page(pfn), count);
+
+}
+#endif /* CONFIG_CMA_BUFFER_LIST */
+
 static int cma_debugfs_get(void *data, u64 *val)
 {
 	unsigned long *p = data;
@@ -126,6 +227,57 @@ static int cma_alloc_write(void *data, u64 val)
 
 DEFINE_SIMPLE_ATTRIBUTE(cma_alloc_fops, NULL, cma_alloc_write, "%llu\n");
 
+#ifdef CONFIG_CMA_BUFFER_LIST
+static ssize_t cma_buffer_list_read(struct file *file, char __user *userbuf,
+				    size_t count, loff_t *ppos)
+{
+	struct cma *cma = file->private_data;
+	struct cma_buffer *cmabuf;
+	char *buf;
+	int ret, n = 0;
+#ifdef CONFIG_CMA_ALLOC_STACKTRACE
+	struct stack_trace trace;
+#endif
+
+	if (*ppos < 0 || !count)
+		return -EINVAL;
+
+	buf = vmalloc(count);
+	if (!buf)
+		return -ENOMEM;
+
+	mutex_lock(&cma->list_lock);
+	list_for_each_entry(cmabuf, &cma->buffer_list, list) {
+		n += snprintf(buf + n, count - n,
+			      "0x%llx - 0x%llx (%lu kB), allocated by pid %u (%s)\n",
+			      (unsigned long long)PFN_PHYS(cmabuf->pfn),
+			      (unsigned long long)PFN_PHYS(cmabuf->pfn +
+				      cmabuf->count),
+			      (cmabuf->count * PAGE_SIZE) >> 10, cmabuf->pid,
+			      cmabuf->comm);
+
+#ifdef CONFIG_CMA_ALLOC_STACKTRACE
+		trace.nr_entries = cmabuf->nr_entries;
+		trace.entries = &cmabuf->trace_entries[0];
+		n += snprint_stack_trace(buf + n, count - n, &trace, 0);
+		n += snprintf(buf + n, count - n, "\n");
+#endif
+	}
+	mutex_unlock(&cma->list_lock);
+
+	ret = simple_read_from_buffer(userbuf, count, ppos, buf, n);
+	vfree(buf);
+
+	return ret;
+}
+
+static const struct file_operations cma_buffer_list_fops = {
+	.open = simple_open,
+	.read = cma_buffer_list_read,
+	.llseek = default_llseek,
+};
+#endif /* CONFIG_CMA_BUFFER_LIST */
+
 static void cma_debugfs_add_one(struct cma *cma, int idx)
 {       
 	struct dentry *tmp;
@@ -148,6 +300,10 @@ static void cma_debugfs_add_one(struct cma *cma, int idx)
 				&cma->count, &cma_debugfs_fops);
 	debugfs_create_file("order_per_bit", S_IRUGO, tmp,
 				&cma->order_per_bit, &cma_debugfs_fops);
+#ifdef CONFIG_CMA_BUFFER_LIST
+	debugfs_create_file("buffers", S_IRUGO, tmp, cma,
+				&cma_buffer_list_fops);
+#endif
 
 	u32s = DIV_ROUND_UP(cma_bitmap_maxno(cma), BITS_PER_BYTE * sizeof(u32));
 	debugfs_create_u32_array("bitmap", S_IRUGO, tmp, (u32*)cma->bitmap, u32s);
-- 
2.1.0

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Michal Nazarewicz]

On Tue, Feb 24 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
> When CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
> /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated
> CMA buffers for each CMA region (N stands for number of CMA region).
>
> Format is:
> <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
>
> When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when
> the allocations are made. The stack traces are added to cma/cma-<N>/buffers
> for each buffer list entry.
>
> Example:
>
> root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
> 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
>  [<c1142c4b>] cma_alloc+0x1bb/0x200
>  [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
>  [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
>  [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
>  [<c1487312>] ahci_port_start+0xe2/0x210
>  [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
>  [<c1473650>] ata_host_activate+0xd0/0x110
>  [<c14881bf>] ahci_host_activate+0x3f/0x170
>  [<c14854e4>] ahci_init_one+0x764/0xab0
>  [<c12e415f>] pci_device_probe+0x6f/0xd0
>  [<c14378a8>] driver_probe_device+0x68/0x210
>  [<c1437b09>] __driver_attach+0x79/0x80
>  [<c1435eef>] bus_for_each_dev+0x4f/0x80
>  [<c143749e>] driver_attach+0x1e/0x20
>  [<c1437197>] bus_add_driver+0x157/0x200
>  [<c14381bd>] driver_register+0x5d/0xf0
> <...>
>
> Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>


> --- a/mm/cma.h
> +++ b/mm/cma.h
> @@ -11,8 +13,32 @@ struct cma {
>  	struct hlist_head mem_head;
>  	spinlock_t mem_head_lock;
>  #endif
> +#ifdef CONFIG_CMA_BUFFER_LIST
> +	struct list_head buffer_list;
> +	struct mutex	list_lock;
> +#endif
>  };
>  
> +#ifdef CONFIG_CMA_BUFFER_LIST
> +struct cma_buffer {
> +	unsigned long pfn;
> +	unsigned long count;
> +	pid_t pid;
> +	char comm[TASK_COMM_LEN];
> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
> +	unsigned long trace_entries[16];
> +	unsigned int nr_entries;
> +#endif
> +	struct list_head list;
> +};

This structure is only ever used in cma_debug.c so is there a reason
to define it in the header file?

> +
> +extern int cma_buffer_list_add(struct cma *cma, unsigned long pfn, int count);
> +extern void cma_buffer_list_del(struct cma *cma, unsigned long pfn, int count);
> +#else
> +#define cma_buffer_list_add(cma, pfn, count) { }
> +#define cma_buffer_list_del(cma, pfn, count) { }
> +#endif /* CONFIG_CMA_BUFFER_LIST */
> +
>  extern struct cma cma_areas[MAX_CMA_AREAS];
>  extern unsigned cma_area_count;


> +#ifdef CONFIG_CMA_BUFFER_LIST
> +static ssize_t cma_buffer_list_read(struct file *file, char __user *userbuf,
> +				    size_t count, loff_t *ppos)
> +{
> +	struct cma *cma = file->private_data;
> +	struct cma_buffer *cmabuf;
> +	char *buf;
> +	int ret, n = 0;
> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
> +	struct stack_trace trace;
> +#endif
> +
> +	if (*ppos < 0 || !count)
> +		return -EINVAL;
> +
> +	buf = vmalloc(count);
> +	if (!buf)
> +		return -ENOMEM;
> +
> +	mutex_lock(&cma->list_lock);
> +	list_for_each_entry(cmabuf, &cma->buffer_list, list) {
> +		n += snprintf(buf + n, count - n,
> +			      "0x%llx - 0x%llx (%lu kB), allocated by pid %u (%s)\n",
> +			      (unsigned long long)PFN_PHYS(cmabuf->pfn),
> +			      (unsigned long long)PFN_PHYS(cmabuf->pfn +
> +				      cmabuf->count),
> +			      (cmabuf->count * PAGE_SIZE) >> 10, cmabuf->pid,
> +			      cmabuf->comm);
> +
> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
> +		trace.nr_entries = cmabuf->nr_entries;
> +		trace.entries = &cmabuf->trace_entries[0];
> +		n += snprint_stack_trace(buf + n, count - n, &trace, 0);
> +		n += snprintf(buf + n, count - n, "\n");
> +#endif
> +	}
> +	mutex_unlock(&cma->list_lock);
> +
> +	ret = simple_read_from_buffer(userbuf, count, ppos, buf, n);
> +	vfree(buf);
> +
> +	return ret;
> +}

So in practice user space must allocate buffer big enough to read the
whole file into memory.  Calling read(2) with some count will never read
anything past the first count bytes of the file.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Stefan Strogin]

Hi Michał,

Thank you for the answer.

On 25/02/15 00:32, Michal Nazarewicz wrote:
> On Tue, Feb 24 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
>> --- a/mm/cma.h
>> +++ b/mm/cma.h
>> @@ -11,8 +13,32 @@ struct cma {
>>  	struct hlist_head mem_head;
>>  	spinlock_t mem_head_lock;
>>  #endif
>> +#ifdef CONFIG_CMA_BUFFER_LIST
>> +	struct list_head buffer_list;
>> +	struct mutex	list_lock;
>> +#endif
>>  };
>>  
>> +#ifdef CONFIG_CMA_BUFFER_LIST
>> +struct cma_buffer {
>> +	unsigned long pfn;
>> +	unsigned long count;
>> +	pid_t pid;
>> +	char comm[TASK_COMM_LEN];
>> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
>> +	unsigned long trace_entries[16];
>> +	unsigned int nr_entries;
>> +#endif
>> +	struct list_head list;
>> +};
> 
> This structure is only ever used in cma_debug.c so is there a reason
> to define it in the header file?
> 

No, there isn't. Thanks. I'll move it to cma_debug.c

>> +
>> +extern int cma_buffer_list_add(struct cma *cma, unsigned long pfn, int count);
>> +extern void cma_buffer_list_del(struct cma *cma, unsigned long pfn, int count);
>> +#else
>> +#define cma_buffer_list_add(cma, pfn, count) { }
>> +#define cma_buffer_list_del(cma, pfn, count) { }
>> +#endif /* CONFIG_CMA_BUFFER_LIST */
>> +
>>  extern struct cma cma_areas[MAX_CMA_AREAS];
>>  extern unsigned cma_area_count;
> 
> 
>> +#ifdef CONFIG_CMA_BUFFER_LIST
>> +static ssize_t cma_buffer_list_read(struct file *file, char __user *userbuf,
>> +				    size_t count, loff_t *ppos)
>> +{
>> +	struct cma *cma = file->private_data;
>> +	struct cma_buffer *cmabuf;
>> +	char *buf;
>> +	int ret, n = 0;
>> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
>> +	struct stack_trace trace;
>> +#endif
>> +
>> +	if (*ppos < 0 || !count)
>> +		return -EINVAL;
>> +
>> +	buf = vmalloc(count);
>> +	if (!buf)
>> +		return -ENOMEM;
>> +
>> +	mutex_lock(&cma->list_lock);
>> +	list_for_each_entry(cmabuf, &cma->buffer_list, list) {
>> +		n += snprintf(buf + n, count - n,
>> +			      "0x%llx - 0x%llx (%lu kB), allocated by pid %u (%s)\n",
>> +			      (unsigned long long)PFN_PHYS(cmabuf->pfn),
>> +			      (unsigned long long)PFN_PHYS(cmabuf->pfn +
>> +				      cmabuf->count),
>> +			      (cmabuf->count * PAGE_SIZE) >> 10, cmabuf->pid,
>> +			      cmabuf->comm);
>> +
>> +#ifdef CONFIG_CMA_ALLOC_STACKTRACE
>> +		trace.nr_entries = cmabuf->nr_entries;
>> +		trace.entries = &cmabuf->trace_entries[0];
>> +		n += snprint_stack_trace(buf + n, count - n, &trace, 0);
>> +		n += snprintf(buf + n, count - n, "\n");
>> +#endif
>> +	}
>> +	mutex_unlock(&cma->list_lock);
>> +
>> +	ret = simple_read_from_buffer(userbuf, count, ppos, buf, n);
>> +	vfree(buf);
>> +
>> +	return ret;
>> +}
> 
> So in practice user space must allocate buffer big enough to read the
> whole file into memory.  Calling read(2) with some count will never read
> anything past the first count bytes of the file.
> 

My fault. You are right.
I'm not sure how to do the output nice... I could use *ppos to point the
number of next list entry to read (like that is used in
read_page_owner()). But in this case the list could be changed before we
finish reading, it's bad.
Or we could use seq_files like in v1, iterating over buffer_list
entries. But seq_print_stack_trace() has to be added.

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Michal Nazarewicz]

On Mon, Mar 02 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
> My fault. You are right.
> I'm not sure how to do the output nice... I could use *ppos to point the
> number of next list entry to read (like that is used in
> read_page_owner()). But in this case the list could be changed before we
> finish reading, it's bad.
> Or we could use seq_files like in v1, iterating over buffer_list
> entries. But seq_print_stack_trace() has to be added.

I’m not that familiar with seq_* so my opinion may be ill-informed, but
I feel that skipping some entries while (de)allocations happen is akin
to process reading a file while some other process modifies it.  This is
a debug function so perhaps it’s acceptable that it may return garbage
if not used carefully?

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Aneesh Kumar K.V]

Stefan Strogin <s.strogin@partner.samsung.com> writes:

> When CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
> /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated
> CMA buffers for each CMA region (N stands for number of CMA region).
>
> Format is:
> <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
>
> When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when
> the allocations are made. The stack traces are added to cma/cma-<N>/buffers
> for each buffer list entry.
>
> Example:
>
> root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
> 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
>  [<c1142c4b>] cma_alloc+0x1bb/0x200
>  [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
>  [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
>  [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
>  [<c1487312>] ahci_port_start+0xe2/0x210
>  [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
>  [<c1473650>] ata_host_activate+0xd0/0x110
>  [<c14881bf>] ahci_host_activate+0x3f/0x170
>  [<c14854e4>] ahci_init_one+0x764/0xab0
>  [<c12e415f>] pci_device_probe+0x6f/0xd0
>  [<c14378a8>] driver_probe_device+0x68/0x210
>  [<c1437b09>] __driver_attach+0x79/0x80
>  [<c1435eef>] bus_for_each_dev+0x4f/0x80
>  [<c143749e>] driver_attach+0x1e/0x20
>  [<c1437197>] bus_add_driver+0x157/0x200
>  [<c14381bd>] driver_register+0x5d/0xf0
> <...>

A perf record -g will also give this information right ? To use this
feature, one need to recompile the kernel anyway. So why not assume that
user can always rerun the test with perf record -g and find the cma
allocation point stack trace ?

-aneesh

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Stefan Strogin]

Hi Aneesh,

On 03/03/15 12:16, Aneesh Kumar K.V wrote:
> Stefan Strogin <s.strogin@partner.samsung.com> writes:
> 
>> When CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
>> /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated
>> CMA buffers for each CMA region (N stands for number of CMA region).
>>
>> Format is:
>> <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
>>
>> When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when
>> the allocations are made. The stack traces are added to cma/cma-<N>/buffers
>> for each buffer list entry.
>>
>> Example:
>>
>> root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
>> 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
>>  [<c1142c4b>] cma_alloc+0x1bb/0x200
>>  [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
>>  [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
>>  [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
>>  [<c1487312>] ahci_port_start+0xe2/0x210
>>  [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
>>  [<c1473650>] ata_host_activate+0xd0/0x110
>>  [<c14881bf>] ahci_host_activate+0x3f/0x170
>>  [<c14854e4>] ahci_init_one+0x764/0xab0
>>  [<c12e415f>] pci_device_probe+0x6f/0xd0
>>  [<c14378a8>] driver_probe_device+0x68/0x210
>>  [<c1437b09>] __driver_attach+0x79/0x80
>>  [<c1435eef>] bus_for_each_dev+0x4f/0x80
>>  [<c143749e>] driver_attach+0x1e/0x20
>>  [<c1437197>] bus_add_driver+0x157/0x200
>>  [<c14381bd>] driver_register+0x5d/0xf0
>> <...>
> 
> A perf record -g will also give this information right ? To use this
> feature, one need to recompile the kernel anyway. So why not assume that
> user can always rerun the test with perf record -g and find the cma
> allocation point stack trace ?
> 
> -aneesh
> 

Excuse me for the delay.
I thought that 'perf record <command>' gathers data only for a command
that it runs, does it? But we want to have information about all the
allocations and releases from the boot time. IMHO it would be more
reasonable to use ftrace for that. But after all the patch enables to
see not a history of allocations and deallocations but a current state
of CMA region.
As to recompilation, for example in our division this feature is enabled
by default among other CONFIG_*_DEBUG features in debug versions of kernel.

Re: [PATCH v3 3/4] mm: cma: add list of currently allocated CMA buffers to debugfs

[Aneesh Kumar K.V]

Stefan Strogin <s.strogin@partner.samsung.com> writes:

> Hi Aneesh,
>
> On 03/03/15 12:16, Aneesh Kumar K.V wrote:
>> Stefan Strogin <s.strogin@partner.samsung.com> writes:
>> 
>>> When CONFIG_CMA_BUFFER_LIST is configured a file is added to debugfs:
>>> /sys/kernel/debug/cma/cma-<N>/buffers contains a list of currently allocated
>>> CMA buffers for each CMA region (N stands for number of CMA region).
>>>
>>> Format is:
>>> <base_phys_addr> - <end_phys_addr> (<size> kB), allocated by <PID> (<comm>)
>>>
>>> When CONFIG_CMA_ALLOC_STACKTRACE is configured then stack traces are saved when
>>> the allocations are made. The stack traces are added to cma/cma-<N>/buffers
>>> for each buffer list entry.
>>>
>>> Example:
>>>
>>> root@debian:/sys/kernel/debug/cma# cat cma-0/buffers
>>> 0x2f400000 - 0x2f417000 (92 kB), allocated by pid 1 (swapper/0)
>>>  [<c1142c4b>] cma_alloc+0x1bb/0x200
>>>  [<c143d28a>] dma_alloc_from_contiguous+0x3a/0x40
>>>  [<c10079d9>] dma_generic_alloc_coherent+0x89/0x160
>>>  [<c14456ce>] dmam_alloc_coherent+0xbe/0x100
>>>  [<c1487312>] ahci_port_start+0xe2/0x210
>>>  [<c146e0e0>] ata_host_start.part.28+0xc0/0x1a0
>>>  [<c1473650>] ata_host_activate+0xd0/0x110
>>>  [<c14881bf>] ahci_host_activate+0x3f/0x170
>>>  [<c14854e4>] ahci_init_one+0x764/0xab0
>>>  [<c12e415f>] pci_device_probe+0x6f/0xd0
>>>  [<c14378a8>] driver_probe_device+0x68/0x210
>>>  [<c1437b09>] __driver_attach+0x79/0x80
>>>  [<c1435eef>] bus_for_each_dev+0x4f/0x80
>>>  [<c143749e>] driver_attach+0x1e/0x20
>>>  [<c1437197>] bus_add_driver+0x157/0x200
>>>  [<c14381bd>] driver_register+0x5d/0xf0
>>> <...>
>> 
>> A perf record -g will also give this information right ? To use this
>> feature, one need to recompile the kernel anyway. So why not assume that
>> user can always rerun the test with perf record -g and find the cma
>> allocation point stack trace ?
>> 
>> -aneesh
>> 
>
> Excuse me for the delay.
> I thought that 'perf record <command>' gathers data only for a command
> that it runs, does it? But we want to have information about all the
> allocations and releases from the boot time. 

>From boot time makes it interesting. Otherwise you could use perf
record. For ex:

./perf record -e kmem:kmalloc -g -a 
./perf script

jbd2/dm-3-8  7666 [000]  4666.621521: kmem:kmalloc: call_site=c0000000003ce108 ptr=0xc000000fcd646360 bytes_req=96 bytes_alloc=96 gfp_flags=GFP_NOFS|GFP_ZERO
                  27f1dc .__kmalloc (/boot/vmlinux)
                       0 [unknown] ([unknown])
                  3de108 .ext4_find_extent (/boot/vmlinux)
                  3e4038 .ext4_ext_map_blocks (/boot/vmlinux)
                  3ab388 .ext4_map_blocks (/boot/vmlinux)
                  3ab970 ._ext4_get_block (/boot/vmlinux)
                  2df7e0 .generic_block_bmap (/boot/vmlinux)
                  3a7f4c .ext4_bmap (/boot/vmlinux)
                  2ba744 .bmap (/boot/vmlinux)
                  4293d4 .jbd2_journal_bmap (/boot/vmlinux)
                  41df84 .jbd2_journal_commit_transaction (/boot/vmlinux)
                  427620 .kjournald2 (/boot/vmlinux)
                   fbfa4 .kthread (/boot/vmlinux)
                   19568 .ret_from_kernel_thread (/boot/vmlinux)

....

>IMHO it would be more
> reasonable to use ftrace for that. But after all the patch enables to
> see not a history of allocations and deallocations but a current state
> of CMA region.
> As to recompilation, for example in our division this feature is enabled
> by default among other CONFIG_*_DEBUG features in debug versions of kernel.

[PATCH v3 4/4] mm: cma: add functions to get region pages counters

[Stefan Strogin]

From: Dmitry Safonov <d.safonov@partner.samsung.com>

Here are two functions that provide interface to compute/get used size
and size of biggest free chunk in cma region. Add that information to debugfs.

Signed-off-by: Dmitry Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>
---
 include/linux/cma.h |  2 ++
 mm/cma.c            | 30 ++++++++++++++++++++++++++++++
 mm/cma_debug.c      | 24 ++++++++++++++++++++++++
 3 files changed, 56 insertions(+)

diff --git a/include/linux/cma.h b/include/linux/cma.h
index 9384ba6..855e6f2 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -18,6 +18,8 @@ struct cma;
 extern unsigned long totalcma_pages;
 extern phys_addr_t cma_get_base(struct cma *cma);
 extern unsigned long cma_get_size(struct cma *cma);
+extern unsigned long cma_get_used(struct cma *cma);
+extern unsigned long cma_get_maxchunk(struct cma *cma);
 
 extern int __init cma_declare_contiguous(phys_addr_t base,
 			phys_addr_t size, phys_addr_t limit,
diff --git a/mm/cma.c b/mm/cma.c
index e97c0ad..51f9d83 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -53,6 +53,36 @@ unsigned long cma_get_size(struct cma *cma)
 	return cma->count << PAGE_SHIFT;
 }
 
+unsigned long cma_get_used(struct cma *cma)
+{
+	unsigned long ret = 0;
+
+	mutex_lock(&cma->lock);
+	/* pages counter is smaller than sizeof(int) */
+	ret = bitmap_weight(cma->bitmap, (int)cma->count);
+	mutex_unlock(&cma->lock);
+
+	return ret << cma->order_per_bit;
+}
+
+unsigned long cma_get_maxchunk(struct cma *cma)
+{
+	unsigned long maxchunk = 0;
+	unsigned long start, end = 0;
+
+	mutex_lock(&cma->lock);
+	for (;;) {
+		start = find_next_zero_bit(cma->bitmap, cma->count, end);
+		if (start >= cma->count)
+			break;
+		end = find_next_bit(cma->bitmap, cma->count, start);
+		maxchunk = max(end - start, maxchunk);
+	}
+	mutex_unlock(&cma->lock);
+
+	return maxchunk << cma->order_per_bit;
+}
+
 static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
 {
 	if (align_order <= cma->order_per_bit)
diff --git a/mm/cma_debug.c b/mm/cma_debug.c
index cb74a0c..f995b07 100644
--- a/mm/cma_debug.c
+++ b/mm/cma_debug.c
@@ -133,6 +133,28 @@ static int cma_debugfs_get(void *data, u64 *val)
 
 DEFINE_SIMPLE_ATTRIBUTE(cma_debugfs_fops, cma_debugfs_get, NULL, "%llu\n");
 
+static int cma_used_get(void *data, u64 *val)
+{
+	struct cma *cma = data;
+
+	*val = cma_get_used(cma);
+
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(cma_used_fops, cma_used_get, NULL, "%llu\n");
+
+static int cma_maxchunk_get(void *data, u64 *val)
+{
+	struct cma *cma = data;
+
+	*val = cma_get_maxchunk(cma);
+
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(cma_maxchunk_fops, cma_maxchunk_get, NULL, "%llu\n");
+
 static void cma_add_to_cma_mem_list(struct cma *cma, struct cma_mem *mem)
 {
 	spin_lock(&cma->mem_head_lock);
@@ -300,6 +322,8 @@ static void cma_debugfs_add_one(struct cma *cma, int idx)
 				&cma->count, &cma_debugfs_fops);
 	debugfs_create_file("order_per_bit", S_IRUGO, tmp,
 				&cma->order_per_bit, &cma_debugfs_fops);
+	debugfs_create_file("used", S_IRUGO, tmp, cma, &cma_used_fops);
+	debugfs_create_file("maxchunk", S_IRUGO, tmp, cma, &cma_maxchunk_fops);
 #ifdef CONFIG_CMA_BUFFER_LIST
 	debugfs_create_file("buffers", S_IRUGO, tmp, cma,
 				&cma_buffer_list_fops);
-- 
2.1.0

Re: [PATCH v3 4/4] mm: cma: add functions to get region pages counters

[Michal Nazarewicz]

On Tue, Feb 24 2015, Stefan Strogin <s.strogin@partner.samsung.com> wrote:
> From: Dmitry Safonov <d.safonov@partner.samsung.com>
>
> Here are two functions that provide interface to compute/get used size
> and size of biggest free chunk in cma region. Add that information to debugfs.
>
> Signed-off-by: Dmitry Safonov <d.safonov@partner.samsung.com>
> Signed-off-by: Stefan Strogin <s.strogin@partner.samsung.com>

Acked-by: Michal Nazarewicz <mina86@mina86.com>

> ---
>  include/linux/cma.h |  2 ++
>  mm/cma.c            | 30 ++++++++++++++++++++++++++++++
>  mm/cma_debug.c      | 24 ++++++++++++++++++++++++
>  3 files changed, 56 insertions(+)

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--