[RFC PATCH 0/7] powerpc/powernv: Nest Instrumentation support

[RFC PATCH 0/7] powerpc/powernv: Nest Instrumentation support

[Madhavan Srinivasan]

This patchset enables Nest Instrumentation support on powerpc.
POWER8 has per-chip Nest Intrumentation which provides various 
per-chip utilisation metrics like memory bandwidth, IO bandwidth 
and many other component metrics.

Nest Instrumentation provides an interface (via PORE Engine)
to configure and move the nest counter data to memory. From
kernel side, OPAL Call interface is used to activate/deactivate
PORE Engine for nest data collection.

OPAL at boot, detects the feature, initializes it and pass on
the nest units and other related information such as memory
region, events supported so on, to kernel via device-tree. 

Patchset uses Intel's uncore framework to enable the consumption 
of Nest instrumentation data on powerpc. First three patches are
derived from arch/x86/kernel/cpu/perf_event_intel_uncore.[c/h].
But patches differ since "box" level abstration is not implemented
As a TODO, need to rework the code to reduce code duplication. 

Fourth patch in the series adds the OPAL call interface to kernel.
Fifth patch defines a generic pmu functions to be shared by all
nest pmu units. And sixth patch defines the Device-tree parsing
functions to populate various structures to aid the pmu registration.
Patch also explains the device-tree layout for nest counter units.

Here is sample perf usage to explain the interface and
values represeneted here are bogus, since platform enablement is
under-developement.

# ./perf list
....
  uncore_mcs_0/mcs_read/                             [Kernel PMU event]
  uncore_mcs_0/mcs_write/                            [Kernel PMU event]
  uncore_mcs_1/mcs_read/                             [Kernel PMU event]
  uncore_mcs_1/mcs_write/                            [Kernel PMU event]
  uncore_mcs_2/mcs_read/                             [Kernel PMU event]
  uncore_mcs_2/mcs_write/                            [Kernel PMU event]
  uncore_mcs_3/mcs_read/                             [Kernel PMU event]
  uncore_mcs_3/mcs_write/                            [Kernel PMU event]
...

[root@ltctul57a-p1 perf]# ./perf stat -e 'uncore_mcs_0/mcs_read/' -e 'uncore_mcs_0/mcs_write/' -a sleep 1

 Performance counter stats for 'system wide':

              64.00 MiB/s	 uncore_mcs_0/mcs_read/                                       [100.00%]
              64.00 MiB/s	 uncore_mcs_0/mcs_write/                                     

       1.000723515 seconds time elapsed

Kindly let me know you comments and feedback.

Madhavan Srinivasan (7):
  powerpc/powernv: Data structure and macros definition 
  powerpc/powernv: uncore foundation code
  powerpc/powernv: uncore cpumask and CPU hotplug
  powerpc/powernv: Add OPAL support for uncore pmu
  powerpc/powernv: Add POWER8 uncore pmu support
  powerpc/powernv: add support to parse dt for uncore pmu
  powerpc/powernv: enable uncore related file in Makefile

 arch/powerpc/include/asm/opal.h                |   3 +
 arch/powerpc/perf/Makefile                     |   2 +-
 arch/powerpc/perf/uncore_pmu.c                 | 260 ++++++++++++++++
 arch/powerpc/perf/uncore_pmu.h                 |  89 ++++++
 arch/powerpc/perf/uncore_pmu_p8.c              | 403 +++++++++++++++++++++++++
 arch/powerpc/platforms/powernv/opal-wrappers.S |   1 +
 6 files changed, 757 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/perf/uncore_pmu.c
 create mode 100644 arch/powerpc/perf/uncore_pmu.h
 create mode 100644 arch/powerpc/perf/uncore_pmu_p8.c

-- 
1.9.1

[RFC PATCH 1/7] powerpc/powernv: Data structure and macros definition

[Madhavan Srinivasan]

Patch adds the data structures and macros 
needed for nest pmu support. "type" structure will 
hold each nest unit and sub-unit information.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/uncore_pmu.h | 89 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 89 insertions(+)
 create mode 100644 arch/powerpc/perf/uncore_pmu.h

diff --git a/arch/powerpc/perf/uncore_pmu.h b/arch/powerpc/perf/uncore_pmu.h
new file mode 100644
index 0000000..ba4b3f4
--- /dev/null
+++ b/arch/powerpc/perf/uncore_pmu.h
@@ -0,0 +1,89 @@
+/*
+ * Uncore Performance Monitor counter support.
+ *
+ * Derived from Intel's uncore framework
+ *   written by: Liang,Kan
+ *               Zheng, Yan
+ *               Andi Keen
+ *
+ *   Copyright Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; under version
+ * 2 of the License.
+ */
+
+#include <linux/perf_event.h>
+#include <linux/cpumask.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <asm/prom.h>
+#include <asm/opal.h>
+#include <asm/cputhreads.h>
+
+/*
+ * Macros for pmu groups
+ */
+#define pmu_group attr_groups[0]
+#define format_group attr_groups[1]
+#define events_group attr_groups[2]
+
+/*
+ * uncore specific macros
+ */
+#define P8_MAX_CHIP		256
+#define MAX_PMU_NAME_LEN 	128
+#define MAX_UNITS_SUPPORTED	16
+#define MAX_SUB_UNITS		8
+#define MAX_TYPE_EVENTS		16
+#define MAX_EVENTS_SUPPORTED	256
+
+/*
+ * Structure to hold event information for a
+ * uncore pmu type. Device-Tree contain events
+ * and memory offsets to access the counter for
+ * a gievn event. Some events will also carry
+ * unit and scale.
+ */
+struct ppc64_uncore_type_events {
+	int ev_offset[MAX_EVENTS_SUPPORTED];
+	const char *ev_name[MAX_EVENTS_SUPPORTED];
+	const char *ev_value[MAX_EVENTS_SUPPORTED];
+};
+
+/*
+ * Structure to hold per chip specific memory address
+ * information for uncore pmus, since uncore pmus are
+ * per-chip.
+ */
+struct ppc64_uncore_unit {
+	uint32_t chip_id;
+	uint64_t preg_base;
+	uint64_t vreg_base;
+	uint32_t size;
+};
+
+/*
+ * Structure for each uncore unit.
+ */
+struct ppc64_uncore_type {
+	const char *name;
+	int num_boxes;
+	struct pmu *pmu;
+	struct ppc64_uncore_pmu *pmus;
+	struct attribute **event_attrs;
+	const struct attribute_group *attr_groups[4];
+	struct ppc64_uncore_type_events event_arry[MAX_SUB_UNITS];
+};
+
+
+struct ppc64_uncore_pmu {
+	struct pmu pmu;
+	char name[MAX_PMU_NAME_LEN];
+	int pmu_idx;
+	struct ppc64_uncore_type *type;
+};
+
+struct ppc64_uncore_pmu *uncore_event_to_pmu(struct perf_event *event);
+extern int uncore_p8_init(void);
-- 
1.9.1

[RFC PATCH 2/7] powerpc/powernv: uncore foundation code

[Madhavan Srinivasan]

Patch adds the foundation code enable nest pmu. It defines
various init routines and these are based on Intel's uncore
framework. But it does differ since "box" structure abstraction
is not implemented here.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/uncore_pmu.c | 104 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)
 create mode 100644 arch/powerpc/perf/uncore_pmu.c

diff --git a/arch/powerpc/perf/uncore_pmu.c b/arch/powerpc/perf/uncore_pmu.c
new file mode 100644
index 0000000..cc544d3
--- /dev/null
+++ b/arch/powerpc/perf/uncore_pmu.c
@@ -0,0 +1,104 @@
+/*
+ * Uncore Performance Monitor counter support.
+ *
+ * Derived from Intel's uncore framework
+ *   written by: Liang,Kan
+ *               Zheng, Yan
+ *               Andi Keen
+ *
+ *   Copyright Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; under version
+ * 2 of the License.
+ */
+
+#include "uncore_pmu.h"
+
+struct ppc64_uncore_type *empty_uncore[] = { NULL, };
+struct ppc64_uncore_type **ppc64_uncore = empty_uncore;
+
+struct ppc64_uncore_pmu *uncore_event_to_pmu(struct perf_event *event)
+{
+	return container_of(event->pmu, struct ppc64_uncore_pmu, pmu);
+}
+
+int __init uncore_type_init(struct ppc64_uncore_type *type)
+{
+	struct ppc64_uncore_pmu *pmus;
+	int i;
+
+	pmus = kzalloc(sizeof(*pmus) * type->num_boxes, GFP_KERNEL);
+	if (!pmus)
+		return -ENOMEM;
+
+	type->pmus = pmus;
+
+	for (i = 0; i < type->num_boxes; i++) {
+		pmus[i].pmu_idx = i;
+		pmus[i].type = type;
+		pmus[i].pmu = *type->pmu;
+		sprintf((char *)pmus[i].name, "uncore_%s_%d",
+				type->name, (int)i);
+	}
+
+	return 0;
+}
+
+int __init uncore_types_init(struct ppc64_uncore_type **types)
+{
+	int i, ret;
+
+	for (i = 0; types[i]; i++) {
+		ret = uncore_type_init(types[i]);
+		if (ret)
+			goto fail;
+	}
+	return 0;
+fail:
+	while (--i >= 0)
+		kfree(types[i]);
+	return ret;
+}
+
+static int __init uncore_pmus_register(void)
+{
+	struct ppc64_uncore_pmu *pmus;
+	struct pmu *pmu;
+	struct ppc64_uncore_type *type;
+	int i, j, ret;
+
+	for (i = 0; ppc64_uncore[i]; i++) {
+		type = ppc64_uncore[i];
+		for (j = 0; j < type->num_boxes; j++) {
+			pmus = &type->pmus[j];
+			pmu = &pmus->pmu;
+			pmu->attr_groups = pmus->type->attr_groups;
+			ret = perf_pmu_register(pmu, pmus->name, -1);
+		}
+	}
+
+	return 0;
+}
+
+static int __init uncore_init(void)
+{
+	int ret = 0;
+
+	if (!cur_cpu_spec->oprofile_cpu_type ||
+	    strcmp(cur_cpu_spec->oprofile_cpu_type, "ppc64/power8") ||
+	    !cpu_has_feature(CPU_FTR_HVMODE))
+		return ret;
+
+	ret = uncore_types_init(ppc64_uncore);
+	if (ret)
+		return ret;
+
+	uncore_pmus_register();
+
+	return ret;
+}
+
+device_initcall(uncore_init);
+
-- 
1.9.1

[RFC PATCH 3/7] powerpc/powernv: uncore cpumask and CPU hotplug

[Madhavan Srinivasan]

Patch to add cpumask attribute for the Nest pmu to
control per-chip counter values to be read by cpus.
Also adds support of cpu hotplug.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/uncore_pmu.c | 152 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 152 insertions(+)

diff --git a/arch/powerpc/perf/uncore_pmu.c b/arch/powerpc/perf/uncore_pmu.c
index cc544d3..67ab6c0 100644
--- a/arch/powerpc/perf/uncore_pmu.c
+++ b/arch/powerpc/perf/uncore_pmu.c
@@ -19,6 +19,32 @@
 struct ppc64_uncore_type *empty_uncore[] = { NULL, };
 struct ppc64_uncore_type **ppc64_uncore = empty_uncore;
 
+/* mask of cpus that collect uncore events */
+static cpumask_t uncore_cpu_mask;
+
+static ssize_t uncore_get_attr_cpumask(struct device *dev,
+			struct device_attribute *attr, char *buf)
+{
+	return cpumap_print_to_pagebuf(true, buf, &uncore_cpu_mask);
+}
+
+/*
+ * cpumask attr used by perf userspace to pick the cpus to execute
+ * in case of -a option. User can still specify -C option to override.
+ * Since these Nest Counters are per-chip, make only one cpu from chip
+ * to read.
+ */
+static DEVICE_ATTR(cpumask, S_IRUGO, uncore_get_attr_cpumask, NULL);
+
+static struct attribute *uncore_pmu_attrs[] = {
+	&dev_attr_cpumask.attr,
+	NULL,
+};
+
+static struct attribute_group uncore_pmu_attr_group = {
+	.attrs = uncore_pmu_attrs,
+};
+
 struct ppc64_uncore_pmu *uncore_event_to_pmu(struct perf_event *event)
 {
 	return container_of(event->pmu, struct ppc64_uncore_pmu, pmu);
@@ -43,6 +69,7 @@ int __init uncore_type_init(struct ppc64_uncore_type *type)
 				type->name, (int)i);
 	}
 
+	type->pmu_group = &uncore_pmu_attr_group;
 	return 0;
 }
 
@@ -82,6 +109,130 @@ static int __init uncore_pmus_register(void)
 	return 0;
 }
 
+static void
+uncore_change_context(struct ppc64_uncore_type **uncores,
+				int old_cpu, int new_cpu)
+{
+	struct ppc64_uncore_type *type;
+	struct ppc64_uncore_pmu *pmu;
+	int i, j;
+
+	for (i = 0; uncores[i]; i++) {
+		type = uncores[i];
+		for (j = 0; j < type->num_boxes; j++) {
+			pmu = &type->pmus[j];
+			if (old_cpu < 0)
+				continue;
+			if (new_cpu >= 0) {
+				perf_pmu_migrate_context(&pmu->pmu,
+					old_cpu, new_cpu);
+			}
+		}
+	}
+}
+
+static void uncore_event_init_cpu(int cpu)
+{
+	int i, phys_id;
+
+	phys_id = topology_physical_package_id(cpu);
+	for_each_cpu(i, &uncore_cpu_mask) {
+		if (phys_id == topology_physical_package_id(i))
+			return;
+	}
+
+	cpumask_set_cpu(cpu, &uncore_cpu_mask);
+
+	uncore_change_context(ppc64_uncore, -1, cpu);
+}
+
+static void uncore_event_exit_cpu(int cpu)
+{
+	int i, phys_id, target;
+
+	/* if exiting cpu is used for collecting uncore events */
+	if (!cpumask_test_and_clear_cpu(cpu, &uncore_cpu_mask))
+		return;
+
+	/* find a new cpu to collect uncore events */
+	phys_id = topology_physical_package_id(cpu);
+	target = -1;
+	for_each_online_cpu(i) {
+		if (i == cpu)
+			continue;
+		if (phys_id == topology_physical_package_id(i)) {
+			target = i;
+			break;
+		}
+	}
+
+	/* migrate uncore events to the new cpu */
+	if (target >= 0)
+		cpumask_set_cpu(target, &uncore_cpu_mask);
+
+	uncore_change_context(ppc64_uncore, cpu, target);
+}
+
+static int uncore_cpu_notifier(struct notifier_block *self,
+				unsigned long action, void *hcpu)
+{
+	unsigned int cpu = (long)hcpu;
+
+	/* select the cpu that collects uncore events */
+	switch (action & ~CPU_TASKS_FROZEN) {
+	case CPU_DOWN_FAILED:
+	case CPU_STARTING:
+		uncore_event_init_cpu(cpu);
+		break;
+	case CPU_DOWN_PREPARE:
+		uncore_event_exit_cpu(cpu);
+		break;
+	default:
+		break;
+	}
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block uncore_cpu_nb = {
+	.notifier_call  = uncore_cpu_notifier,
+	/*
+	 * to migrate uncore events, our notifier should be executed
+	 * before perf core's notifier.
+	 */
+	.priority	= CPU_PRI_PERF + 1,
+};
+
+static void __init cpumask_per_chip_init(void)
+{
+	int cpu;
+
+	if (!cpumask_empty(&uncore_cpu_mask))
+		return;
+
+	cpu_notifier_register_begin();
+
+	for_each_online_cpu(cpu) {
+		int i, phys_id = topology_physical_package_id(cpu);
+
+		for_each_cpu(i, &uncore_cpu_mask) {
+			if (phys_id == topology_physical_package_id(i)) {
+				phys_id = -1;
+				break;
+			}
+		}
+		if (phys_id < 0)
+			continue;
+
+		uncore_event_init_cpu(cpu);
+	}
+
+	__register_cpu_notifier(&uncore_cpu_nb);
+
+	cpu_notifier_register_done();
+}
+
+
 static int __init uncore_init(void)
 {
 	int ret = 0;
@@ -95,6 +246,7 @@ static int __init uncore_init(void)
 	if (ret)
 		return ret;
 
+	cpumask_per_chip_init();
 	uncore_pmus_register();
 
 	return ret;
-- 
1.9.1

[RFC PATCH 4/7]powerpc/powernv: Add OPAL support for Nest pmu

[Madhavan Srinivasan]

Nest Counters can be configured via PORE Engine and OPAL
provides an interface call to it. PORE Engine also does the 
work of moving the counter data to memory.  

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/opal.h                | 3 +++
 arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
 2 files changed, 4 insertions(+)

diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9ee0a30..8ac586b 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -169,6 +169,7 @@ struct opal_sg_list {
 #define OPAL_IPMI_SEND				107
 #define OPAL_IPMI_RECV				108
 #define OPAL_I2C_REQUEST			109
+#define OPAL_UNCORE_CONTROL			114
 
 /* Device tree flags */
 
@@ -932,6 +933,8 @@ int64_t opal_ipmi_recv(uint64_t interface, struct opal_ipmi_msg *msg,
 int64_t opal_i2c_request(uint64_t async_token, uint32_t bus_id,
 			 struct opal_i2c_request *oreq);
 
+int64_t opal_uncore_control(uint32_t value);
+
 /* Internal functions */
 extern int early_init_dt_scan_opal(unsigned long node, const char *uname,
 				   int depth, void *data);
diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
index 0509bca..df709c3 100644
--- a/arch/powerpc/platforms/powernv/opal-wrappers.S
+++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
@@ -292,3 +292,4 @@ OPAL_CALL(opal_tpo_read,			OPAL_READ_TPO);
 OPAL_CALL(opal_ipmi_send,			OPAL_IPMI_SEND);
 OPAL_CALL(opal_ipmi_recv,			OPAL_IPMI_RECV);
 OPAL_CALL(opal_i2c_request,			OPAL_I2C_REQUEST);
+OPAL_CALL(opal_uncore_control,			OPAL_UNCORE_CONTROL);
-- 
1.9.1

[RFC PATCH 5/7]powerpc/powernv: Add POWER8 specific nest pmu support

[Madhavan Srinivasan]

Patch enables POWER8 specific nest pmu support. It defines
pmu functions in a generic way that it can be shared across 
different nest units. Event id is used, to identify the 
offset in memory to read from. And the offset information
is saved in the per-chip data strucutres which are populated
at the time of device-tree parsing.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/uncore_pmu.c    |   4 +
 arch/powerpc/perf/uncore_pmu_p8.c | 167 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 171 insertions(+)
 create mode 100644 arch/powerpc/perf/uncore_pmu_p8.c

diff --git a/arch/powerpc/perf/uncore_pmu.c b/arch/powerpc/perf/uncore_pmu.c
index 67ab6c0..504c6ac 100644
--- a/arch/powerpc/perf/uncore_pmu.c
+++ b/arch/powerpc/perf/uncore_pmu.c
@@ -242,6 +242,10 @@ static int __init uncore_init(void)
 	    !cpu_has_feature(CPU_FTR_HVMODE))
 		return ret;
 
+	ret = uncore_p8_init();
+	if (ret)
+		return ret;
+
 	ret = uncore_types_init(ppc64_uncore);
 	if (ret)
 		return ret;
diff --git a/arch/powerpc/perf/uncore_pmu_p8.c b/arch/powerpc/perf/uncore_pmu_p8.c
new file mode 100644
index 0000000..411c077
--- /dev/null
+++ b/arch/powerpc/perf/uncore_pmu_p8.c
@@ -0,0 +1,167 @@
+/*
+ * Uncore Performance Monitor counter support for POWER8 processors.
+ *
+ * Copyright 2015, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version
+ * 2 of the License
+ */
+
+#include "uncore_pmu.h"
+
+#define P8_UNCORE_EVENT_MASK    0xFF
+#define P8_UNCORE_ENGINE_START	0x1
+#define P8_UNCORE_ENGINE_STOP	0
+
+extern struct ppc64_uncore_type **ppc64_uncore;
+static struct ppc64_uncore_unit uncore_per_chip[P8_MAX_CHIP];
+
+struct attribute *p8_uncore_event_attrs[MAX_TYPE_EVENTS];
+struct ppc64_uncore_type *p8_uncore[MAX_UNITS_SUPPORTED];
+
+/*
+ * percpu variable for refcount for uncore events.
+ */
+DEFINE_PER_CPU(uint32_t, uncore_refcnt);
+
+PMU_FORMAT_ATTR(event, "config:0-7");
+
+static struct attribute *p8_uncore_format_attrs[] = {
+	&format_attr_event.attr,
+	NULL,
+};
+
+static struct attribute_group p8_uncore_format_group = {
+	.name = "format",
+	.attrs = p8_uncore_format_attrs,
+};
+
+int p8_uncore_event_init(struct perf_event *event)
+{
+	struct ppc64_uncore_pmu *pmu;
+	struct ppc64_uncore_type *type;
+	int chip_id, cfg;
+
+	if (event->attr.type != event->pmu->type)
+		return -ENOENT;
+
+	/* Sampling not supported yet */
+	if (event->hw.sample_period)
+		return -EINVAL;
+
+	/* unsupported modes and filters */
+	if (event->attr.exclude_user   ||
+	    event->attr.exclude_kernel ||
+	    event->attr.exclude_hv     ||
+	    event->attr.exclude_idle   ||
+	    event->attr.exclude_host   ||
+	    event->attr.exclude_guest  ||
+	    event->attr.sample_period) /* no sampling */
+		return -EINVAL;
+
+	if (event->cpu < 0)
+		return -EINVAL;
+
+	pmu = uncore_event_to_pmu(event);
+	if (!pmu)
+		return -EINVAL;
+
+	if (event->attr.config & ~P8_UNCORE_EVENT_MASK)
+		return -EINVAL;
+
+	/* Event to look for in the offset strucutre */
+	cfg = event->attr.config & P8_UNCORE_EVENT_MASK;
+
+	type = pmu->type;
+	chip_id = topology_physical_package_id(event->cpu);
+
+	/* Mem access address calculation for this event */
+	event->hw.event_base = uncore_per_chip[chip_id].vreg_base;
+	event->hw.event_base += type->event_arry[pmu->pmu_idx].ev_offset[(cfg - 1)];
+
+	return 0;
+}
+
+void p8_uncore_read_counter(struct perf_event *event)
+{
+	uint64_t *ptr;
+
+	ptr = (uint64_t *)event->hw.event_base;
+	local64_set(&event->hw.prev_count, __be64_to_cpu((uint64_t)*ptr));
+}
+
+void p8_uncore_perf_event_update(struct perf_event *event)
+{
+	u64 counter_prev, counter_new, final_count;
+	uint64_t *ptr;
+
+	ptr = (uint64_t *)event->hw.event_base;
+	counter_prev = cpu_to_be64(local64_read(&event->hw.prev_count));
+	counter_new = __be64_to_cpu((uint64_t)*ptr);
+	final_count = counter_new - counter_prev;
+
+	local64_set(&event->hw.prev_count, counter_new);
+	local64_add(final_count, &event->count);
+}
+
+void p8_uncore_event_start(struct perf_event *event, int flags)
+{
+	uint32_t *refcnt = &get_cpu_var(uncore_refcnt);
+
+	event->hw.state = 0;
+	*refcnt += 1;
+
+	if (*refcnt == 1)
+		opal_uncore_control(P8_UNCORE_ENGINE_START);
+
+	p8_uncore_read_counter(event);
+	put_cpu_var(uncore_refcnt);
+}
+
+void p8_uncore_event_stop(struct perf_event *event, int flags)
+{
+	uint32_t *refcnt = &get_cpu_var(uncore_refcnt);
+
+	*refcnt -= 1;
+	if (*refcnt == 0)
+		opal_uncore_control(P8_UNCORE_ENGINE_STOP);
+
+	p8_uncore_perf_event_update(event);
+	put_cpu_var(uncore_refcnt);
+}
+
+int p8_uncore_event_add(struct perf_event *event, int flags)
+{
+	p8_uncore_event_start(event, flags);
+	return 0;
+}
+
+void p8_uncore_event_del(struct perf_event *event, int flags)
+{
+	p8_uncore_event_stop(event, flags);
+}
+
+struct pmu p8_uncore_pmu = {
+	.task_ctx_nr	= perf_invalid_context,
+	.event_init	= p8_uncore_event_init,
+	.add		= p8_uncore_event_add,
+	.del		= p8_uncore_event_del,
+	.start		= p8_uncore_event_start,
+	.stop		= p8_uncore_event_stop,
+	.read		= p8_uncore_perf_event_update,
+};
+
+static int uncore_init(void)
+{
+	return 0;
+}
+
+int uncore_p8_init(void)
+{
+	ppc64_uncore = p8_uncore;
+	return uncore_init();
+}
+
+
-- 
1.9.1

[RFC PATCH 6/7]powerpc/powernv: add support to parse dt for nest pmu

[Madhavan Srinivasan]

Nest counter unit informations are passed on as part of device-tree
to kernel. Here is the Device-tree layout.

DT root folder  /:
 /
 |
 -uncore@<chip-id>
   |
   -phb  <--dt node for pcie host bridge (uncore pmu type)
   -...
   -mcs  <--dt node for memory controller (uncore pmu type)
     |
     -mcs@0   <-- dt node for memory controller sub units
     -mcs@1
     -mcs@2
     -mcs@3
      |
      - mcs_read                <-- Sample event file
      - unit.mcs_read.unit      <-- Sample event unit file.
      - scale.mcs_read.scale    <-- Sample event scale file

Patch implements generic function to creates type structure and populate 
events and event related information from DT. For each type structure, 
new pmu is registered.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/uncore_pmu_p8.c | 240 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 238 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/perf/uncore_pmu_p8.c b/arch/powerpc/perf/uncore_pmu_p8.c
index 411c077..6a347d6 100644
--- a/arch/powerpc/perf/uncore_pmu_p8.c
+++ b/arch/powerpc/perf/uncore_pmu_p8.c
@@ -153,8 +153,246 @@ struct pmu p8_uncore_pmu = {
 	.read		= p8_uncore_perf_event_update,
 };
 
+/*
+ * Populate event name and string in attribute
+ */
+struct attribute *dev_str_attr(char *name, char *str)
+{
+	struct perf_pmu_events_attr *attr;
+
+	attr = kzalloc(sizeof(*attr), GFP_KERNEL);
+
+	attr->event_str = (const char *)str;
+	attr->attr.attr.name = name;
+	attr->attr.attr.mode = 0444;
+	attr->attr.show = perf_event_sysfs_show;
+
+	return &attr->attr.attr;
+}
+
+int update_uncore_unit_event_group(
+		struct ppc64_uncore_type *ptr)
+{
+	struct attribute_group *attr_group;
+	struct attribute **attrs;
+	struct ppc64_uncore_type_events *ev_arry = ptr->event_arry;
+	char *ev_val;
+	int i, j;
+
+	/*
+	 * Count the total # of events for this uncore pmu
+	 */
+	for (j = 0; ev_arry[0].ev_offset[j] != 0; j++)
+		;
+
+	attr_group = kzalloc(sizeof(struct attribute *) * (j) +
+		sizeof(*attr_group), GFP_KERNEL);
+	if (!attr_group)
+		return -ENOMEM;
+
+	attrs = (struct attribute **)(attr_group + 1);
+	attr_group->name = "events";
+	attr_group->attrs = attrs;
+
+	for (i = 0; i < j; i++) {
+		if (ev_arry[0].ev_offset[i] < 0) {
+			attrs[i] = dev_str_attr((char *)ev_arry[0].ev_name[i], (char *)ev_arry[0].ev_value[i]);
+		} else {
+			ev_val = kzalloc(MAX_PMU_NAME_LEN, GFP_KERNEL);
+			if (!ev_val)
+				return -ENOMEM;
+			sprintf((char *)ev_val, "event=0x%02x", (i+1));
+			attrs[i] = dev_str_attr((char *)ev_arry[0].ev_name[i], ev_val);
+		}
+	}
+
+	ptr->events_group = attr_group;
+	return 0;
+}
+
+/*
+ * Parse the child node and identify events associated
+ * with this pmu.
+ */
+static int populate_events(struct device_node *dev,
+			struct ppc64_uncore_type *ptr, int idx)
+{
+	struct ppc64_uncore_type_events *ev_arry = ptr->event_arry;
+	struct property *pp;
+	int ev_idx = 0, len = 0;
+	const char *start, *end, *out_str;
+	char *buf;
+	const __be32 *lval;
+	u32 val;
+
+	/*
+	 * Loop through each property
+	 */
+	for_each_property_of_node(dev, pp) {
+		start = pp->name;
+		end = start + strlen(start);
+		len = strlen(start);
+
+		/* Skip these, we dont need it */
+		if (!strcmp(pp->name, "name") ||
+		    !strcmp(pp->name, "phandle") ||
+		    !strcmp(pp->name, "ibm,dev-id") ||
+		    !strcmp(pp->name, "linux,phandle"))
+			continue;
+
+		buf = kzalloc(MAX_PMU_NAME_LEN, GFP_KERNEL);
+		if (!buf)
+			return -ENOMEM;
+
+		/*
+		 * Event may have corresponding unit and scale
+		 * information. Incase of unit and scale, populate
+		 * the offset value to negative.
+		 */
+		if (strncmp(pp->name, "unit.", 5) == 0) {
+			of_property_read_string(dev, pp->name, &out_str);
+			start += 5;
+			len = strlen(start);
+			strncpy(buf, start, strlen(start));
+			of_property_read_string(dev, pp->value, &out_str);
+
+			ev_arry[idx].ev_offset[ev_idx] = -1;
+			ev_arry[idx].ev_name[ev_idx] = buf;
+			ev_arry[idx].ev_value[ev_idx] = out_str;
+			ev_idx++;
+			continue;
+		}
+
+		if (strncmp(pp->name, "scale.", 6) == 0) {
+			of_property_read_string(dev, pp->name, &out_str);
+			start += 6;
+			len = strlen(start);
+			strncpy(buf, start, strlen(start));
+			of_property_read_string(dev, pp->value, &out_str);
+
+			ev_arry[idx].ev_offset[ev_idx] = -1;
+			ev_arry[idx].ev_name[ev_idx] = buf;
+			ev_arry[idx].ev_value[ev_idx] = out_str;
+			ev_idx++;
+			continue;
+		}
+
+		/*
+		 * Event support by this pmu.
+		 */
+		lval = of_get_property(dev, pp->name, NULL);
+		val = (uint32_t)be32_to_cpup(lval);
+		strncpy(buf, start, len);
+		ev_arry[idx].ev_offset[ev_idx] = val;
+		ev_arry[idx].ev_name[ev_idx] = buf;
+		ev_arry[idx].ev_value[ev_idx] = "\0";
+		ev_idx++;
+	}
+	return 0;
+}
+
+static int uncore_unit_parse(struct device_node *parent,
+				struct ppc64_uncore_type *ptr)
+{
+	struct device_node *child;
+	const __be32 *unit_idx;
+	int idx;
+
+	for_each_available_child_of_node(parent, child) {
+		unit_idx = of_get_property(child, "ibm,dev-id", NULL);
+		idx = (uint32_t)be32_to_cpup(unit_idx);
+		if (populate_events(child, ptr, idx))
+			return -1;
+	}
+
+	if (update_uncore_unit_event_group(ptr))
+		return -1;
+
+	return 0;
+}
+
+static int uncore_create_types(struct device_node *parent)
+{
+	struct device_node *dev_type;
+	struct ppc64_uncore_type *uncore_type;
+	const __be32 *uncore_type_units;
+	int idx = 0, i;
+
+	for_each_child_of_node(parent, dev_type) {
+		uncore_type = (struct ppc64_uncore_type *)
+				kzalloc(sizeof(struct ppc64_uncore_type),
+					GFP_KERNEL);
+		if (!uncore_type)
+			goto fail;
+
+		/*
+		 * Populate the uncore pmu type name and
+		 * # of sub units.
+		 */
+		uncore_type->name = of_get_property(dev_type,
+					"name", NULL);
+		uncore_type_units = of_get_property(dev_type,
+					"sub_units", NULL);
+		uncore_type->num_boxes = be32_to_cpup(uncore_type_units);
+
+		if (uncore_unit_parse(dev_type, uncore_type))
+			goto fail;
+
+		uncore_type->format_group = &p8_uncore_format_group;
+		uncore_type->pmu = &p8_uncore_pmu;
+
+		p8_uncore[idx++] = uncore_type;
+	}
+
+	return 0;
+
+fail:
+	for (i = 0; i < idx; i++)
+		kfree(p8_uncore[i]);
+	return -1;
+}
+
 static int uncore_init(void)
 {
+	struct device_node *dev;
+	const __be32 *gcid;
+	const __be64 *chip_ima_reg;
+	const __be32 *chip_ima_size;
+	int idx = 0;
+
+	dev = of_find_node_with_property(NULL, "ibm,uncore");
+	if (!dev)
+		return -EINVAL;
+
+	for_each_node_with_property(dev, "ibm,uncore") {
+		gcid = of_get_property(dev, "ibm,chip-id", NULL);
+		chip_ima_reg = of_get_property(dev, "reg", NULL);
+		chip_ima_size = of_get_property(dev, "size", NULL);
+		if ((!gcid) || (!chip_ima_reg) || (!chip_ima_size)) {
+			pr_err("%s: device %s missing property \n",
+				__func__, dev->full_name);
+			return -EINVAL;
+		}
+
+		/*
+		 * Get the chip id, populate
+		 * address range/size and translate it.
+		 */
+		idx = (uint32_t)be32_to_cpup(gcid);
+		uncore_per_chip[idx].chip_id = (uint32_t)be32_to_cpup(gcid);
+		uncore_per_chip[idx].preg_base = be64_to_cpup(chip_ima_reg);
+		uncore_per_chip[idx].size = be32_to_cpup(chip_ima_size);
+		uncore_per_chip[idx].vreg_base = (u64)ioremap(
+			(phys_addr_t)uncore_per_chip[idx].preg_base,
+				uncore_per_chip[idx].size);
+
+		/*
+		 * looks for uncore pmu types supported here.
+		 */
+		if (uncore_create_types(dev))
+			return -EINVAL;
+	}
+
 	return 0;
 }
 
@@ -163,5 +401,3 @@ int uncore_p8_init(void)
 	ppc64_uncore = p8_uncore;
 	return uncore_init();
 }
-
-
-- 
1.9.1

[RFC PATCH 7/7]powerpc/powernv: enable nest pmu related file in Makefile

[Madhavan Srinivasan]

Patch to enable the nest pmu code files in Makefile.

Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
 arch/powerpc/perf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/perf/Makefile b/arch/powerpc/perf/Makefile
index f9c083a..4b0bfd8 100644
--- a/arch/powerpc/perf/Makefile
+++ b/arch/powerpc/perf/Makefile
@@ -5,7 +5,7 @@ obj-$(CONFIG_PERF_EVENTS)	+= callchain.o
 obj-$(CONFIG_PPC_PERF_CTRS)	+= core-book3s.o bhrb.o
 obj64-$(CONFIG_PPC_PERF_CTRS)	+= power4-pmu.o ppc970-pmu.o power5-pmu.o \
 				   power5+-pmu.o power6-pmu.o power7-pmu.o \
-				   power8-pmu.o
+				   power8-pmu.o uncore_pmu.o uncore_pmu_p8.o
 obj32-$(CONFIG_PPC_PERF_CTRS)	+= mpc7450-pmu.o
 
 obj-$(CONFIG_FSL_EMB_PERF_EVENT) += core-fsl-emb.o
-- 
1.9.1

Re: [RFC PATCH 4/7]powerpc/powernv: Add OPAL support for Nest pmu

[Stewart Smith]

Madhavan Srinivasan <maddy@linux.vnet.ibm.com> writes:
> Nest Counters can be configured via PORE Engine and OPAL
> provides an interface call to it. PORE Engine also does the 
> work of moving the counter data to memory.  

Do you have the associated skiboot patch that implements this firmware
call? I haven't seen it on the skiboot list yet :)

Re: [RFC PATCH 4/7]powerpc/powernv: Add OPAL support for Nest pmu

[maddy]

On Thursday 12 March 2015 04:27 AM, Stewart Smith wrote:
> Madhavan Srinivasan <maddy@linux.vnet.ibm.com> writes:
>> Nest Counters can be configured via PORE Engine and OPAL
>> provides an interface call to it. PORE Engine also does the
>> work of moving the counter data to memory.
> Do you have the associated skiboot patch that implements this firmware
> call? I haven't seen it on the skiboot list yet :)
Hi Stewart,

     OPAL side code is under-developement. Will post the patches soon to 
the skiboot mailing list.

Regards
Maddy