Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andrew Lunn <andrew@lunn.ch>
To: Jonathan Adams <jwadams@google.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	netdev@vger.kernel.org, kvm@vger.kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	Jim Mattson <jmattson@google.com>,
	David Rientjes <rientjes@google.com>
Subject: Re: [RFC PATCH 0/7] metricfs metric file system and examples
Date: Sat, 8 Aug 2020 04:06:17 +0200	[thread overview]
Message-ID: <20200808020617.GD2028541@lunn.ch> (raw)
In-Reply-To: <20200807212916.2883031-1-jwadams@google.com>

> net/dev/stats/tx_bytes/annotations
>   DESCRIPTION net\ device\ transmited\ bytes\ count
>   CUMULATIVE
> net/dev/stats/tx_bytes/fields
>   interface value
>   str int
> net/dev/stats/tx_bytes/values
>   lo 4394430608
>   eth0 33353183843
>   eth1 16228847091

This is a rather small system. Have you tested it at scale? An
Ethernet switch with 64 physical interfaces, and say 32 VLAN
interfaces stack on top. So this one file will contain 2048 entries?

And generally, you are not interested in one statistic, but many
statistics. So you will need to cat each file, not just one file. And
the way this is implemented:

+static void dev_stats_emit(struct metric_emitter *e,
+                          struct net_device *dev,
+                          struct metric_def *metricd)
+{
+       struct rtnl_link_stats64 temp;
+       const struct rtnl_link_stats64 *stats = dev_get_stats(dev, &temp);
+
+       if (stats) {
+               __u8 *ptr = (((__u8 *)stats) + metricd->off);
+
+               METRIC_EMIT_INT(e, *(__u64 *)ptr, dev->name, NULL);
+       }
+}

means you are going to be calling dev_get_stats() for each file, and
there are 23 files if i counted correctly. So dev_get_stats() will be
called 47104 times, in this made up example. And this is not always
cheap, these counts can be atomic.

So i personally don't think netdev statistics is a good idea, i doubt
it scales.

I also think you are looking at the wrong set of netdev counters. I
would be more interested in ethtool -S counters. But it appears you
make the assumption that each object you are collecting metrics for
has the same set of counters. This is untrue for network interfaces,
where each driver can export whatever counters it wants, and they can
be dynamic.

	Andrew

  parent reply	other threads:[~2020-08-08  2:06 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-07 21:29 Jonathan Adams
2020-08-07 21:29 ` [RFC PATCH 1/7] core/metricfs: Create metricfs, standardized files under debugfs Jonathan Adams
2020-08-08  5:41   ` Greg KH
2020-08-07 21:29 ` [RFC PATCH 2/7] core/metricfs: add support for percpu metricfs files Jonathan Adams
2020-08-08  5:43   ` Greg KH
2020-08-07 21:29 ` [RFC PATCH 3/7] core/metricfs: metric for kernel warnings Jonathan Adams
2020-08-08  5:45   ` Greg KH
2020-08-07 21:29 ` [RFC PATCH 4/7] core/metricfs: expose softirq information through metricfs Jonathan Adams
2020-08-08  5:46   ` Greg KH
2020-08-07 21:29 ` [RFC PATCH 5/7] core/metricfs: expose scheduler stat " Jonathan Adams
2020-08-07 21:29 ` [RFC PATCH 6/7] core/metricfs: expose x86-specific irq " Jonathan Adams
2020-08-13 10:11   ` Thomas Gleixner
2020-08-13 11:47     ` Paolo Bonzini
2020-08-13 12:13       ` Thomas Gleixner
2020-08-13 14:10         ` Paolo Bonzini
2020-08-13 14:21           ` Thomas Gleixner
2020-08-07 21:29 ` [RFC PATCH 7/7] net-metricfs: Export /proc/net/dev via metricfs Jonathan Adams
2020-08-08  2:06 ` Andrew Lunn [this message]
2020-08-08 15:59   ` [RFC PATCH 0/7] metricfs metric file system and examples David Ahern
2020-08-10 18:20     ` Jakub Kicinski
2020-08-10  9:23 ` Pavel Machek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200808020617.GD2028541@lunn.ch \
    --to=andrew@lunn.ch \
    --cc=gregkh@linuxfoundation.org \
    --cc=jmattson@google.com \
    --cc=jwadams@google.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=rientjes@google.com \
    --subject='Re: [RFC PATCH 0/7] metricfs metric file system and examples' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).