Difference between revisions of "Roadmap"

Latest revision as of 17:17, 5 April 2013

Description

Life goes on. Which features are planned in future releases? This page is more a list of ideas than a fixed plan for the future, but it may give you an idea where the journey is going. If you want to contribute but don't know what is to be done, simply drop us a short mail and we'll get you started.

This file is not automatically generated and may thus be out of date from time to time. To give you an idea how fresh this information is please take a look at its history.

Implemented features

The following features have been implemented and will be released with the next minor version. You can check out our Git repository to test these features. They can be found in the master branch. Any feedback and bug reports are welcome!

Varnish plugin

Jérôme Renard has implemented a plugin which reads statistics from Varnish, a caching proxy server for web applications. His code is available from jeromer/collectd. —octo 11:17, 9 June 2010 (UTC)

Redis plugin

A new Redis plugin has been written by Andrés J. Díaz. It connects to one or more Redis databases and queries statistics such as commands received, memory allocated and connected clients. The code was merged into the master branch on August 17.

Write Redis plugin

A plugin for writing collectd data to a Redis database was written by Florian Forster. It was merged into the master branch on August 17.

Thresholds

Andrés J. Díaz has written a patch that introduces a global timeout_g variable rather than the hard-coded default (two). The patch has been sent to the mailing list on December 17, 2009.

• Available in the master branch since August 25. —octo 00:11, 4 November 2010 (UTC)

AMQP plugin

→ spahl/collectd/amqp, http://github.com/spahl/collectd/commits/amqp

Sebastien Pahl has written a write plugin which sends values via the Advanced Message Queuing Protocol (AMQP). It is using librabbitmq to do so. The code is available from the amqp branch on github.

• I've rebase the code to the master branch and contacted Sebastien for some feedback. —octo 13:26, 4 August 2010 (UTC)
• Sending and receiving data via AMQP is now possible. Possible future todos:
  • Use the parsetree interface of YAJL to parse received JSON data. Currently, the parsetree code has not yet been merged into YAJL's master branch.
  • Possibly rename the AMQP plugin to Write AMQP. For consistency's sake it might be best to rename the CSV plugin, RRDtool plugin and RRDCacheD plugin, too. —octo 06:55, 18 August 2010 (UTC)
    • No, since the AMQP plugin can send and receive values, renaming the plugin is not appropriate. —octo 20:43, 4 September 2010 (UTC)
• Merged to the master branch. Parsing JSON still not possible, though. —octo 08:42, 6 November 2010 (UTC)

LPAR plugin

Aurélien Reynaud has sent a patch for an Logical Partitions (LPAR) plugin to the mailing list on August 10. It collects information about the (a?) virtualization infrastructure of AIX. —octo 07:23, 18 August 2010 (UTC)

• Merged to the master branch. —octo 08:44, 6 November 2010 (UTC)

Per-plugin interval configuration

Sebastian 'tokkee' Harl implemented per-plugin interval support. It is included in the core since version 5.2.

# foobar is updated every 42 seconds.
<LoadPlugin "foobar">
  Interval 42
</LoadPlugin>
# qux is updated with the global default interval.
LoadPlugin "qux"

Planned features

The following features are actively being worked upon. They are in various stages of completion and may or may not make it into the next release. They should have a contact (name) associated with them, so people who are interested can contact the person working on a feature. Please don't add email addresses of other people without their permission. If you want to contact a person below, please look for the email address using the mailing list archives.

Plans for 5.0

Backwards incompatible changes have been collected on the Version 5.0/Plans page. Make sure every point raised there has either been addressed or has actively been dismissed (i.e. not forgotten about).

perfmon output

Dave McLellan is looking into a text output plugin which creates perfmon compatible output (last contact: April 15^th, 2009). See also: #output_text_plugin

Library for binary network protocol

Write a library for the binary network protocol, so that external applications have an easier job interfacing with collectd. The new authentication and encryption feature (added in version 4.7) makes this even more desirable. Octo is working on it.

• Sending plain and signed network packets is now possible. The code is available from the ff/netlib branch. --octo 06:51, 18 August 2010 (UTC)
• Sending encrypted packages is now possible, too. —octo 13:37, 19 August 2010 (UTC)

FreeSWITCH plugin

See also: ldr/collectd-freeswitch

Plugin which connects to FreeSWITCH, a free IP-telephony software, issues commands periodically and parses the returned output. The plugin was originally written by Leon de Rooij and is currently available from the ldr/freeswitch branch.

Match and target for meta data

It'd be nice to match values by the meta data attached to it and to attach or re-set meta data for values. At least something simple (see the MARK match and target in ip_tables) would be nice.

Event infrastructure

Add an EVENT command to the UnixSock plugin. This command can be used to count events, such as page loads and the like. The goal is to make it possible for applications to implement statistics collection with the minimum of overhead. powdahound is looking into it.

NetCmd plugin

The UnixSock plugin is limited to UNIX domain sockets. It'd be great if it was possible to access this infrastructure from remote hosts, for example for receiving values. Some initial code lives in the ff/netcmd branch but isn't ready yet. The idea is basically to take the UnixSock plugin and make it possible to use network sockets, too. Several people have expressed interest in picking that up, but so far no progress has been made. It's safest to keep development cycles short and provide many small patches as soon as possible.

Erlang plugin

octo is working on an Erlang plugin. The plugin implements a C-node which can communicate with Erlang processes using message-passing. The code is available in the ff/erlang branch.

InnoDB addon to MySQL plugin

Morten Hanshaugen is working on adding InnoDB statistics to the MySQL plugin. Last contact was on January 27^th via email.

SNORT plugin

Benjamin Petrin is working on a plugin that will gather statistics from SNORT, an open source network intrusion prevention and detection system.

WPAR plugin

Manuel Luis Sanmartín Rozada has sent patches for a workload partitioning (WPAR) plugin for AIX to the mailing list on July 20th. —octo 13:40, 4 August 2010 (UTC)

The patches are now available from the octo/collectd/ms/wpar branch on Github. —octo 14:02, 4 August 2010 (UTC)

MongoDB plugin

There's a MongoDB plugin by Ryan Cox available from ryancox/collectd. The changes have since been rebased to the octo/collectd/rc/mongodb branch on Github. —octo 00:07, 4 November 2010 (UTC)

Write MongoDB plugin

A write plugin for MongoDB has been developed by Akkarit Sangpetch. The current version is available from the octo/collectd/as/mongodb branch on Gibhub. —octo 00:07, 4 November 2010 (UTC)

ZeroMQ plugin

Julien Ammous is working on a communication plugin which uses ØMQ (aka. ZeroMQ) as transport mechanism. Apparently it works, but it'd be great if it was merged with the the ff/netlib branch (see: #Library for binary network protocol) before including it in master. The code lives in the zeromq branch on Github. —octo 08:12, 5 November 2010 (UTC)

Lua plugin

Julien Ammous started writing a plugin which will eventually allow users to write plugins in the scripting language Lua. The code is available from the ja/lua branch and schmurfy/collectd/lua. —octo 08:02, 26 November 2010 (UTC)

Modbus plugin

Ivo De Decker has implemented a grouping feature for the <Data /> blocks of the Modbus plugin. The code is available from the id/modbus branch. Last contact in a private email on January 12th, 2011. —octo 10:25, 14 January 2011 (UTC)

Gearman

Find out what this does and expand the page. —octo 22:32, 11 March 2011 (UTC)

Gearman allows to distribute work-loads to multiple workers. A (Gearman) client dispatches jobs to the Gearman job server which then distributes them to Gearman workers. The worker executes the requested task and sends back the result to the client (thru the Gearman job server). I'm not sure how collectd might benefit from that. -- tokkee 11:58, 7 July 2011 (UTC)

It might make (some) sense to implement a gearman worker plugin which receives Nagios perfdata sent by the Nagios mod-gearman add-on and dispatches that data to collectd. I think the one challenging question is how to do the data mapping. The following approaches come to my mind:

• Use the same approach as in most generic plugins, i.e. let the user specify each and every data set (check command and/or service description, perfdata label) plus how to convert that info collectd types.
• Use a similar approach to PNP4Nagios, i.e. use default values (e.g. host = Nagios hostname, plugin = mod_gearman, plugin instance = check command or service description, type = gauge, type instance = perfdata label) and let the user overwrite (some of those) information, c.f. PNP's custom templates.
• Combine the two approaches by defining a default data-mapping and (maybe) let other mappings "inherit" from that.

tokkee 10:40, 9 January 2013 (UTC)

LXC plugin

François-Xavier Bourlet is working on a LXC plugin. LXC is a linux-specific containers implementation (à la OpenVZ) based on kernel cgroups and namespaces. The code is available from dotcloud/collectd on Github.

Wishlist / Ideas

The following is a list of ideas we had but so far nobody has stepped up to implement them. Feel free to add your own “cool stuff” :)

RRDtool plugin: Type-specific configuration

This would allow to keep interesting data longer and less interesting data only for a short time. Also seasonal data could be added. It's not reasonable to add seasonal data to all RRD files, because it operates on unconsolidated primary datapoints (thus taking up a lot of disk space) and you can't specify one seasonal period that fits all kinds of data. A discussion of this feature can be found in the mailinglist's archive.

Maybe an option that can be passed to write functions by the Chains mechanism would be a good idea for implementing this.

Distributed setups

Make it possible to have several "servers" with shared storage. When one server dies another one takes its place, so that you don't have gaps in your statistics. This is probably best implemented as a match that matches only those values the host is responsible for. A really simple first step is available in form of the Hashed match as of version 4.9. Adding automatic fail-over would be a nice second step.

A solution for non-shared storage is probably even harder to implement and likely not worth the effort.

This can be done with DRBD together with Linux HA. A lot of solutions based on this thechnologies works very reliable in production environments.

Embed more scripting languages

ECMA Common Language Infrastructure

Work in progress, for information contact octo.

Best known implementation is likely Microsoft .NET; for UNIX there's a free implementation called Mono.

Target language of a variety of programming languages, for example C#.

http://mono-project.com/

http://mono-project.com/Embedding_Mono

JavaScript

It looks like SpiderMonkey is relatively easy to embed. Using JavaScript may especially interesting for implementing matches and targets, though it's likely to compete with Lua in this respect.

https://developer.mozilla.org/en/How_to_embed_the_JavaScript_engine

Lua

Work in progress, see #Lua plugin above.

http://www.lua.org/

Parrot

It is now (November 2010) at version 2.10 and provides a stable API. Whether the interpreter has become thread- and reentrant-safe has to be determined again.

http://www.parrot.org/

Ruby

http://www.ruby-lang.org/

Store to RDBMS

Being able to store values in an RDBMS would be nice. Bob Cotton has put together some SQL for PostgreSQL and a Perl based plugin to store values into it. This could serve as a starting point as a C based plugin with the same SQL backend.

See also: http://github.com/bcotton/collectd_dbstore/tree/master

Starting with version 5.2, the PostgreSQL plugin supports storing data in PostgreSQL. This has been done using a generic approach by letting the user specify the query that is used for storing a value-list and, thus, allowing arbitrary database layouts. See the manpage for details.

Query wprobe

wprobe is a interface to query/manage wireless links, written by the guys over at OpenWrt.

See also: https://dev.openwrt.org/browser/trunk/package/wprobe

output_text plugin

Sebastian has suggested a output_text plugin that can write values to arbitrarily formatted (text-)files. See also: #perfmon_output

Apache plugin: Per virtualhost stats

mod_watch gives access to stats like:

domain.com 8190 539056 20965428 4813 4813 0 14.520 0 4286618408
domain2.com 8190 539056 20965428 4813 4813 0 14.520 0 4286618408

Why not to use it?

Couldn't find a hopepage for this project. Where does it live? Is it still alive? -- octo 21:16, 22 September 2009 (UTC)

pf (packetfilter) plugin

pf is a firewall available on OpenBSD and FreeBSD. There are quite a lot of counters which should be available (packets/seconds, octets/seconds, by states, tables, ...). Some munin plugins for reference : pf_bytes, pf

Drop privileges

A couple of plugins require special privileges. It'd be great if we could find a way to allow these plugins to do their job without requiring collectd to run with privileges.

Under Linux, a solution might be to use capabilities(7) to deal with this problem: Assign the collectd binary the permission to a certain capability, for example to open a raw socket. The plugin can then active this capability if needed. For example:

setcap cap_net_raw=p $prefix/bin/collectd

For other UNIXes, this solution is not applicable. In order to drop privileges in the traditional UNIX way, plugins such as the Ping plugin would need to live in a separate process space, since it needs to open new sockets whenever the address of a pinged hostname changes.

Also see the discussion in Debian bug #680660.

Summarize CPU information

Implement a configuration option in the CPU plugin, which allows to dispatch the sum of all CPUs, rather than information about individual CPUs. —octo 06:31, 7 December 2010 (UTC) Please see patch http://mailman.verplant.org/pipermail/collectd/2011-March/004415.html

Also see the generic approach provided by the Aggregation plugin.

CPU percent usage

It will be nice in FreeBSD (may be other unixes) divide values by stathz and multiply by 100 to get usage more like percent usage (munin does like this, see sysctl -n kern.clockrate). In linux it can be achieved dividing to HZ value. Please see patch http://mailman.verplant.org/pipermail/collectd/2011-March/004415.html

apcupsd

In many cases on one server runned many apcupsd on different tcp ports. Plugin should be able to poll many apcupsd.

Swap plugin

Linux

The ReportByDevice option has been added to the Solaris code of the Swap plugin in version 5.0. Similar information can be read from /proc/swaps under Linux. Make this option available there, too. —octo 07:55, 2 January 2011 (UTC)

FreeBSD and OpenBSD

The prototype of the swapctl(2) function on BSD systems differs from the same function on SunOS. Its prototype is:

 int swapctl(int cmd, const void *arg, int misc);

The configure script can already check for this second version, but the Swap plugin can not yet handle it. Implement this.

More information is available from swapctl(2). —octo 13:31, 2 February 2011 (UTC)

collectd-nagios

Support querying multiple data-sets; apply consolidation to selected data-sets. --tokkee 14:46, 5 January 2011 (UTC)

Also see the ideas / discussion on Collectd-nagios.

NTPd

It'd be great to be able to see only the declared peer instead of all peer (declared and candidate). Actually we can't see which server is declared as peer.

Reverse DNS Check

XANi suggested to ignore values if the reverse DNS of the hostname submitted over the network does not match the sending IP. This could be implemented by adding the remote IP as meta-data to received value-lists in the network plugin and then add some match plugin to check that kind of information. --tokkee 06:44, 16 August 2011 (UTC)