A new approach for monitoring simulation data

Hello,

Ontje and I want to present and discuss a new way for monitoring SimPy
simulations. We have attached a commented sample implementation of our approach
with some usage examples.

Before we describe our idea, we want to point out some things we didn’t like
about SimPy’s ``Monitor`` and ``Tally`` classes:

* They use global variables.
* If you monitor multiple attributes, timestamps are stored with each of them
  (data duplication, requires more memory as necessary).
* They combine data collection and analysis in one class. These are different
  functional aspects and should in our opinion be separated.
* Coupling with Histogram and GUI stuff.

We never used any of both for our projects, because they never really fitted our
needs. So we debated intensely how an optimal monitoring would look like. It
should meet the following requirements:

* Easy to use (simple API, little typing)
* Memory and CPU efficiency

  * No impact on simulation speed if you don’t use it.
  * As little impact as possible if you use it.

* Flexibility and easy extensibility
* Separation of data collection and data analysis

Description of our approach
===========================

We know that there are many ways one might want to monitor simulations. We
focused on the use case, that you want to monitor several attributes for one
process—maybe with timestamp—so that your data looks something like this:

    ([t0, t1, t2], [x0, x1, x2], [y0, y1, y2])

Our approach can easily be extended to meet other use cases, too, but more on
this later. We also state that our approach does not cover data analysis. This
is something that should in our opinion be done by specialized frameworks like
e.g. NumPy, SciPy and Matplotlib.

Basic idea
----------

For performance reasons our monitoring works on SimPy processes rather than on
the simulation itself. This has the advantage of a minimized or even nonexisting
performance impact on the simulation speed if you monitor only a few processes or
don’t monitor anything at all.

We also wanted a very easy API: Create a monitor like ``m = Monitor(config)``
and trigger the monitoring of a process’ current state by just calling ``m()``.

How to exactly use it
---------------------

Monitors should be instanciated in ``MyProc.__init__()``. You must pass
some configuration for the *series* you want to create. Each *series* is
defined by a *name* and a *collector* function::

    class MyProc(Process):
        def __init__(self):
            # ...
            self.monitor = Monitor(
                ('a', func1),
                ('b', func2),
                # ...
            )

You can then retrieve the data for e.g. the series “a” via ``self.monitor.a``.
The collector functions describe, how the data should be collected. The two
common cases here are:

1. Automatically get an attribute’s value (e.g. ``self.a``)
2. Manually pass the value to be monitored

You can solve both cases with custom lambda functions, but we’ve also create a
shortcut for each of them.

And advanced example::

    class MyProc(Process):
        def __init__(self, sim):
            # ...
            self.monitor = Monitor(
                ('time', sim.now),
                get(self, 'a', 'b'),
                ('diff', manual)
            )

This will create a monitor containing the series ``time``, ``a``, ``b`` and
``diff``, where ``time`` stores the values returned by ``sim.now()``, ``self.a``
and ``self.b`` get collected automatically and and a value for ``diff`` has to
be passed manually. All this happens with the PEM of a process::

    def run(self)
        while True:
            # ...
            self.monitor(diff=self.get_diff()

That’s it!

Conclusion
==========

We hope we could make our idea clear. We think our solution is quite clean and
flexible. It can easily be extended (e.g. by adding other monitor classes) and
also easily be added to SimPy without interfering with any existing
functionality. Since it isn’t integrated into the simulation core, it won’t slow
down the simulation, if you don’t want monitoring.

The attached script contains the whole monitoring framework and an example
simulation that tests and shows how to use it. You need SimPy, NumPy and
Matplotlib to execute it (NumPy and Matplotlib are only required for the example
analysis part in the end, so if you uncomment these lines, you don’t need them).

Other monitor classes we thought of are:

* a *SimpleMonitor* that just creates one list of values (e.g. [x0, x1, ...]),
* a *GroupedMonitor* that collects groups of series (useful for a central
  collector process which just monitors other processes in the simulation) and
* aggregate monitors like e.g. and AvgMonitor. This could be useful if you are
  just interested in the average of all values (and not the values themselves),
  so it only needs to store the sum and count of all values.

What do you think of it?

Cheers,
Ontje and Stefan

------------------------------------------------------------------------------
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for Conference
attendees to learn about information security's most important issues through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev

_______________________________________________
Simpy-users mailing list
Simpy-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simpy-users