March 28, 2022

Mirabelle 0.10.0: Prometheus remote write and new features

I released yesterday the version 0.10.0 of Mirabelle, a powerful stream processing engine which can work on events, metrics or logs. I will detail in this article what’s new in this release.

Prometheus remote write input

Mirabelle now supports receiving events from Prometheus (or any Prometheus-compatible tool). You only have to add the Mirabelle remote write endpoint into the Prometheus remote write configuration and you’re done:

remote_write:
  - url: 'http://mirabelle-address:5558/api/v1/prometheus/remote-write/default'

Prometheus will forward scraped metrics to Mirabelle. default in this URL is the Mirabelle stream to target.

For example, if Prometheus scrapes this metric:

http_requests_duration_second_bucket{method="GET",path="/metrics",le="0.05"} 29

Mirabelle will receive:

{
  :instance "localhost:9013"
  :job "my-prometheus-job"
  :le "0.05"
  :method "GET"
  :path "/metrics"
  :service "http_requests_duration_second_bucket"
  :metric 29.0
  :time" 1647297445.056
}

You can then use Mirabelle’s powerful stream processing language to compute things or alert based on these events, and/or forward them to external systems (like InfluxDB for example).

New websocket implementation

Mirabelle historically had one HTTP server for its API and one for the websocket protocol.

Websocket is for me one of the coolest feature in Mirabelle. I really liked this feature when I worked with Riemann and I improved it in Mirabelle (see the documentation if you are interested about this topic).

Mirabelle now uses the same server for its API and for websocket.

InfluxDB output refactoring

The InfluxDB output was simplified, and tags are now automatically deducted from the Mirabelle event. Everything is described in the Mirabelle documentation.

Elasticsearch output: optional index pattern

The Mirabelle Elasticsearch output can now be configured without the default-index-pattern option. You had before to configure two options to send events to Elasticsearch:

;; The default Elasticsearch index name
:default-index "abc"
;; The default index pattern which will be added to the index name
:default-index-pattern "yyyy-MM-dd"

:default-index-pattern is now optional.

Support delay on several streams

There is a major issue with stream processing engines: how to deal with late events ?

Let’s say you want to create 10 seconds time windows. When should the windows be flushed to downstream actions ? How late events (arriving after the window end) should be handled ?
Mirabelle historically ignored late events.

10 seconds time windows with a late event is dropped

Dropping late event is an issue when you aggregate events from multiple hosts. Indeed, we can consider that events coming from a single host will be ordered by time (except if you have clock issues :D), but what if you want to compute something on events from 10 hosts ? The risk to have out-of-order events is high.

Most Mirabelle actions which work on multiple events (like time windows actions) now accepts a :delay configuration. For example, fixed-time-window {:duration 60 :delay 30}) will only flush windows 30 seconds after its end, in order to wait for late events which can then be included into the right time window.

New actions

New actions were added to Mirabelle:

  • coll-increase: Transforms an always increasing counter into an event with the increase value as metric.

  • smax and smin: Send downstream the event with the highest (or smallest) event :metric field evey time it receives an event.

  • aggr-max and aggr-min: These actions takes a duration period (and optionally a delay) and will send downstream the event with the highest or smallest metric: (aggr-max {:duration 60 :delay 5}).

  • extract: Replaces an event by the content of one of its field. Using for example (extract :base-event) with exception-stream (an action to handle errors and which store the event which produced it in the :base-event key) can for example be useful.

What’s next

I would like to work on two things in priority:

  • Kafka integration (consumer and producer).

  • An action to work on Prometheus histograms (the equivalent of histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le)) in Prometheus for example).

  • Create a performance benchmark for the project.

I also have more ideas for the future of Mirabelle but I’m not ready to share them now ;)

Tags: cloud projects english

Add a comment






If you have a bug/issue with the commenting system, please send me an email (my email is in the "About" section).

Top of page