generate

Create new events, specifically for alerts

This step requires special attention, because it is a way to create custom events and alerts that are aware of the history of the data.

As a stream of JSON events is read and passed through. generate saves these records in a SQLite database so it can use the full power of SQL to generate historical queries over aggregates such as averages and maximums.

It takes free-form fields, but the values are documented in the fields array below.

Example: A contrived example

  • We take the 3s average (avg) over event field, number
  • We create an new event (alert) when threshold is greater than 2
  • add indicates what fields will be added to the new event, should the when be met

input:

input:
  exec:
    command: |
      echo '{"number":1}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":1}';
      sleep 1;
      echo '{"number":2}';
    raw: true
    no-strip-linefeeds: true

action:

generate:
  high average:
    let:
       avg: AVG(number, 3s)
    when: avg > 2
    add:
      - title: average too high
      - text: average threshold 2 exceeded over time rage=3s interval is higher than threshold

output:

{"number":1}
{"number":2}
{"number":2}
{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time rage=3s interval is higher than threshold","title":"average too high","aggregation_key":"high average","severity":"info","alert_count":1,"@timestamp":"2020-03-06T11:21:04.653Z"}
{"number":1}
{"number":2}

Example: another example

input:

input:
  exec:
    command: |
      echo '{"number":1}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":1}';
      sleep 1;
      echo '{"number":2}';
    raw: true
    no-strip-linefeeds: true

action:

generate:
  high average:
    let:
       avg: AVG(number, 3s)
    when: avg > 2
    add:
      - title: average too high
      - text: average threshold=2 exceeded (over time range=3s)

output:

{"number":1}
{"number":2}
{"number":2}
{"number":3}
{"type":"alert","text":"average threshold=2 exceeded (over time range=3s)","title":"average too high","aggregation_key":"high average","severity":"info","alert_count":1,"@timestamp":"2020-03-12T10:23:56.271Z"}
{"number":1}
{"number":2}

Example: A more real example

  • We take the 5m average over incomingBytesPerInterval (an event field)
  • We take the 60m max value over the same field,
  • We create an new event (alert) when the ratio of the former (avg_incoming) over the latter (max_incoming) is greater than threshold.
  • add indicates what fields will be added to the new event, should the when be met

action:

generate:
  bbox.linkutilisation.incoming:
    let:
       avg_incoming: AVG(incomingBytesPerInterval, 5m)
       max_incoming: MAX(incomingBytesPerInterval, 60m)
       threshold: 90.0/100.0
    when: (avg_incoming / max_incoming) > threshold
    add:
      - severity: warning
      - kind: alert
      - title: "Line Utilisation incoming over 90%"
      - text: "average incoming ${avg_incoming}kb close to max incoming ${max_incoming}kb: ratio: ${threshold:1}"

Field NameDescriptionTypeDefault
letA section to generate calculations--
whenDetermines if an event should be generated--
addDescribes the new fields to be added, whenever the event is generatedarray of key-value pairs-
notificationHow often the event should be generated, to help limit event countduration-
at_endbool-
group-bystring-

let

A section to generate calculations

when

Determines if an event should be generated

add

Describes the new fields to be added, whenever the event is generated

Type: array of key-value pairs

Field NameDescriptionTypeDefault
titleTitle of eventstring-
textText describing the eventstring-
typeText describing the event type, typically eventstring-
severityText describing alert level, examples being warningstring-

title

Title of event

Type: string

text

Text describing the event

Type: string

type

Text describing the event type, typically event

Type: string

severity

Text describing alert level, examples being warning

Type: string

notification

How often the event should be generated, to help limit event count

Type: duration

Example: A contrived example

  • We take the 3s average (avg) over event field, number
  • We create an new event (alert) when threshold is greater than 2
  • add indicates what fields will be added to the new event, should the when be met
  • We create the event only when notification duration has been exceeded

input:

input:
  exec:
    command: |
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":3}';
    raw: true
    no-strip-linefeeds: true

action:

generate:
  high.average:
    let:
       avg: AVG(number, 3s)
    when: avg > 2
    add:
      - title: average too high
      - text: average threshold 2 exceeded over time range=3s interval is higher than threshold
    notification: 3s

output:

{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time range=3s interval is higher than threshold","title":"average too high","aggregation_key":"high.average","severity":"info","alert_count":1,"@timestamp":"2020-03-12T10:27:50.321Z"}
{"number":2}
{"number":2}
{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time range=3s interval is higher than threshold","title":"average too high","aggregation_key":"high.average","severity":"info","alert_count":4,"@timestamp":"2020-03-12T10:27:53.327Z"}
{"number":2}
{"number":2}
{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time range=3s interval is higher than threshold","title":"average too high","aggregation_key":"high.average","severity":"info","alert_count":7,"@timestamp":"2020-03-12T10:27:56.332Z"}
{"number":2}
{"number":3}
{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time range=3s interval is higher than threshold","title":"average too high","aggregation_key":"high.average","severity":"info","alert_count":10,"@timestamp":"2020-03-12T10:27:59.337Z"}

at_end

???

Type: bool

Example

input:

input:
  exec:
    command: |
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":2}';
      sleep 1;
      echo '{"number":3}';
      sleep 1;
      echo '{"number":3}';
    raw: true
    no-strip-linefeeds: true

action:

generate:
  high.average:
    let:
       avg: AVG(number, 3s)
    when: avg > 2
    add:
      - title: average too high
      - text: average threshold 2 exceeded over time rage=3s interval is higher than threshold
    at_end: true

output:

{"number":3}
{"number":2}
{"number":2}
{"number":3}
{"number":2}
{"number":2}
{"number":3}
{"number":2}
{"number":3}
{"number":3}
{"type":"alert","text":"average threshold 2 exceeded over time rage=3s interval is higher than threshold","title":"average too high","aggregation_key":"high.average","severity":"info","alert_count":1,"@timestamp":"2020-03-12T10:24:16.648Z"}

group-by

???

Type: string