redis
Read from Redis in-memory key-value store
Only reading key value (hash-value and hash) are scheduled
| Field Name | Description | Type | Default |
|---|---|---|---|
| uri | Redis address to listen to | string | redis://localhost/ |
| raw | Assume incoming data is in JSON format | bool | false |
| interval | How often to run the command | duration | - |
| cron | How often to run the command | cron | - |
| immediate | Run as soon as invoked, instead of waiting for the specified cron interval | bool | false |
| random-offset | Sets a random offset to the schedule, then sticks to it | duration | - |
| window | For resources that need a time window to be specified | - | - |
| Sub Action | Description |
|---|---|
| redis-list | Name of a Redis list (queue) |
| redis-hash-value | Read from a Redis hash |
| redis-hash | Read a Redis hash as a JSON event |
| redis-subscribe | Subscribe to Redis channels |
redis list
Name of a Redis list (queue)
Type: string
Example
action:
redis:
get:
list: some-field
redis hash-value
Read from a Redis hash
This can be scheduled. Event field expansions allowed in the hash field
Type: array of string (hash key and field)
Example
action:
redis:
get:
hash-value:
- my-hash
- the-field
interval: 5m
redis hash
Read a Redis hash as a JSON event
Scheduled
Type: string
Example
action:
redis:
get:
hash: some-hash
interval: 2s
redis subscribe
Subscribe to Redis channels
Type: array of strings
Example: Can subscribe to multple channels
action:
redis:
get:
subscribe:
- some-channel
- some-other-channel
Example: Cab subscribe to all sub-channels of a particular channel
action:
redis:
get:
subscribe:
- some-channel.*
uri
Redis address to listen to
Type: string
Example
action:
redis:
get:
subscribe:
- topic
uri: redis://localhost/
raw
Assume incoming data is in JSON format
Type: bool
Example
action:
redis:
get:
list: queue
raw: true
interval
How often to run the command
By default, interval: 0s which means: once.
Note that scheduled inputs set document markers.
See full discussion
Type: duration
Example
action:
exec:
command: echo 'once a day'
interval: 1d
cron
How often to run the command
Type: cron
Example: Once a day
action:
exec:
command: echo 'once a day'
cron: '0 0 0 * * *'
Example: Once a day, using a convenient shortcut
action:
exec:
command: echo 'once a day'
cron: '@daily'
immediate
Run as soon as invoked, instead of waiting for the specified cron interval
Type: bool
Example: Run immediately on invocation, and thereafter at 10h every morning
action:
exec:
command: echo 'hello'
immediate: true
cron: '0 0 10 * * *'
random-offset
Sets a random offset to the schedule, then sticks to it
This can help avoid the thundering herd problem, where you do not, for example, want to overload some service at 00:00:00
Type: duration
Example: Would fire up to a minute after every hour
action:
exec:
command: echo 'hello'
random-offset: 1m
cron: '0 0 * * * *'
window
For resources that need a time window to be specified
| Field Name | Description | Type | Default |
|---|---|---|---|
| size | Window size | duration | - |
| offset | Window offset | duration | - |
| start-time | Allows the windowing to start at a specified time | time | - |
| highwatermark-file | Specify file where timestamp would be stored in order to resume, for when Pipe has been restarted | string | - |
size
Window size
Type: duration
Example
action:
exec:
command: echo 'one two'
window:
size: 1m
offset
Window offset
Type: duration
Example
action:
exec:
command: echo 'one two'
window:
size: 1m
offset: 10s
start-time
Allows the windowing to start at a specified time
It should in the following format: 2019-07-10 18:45:00.000 +0200
Type: time
Example
action:
exec:
command: echo 'one two'
window:
size: 1m
start-time: 10s
highwatermark-file
Specify file where timestamp would be stored in order to resume, for when Pipe has been restarted
Type: string
Example
action:
exec:
command: echo 'one two'
window:
size: 1m
highwatermark-file:: /tmp/mark.txt