Quickstart Guide

Installation

Obtaining the software

A file hotrod-VERSION-PLATFORM-ARCHITECTURE.zip will be made available to you. For example: hotrod-2.3.0-Linux-x64.zip

Installing the software

The zip file consists of the following programs

  • hotrodd - this is the heart of the system and runs on the server. It exposes a REST-style API over HTTP and so can be accessed remotely.
  • hotrod - the Hotrod Command-line interface (CLI). This speaks to hotrodd using the API with an authenticated connection.
  • hotrod-agent - each target has a running instance. It constantly communicates to the central hotrodd and checks for pipe updates. On changes it will pull a tarball of the pipe scripts and any associated files, and start/stop pipe services accordingly. It can do both systemd and Docker services.
  • pipe - the executables used in action pipe processes.

Copy these executables into your $PATH (FHS recomments /usr/local/bin).

Running the Hotrod

NOTE: Each of these executables come with built-in help, and you can access it with the --help command line flag.

Create a directory for testing. This is recommended since hotrodd creates some files and directories for its use.

Start the executable:

hotrodd

It will start listening on localhost (by default port 3000).

Among the logs seen will be the password, which you wil use in the next step, where you will need to login. Open another terminal for this.

$ hotrod login admin
Type password:
Login successful

Once that is done, a file named ~/.config/hotrod/tokens.yml gets created. It stores token that allows accessing the Hotrod without needing to re-enter the password repeatedly.

If you wanted to change the password to something more memorable...

$ hotrod users chpasswd admin
Type old password:
Type new password:
Retype password:

Now we can add a target to hotrodd. The target has a convenient name, and a unique, aut--generated identifier.

hotrod targets add 'Panoptix office' --id 0001

Now you can say hotrod targets list and see the new target.

Adding a pipe

Create a file called uptime.yml:

name: uptime
input:
    exec:
        command: uptime
        interval: 2s
output:
    write: console

The name of the file MUST be the name of the Pipe plus .yml extension. Pipe names must be unique.

We first upload the pipe definition to hotrodd and then attach that Pipe to the "Panoptix office" Target.

NOTE: If you have typing errors, you will find out about them when trying to add the pipe definition; as far as possible, we do not allow invalid pipes to actually run on targets.

hotrod pipes add --file uptime.yml
hotrod targets update office --add-pipe uptime

Running hotrod-agent

The target does not actually exist yet - this is where we need hotrod-agent. This needs an API key to communicate back to the hotrodd. We ask hotrodd for a new API key and copy it to a little file called .env. The agent will read this and find out the value of the environment variable HOTROD_API_KEY. You can also of course just use export here.

$ hotrod api-key issue agent
API-KEY(agent;api_read;default) XXXXXXXX
$ echo "HOTROD_API_KEY=XXXXXX" > .env

NOTE: In a BMS install, these details are managed for you, but it's useful to see how the parts fit together.

You might need to visudo:

env_keep+=HOTROD_API_KEY, !requiretty
sudo hotrod-agent --systemd --poll-interval 5 --url http://localhost:3000 --target-id 0001 --pipes-dir $PWD/pipes

So in another terminal in the same directory as that .env file, type:

mkdir pipes
sudo hotrod-agent --systemd --poll-interval 5 --url http://localhost:3000 --target-id 0001 --pipes-dir $PWD/pipes
2019-11-07T18:10:07.631 INFO  hotrod_client_sdk::hotrod_client > Got target tarball: 188 bytes
2019-11-07T18:10:07.631 INFO  hotrod_client_sdk::hotrod_client > Extracting "/tmp/.tmpPpZS4L/hotrod_target.tar.gz" tarball to "/root/hotrod-agent/pipes"

The agent will now poll that URL every 5 seconds, with our id 0001. It will use the directory we created to keep the pipe service files (The $PWD is because systemd wants an absolute directory path for scripts). In this mode, it will create systemd services.

The first time the agent polls, it finds that there's already a Pipe defined for it, and immediately downloads a little tarball and extracts it in that pipes directory. The key message is "Creating and running hotrod-pipe-uptime service". The agent will always create services with names in the format "hotrod-pipe-PIPE".

You can now say journalctl --unit hotrod-pipe-uptime --follow and see the pipe running! Here is the output on some old laptop:

Jan 31 12:48:05 steve-HP-ZBook-14 hotrod-pipe-uptime[4635]: {"_raw":" 12:48:05 up 49 min,  4 users,  load average: 0.12, 0.08, 0.02"}

(The default format is "DATE MACHINE SERVICE[PID]: DATA".)

This invocation just gives you the data without the log fields:

journalctl --unit hotrod-pipe-uptime --output cat --follow

You will notice that the output of uptime is wrapped up as a JSON field '_raw'. Generally speaking, Pipes work with JSON data.

The next step is to extract the load averages. The extract action uses a regular expression to extract the values, which become named JSON fields.

# uptime.yml
name: uptime
input:
  exec:
    command: uptime
    interval: 2s
actions:
  - extract:
      pattern: 'load average: (\S+), (\S+), (\S+)'
      output-fields: [m1, m5, m15]
output:
  write: console

Then just do hotrod pipes update --file uptime.yml (NOT add) and in a few seconds the updated Pipe is in action:

journalctl -u hotrod-pipe-uptime -o cat --follow
{"_raw":" 13:08:47 up  1:10,  4 users,  load average: 0.07, 0.07, 0.02"}
{"_raw":" 13:08:49 up  1:10,  4 users,  load average: 0.07, 0.07, 0.02"}
{"_raw":" 13:08:51 up  1:10,  4 users,  load average: 0.14, 0.09, 0.02"}
Stopping hotrod-pipe-uptime service...
Stopped hotrod-pipe-uptime service.
Started hotrod-pipe-uptime service.
{"_raw":" 13:08:53 up  1:10,  4 users,  load average: 0.14, 0.09, 0.02","m1":"0.14","m5":"0.09","m15":"0.02"}
{"_raw":" 13:08:55 up  1:10,  4 users,  load average: 0.14, 0.09, 0.02","m1":"0.14","m5":"0.09","m15":"0.02"}
{"_raw":" 13:08:57 up  1:10,  4 users,  load average: 0.14, 0.08, 0.02","m1":"0.13","m5":"0.08","m15":"0.02"}

Every registered target that has the uptime pipe running will be updated, next time they poll the server.

By default, extract is going to augment the data by adding additional fields. In many cases, you actually want to replace the input with the parsed output. To do that, we add remove: true to our extract configuration.

# uptime.yml
name: uptime
input:
  exec:
    command: uptime
    interval: 2s
actions:
  - extract:
      remove: true
      pattern: 'load average: (\S+), (\S+), (\S+)'
      output-fields: [m1, m5, m15]
output:
  write: console

After this we again update the pipe with hotrod pipes update --file uptime.yml, and in a few seconds, the updated pipe is in action, now without the _raw field:

journalctl -u hotrod-pipe-uptime -o cat --follow
{"m1":"0.14","m5":"0.09","m15":"0.02"}
{"m1":"0.14","m5":"0.09","m15":"0.02"}
{"m1":"0.13","m5":"0.08","m15":"0.02"}