towk/magellan
1
0
Fork 0
mirror of https://github.com/davidallendj/magellan.git synced 2025-12-20 03:27:03 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

Updated README.md (WIP)

Browse source
This commit is contained in:
David J. Allen 2024-05-10 10:08:26 -06:00
parent fba9b547ee
commit ebee86ccbe
No known key found for this signature in database
GPG key ID: 717C593FF60A2ACC
1 changed files with 146 additions and 59 deletions
Download patch file Download diff file Expand all files Collapse all files

205
README.md
View file

@ -1,88 +1,155 @@
# Magellan # Magellan
Magellan is a board management controller discovery tool designed to scan a network The `magellan` CLI tool is a Redfish-based, board management controller (BMC) discovery tool designed to scan networks and is written in Go. The tool collects information from BMC nodes using the provided Redfish RESTful API with [`gofish`](https://github.com/stmcginnis/gofish) and loads the queried data into an [SMD](https://github.com/OpenCHAMI/smd/tree/master) instance. The tool strives to be more flexible by implementing multiple methods of discovery to work for a wider range of systems (WIP) and is capable of using independently of other tools or services.
and collect information about a BMC node and load that data into an
[`hms-smd`](https://github.com/bikeshack/smd/tree/master) instance.
## How It Works ## Getting Started
Magellan is designed to do three things: [Build](#building) and [run on bare metal](#running-the-tool) or run and test with Docker using the [latest prebuilt image](#running-with-docker).
1. Scan for BMC nodes in cluster available on a network
2. Query information about each BMC node
3. Store queried information into a database
Magellan first tries to probe for specified hosts using the [`dora`](https://github.com/bmc-toolbox/dora)
API. If that fails, it then tries to use its own built-in, simpler scanner as a fallback.
This is done by sending a raw TCP request to a number of potential hosts over a
network, and noting which requests are successful. At this point, `magellan` sees
no difference between a services.
Next, it tries to query information about the BMC node using `bmclib` functions,
but requires access to a redfish interface on the node to work. Once the BMC
information is received, it is then stored into `hms-smd` using its API.
In summary, `magellan` needs at minimum the following configured to work on each node:
1. Available redfish interface with its host and port
2. A running instance of `hms-smd` with its host and port
3. Additional dependencies for `bmclib` such as `ipmitool`
## Building ## Building
Install Go, clone the repo, and then run the following in the project root: The `magellan` tool can be built to run on bare metal. Install the required Go tools, clone the repo, and then build the binary in the root directory with the following:
```bash ```bash
git clone https://github.com/bikeshack/magellan git clone https://github.com/OpenCHAMI/magellan
cd magellan cd magellan
go mod tidy && go build go mod tidy && go build
``` ```
This should find and download all of the required dependencies. Although other And that's it. The last line should find and download all of the required dependencies to build the project. Although other versions of Go may work, the project has been tested to work with versions v1.20 and later on MacOS and Linux.
versions of Go may work, the project has only been tested with v1.20.
To build the Docker container, run `docker build -t magellan:latest .` in the ### Docker
project's directory.
The tool can also run using Docker. To build the Docker container, run `docker build -t magellan:testing .` in the project's directory. This is useful if you to run `magellan` on a different system through Docker desktop without having to install and build with Go (or if you can't do so for some reason). [Prebuilt images](https://github.com/OpenCHAMI/magellan/pkgs/container/magellan) are available as well on `ghcr`. Images can be pulled directly from the repository:
```bash
docker pull ghcr.io/openchami/magellan:latest
```
See the ["Running with Docker"](#running-with-docker) section below about running with the Docker container.
## Usage ## Usage
There are three main commands to use with the tool: `scan`, `list`, and `collect`. The sections below assume that the BMC nodes have an IP address available to query Redfish. Currently, `magellan` does not support discovery with MAC addresses although that may change in the future.
To scan a network for BMC nodes, use the `scan` command. If the port is not specified,
`magellan` will probe ports 623, 442 (redfish and IPMI) by default: ### Checking for Redfish
Before using the tool, confirm that the identified node has Redfish with `curl`. Assuming the IP address for the BMC node is `172.16.0.10`, we can send a request to see if it we get a response. You might need to pass the `-k` flag if the node uses TLS or point to the appropriate certificate.
```bash ```bash
./magellan scan --subnet 192.168.0.0 --db.path data/assets.db --port 623 curl -k https://172.16.0.10/redfish/v1 --cacert cacert.pem | jq
``` ```
This will scan the `192.168.0.0` subnet returning the host and port that return a response This should return a JSON response with general information. The output below has been truncated:
and store the results in database with path `data/assets.db`. Additional flags can
be set such as `host` to add more hosts to scan not included on the subnet, `timeout` to set how long
to wait for a response from the BMC node, or `threads` to set the number of requests
to make concurrently. Try using `./magellan help scan` for a complete set of options.
To see the available BMC nodes found from the scan, use the `list` command. Make ```json
sure to point to the same database used before: {
"@odata.context": "/redfish/v1/$metadata#ServiceRoot.ServiceRoot",
"@odata.etag": "W/\"1715279084\"",
"@odata.id": "/redfish/v1/",
"@odata.type": "#ServiceRoot.v1_5_2.ServiceRoot",
"AccountService": {
"@odata.id": "/redfish/v1/AccountService"
},
"CertificateService": {
"@odata.id": "/redfish/v1/CertificateService"
},
"Chassis": {
"@odata.id": "/redfish/v1/Chassis"
},
...
}
```
To see all of the available commands, run `magellan` with the `help` subcommand:
```bash
./magellan help
Tool for BMC discovery
Usage:
magellan [flags]
magellan [command]
Available Commands:
collect Query information about BMC
completion Generate the autocompletion script for the specified shell
help Help about any command
list List information from scan
login Log in with identity provider for access token
scan Scan for BMC nodes on a network
update Update BMC node firmware
Flags:
--access-token string set the access token
-c, --config string set the config file path
--db.path string set the probe storage path (default "/tmp/magellan/magellan.db")
-h, --help help for magellan
--threads int set the number of threads (default -1)
--timeout int set the timeout (default 30)
-v, --verbose set verbose flag
Use "magellan [command] --help" for more information about a command.
```
### Running the Tool
There are three main commands to use with the tool: `scan`, `list`, and `collect`. To start a network scan for BMC nodes, use the `scan` command. If the port is not specified, `magellan` will probe ports 623 and 443 by default:
```bash
./magellan scan \
--subnet 172.16.0.0 \
--subnet-mask 255.255.255.0 \
--db.path data/assets.db --port 443
```
This will scan the `172.16.0.0` subnet returning the host and port that return a response and store the results in a local cache with at the `data/assets.db` path. Additional flags can be set such as `host` to add more hosts to scan not included on the subnet, `timeout` to set how long to wait for a response from the BMC node, or `threads` to set the number of requests to make concurrently. Try using `./magellan help scan` for a complete set of options this subcommand.
To inspect the cache, use the `list` command. Make sure to point to the same database used before:
```bash ```bash
./magellan list --db.path data/assets.db ./magellan list --db.path data/assets.db
``` ```
This will print a list of IP address and ports found and stored from the scan. This will print a list of node info found and stored from the scan.
Finally, run the `collect` command to store BMC info into `hms-smd`:
Finally, set the `MAGELLAN_ACCESS_TOKEN`run the `collect` command to query the node from cache and send the info to be stored into SMD:
```bash ```bash
./magellan collect --db.path data/assets.db --driver ipmi --timeout 5 --user admin --pass password ./magellan collect \
--db.path data/assets.db \
--timeout 5 \
--user admin \
--pass password \
--host https://example.openchami.cluster \
--port 27779 \
--ca-cert cacert.pem
``` ```
This uses the info store in the database above to request information about each This uses the info stored in cache to request information about each BMC node if possible. Like with the scan, the time to wait for a response can be set with the `timeout` flag as well. This command also requires the `user` and `pass/password` flag to be set if access the Redfish service requires basic authentication. Additionally, it may be necessary to set the `host` and `port` flags for `magellan` to find the SMD API (not the root API endpoint /hsm/v2).
BMC node if possible. It uses the driver specified by the `driver` flag which is
passed to and set in `bmclib`. Like with the scan, the time to wait for a response
can be set with the `timeout` flag as well. This command also requires the `user`
and `pass/password` flag to be set to use `ipmitool` (which must installed as well).
Additionally, it may be necessary to set the `host` and `port` flags for `magellan`
to find the `hms-smd` API.
Note: If the `db.path` flag is not set, `magellan` will use /tmp/magellan.db by default. Note: If the `db.path` flag is not set, `magellan` will use "/tmp/$USER/magellan.db" by default.
### Getting an Access Token (WIP)
The `magellan` tool has a `login` subcommand that works with the [`opaal`](https://github.com/OpenCHAMI/opaal) service to obtain a token needed to access the SMD service. If the SMD instance requires authentication, set the `MAGELLAN_ACCESS_TOKEN` environment variable to have `magellan` include it in the header for HTTP requests to SMD.
```bash
./magellan login --url https://opaal:4444/login
# ...complete login flow
export MAGELLAN_ACCESS_TOKEN=
```
Alternatively, if you are running the OpenCHAMI quickstart, you can run the provided script to generate a token and set the environment variable that way.
```bash
quickstart_dir=path/to/deployment/recipes/quickstart
source $quickstart_dir/bash_functions.sh
export MAGELLAN_ACCESS_TOKEN=$(gen_access_token)
```
### Running with Docker
Both the `scan` and `collect` commands can be ran via Docker after pulling the image: Both the `scan` and `collect` commands can be ran via Docker after pulling the image:
@ -91,17 +158,37 @@ docker pull bikeshack/magellan:latest
docker run bikeshack/magellan:latest /magellan.sh --scan "--subnet 172.16.0.0 --port 443 --timeout 3" --collect "--user admin --pass password --host http://vm01 --port 27779" docker run bikeshack/magellan:latest /magellan.sh --scan "--subnet 172.16.0.0 --port 443 --timeout 3" --collect "--user admin --pass password --host http://vm01 --port 27779"
``` ```
## How It Works
At its core, `magellan` is designed to do three basic things:
1. Scan for BMC nodes in cluster available on a network
2. Query information about each BMC node through Redfish API
3. Store queried information into a system management database
First, the tool performs a scan to find running services on a network. This is done by sending a raw TCP packet to all specified hosts (either IP or host name) and taking note which services respond. At this point, `magellan` has no way of knowing whether this is a Redfish service or not, so another HTTP request is made to verify. Once the BMC responds with an OK status code, `magellan` will store the necessary information in a local cache database to allow collecting more information about the node later. This allows for users to only have to scan their cluster once to find systems that are currently available and scannable.
Next, the tool queries information about the BMC node using `gofish` API functions, but requires access to BMC node found in the scanning step mentioned above to work. If the node requires basic authentication, a user name and password is required to be supplied as well. Once the BMC information is retrived from each node, the info is aggregated and a HTTP request is made to a SMD instance to be stored. Optionally, the information can be written to disk for inspection and debugging purposes.
In summary, `magellan` needs at minimum the following configured to work on each node:
1. Available Redfish service with its known host and port
2. A running instance of SMD service with its known host and port
3. Docker to pull and run containers or Go to build binaries
## TODO ## TODO
List of things left to fix, do, or ideas... See the [issue list](https://github.com/OpenCHAMI/magellan/issues) for plans for `magellan`. Here is a list of other features left to add, fix, or do (and some ideas!):
* [ ] Switch to internal scanner if `dora` fails * [X] Confirm loading different components into SMD
* [ ] Set default port automatically depending on the driver used to scan
* [X] Test using different `bmclib` supported drivers (mainly 'redfish')
* [X] Confirm loading different components into `hms-smd`
* [X] Add ability to set subnet mask for scanning * [X] Add ability to set subnet mask for scanning
* [ ] Add ability to scan with other protocols like LLDP
* [ ] Add more debugging messages with the `-v/--verbose` flag
* [ ] Separate `collect` subcommand with making request to endpoint
* [X] Support logging in with `opaal` to get access token
* [X] Support using CA certificates with HTTP requests to SMD
* [ ] Add unit tests for `scan`, `list`, and `collect` commands * [ ] Add unit tests for `scan`, `list`, and `collect` commands
* [X] Clean up, remove unused, and tidy code * [ ] Clean up, remove unused, and tidy code
## Copyright ## Copyright