towk/configurator
1
0
Fork 0
mirror of https://github.com/davidallendj/configurator.git synced 2025-12-20 03:27:02 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #5 from OpenCHAMI/rewrite

Browse source 
Rewrite to use generator plugins and update README
This commit is contained in:
David Allen 2024-06-19 14:23:11 -06:00 • committed by GitHub
commit 843a248559
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 955 additions and 197 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -1,3 +1,4 @@
**configurator**
**.yaml
**.yml
**.so

17
Makefile Normal file
View file

@ -0,0 +1,17 @@
# build everything at once
all: plugins exe
# build the main executable to make configs
main: exe
exe:
go build --tags=all -o configurator
# build all of the generators into plugins
plugins:
mkdir -p lib
go build -buildmode=plugin -o lib/conman.so internal/generator/plugins/conman/conman.go
go build -buildmode=plugin -o lib/coredhcp.so internal/generator/plugins/coredhcp/coredhcp.go
go build -buildmode=plugin -o lib/dnsmasq.so internal/generator/plugins/dnsmasq/dnsmasq.go
go build -buildmode=plugin -o lib/powerman.so internal/generator/plugins/powerman/powerman.go
go build -buildmode=plugin -o lib/syslog.so internal/generator/plugins/syslog/syslog.go

116
README.md
View file

@ -1,26 +1,38 @@
# OpenCHAMI Configurator
Configurator is a tool that fetchs data from an instance of [SMD](https://github.com/OpenCHAMI/smd) to generate commonly used config files.
The `configurator` (portmanteau of config + generator) is a tool that fetchs data from an instance of [SMD](https://github.com/OpenCHAMI/smd) to generate commonly used config files. The tool is also capable of some templating using the Jinja 2 syntax with generator plugins.
## Building and Usage
Configurator is built using standard `go` build tools. The project separates the client and server with build tags. To get started, clone the project, download the dependencies, and build the project:
The `configurator` is built using standard `go` build tools. The project separates the client and server with build tags. To get started, clone the project, download the dependencies, and build the project:
```bash
git clone https://github.com/OpenCHAMI/configurator.git
go mod tidy
go build --tags all # equivalent to `go build --tags client,server``
## ...or just run `make` in project directory
```
To use the tool, run the following:
This will build the main driver program, but also requires generator plugins to define how new config files are generated. The default plugins can be built using the following build command:
```bash
./configurator generate --config config.yaml --target dhcp:dnsmasq
go build -buildmode=plugin -o lib/conman.so internal/generator/plugins/conman/conman.go
go build -buildmode=plugin -o lib/coredhcp.so internal/generator/plugins/coredhcp/coredhcp.go
go build -buildmode=plugin -o lib/dnsmasq.so internal/generator/plugins/dnsmasq/dnsmasq.go
go build -buildmode=plugin -o lib/powerman.so internal/generator/plugins/powerman/powerman.go
go build -buildmode=plugin -o lib/syslog.so internal/generator/plugins/syslog/syslog.go
```
This will generate a new DHCP `dnsmasq` config file based on the Jinja 2 template specified in the config file for "dnsmasq". The `--target` flag is set by passing an argument in the form of "type:template" to specify the type of config file being generate and the template file to use respectively. The configurator requires valid access token when making requests to an instance of SMD that has protected routes.
These commands will build the default plugins and store them in the "lib" directory by default. Alternatively, the plugins can be built using `make plugins` if GNU make is installed and available. After you build the plugins, run the following to use the tool:
The tool can also be ran as a microservice:
```bash
./configurator generate --config config.yaml --target dnsmasq -o dnsmasq.conf
```
This will generate a new `dnsmasq` config file based on the Jinja 2 template specified in the config file for "dnsmasq". The `--target` flag specifies the type of config file to generate by its name (see the [`Creating Generator Plugins`](#creating-generator-plugins) section for details). The `configurator` tool requires a valid access token when making requests to an instance of SMD that has protected routes.
The tool can also run as a microservice:
```bash
./configurator serve --config config.yaml
@ -34,27 +46,103 @@ curl http://127.0.0.1:3334/target?type=dhcp&template=dnsmasq
This will do the same thing as the `generate` subcommand, but remotely.
### Creating Generator Plugins
The `configurator` uses generator plugins to define how config files are generated using a `Generator` interface. The interface is defined like so:
```go
type Generator interface {
GetName() string
GetGroups() []string
Generate(config *configurator.Config, opts ...util.Option) ([]byte, error)
}
```
A new plugin can be created by implementing the methods from interface and exporting a symbol with `Generator` as the name and the plugin struct as the type. The `GetName()` function returns the name that is used for looking up the corresponding template set in your config file. The `GetGroups()` function is used to look all of the groups that the plugin is included. The `Generate` function is where the magic happens to build the config file from a template.
```go
package main
type MyGenerator struct {}
func (g *MyGenerator) GetName() string {
return "my-generator"
}
func (g *MyGenerator) GetGroups() []string {
return []string{ "my-generator" }
}
func (g *MyGenerator) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
// do config generation stuff here...
var (
params = generator.GetParams(opts...)
client = generator.GetClient(params)
output = ""
)
if client {
eths, err := client.FetchEthernetInterfaces(opts...)
// ... blah, blah, blah, format output, and so on...
}
// apply the template and get substituted output as byte array
return generator.ApplyTemplate(path, generator.Mappings{
"hosts": output,
})
}
// this MUST be named "Generator" for symbol lookup
var Generator MyGenerator
```
Finally, build the plugin and put it somewhere specified by `plugins` in your config. Make sure that the package is `main` before building.
```bash
go build -buildmode=plugin -o lib/mygenerator.so path/to/mygenerator.go
```
Now your plugin should be available to use with the `configurator` main driver.
## Configuration
Here is an example config file to start using configurator:
```yaml
version: ""
smd-host: http://127.0.0.1
smd-port: 27779
access-token:
server:
host: 127.0.0.1
port: 3334
jwks:
uri: ""
retries: 5
smd:
host: http://127.0.0.1
port: 27779
templates:
coredhcp: templates/dhcp/coredhcp.config.jinja
dnsmasq: templates/dhcp/dnsmasq.conf.jinja
dnsmasq: templates/dnsmasq.jinja
coredhcp: templates/coredhcp.jinja
syslog: templates/syslog.jinja
ansible: templates/ansible.j2
ansible: templates/ansible.jinja
powerman: templates/powerman.jinja
conman: templates/conman.jinja
groups:
warewulf:
- dnsmasq
- syslog
- ansible
- powerman
- conman
plugins:
- "lib/"
```
The `server` section sets the properties for running the `configurator` tool as a service and is not required if you're only using the CLI. Also note that the `jwks-uri` parameter is only needs for protecting endpoints. If it is not set, then the API is entirely public. The `smd` section tells the `configurator` tool where to find SMD to pull state management data used by the internal client. The `templates` section is where the paths are mapped to each generator plugin by its name (see the [`Creating Generator Plugins`](#creating-generator-plugins) section for details). The `plugins` is a list of paths to load generator plugins.
## Known Issues
- Adds a new `OAuthClient` with every token request
## TODO
## TODO
- Add group functionality
- Extend SMD client functionality
- Redo service API with authorization

198
cmd/generate.go
View file

@ -4,10 +4,11 @@
package cmd
import (
"encoding/json"
"fmt"
"maps"
"os"
"path/filepath"
"strings"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/generator"
@ -17,16 +18,52 @@ import (
var (
tokenFetchRetries int
pluginPaths []string
)
var generateCmd = &cobra.Command{
Use: "generate",
Short: "Generate a config file from system state",
Short: "Generate a config file from state management",
Run: func(cmd *cobra.Command, args []string) {
client := configurator.SmdClient{
Host: config.SmdHost,
Port: config.SmdPort,
AccessToken: config.AccessToken,
// load generator plugins to generate configs or to print
var (
generators = make(map[string]generator.Generator)
client = configurator.SmdClient{
Host: config.SmdClient.Host,
Port: config.SmdClient.Port,
AccessToken: config.AccessToken,
}
)
for _, path := range pluginPaths {
if verbose {
fmt.Printf("loading plugins from '%s'\n", path)
}
gens, err := generator.LoadPlugins(path)
if err != nil {
fmt.Printf("failed to load plugins: %v\n", err)
err = nil
continue
}
// add loaded generator plugins to set
maps.Copy(generators, gens)
}
// show config as JSON and generators if verbose
if verbose {
b, err := json.MarshalIndent(config, "", " ")
if err != nil {
fmt.Printf("failed to marshal config: %v\n", err)
}
fmt.Printf("%v\n", string(b))
}
// show available targets then exit
if len(args) == 0 && len(targets) == 0 {
for g := range generators {
fmt.Printf("\tplugin: %s, name:\n", g)
}
os.Exit(0)
}
// make sure that we have a token present before trying to make request
@ -34,12 +71,14 @@ var generateCmd = &cobra.Command{
// TODO: make request to check if request will need token
// check if OCHAMI_ACCESS_TOKEN env var is set if no access token is provided and use that instead
accessToken := os.Getenv("OCHAMI_ACCESS_TOKEN")
accessToken := os.Getenv("ACCESS_TOKEN")
if accessToken != "" {
config.AccessToken = accessToken
} else {
// TODO: try and fetch token first if it is needed
fmt.Printf("No token found. Attempting to generate config without one...\n")
if verbose {
fmt.Printf("No token found. Attempting to generate config without one...\n")
}
}
}
@ -58,82 +97,102 @@ var generateCmd = &cobra.Command{
for _, target := range targets {
// split the target and type
tmp := strings.Split(target, ":")
// tmp := strings.Split(target, ":")
// make sure each target has at least two args
if len(tmp) < 2 {
message := "target"
if len(tmp) == 1 {
message += fmt.Sprintf(" '%s'", tmp[1])
}
message += " does not provide enough arguments (args: \"type:template\")"
logrus.Errorf(message)
continue
}
g := generator.Generator{
Type: tmp[0],
Template: tmp[1],
}
// if len(tmp) < 2 {
// message := "target"
// if len(tmp) == 1 {
// message += fmt.Sprintf(" '%s'", tmp[0])
// }
// message += " does not provide enough arguments (args: \"type:template\")"
// logrus.Errorf(message)
// continue
// }
// var (
// _type = tmp[0]
// _template = tmp[1]
// )
// g := generator.Generator{
// Type: tmp[0],
// Template: tmp[1],
// }
// check if another param is specified
targetPath := ""
if len(tmp) > 2 {
targetPath = tmp[2]
// targetPath := ""
// if len(tmp) > 2 {
// targetPath = tmp[2]
// }
// run the generator plugin from target passed
gen := generators[target]
if gen == nil {
fmt.Printf("invalid generator target (%s)\n", target)
continue
}
output, err := gen.Generate(
&config,
generator.WithTemplate(gen.GetName()),
generator.WithClient(client),
)
if err != nil {
fmt.Printf("failed to generate config: %v\n", err)
continue
}
// NOTE: we probably don't want to hardcode the types, but should do for now
ext := ""
contents := []byte{}
if g.Type == "dhcp" {
// fetch eths from SMD
eths, err := client.FetchEthernetInterfaces()
if err != nil {
logrus.Errorf("failed to fetch DHCP metadata: %v\n", err)
continue
}
if len(eths) <= 0 {
continue
}
// generate a new config from that data
contents, err = g.GenerateDHCP(&config, eths)
if err != nil {
logrus.Errorf("failed to generate DHCP config file: %v\n", err)
continue
}
ext = "conf"
} else if g.Type == "dns" {
// TODO: fetch from SMD
// TODO: generate config from pulled info
// ext := ""
// contents := []byte{}
// if _type == "dhcp" {
// // fetch eths from SMD
// eths, err := client.FetchEthernetInterfaces()
// if err != nil {
// logrus.Errorf("failed to fetch DHCP metadata: %v\n", err)
// continue
// }
// if len(eths) <= 0 {
// continue
// }
// // generate a new config from that data
// contents, err = g.GenerateDHCP(&config, eths)
// if err != nil {
// logrus.Errorf("failed to generate DHCP config file: %v\n", err)
// continue
// }
// ext = "conf"
// } else if g.Type == "dns" {
// // TODO: fetch from SMD
// // TODO: generate config from pulled info
} else if g.Type == "syslog" {
// } else if g.Type == "syslog" {
} else if g.Type == "ansible" {
// } else if g.Type == "ansible" {
} else if g.Type == "warewulf" {
// } else if g.Type == "warewulf" {
}
// }
// write config output if no specific targetPath is set
if targetPath == "" {
if outputPath == "" {
// write only to stdout
fmt.Printf("%s\n", "")
} else if outputPath != "" && targetCount == 1 {
// write just a single file using template name
err := os.WriteFile(outputPath, contents, 0o644)
if err != nil {
logrus.Errorf("failed to write config to file: %v", err)
continue
}
} else if outputPath != "" && targetCount > 1 {
// write multiple files in directory using template name
err := os.WriteFile(fmt.Sprintf("%s/%s.%s", filepath.Clean(outputPath), g.Template, ext), contents, 0o644)
if err != nil {
logrus.Errorf("failed to write config to file: %v", err)
continue
}
// if targetPath == "" {
if outputPath == "" {
// write only to stdout
fmt.Printf("%s\n", string(output))
} else if outputPath != "" && targetCount == 1 {
// write just a single file using template name
err := os.WriteFile(outputPath, output, 0o644)
if err != nil {
logrus.Errorf("failed to write config to file: %v", err)
continue
}
} else if outputPath != "" && targetCount > 1 {
// write multiple files in directory using template name
err := os.WriteFile(fmt.Sprintf("%s/%s.%s", filepath.Clean(outputPath), target, ".conf"), output, 0o644)
if err != nil {
logrus.Errorf("failed to write config to file: %v", err)
continue
}
}
// }
} // for targets
}
},
@ -141,6 +200,7 @@ var generateCmd = &cobra.Command{
func init() {
generateCmd.Flags().StringSliceVar(&targets, "target", nil, "set the target configs to make")
generateCmd.Flags().StringSliceVar(&pluginPaths, "plugin", nil, "set the generator plugins directory path to shared libraries")
generateCmd.Flags().StringVarP(&outputPath, "output", "o", "", "set the output path for config targets")
generateCmd.Flags().IntVar(&tokenFetchRetries, "fetch-retries", 5, "set the number of retries to fetch an access token")

4
cmd/root.go
View file

@ -12,6 +12,7 @@ import (
var (
configPath string
config configurator.Config
verbose bool
targets []string
outputPath string
)
@ -36,7 +37,8 @@ func Execute() {
func init() {
cobra.OnInitialize(initConfig)
rootCmd.PersistentFlags().StringVarP(&configPath, "config", "c", "", "set the config path")
rootCmd.PersistentFlags().StringVarP(&configPath, "config", "c", "./config.yaml", "set the config path")
rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "set to enable verbose output")
}
func initConfig() {

2
cmd/serve.go
View file

@ -33,5 +33,7 @@ var serveCmd = &cobra.Command{
func init() {
serveCmd.Flags().StringVar(&config.Server.Host, "host", config.Server.Host, "set the server host")
serveCmd.Flags().IntVar(&config.Server.Port, "port", config.Server.Port, "set the server port")
serveCmd.Flags().StringVar(&config.Options.JwksUri, "jwks-uri", config.Options.JwksUri, "set the JWKS url to fetch public key")
serveCmd.Flags().IntVar(&config.Options.JwksRetries, "jwks-fetch-retries", config.Options.JwksRetries, "set the JWKS fetch retry count")
rootCmd.AddCommand(serveCmd)
}

1
go.mod
View file

@ -9,6 +9,7 @@ require (
github.com/nikolalohinski/gonja/v2 v2.2.0
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.8.0
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8
gopkg.in/yaml.v2 v2.4.0
)

10
go.sum
View file

@ -19,8 +19,8 @@ github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEe
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 h1:yAJXTCF9TqKcTiHJAE8dj7HMvPfh66eeA2JYW7eFpSE=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
@ -92,6 +92,8 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8 h1:yixxcjnhBmY0nkL253HFVIm0JsFHwrHdT3Yh6szTnfY=
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8/go.mod h1:jj3sYF3dwk5D+ghuXyeI3r5MFf+NT2An6/9dOA95KSI=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@ -132,8 +134,8 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc=
golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg=
golang.org/x/tools v0.22.0 h1:gqSGLZqv+AI9lIQzniJ0nZDRG5GBPsSi+DRNHWNz6yA=
golang.org/x/tools v0.22.0/go.mod h1:aCwcsjqvq7Yqt6TNyX7QMU2enbQ/Gt0bo6krSeEri+c=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=

114
internal/auth.go Normal file
View file

@ -0,0 +1,114 @@
package configurator
import (
"context"
"encoding/json"
"fmt"
"net/http"
"slices"
"github.com/OpenCHAMI/jwtauth/v5"
"github.com/lestrrat-go/jwx/jwk"
)
func VerifyClaims(testClaims []string, r *http.Request) (bool, error) {
// extract claims from JWT
_, claims, err := jwtauth.FromContext(r.Context())
if err != nil {
return false, fmt.Errorf("failed to get claims(s) from token: %v", err)
}
// verify that each one of the test claims are included
for _, testClaim := range testClaims {
_, ok := claims[testClaim]
if !ok {
return false, fmt.Errorf("failed to verify claim(s) from token: %s", testClaim)
}
}
return true, nil
}
func VerifyScope(testScopes []string, r *http.Request) (bool, error) {
// extract the scopes from JWT
var scopes []string
_, claims, err := jwtauth.FromContext(r.Context())
if err != nil {
return false, fmt.Errorf("failed to get claim(s) from token: %v", err)
}
appendScopes := func(slice []string, scopeClaim any) []string {
switch scopeClaim.(type) {
case []any:
// convert all scopes to str and append
for _, s := range scopeClaim.([]any) {
switch s.(type) {
case string:
slice = append(slice, s.(string))
}
}
case []string:
slice = append(slice, scopeClaim.([]string)...)
}
return slice
}
// check for and append both "scp" and "scope" claims
v, ok := claims["scp"]
if ok {
scopes = appendScopes(scopes, v)
}
v, ok = claims["scope"]
if ok {
scopes = appendScopes(scopes, v)
}
// check for both 'scp' and 'scope' claims for scope
scopeClaim, ok := claims["scp"]
if ok {
switch scopeClaim.(type) {
case []any:
// convert all scopes to str and append
for _, s := range scopeClaim.([]any) {
switch s.(type) {
case string:
scopes = append(scopes, s.(string))
}
}
case []string:
scopes = append(scopes, scopeClaim.([]string)...)
}
}
scopeClaim, ok = claims["scope"]
if ok {
scopes = append(scopes, scopeClaim.([]string)...)
}
// verify that each of the test scopes are included
for _, testScope := range testScopes {
index := slices.Index(scopes, testScope)
if index < 0 {
return false, fmt.Errorf("invalid or missing scope")
}
}
// NOTE: should this be ok if no scopes were found?
return true, nil
}
func FetchPublicKeyFromURL(url string) (*jwtauth.JWTAuth, error) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
set, err := jwk.Fetch(ctx, url)
if err != nil {
return nil, fmt.Errorf("%v", err)
}
jwks, err := json.Marshal(set)
if err != nil {
return nil, fmt.Errorf("failed to marshal JWKS: %v", err)
}
tokenAuth, err := jwtauth.NewKeySet(jwks)
if err != nil {
return nil, fmt.Errorf("failed to initialize JWKS: %v", err)
}
return tokenAuth, nil
}

111
internal/client.go
View file

@ -6,49 +6,110 @@ import (
"fmt"
"io"
"net/http"
"github.com/OpenCHAMI/configurator/internal/util"
)
type SmdClient struct {
http.Client
Host string
Port int
AccessToken string
Host string `yaml:"host"`
Port int `yaml:"port"`
AccessToken string `yaml:"access-token"`
}
func (client *SmdClient) FetchDNS(config *Config) error {
// fetch DNS related information from SMD's endpoint:
return nil
type Params = map[string]any
type Option func(Params)
func WithVerbose() Option {
return func(p util.Params) {
p["verbose"] = true
}
}
func (client *SmdClient) FetchEthernetInterfaces() ([]EthernetInterface, error) {
func NewParams() Params {
return Params{
"verbose": false,
}
}
// Fetch the ethernet interfaces from SMD service using its API. An access token may be required if the SMD
// service SMD_JWKS_URL envirnoment variable is set.
func (client *SmdClient) FetchEthernetInterfaces(opts ...util.Option) ([]EthernetInterface, error) {
// make request to SMD endpoint
b, err := client.makeRequest("/Inventory/EthernetInterfaces")
if err != nil {
return nil, fmt.Errorf("failed to read HTTP response: %v", err)
}
// unmarshal response body JSON and extract in object
eths := []EthernetInterface{} // []map[string]any{}
err = json.Unmarshal(b, &eths)
if err != nil {
return nil, fmt.Errorf("failed to unmarshal response: %v", err)
}
// print what we got if verbose is set
params := util.GetParams(opts...)
if verbose, ok := params["verbose"].(bool); ok {
if verbose {
fmt.Printf("Ethernet Interfaces: %v\n", string(b))
}
}
return eths, nil
}
// Fetch the components from SMD using its API. An access token may be required if the SMD
// service SMD_JWKS_URL envirnoment variable is set.
func (client *SmdClient) FetchComponents(opts ...util.Option) ([]Component, error) {
// make request to SMD endpoint
b, err := client.makeRequest("/State/Components")
if err != nil {
return nil, fmt.Errorf("failed to read HTTP response: %v", err)
}
// unmarshal response body JSON and extract in object
comps := []Component{}
err = json.Unmarshal(b, &comps)
if err != nil {
return nil, fmt.Errorf("failed to unmarshal response: %v", err)
}
// print what we got if verbose is set
params := util.GetParams(opts...)
if verbose, ok := params["verbose"].(bool); ok {
if verbose {
fmt.Printf("Components: %v\n", string(b))
}
}
return comps, nil
}
func (client *SmdClient) makeRequest(endpoint string) ([]byte, error) {
if client == nil {
return nil, fmt.Errorf("client is nil")
}
// fetch DHCP related information from SMD's endpoint:
url := fmt.Sprintf("%s:%d/hsm/v2/Inventory/EthernetInterfaces", client.Host, client.Port)
req, err := http.NewRequest(http.MethodGet, url, bytes.NewBuffer([]byte{}))
// include access token in authorzation header if found
if client.AccessToken != "" {
req.Header.Add("Authorization", "Bearer "+client.AccessToken)
}
// fetch DHCP related information from SMD's endpoint:
url := fmt.Sprintf("%s:%d/hsm/v2%s", client.Host, client.Port, endpoint)
req, err := http.NewRequest(http.MethodGet, url, bytes.NewBuffer([]byte{}))
if err != nil {
return nil, fmt.Errorf("failed to create new HTTP request: %v", err)
}
// include access token in authorzation header if found
// NOTE: This shouldn't be needed for this endpoint since it's public
if client.AccessToken != "" {
req.Header.Add("Authorization", "Bearer "+client.AccessToken)
}
// make the request to SMD
res, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to make request: %v", err)
}
b, err := io.ReadAll(res.Body)
if err != nil {
return nil, fmt.Errorf("failed to read HTTP response: %v", err)
}
// unmarshal JSON and extract
eths := []EthernetInterface{} // []map[string]any{}
json.Unmarshal(b, &eths)
fmt.Printf("ethernet interfaces: %v\n", string(b))
return eths, nil
// read the contents of the response body
return io.ReadAll(res.Body)
}

44
internal/config.go
View file

@ -8,45 +8,53 @@ import (
"gopkg.in/yaml.v2"
)
type Options struct {
JwksUri string `yaml:"jwks-uri"`
JwksRetries int `yaml:"jwks-retries"`
type Options struct{}
type Jwks struct {
Uri string `yaml:"uri"`
Retries int `yaml:"retries"`
}
type Server struct {
Host string `yaml:"host"`
Port int `yaml:"port"`
Jwks Jwks `yaml:"jwks"`
}
type Config struct {
Version string `yaml:"version"`
SmdHost string `yaml:"smd-host"`
SmdPort int `yaml:"smd-port"`
Server Server `yaml:"server"`
SmdClient SmdClient `yaml:"smd"`
AccessToken string `yaml:"access-token"`
TemplatePaths map[string]string `yaml:"templates"`
Server Server `yaml:"server"`
Plugins []string `yaml:"plugins"`
Options Options `yaml:"options"`
}
func NewConfig() Config {
return Config{
Version: "",
SmdHost: "http://127.0.0.1",
SmdPort: 27779,
TemplatePaths: map[string]string{
"dnsmasq": "templates/dhcp/dnsmasq.conf",
"syslog": "templates/syslog/",
"ansible": "templates/ansible",
"powerman": "templates/powerman",
"conman": "templates/conman",
SmdClient: SmdClient{
Host: "http://127.0.0.1",
Port: 27779,
},
TemplatePaths: map[string]string{
"dnsmasq": "templates/dnsmasq.jinja",
"syslog": "templates/syslog.jinja",
"ansible": "templates/ansible.jinja",
"powerman": "templates/powerman.jinja",
"conman": "templates/conman.jinja",
},
Plugins: []string{},
Server: Server{
Host: "127.0.0.1",
Port: 3334,
Jwks: Jwks{
Uri: "",
Retries: 5,
},
},
Options: Options{
JwksUri: "",
JwksRetries: 5,
},
Options: Options{},
}
}

11
internal/configurator.go
View file

@ -15,8 +15,11 @@ type EthernetInterface struct {
IpAddresses []IPAddr
}
type DHCP struct {
Hostname string
MacAddress string
IpAddress []IPAddr
type Component struct {
}
type Node struct {
}
type BMC struct {
}

166
internal/generator/generator.go
View file

@ -2,50 +2,162 @@ package generator
import (
"bytes"
"fmt"
"os"
"plugin"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/util"
"github.com/nikolalohinski/gonja/v2"
"github.com/nikolalohinski/gonja/v2/exec"
)
type Generator struct {
Type string
Template string
type Mappings = map[string]any
type Generator interface {
GetName() string
GetGroups() []string
Generate(config *configurator.Config, opts ...util.Option) ([]byte, error)
}
func New() *Generator {
return &Generator{}
func LoadPlugin(path string) (Generator, error) {
p, err := plugin.Open(path)
if err != nil {
return nil, fmt.Errorf("failed to load plugin: %v", err)
}
symbol, err := p.Lookup("Generator")
if err != nil {
return nil, fmt.Errorf("failed to look up symbol: %v", err)
}
gen, ok := symbol.(Generator)
if !ok {
return nil, fmt.Errorf("failed to load the correct symbol type")
}
return gen, nil
}
func (g *Generator) GenerateDNS(config *configurator.Config) {
// generate file using jinja template
// TODO: load template file for DNS
// TODO: substitute DNS data fetched from SMD
// TODO: print generated config file to STDOUT
func LoadPlugins(dirpath string, opts ...util.Option) (map[string]Generator, error) {
// check if verbose option is supplied
var (
gens = make(map[string]Generator)
params = util.GetParams(opts...)
)
items, _ := os.ReadDir(dirpath)
var LoadGenerator = func(path string) (Generator, error) {
// load each generator plugin
p, err := plugin.Open(path)
if err != nil {
return nil, fmt.Errorf("failed to load plugin: %v", err)
}
// lookup symbol in plugin
symbol, err := p.Lookup("Generator")
if err != nil {
return nil, fmt.Errorf("failed to look up symbol: %v", err)
}
// assert that the loaded symbol is the correct type
gen, ok := symbol.(Generator)
if !ok {
return nil, fmt.Errorf("failed to load the correct symbol type")
}
return gen, nil
}
for _, item := range items {
if item.IsDir() {
subitems, _ := os.ReadDir(item.Name())
for _, subitem := range subitems {
if !subitem.IsDir() {
gen, err := LoadGenerator(subitem.Name())
if err != nil {
fmt.Printf("failed to load generator in directory '%s': %v\n", item.Name(), err)
continue
}
if verbose, ok := params["verbose"].(bool); ok {
if verbose {
fmt.Printf("found plugin '%s'\n", item.Name())
}
}
gens[gen.GetName()] = gen
}
}
} else {
gen, err := LoadGenerator(dirpath + item.Name())
if err != nil {
fmt.Printf("failed to load generator: %v\n", err)
continue
}
if verbose, ok := params["verbose"].(bool); ok {
if verbose {
fmt.Printf("found plugin '%s'\n", dirpath+item.Name())
}
}
gens[gen.GetName()] = gen
}
}
return gens, nil
}
func (g *Generator) GenerateDHCP(config *configurator.Config, eths []configurator.EthernetInterface) ([]byte, error) {
// generate file using gonja template
path := config.TemplatePaths[g.Template]
fmt.Printf("path: %s\neth count: %v\n", path, len(eths))
func WithTemplate(_template string) util.Option {
return func(p util.Params) {
if p != nil {
p["template"] = _template
}
}
}
func WithType(_type string) util.Option {
return func(p util.Params) {
if p != nil {
p["type"] = _type
}
}
}
func WithClient(client configurator.SmdClient) util.Option {
return func(p util.Params) {
p["client"] = client
}
}
// Syntactic sugar generic function to get parameter from util.Params.
func Get[T any](params util.Params, key string) *T {
if v, ok := params[key].(T); ok {
return &v
}
return nil
}
// Helper function to get client in generator plugins.
func GetClient(params util.Params) *configurator.SmdClient {
return Get[configurator.SmdClient](params, "client")
}
func GetParams(opts ...util.Option) util.Params {
params := util.Params{}
for _, opt := range opts {
opt(params)
}
return params
}
func Generate(g Generator, config *configurator.Config, opts ...util.Option) {
g.Generate(config, opts...)
}
func ApplyTemplate(path string, mappings map[string]any) ([]byte, error) {
data := exec.NewContext(mappings)
// load jinja template from file
t, err := gonja.FromFile(path)
if err != nil {
return nil, fmt.Errorf("failed to read template from file: %v", err)
}
template := "# ========== GENERATED BY OCHAMI CONFIGURATOR ==========\n"
for _, eth := range eths {
if eth.Type == "NodeBMC" {
template += "dhcp-host=" + eth.MacAddress + "," + eth.ComponentId + "," + eth.IpAddresses[0].IpAddress + "\n"
} else {
template += "dhcp-host=" + eth.MacAddress + "," + eth.ComponentId + "," + eth.IpAddresses[0].IpAddress + "\n"
}
}
template += "# ======================================================"
data := exec.NewContext(map[string]any{
"hosts": template,
})
// execute/render jinja template
b := bytes.Buffer{}
if err = t.Execute(&b, data); err != nil {
return nil, fmt.Errorf("failed to execute: %v", err)

46
internal/generator/plugins/conman/conman.go Normal file
View file

@ -0,0 +1,46 @@
package main
import (
"bytes"
"fmt"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/generator"
"github.com/OpenCHAMI/configurator/internal/util"
"github.com/nikolalohinski/gonja/v2"
"github.com/nikolalohinski/gonja/v2/exec"
)
type Conman struct{}
func (g *Conman) GetName() string {
return "conman"
}
func (g *Conman) GetGroups() []string {
return []string{"conman"}
}
func (g *Conman) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
params := generator.GetParams(opts...)
var (
template = params["template"].(string)
path = config.TemplatePaths[template]
)
data := exec.NewContext(map[string]any{})
t, err := gonja.FromFile(path)
if err != nil {
return nil, fmt.Errorf("failed to read template from file: %v", err)
}
output := "# ========== GENERATED BY OCHAMI CONFIGURATOR ==========\n"
output += "# ======================================================"
b := bytes.Buffer{}
if err = t.Execute(&b, data); err != nil {
return nil, fmt.Errorf("failed to execute: %v", err)
}
return b.Bytes(), nil
}
var Generator Conman

22
internal/generator/plugins/coredhcp/coredhcp.go Normal file
View file

@ -0,0 +1,22 @@
package main
import (
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/util"
)
type CoreDhcp struct{}
func (g *CoreDhcp) GetName() string {
return "coredhcp"
}
func (g *CoreDhcp) GetGroups() []string {
return []string{"coredhcp"}
}
func (g *CoreDhcp) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
return nil, nil
}
var Generator CoreDhcp

89
internal/generator/plugins/dnsmasq/dnsmasq.go Normal file
View file

@ -0,0 +1,89 @@
package main
import (
"fmt"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/generator"
"github.com/OpenCHAMI/configurator/internal/util"
)
type DnsMasq struct{}
func TestGenerateDnsMasq() {
var (
g = DnsMasq{}
config = &configurator.Config{}
client = configurator.SmdClient{}
)
g.Generate(
config,
generator.WithTemplate("dnsmasq"),
generator.WithClient(client),
)
}
func (g *DnsMasq) GetName() string {
return "dnsmasq"
}
func (g *DnsMasq) GetGroups() []string {
return []string{"dnsmasq"}
}
func (g *DnsMasq) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
// make sure we have a valid config first
if config == nil {
return nil, fmt.Errorf("invalid config (config is nil)")
}
// set all the defaults for variables
var (
params = generator.GetParams(opts...)
template = params["template"].(string) // required param
path = config.TemplatePaths[template]
eths []configurator.EthernetInterface = nil
err error = nil
)
// if we have a client, try making the request for the ethernet interfaces
if client, ok := params["client"].(configurator.SmdClient); ok {
eths, err = client.FetchEthernetInterfaces(opts...)
if err != nil {
return nil, fmt.Errorf("failed to fetch ethernet interfaces with client: %v", err)
}
}
// check if we have the required params first
if eths == nil {
return nil, fmt.Errorf("invalid ethernet interfaces (variable is nil)")
}
if len(eths) <= 0 {
return nil, fmt.Errorf("no ethernet interfaces found")
}
// print message if verbose param found
if verbose, ok := params["verbose"].(bool); ok {
if verbose {
fmt.Printf("path: %s\neth count: %v\n", path, len(eths))
}
}
// format output to write to config file
output := "# ========== GENERATED BY OCHAMI CONFIGURATOR ==========\n"
for _, eth := range eths {
if eth.Type == "NodeBMC" {
output += "dhcp-host=" + eth.MacAddress + "," + eth.ComponentId + "," + eth.IpAddresses[0].IpAddress + "\n"
} else {
output += "dhcp-host=" + eth.MacAddress + "," + eth.ComponentId + "," + eth.IpAddresses[0].IpAddress + "\n"
}
}
output += "# ======================================================"
// apply template substitutions and return output as byte array
return generator.ApplyTemplate(path, generator.Mappings{
"hosts": output,
})
}
var Generator DnsMasq

22
internal/generator/plugins/powerman/powerman.go Normal file
View file

@ -0,0 +1,22 @@
package main
import (
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/util"
)
type Powerman struct{}
func (g *Powerman) GetName() string {
return "powerman"
}
func (g *Powerman) GetGroups() []string {
return []string{"powerman"}
}
func (g *Powerman) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
return nil, nil
}
var Generator Powerman

22
internal/generator/plugins/syslog/syslog.go Normal file
View file

@ -0,0 +1,22 @@
package main
import (
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/util"
)
type Syslog struct{}
func (g *Syslog) GetName() string {
return "syslog"
}
func (g *Syslog) GetGroups() []string {
return []string{"log"}
}
func (g *Syslog) Generate(config *configurator.Config, opts ...util.Option) ([]byte, error) {
return nil, nil
}
var Generator Syslog

44
internal/generator/plugins/warewulf/warewulf.go Normal file
View file

@ -0,0 +1,44 @@
package main
import (
"bytes"
"fmt"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/nikolalohinski/gonja/v2"
"github.com/nikolalohinski/gonja/v2/exec"
)
type Warewulf struct{}
func (g *Warewulf) GetName() string {
return "warewulf"
}
func (g *Warewulf) GetGroups() []string {
return []string{"warewulf"}
}
func (g *Warewulf) Generate(config *configurator.Config, template string) ([]byte, error) {
var (
path = config.TemplatePaths[template]
)
t, err := gonja.FromFile(path)
if err != nil {
return nil, fmt.Errorf("failed to read template from file: %v", err)
}
output := "# ========== GENERATED BY OCHAMI CONFIGURATOR ==========\n"
output += "# ======================================================"
data := exec.NewContext(map[string]any{
"hosts": output,
})
b := bytes.Buffer{}
if err = t.Execute(&b, data); err != nil {
return nil, fmt.Errorf("failed to execute: %v", err)
}
return nil, nil
}
var Generator Warewulf

2
internal/schema.go Normal file
View file

@ -0,0 +1,2 @@
// TODO: implement a way to fetch schemas from node orchestrator
package configurator

73
internal/server/server.go
View file

@ -9,7 +9,6 @@ import (
"time"
configurator "github.com/OpenCHAMI/configurator/internal"
"github.com/OpenCHAMI/configurator/internal/generator"
"github.com/OpenCHAMI/jwtauth/v5"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
@ -22,29 +21,33 @@ var (
type Server struct {
*http.Server
JwksUri string `yaml:"jwks-uri"`
}
func New() *Server {
return &Server{
Server: &http.Server{},
Server: &http.Server{
Addr: "localhost:3334",
},
JwksUri: "",
}
}
func (s *Server) Start(config *configurator.Config) error {
// create client just for the server to use to fetch data from SMD
client := &configurator.SmdClient{
Host: config.SmdHost,
Port: config.SmdPort,
_ = &configurator.SmdClient{
Host: config.SmdClient.Host,
Port: config.SmdClient.Port,
}
// set the server address with config values
s.Server.Addr = fmt.Sprintf("%s:%d", config.Server.Host, config.Server.Port)
// fetch JWKS public key from authorization server
if config.Options.JwksUri != "" && tokenAuth == nil {
for i := 0; i < config.Options.JwksRetries; i++ {
if config.Server.Jwks.Uri != "" && tokenAuth == nil {
for i := 0; i < config.Server.Jwks.Retries; i++ {
var err error
tokenAuth, err = configurator.FetchPublicKeyFromURL(config.Options.JwksUri)
tokenAuth, err = configurator.FetchPublicKeyFromURL(config.Server.Jwks.Uri)
if err != nil {
logrus.Errorf("failed to fetch JWKS: %w", err)
continue
@ -58,42 +61,42 @@ func (s *Server) Start(config *configurator.Config) error {
router.Use(middleware.RedirectSlashes)
router.Use(middleware.Timeout(60 * time.Second))
router.Group(func(r chi.Router) {
if config.Options.JwksUri != "" {
if config.Server.Jwks.Uri != "" {
r.Use(
jwtauth.Verifier(tokenAuth),
jwtauth.Authenticator(tokenAuth),
)
}
r.HandleFunc("/target", func(w http.ResponseWriter, r *http.Request) {
g := generator.Generator{
Type: r.URL.Query().Get("type"),
Template: r.URL.Query().Get("template"),
}
// g := generator.Generator{
// Type: r.URL.Query().Get("type"),
// Template: r.URL.Query().Get("template"),
// }
// NOTE: we probably don't want to hardcode the types, but should do for now
if g.Type == "dhcp" {
// fetch eths from SMD
eths, err := client.FetchEthernetInterfaces()
if err != nil {
logrus.Errorf("failed to fetch DHCP metadata: %v\n", err)
w.Write([]byte("An error has occurred"))
return
}
if len(eths) <= 0 {
logrus.Warnf("no ethernet interfaces found")
w.Write([]byte("no ethernet interfaces found"))
return
}
// generate a new config from that data
// if _type == "dhcp" {
// // fetch eths from SMD
// eths, err := client.FetchEthernetInterfaces()
// if err != nil {
// logrus.Errorf("failed to fetch DHCP metadata: %v\n", err)
// w.Write([]byte("An error has occurred"))
// return
// }
// if len(eths) <= 0 {
// logrus.Warnf("no ethernet interfaces found")
// w.Write([]byte("no ethernet interfaces found"))
// return
// }
// // generate a new config from that data
b, err := g.GenerateDHCP(config, eths)
if err != nil {
logrus.Errorf("failed to generate DHCP: %v", err)
w.Write([]byte("An error has occurred."))
return
}
w.Write(b)
}
// // b, err := g.GenerateDHCP(config, eths)
// if err != nil {
// logrus.Errorf("failed to generate DHCP: %v", err)
// w.Write([]byte("An error has occurred."))
// return
// }
// w.Write(b)
// }
})
r.HandleFunc("/templates", func(w http.ResponseWriter, r *http.Request) {
// TODO: handle GET request

37
internal/util/params.go Normal file
View file

@ -0,0 +1,37 @@
package util
import (
"slices"
"golang.org/x/exp/maps"
)
type Params = map[string]any
type Option func(Params)
func GetParams(opts ...Option) Params {
params := Params{}
for _, opt := range opts {
opt(params)
}
return params
}
func OptionExists(params Params, opt string) bool {
var k []string = maps.Keys(params)
return slices.Contains(k, opt)
}
// Assert that the options exists within the params map
func AssertOptionsExist(params Params, opts ...string) []string {
foundKeys := []string{}
for k := range params {
index := slices.IndexFunc(opts, func(s string) bool {
return s == k
})
if index >= 0 {
foundKeys = append(foundKeys, k)
}
}
return foundKeys
}