209 lines
No EOL
7.1 KiB
Markdown
209 lines
No EOL
7.1 KiB
Markdown
# << Makeshift >>
|
|
|
|
The `makeshift` tool is a service that serves files and CLI that downloads them with a couple of handy features baked-in. Although the CLI and server component function more like a glorified FTP, the power of this tool comes from the plugin system. For example, the file cobbler is built to run external plugins for more advanced processing files before serving them (e.g. fetching from a data source, rendering Jinja 2 templates, etc.).
|
|
|
|
|
|
## Building the Tool
|
|
|
|
|
|
The `makeshift` tool is built using standard `go` build tools. To get started, clone the project, download the dependencies, and build the project:
|
|
|
|
```bash
|
|
git clone https://git.towk2.me/towk/makeshift.git
|
|
go mod tidy
|
|
go build
|
|
```
|
|
|
|
This will build the main driver program with the default generators that are found in the `pkg/generators` directory.
|
|
|
|
> [!NOTE]
|
|
> The project does not current separate the client, server, and plugin components using build tags, but will eventually. This will allow users to only compile and distribute specific parts of the tool with limited functionality.
|
|
|
|
|
|
## Basic Examples
|
|
|
|
Here are some of the common commands you may want to try right off the bat (aside from `makeshift help` of course). The commands below that do not include the `--host`, `--path`, or `--root` flags are set using the environment variables.
|
|
|
|
```bash
|
|
export MAKESHIFT_HOST=localhost
|
|
export MAKESHIFT_PATH=/test
|
|
export MAKESHIFT_ROOT=./test
|
|
```
|
|
|
|
Start the server. The `--init` flag with create the default files and directory to get started at the `--root` path.
|
|
|
|
```bash
|
|
makeshift serve --root $HOME/apps/makeshift/server --init
|
|
```
|
|
|
|
From here, you might want to see what files are available by default.
|
|
|
|
```bash
|
|
# list the files in the root directory
|
|
makeshift list
|
|
|
|
# list files store in the template directory with a specified host
|
|
makeshift list --host http://localhost:5050 --path templates
|
|
|
|
# list all available plugins
|
|
makeshift list plugins
|
|
|
|
# list specific plugin (for plugin info, use 'makeshift plugin info` instead)
|
|
makeshift list plugins jinja2
|
|
|
|
# list all available profiles
|
|
makeshift list profiles
|
|
|
|
# list specific profile information
|
|
makeshift list profiles default
|
|
```
|
|
|
|
Then, we can start downloading some files or directories (as archives).
|
|
|
|
```bash
|
|
# download all data (notice --host and --port are not set here)
|
|
makeshift download
|
|
|
|
# download the 'help.txt' file without processing (i.e. using plugins)
|
|
makeshift download --host http://localhost:5050 --path help.txt
|
|
|
|
# download files with rendering using Jinja 2 plugin and default profile
|
|
makeshift download -p help.txt --plugins jinja2 --profile default
|
|
|
|
# download directory with rendering using plugins to fetch data and render
|
|
# using a custom 'compute' profile
|
|
makeshift download -p templates --plugins smd,jinja2 --profile compute
|
|
|
|
# do everything in the above example but extract and remove archive
|
|
makeshift download -p templates --plugins smd,jinja2 --profile compute -xr
|
|
|
|
# download a raw plugin
|
|
makeshift download plugin jinja2
|
|
|
|
# download a profile
|
|
makeshift download profile default
|
|
```
|
|
|
|
> [!NOTE]
|
|
> Plugins are ran in order specified with the `--plugins` flag, which means if you're creating a plugin to write to a data store and then read in a subsequent plugin, the order specified with the CLI matters!
|
|
|
|
(WIP) Files, directories, profiles, and plugins will eventually be able to be uploaded to the server.
|
|
|
|
```bash
|
|
# upload a file or directory (recursively)
|
|
makeshift upload
|
|
makeshift upload --host http://localhost:5050 --path help.txt
|
|
```
|
|
|
|
> [!NOTE]
|
|
> Although every command has a `curl` equivalent, it is better to use the CLI since it has other features such as extracting and remove archives after downloading and saving archives as files automatically.
|
|
|
|
## Creating Plugins
|
|
|
|
The `makeshift` tool defines a plugin as an interface that can be implemented and compiled.
|
|
|
|
```go
|
|
type Plugin interface {
|
|
Name() string
|
|
Version() string
|
|
Description() string
|
|
Metadata() Metadata
|
|
|
|
Init() error
|
|
Run(data storage.KVStore, args []string) error
|
|
Cleanup() error
|
|
}
|
|
```
|
|
|
|
Plugins can *literally* contain whatever you want and is written in Go. Here is a simple example implementation to demonstrate how that is done which we will save at `src/example.go`.
|
|
|
|
```go
|
|
type Example struct{}
|
|
|
|
func (p *Example) Name() string { return "example" }
|
|
func (p *Example) Version() string { return "v0.0.1-alpha" }
|
|
func (p *Example) Description() string { return "An example plugin" }
|
|
func (p *Example) Metadata() map[string]string {
|
|
return makeshift.Metadata{
|
|
"author": map[string]any{
|
|
"name": "John Smith",
|
|
"email": "john.smith@example",
|
|
"links": []string{
|
|
"https://example.com",
|
|
},
|
|
},
|
|
}
|
|
}
|
|
|
|
func (p *Example) Init() error {
|
|
// Initialize the plugin if necessary.
|
|
return nil
|
|
}
|
|
|
|
func (p *Example) Run(data storage.KVStore, args []string) error {
|
|
// Plugins can read and write to a data stores passed in.
|
|
// See the 'jinja2' plugin for reading and 'smd' plugin for writing.
|
|
return nil
|
|
}
|
|
|
|
func (p *Example) Clean() error {
|
|
// Clean up resources if necessary.
|
|
return nil
|
|
}
|
|
|
|
// This MUST be included to find the symbol in the main driver executable.
|
|
var Makeshift Example
|
|
```
|
|
|
|
Then, we can use the built-in `makeshift plugins compile` command to compile it.
|
|
|
|
```bash
|
|
makeshift plugins compile src/example.go -o $MAKESHIFT_ROOT/plugins/example.so
|
|
```
|
|
|
|
> [!TIP]
|
|
> Make sure you move all of your plugins to `$MAKESHIFT_ROOT/plugins` to use them and should have an `*.so` name for lookup. For example, to use a custom plugin with `makeshift download -p templates/hosts.j2 --plugins my-plugin`, there has to a plugin `$MAKESHIFT_ROOT/plugins/my-plugin.so`.
|
|
|
|
## Creating Profiles
|
|
|
|
On the other hand, profiles are simply objects that contain data used to populate data stores. The `makeshift` tool does not currently use all fields of a profile which will likely be removed in the near future.
|
|
|
|
```go
|
|
type Profile struct {
|
|
ID string `json:"id"` // profile ID
|
|
Description string `json:"description,omitempty"` // profile description
|
|
Tags []string `json:"tags,omitempty"` // tags used for ...
|
|
Paths []string `json:"paths,omitempty"` // paths to download
|
|
Plugins []string `json:"plugins,omitempty"` // plugins to run
|
|
Data map[string]any `json:"data,omitempty"` // include render data
|
|
}
|
|
```
|
|
|
|
Profiles can be created using JSON. See the example in `$MAKESHIFT_ROOT/profiles/default.json`.
|
|
|
|
```json
|
|
{
|
|
"id": "default",
|
|
"description": "Makeshift default profile",
|
|
"data": {
|
|
"host": "localhost",
|
|
"path": "/test",
|
|
"server_root": "./test"
|
|
}
|
|
}
|
|
```
|
|
|
|
> [!TIP]
|
|
> Make sure that you store your custom profiles in `$MAKESHIFT_ROOT/profiles` and that you set the name you want to use for lookup with a `*.json` extension (e.g. `compute.json`).
|
|
|
|
## TODO: Missing Features
|
|
|
|
There are some features still missing that will be added later.
|
|
|
|
1. Uploading files and directories
|
|
2. Uploading new profiles and plugins
|
|
3. Running `makeshift` locally with profiles and plugins
|
|
4. Plugin to add user data for one-time use without creating a profile
|
|
5. Protected routes that require authentication
|
|
6. Configuration file for persistent runs
|
|
7. `Dockerfile` and `docker-compose.yml` files |