424 lines
13 KiB
Markdown
424 lines
13 KiB
Markdown
# Proxmox Deploy
|
|
[](https://drone.broillet.ch/mathieu/ProxmoxDeploy)
|
|
|
|
|
|
|
|
## Description
|
|
Proxmox Deploy is a little script to manage my HomeLab with JSON file.
|
|
|
|
## Why?
|
|
As my homelab was growing I realised that it was harder and harder to keep everything in sync and up to date.
|
|
So I decided to create a script to manage my Proxmox homelab.
|
|
|
|
# How it works
|
|
The concept is simple, you have a Git repository with a the following structure:
|
|
```
|
|
.
|
|
├── config.json
|
|
├── lxc
|
|
│ ├── <id>
|
|
│ │ ├── config.json
|
|
│ │ ├── <your files>
|
|
│ │ └── <your folders>
|
|
│ └── <id>
|
|
│ ├── ...
|
|
│── qemu
|
|
│ ├── <id>
|
|
│ │ ├── config.json
|
|
│ │ ├── <your files>
|
|
│ │ └── <your folders>
|
|
│ └── <id>
|
|
│ ├── ...
|
|
│── scripts
|
|
│ ├── <your scripts>
|
|
│ └── ...
|
|
```
|
|
*See below for more information about the structure and differents files*
|
|
|
|
PDJ (Proxmox Deploy JSON) is a program that will read that repository files and execute the necessary commands to create/update your LXC/VM.
|
|
Now ideally you have some sort of Git actions like Drone/GitHub(/Gitea) Actions to run PDJ automatically when you push a change, which will result in your homelab being updated almost instantly.
|
|
|
|
If you don't, no big deal, you'll just have to manually clone and update your repo, then start PDJ *(you could use a crontab with regular intervals)*
|
|
|
|
# Usage
|
|
## Download
|
|
Download the pre-compiled binaries from the release page or build it yourself.
|
|
|
|
## Build it yourself
|
|
```bash
|
|
# Build on Debian
|
|
git clone <url of this repo>
|
|
cd ProxmoxDeploy
|
|
apt update && apt install -y build-essential patchelf
|
|
pip install nuitka
|
|
pip install -r requirements.txt
|
|
python -m nuitka --onefile run.py --output-filename="ProxmoxDeploy"
|
|
```
|
|
*Also see the ``Dockerfile`` and ``.drone.yml`` for more information.*
|
|
|
|
## Run it
|
|
```bash
|
|
# Run it
|
|
./ProxmoxDeploy --repo /path/to/repo
|
|
```
|
|
|
|
|
|
# Documentation
|
|
## Configuration
|
|
### General
|
|
Before configuring your LXC and VM, you must decide how you'll run this program.
|
|
As it requires SSH for some actions, you have two options:
|
|
- Run it directly on the Proxmox VE host **(recommended)** *(refered as `local`)*
|
|
- No configuration needed
|
|
- You can use Git(ea)/Drone actions to run it automatically as soon as a change is pushed.
|
|
- Run it on another machine and connect via SSH
|
|
- You'll need to setup passwordless SSH connection between your machine and the Proxmox VE host.
|
|
|
|
### Proxmox VE
|
|
The Proxmox VE configuration is located in the `config.json` file.
|
|
```json
|
|
{
|
|
"pve":{
|
|
"host": "<pve host ip/hostname>",
|
|
"user": "<user to connect via ssh>", // usually root
|
|
"port": 22, // ssh port
|
|
"local": false // set to true if this program is running directly on the PVE without needing ssh
|
|
}
|
|
}
|
|
```
|
|
|
|
### LXC
|
|
To create a LXC, you need to create a JSON file in the `resources/lxc/<id of lxc>` folder.
|
|
You can look at the example with ID 100 in the `resources/lxc/100` folder.
|
|
|
|
Here is an example config with all available parameters :
|
|
*Note : this is not valid JSON, it's just for documentation purposes, if you want to copy the file, use the one in the resources folder*
|
|
```json
|
|
{
|
|
"lxc_hostname": "traefik", //hostname of the lxc
|
|
|
|
// the os block contains the os type
|
|
// alpine | archlinux | centos | debian | devuan | fedora | gentoo | nixos | opensuse | ubuntu | unmanaged
|
|
"os": {
|
|
"name": "alpine",
|
|
"release": "3.17"
|
|
},
|
|
|
|
// the resources block contains the resources given to the lxc
|
|
"resources": {
|
|
"cpu": "2", // number of cpu cores
|
|
"memory": "1024", // memory (ram) in MB
|
|
"swap": "256", // swap in MB
|
|
"disk": "8", // disk size in GB
|
|
"storage": "local-lvm" // the proxmox storage to use
|
|
},
|
|
|
|
// the network block contains the network configuration
|
|
"network": {
|
|
"bridge": "vmbr0", // the proxmox bridge to use, vmbr0 is the default one
|
|
"ipv4": "dhcp", // ipv4 address, dhcp for dhcp, auto for auto, or an ip address
|
|
"ipv6": "auto", // ipv6 address, dhcp for dhcp, auto for auto, or an ip address
|
|
"mac": "92:A6:71:77:8E:D8", // mac address, leave empty for random
|
|
"gateway4": "", // ipv4 gateway, leave empty for auto/dhcp
|
|
"gateway6": "", // ipv6 gateway, leave empty for auto/dhcp
|
|
"vlan": "" // vlan id, leave empty for no vlan
|
|
},
|
|
|
|
// the options block contains various options for the lxc
|
|
"options": {
|
|
"privileged": "false", // set to true to run the lxc in privileged mode
|
|
"start_on_boot": "false", // set to true to start the lxc on boot
|
|
"startup_order": 2, // the startup order of the lxc
|
|
"password": "qwertz1234", // the password of the root user, leave empty for none
|
|
"tags": "2-proxy+auth" // tags for the lxc to display in the proxmox web ui
|
|
},
|
|
```
|
|
*Tip : have a look at the [man page](https://pve.proxmox.com/pve-docs/pct.1.html) of the `pct` command to see how the values should look.*
|
|
|
|
## Creation
|
|
In the creation section, you have the option to define conditions for checking whether your LXC/VM has been previously configured.
|
|
If all of these conditions are met, the script will proceed directly to the deploy section and execute the necessary steps to update it. *(This indicates that your LXC/VM has already been configured, and we want to avoid erasing any existing settings/config.)*
|
|
However, if **any** of the conditions fail to match, the script will process to the creation steps section and execute **all** the commands again.
|
|
|
|
## Conditions
|
|
### File
|
|
The file condition will check if a file exists or not inside the LXC/VM.
|
|
```json
|
|
"conditions": {
|
|
// use a single file
|
|
"file": "/var/data/traefikv2/traefik.toml"
|
|
// or use a list
|
|
"file": ["/var/data/traefikv2/traefik.toml", "/var/data/config/traefikv2/docker-compose.yml"]
|
|
}
|
|
```
|
|
It can be an array of file using ``["file1", "file2"]`` or just one file in double quote ``"file"``.
|
|
|
|
### Folder
|
|
The folder condition will check if a folder exists or not inside the LXC/VM.
|
|
```json
|
|
"conditions": {
|
|
// use a single folder
|
|
"folder": "/var/data/traefikv2"
|
|
// or use a list
|
|
"folder": ["/var/data/traefikv2", "/var/data/config/traefikv2"]
|
|
}
|
|
```
|
|
It can be an array of folders using ``["folder1", "folder2"]`` or just one folder in double quote ``"file"``.
|
|
|
|
### Programs
|
|
The programs condition will check if a program is installed or not in the LXC/VM.
|
|
```json
|
|
"conditions": {
|
|
// use a single program
|
|
"program": "docker"
|
|
// or use a list
|
|
"program": ["docker", "docker-compose"]
|
|
}
|
|
```
|
|
It can be an array of programs using ``["program1", "program2"]`` or just one program in double quote ``"program"``.
|
|
*Note:This uses `which` to check if the program matches to anything*
|
|
|
|
### Command
|
|
The command condition will check if a command returns a specific value.
|
|
```json
|
|
"conditions": {
|
|
"command": "whoami",
|
|
"value": "root"
|
|
}
|
|
```
|
|
*Note: This uses `bash -c` to execute the command*
|
|
*Note: If value is an integer, it will look for the return code, if it's a string it's check the output.*
|
|
|
|
### Docker
|
|
The docker condition will check if a docker (or podman) container is running or not.
|
|
```json
|
|
"conditions": {
|
|
// use a single container
|
|
"container": "traefikv2"
|
|
// or use a list
|
|
"container": ["traefikv2", "portainer"]
|
|
}
|
|
```
|
|
|
|
## Steps
|
|
|
|
**Note: Paths**
|
|
When you have to specify a path, you can use ``/global/`` to use the global folder (`resources/<scripts|file>/`).
|
|
If nothing is specified, the script will use the VM/LXC folder (`resources/lxc/<id>/`).
|
|
|
|
*The comments in the JSON below are only for documentation purposes and are not valid JSON, remove them before running.*
|
|
### Script
|
|
The script step will execute a script.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "script",
|
|
// use path for local scripts
|
|
"local_path": "global/install-docker.sh" // local path (here: resources/scripts/install-docker.sh)
|
|
// or use url for remote scripts
|
|
"url": "https://xyz.abc/scripts/install-docker.sh" // remote url
|
|
// or use lxc_path for scripts inside the lxc
|
|
"lxc_path": "/root/install-docker.sh" // lxc path
|
|
}
|
|
]
|
|
```
|
|
|
|
### File
|
|
The file step will create / copy a file to the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "file_create",
|
|
"path": "/var/data/traefikv2/traefik.toml", // lxc/vm path
|
|
"permissions": "644" // (optional) permissions of the file
|
|
}
|
|
]
|
|
```
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "file_copy",
|
|
"path": "traefik.toml", // local path (here: resources/lxc/<id>/traefik.toml)
|
|
"destination": "/var/data/traefikv2/", // lxc/vm path
|
|
"permissions": "644" // (optional) permissions of the file
|
|
}
|
|
]
|
|
```
|
|
*Note: `file_copy` creates the folder if it doesn't exist*
|
|
|
|
### Folder
|
|
The folder step will copy a folder to the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "folder_create",
|
|
"path": "/var/data/traefikv2", // lxc/vm path
|
|
"permissions": "755" // (optional) permissions of the folder
|
|
}
|
|
]
|
|
```
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "folder_copy",
|
|
"path": "data/", // local path (here: resources/lxc/<id>/data/)
|
|
"destination": "/var/", // lxc/vm path
|
|
"permissions": "755" // (optional) permissions of the folder
|
|
}
|
|
]
|
|
```
|
|
*Note: `folder_copy` creates the folder if it doesn't exist*
|
|
|
|
### Command
|
|
The command step will execute a command on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "command",
|
|
"command": "whoami", // command to execute
|
|
"working_directory": "/var/data/config/traefikv2" // (optional) lxc/vm path
|
|
}
|
|
]
|
|
```
|
|
|
|
### Docker
|
|
The docker step will execute a command inside a docker container running on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "docker",
|
|
"container": "<container-name>", // docker container name
|
|
"command": "<command-to-run-inside>" // docker command to execute
|
|
}
|
|
]
|
|
```
|
|
|
|
### Docker-compose
|
|
The docker-compose step will execute a docker-compose command on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "docker-compose",
|
|
"command": "up -d", // docker-compose command to execute
|
|
"working_directory": "/var/data/config/traefikv2" // lxc/vm path
|
|
}
|
|
]
|
|
```
|
|
*Note : Here, why not use the `command` step? Docker Compose can sometimes be executed with `docker-compose` or `docker compose`, the script will take care of choosing the correct one.*
|
|
|
|
### Git
|
|
The git step will clone a git repo on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "git",
|
|
"url": "https://git.abc.xyz/abc/abc.git", // git url to clone
|
|
"destination": "/root/" // lxc/vm path
|
|
}
|
|
]
|
|
```
|
|
*Note: At the moment, no authentication is supported, so only works on public repo.*
|
|
|
|
### Download
|
|
The download step will download a file on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "download",
|
|
// download a single file
|
|
"url": "https://git.abc.xyz/file.tar.gz", // download url
|
|
// or use a list of urls
|
|
"url": ["https://git.abc.xyz/file1.tar.gz", "https://git.abc.xyz/file2.tar.gz"] // download urls
|
|
"destination": "/tmp/" // lxc/vm path
|
|
}
|
|
]
|
|
```
|
|
|
|
### Extract archive (tar/zip)
|
|
The unzip step will unzip a file in the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "unzip",
|
|
"path": "/tmp/file.tar.gz", // lxc/vm path to the archive
|
|
"destination": "/var/data/config/traefikv2" // (optional) lxc/vm path to extract the archive, will use archive parent directory if blank
|
|
}
|
|
]
|
|
```
|
|
*Note : At the the moment, only tar and zip are supported.*
|
|
|
|
### Install package
|
|
The install-package step will install a package on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "install-package",
|
|
// install a single package
|
|
"package": "git",
|
|
// or use a list of packages
|
|
"package": ["git", "docker"]
|
|
}
|
|
]
|
|
```
|
|
*Note : At the the moment, only apt, apk, dnf, yum are supported.*
|
|
*Warning : Packages can have different names depending on the Linux distribution.*
|
|
|
|
### Remove package
|
|
The remove-package step will remove a package on the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "remove-package",
|
|
// remove a single package
|
|
"package": "git"
|
|
// or use a list of packages
|
|
"package": ["git", "docker"]
|
|
}
|
|
]
|
|
```
|
|
*Note : At the the moment, only apt, apk, dnf, yum are supported.*
|
|
*Warning : Packages can have different names depending on the Linux distribution.*
|
|
|
|
### Power
|
|
The reboot step will reboot the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "reboot"
|
|
}
|
|
]
|
|
```
|
|
The start step will start the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "start"
|
|
}
|
|
]
|
|
```
|
|
|
|
The stop step will stop the VM/LXC.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "stop"
|
|
}
|
|
]
|
|
```
|
|
|
|
### Replace in file
|
|
The replace-in-file step will replace a string in a file.
|
|
```json
|
|
"steps": [
|
|
{
|
|
"type": "replace-in-file",
|
|
// replace in a single file
|
|
"path": "/var/data/config/traefikv2/traefik.toml", // inside lxc/vm
|
|
// or use a list of files
|
|
"path": ["/var/data/config/traefikv2/traefik.toml", "/var/data/config/traefikv2/traefik2.toml"] // inside lxc/vm
|
|
"search": "abc", // string to search
|
|
"replace": "xyz", // string to replace
|
|
"case_sensitive": true, // (optional) case sensitive? default: true
|
|
}
|
|
]
|
|
``` |