ProxmoxDeploy/README.md

# Proxmox Deploy
[![Build Status](https://drone.broillet.ch/api/badges/mathieu/ProxmoxDeploy/status.svg)](https://drone.broillet.ch/mathieu/ProxmoxDeploy)

![Logo](./.git-images/logo.png)

## Description
Proxmox Deploy is a little script to manage my HomeLab with JSON file.

## Why?
As my homelab was growing I realised that it was harder and harder to keep everything in sync and up to date.
So I decided to create a script to manage my Proxmox homelab.

# How it works
The concept is simple, you have a Git repository with a the following structure:
```
.
├── config.json
├── lxc
│   ├── <id>
│   │   ├── config.json
│   │   ├── <your files>
│   │   └── <your folders>
│   └── <id>
│       ├── ...
│── qemu
│   ├── <id>
│   │   ├── config.json
│   │   ├── <your files>
│   │   └── <your folders>
│   └── <id>
│       ├── ...
│── scripts
│   ├── <your scripts>
│   └── ...
```
*See below for more information about the structure and differents files*

PDJ (Proxmox Deploy JSON) is a program that will read that repository files and execute the necessary commands to create/update your LXC/VM.
Now ideally you have some sort of Git actions like Drone/GitHub(/Gitea) Actions to run PDJ automatically when you push a change, which will result in your homelab being updated almost instantly.

If you don't, no big deal, you'll just have to manually clone and update your repo, then start PDJ *(you could use a crontab with regular intervals)*

# Usage
## Download
Download the pre-compiled binaries from the release page or build it yourself.

## Build it yourself
```bash
# Build on Debian
git clone <url of this repo>
cd ProxmoxDeploy
apt update && apt install -y build-essential patchelf
pip install nuitka
pip install -r requirements.txt
python -m nuitka --onefile run.py --output-filename="ProxmoxDeploy"
```
*Also see the ``Dockerfile`` and ``.drone.yml`` for more information.*

## Run it
```bash
# Run it
./ProxmoxDeploy --repo /path/to/repo
```


# Documentation
## Configuration
### General
Before configuring your LXC and VM, you must decide how you'll run this program.
As it requires SSH for some actions, you have two options:
- Run it directly on the Proxmox VE host **(recommended)** *(refered as `local`)*
  - No configuration needed
  - You can use Git(ea)/Drone actions to run it automatically as soon as a change is pushed.
- Run it on another machine and connect via SSH
  - You'll need to setup passwordless SSH connection between your machine and the Proxmox VE host.

### Proxmox VE
The Proxmox VE configuration is located in the `config.json` file.  
```json
{
  "pve":{
    "host": "<pve host ip/hostname>",
    "user": "<user to connect via ssh>", // usually root
    "port": 22, // ssh port 
    "local": false // set to true if this program is running directly on the PVE without needing ssh
  }
}
```

### LXC
To create a LXC, you need to create a JSON file in the `resources/lxc/<id of lxc>` folder.
You can look at the example with ID 100 in the `resources/lxc/100` folder.

Here is an example config with all available parameters :  
*Note : this is not valid JSON, it's just for documentation purposes, if you want to copy the file, use the one in the resources folder*
```json
{
  "lxc_hostname": "traefik", //hostname of the lxc
  
  // the os block contains the os type
  // alpine | archlinux | centos | debian | devuan | fedora | gentoo | nixos | opensuse | ubuntu | unmanaged
  "os": {
    "name": "alpine",
    "release": "3.17"
  },
  
  // the resources block contains the resources given to the lxc
  "resources": {
    "cpu": "2", // number of cpu cores
    "memory": "1024", // memory (ram) in MB
    "swap": "256", // swap in MB
    "disk": "8", // disk size in GB
    "storage": "local-lvm" // the proxmox storage to use
  },
  
  // the network block contains the network configuration
  "network": {
    "bridge": "vmbr0", // the proxmox bridge to use, vmbr0 is the default one
    "ipv4": "dhcp", // ipv4 address, dhcp for dhcp, auto for auto, or an ip address
    "ipv6": "auto", // ipv6 address, dhcp for dhcp, auto for auto, or an ip address
    "mac": "92:A6:71:77:8E:D8", // mac address, leave empty for random
    "gateway4": "", // ipv4 gateway, leave empty for auto/dhcp
    "gateway6": "", // ipv6 gateway, leave empty for auto/dhcp
    "vlan": "" // vlan id, leave empty for no vlan
  },
  
  // the options block contains various options for the lxc
  "options": {
    "privileged": "false", // set to true to run the lxc in privileged mode
    "start_on_boot": "false", // set to true to start the lxc on boot
    "startup_order": 2, // the startup order of the lxc
    "password": "qwertz1234", // the password of the root user, leave empty for none
    "tags": "2-proxy+auth" // tags for the lxc to display in the proxmox web ui
  },
```
*Tip : have a look at the [man page](https://pve.proxmox.com/pve-docs/pct.1.html) of the `pct` command to see how the values should look.*

## Creation
In the creation section, you have the option to define conditions for checking whether your LXC/VM has been previously configured.  
If all of these conditions are met, the script will proceed directly to the deploy section and execute the necessary steps to update it. *(This indicates that your LXC/VM has already been configured, and we want to avoid erasing any existing settings/config.)*  
However, if **any** of the conditions fail to match, the script will process to the creation steps section and execute **all** the commands again.  

## Conditions
### File
The file condition will check if a file exists or not inside the LXC/VM.
```json
"conditions": {
  // use a single file
  "file": "/var/data/traefikv2/traefik.toml"
  // or use a list
  "file": ["/var/data/traefikv2/traefik.toml", "/var/data/config/traefikv2/docker-compose.yml"]
}
```
It can be an array of file using ``["file1", "file2"]`` or just one file in double quote ``"file"``. 

### Folder
The folder condition will check if a folder exists or not inside the LXC/VM.
```json
"conditions": {
  // use a single folder
  "folder": "/var/data/traefikv2"
  // or use a list
  "folder": ["/var/data/traefikv2", "/var/data/config/traefikv2"]
}
```
It can be an array of folders using ``["folder1", "folder2"]`` or just one folder in double quote ``"file"``. 

### Programs
The programs condition will check if a program is installed or not in the LXC/VM.
```json
"conditions": {
  // use a single program
  "program": "docker"
  // or use a list
  "program": ["docker", "docker-compose"]
}
```
It can be an array of programs using ``["program1", "program2"]`` or just one program in double quote ``"program"``.  
*Note:This uses `which` to check if the program matches to anything*

### Command
The command condition will check if a command returns a specific value.
```json
"conditions": {
  "command": "whoami",
  "value": "root"
}
```
*Note: This uses `bash -c` to execute the command*  
*Note: If value is an integer, it will look for the return code, if it's a string it's check the output.*

### Docker
The docker condition will check if a docker (or podman) container is running or not.
```json
"conditions": {
  // use a single container
  "container": "traefikv2"
  // or use a list
  "container": ["traefikv2", "portainer"]
}
```

## Steps

**Note: Paths**  
When you have to specify a path, you can use ``/global/`` to use the global folder (`resources/<scripts|file>/`).  
If nothing is specified, the script will use the VM/LXC folder (`resources/lxc/<id>/`).

*The comments in the JSON below are only for documentation purposes and are not valid JSON, remove them before running.*
### Script
The script step will execute a script.
```json
"steps": [
  {
    "type": "script",
    // use path for local scripts
    "path": "global/install-docker.sh" // path in repo (here: resources/scripts/install-docker.sh)
    // or use url for remote scripts
    "url": "https://xyz.abc/scripts/install-docker.sh" // remote url
    // or use lxc_path for scripts inside the lxc
    "lxc_path": "/root/install-docker.sh" // lxc path
  }
]
```

### File
The file step will create / copy a file to the VM/LXC.
```json
"steps": [
  {
    "type": "file_create",
    "path": "/var/data/traefikv2/traefik.toml", // lxc/vm path
    "permissions": "644" // (optional) permissions of the file
  }
]
```
```json
"steps": [
  {
    "type": "file_copy",
    "path": "traefik.toml", // local path (here: resources/lxc/<id>/traefik.toml)
    "destination": "/var/data/traefikv2/", // lxc/vm path
    "permissions": "644" // (optional) permissions of the file
  }
]
```
*Note: `file_copy` creates the folder if it doesn't exist*

### Folder
The folder step will copy a folder to the VM/LXC.
```json
"steps": [
  {
    "type": "folder_create",
    "path": "/var/data/traefikv2", // lxc/vm path
    "permissions": "755" // (optional) permissions of the folder
  }
]
```
```json
"steps": [
  {
    "type": "folder_copy",
    "path": "data/", // local path (here: resources/lxc/<id>/data/)
    "destination": "/var/", // lxc/vm path
    "permissions": "755" // (optional) permissions of the folder
  }
]
```
*Note: `folder_copy` creates the folder if it doesn't exist*

### Command
The command step will execute a command on the VM/LXC.
```json
"steps": [
  {
    "type": "command",
    "command": "whoami", // command to execute
    "working_directory": "/var/data/config/traefikv2" // (optional) lxc/vm path
  }
]
```

### Docker
The docker step will execute a command inside a docker container running on the VM/LXC.
```json
"steps": [
  {
    "type": "docker",
    "container": "<container-name>", // docker container name
    "command": "<command-to-run-inside>" // docker command to execute
  }
]
```

### Docker-compose
The docker-compose step will execute a docker-compose command on the VM/LXC.
```json
"steps": [
  {
    "type": "docker-compose",
    "command": "up -d", // docker-compose command to execute
    "working_directory": "/var/data/config/traefikv2" // lxc/vm path
  }
]
```
*Note : Here, why not use the `command` step? Docker Compose can sometimes be executed with `docker-compose` or `docker compose`, the script will take care of choosing the correct one.*

### Git
The git step will clone a git repo on the VM/LXC.
```json
"steps": [
  {
    "type": "git",
    "url": "https://git.abc.xyz/abc/abc.git", // git url to clone
    "destination": "/root/" // lxc/vm path
  }
]
```
*Note: At the moment, no authentication is supported, so only works on public repo.*

### Download
The download step will download a file on the VM/LXC.
```json
"steps": [
  {
    "type": "download",
    // download a single file
    "url": "https://git.abc.xyz/file.tar.gz", // download url
    // or use a list of urls
    "url": ["https://git.abc.xyz/file1.tar.gz", "https://git.abc.xyz/file2.tar.gz"] // download urls
    "destination": "/tmp/" // lxc/vm path
  }
]
```

### Extract archive (tar/zip)
The unzip step will unzip a file in the VM/LXC.
```json
"steps": [
  {
    "type": "unzip",
    "path": "/tmp/file.tar.gz", // lxc/vm path to the archive
    "destination": "/var/data/config/traefikv2" // (optional) lxc/vm path to extract the archive, will use archive parent directory if blank
  }
]
```
*Note : At the the moment, only tar and zip are supported.*

### Install package
The install-package step will install a package on the VM/LXC.
```json
"steps": [
  {
    "type": "install-package",
    // install a single package
    "package": "git",
    // or use a list of packages
    "package": ["git", "docker"]
  }
]
```
*Note : At the the moment, only apt, apk, dnf, yum are supported.*  
*Warning : Packages can have different names depending on the Linux distribution.*

### Remove package
The remove-package step will remove a package on the VM/LXC.
```json
"steps": [
  {
    "type": "remove-package",
    // remove a single package
    "package": "git"
    // or use a list of packages
    "package": ["git", "docker"]
  }
]
```
*Note : At the the moment, only apt, apk, dnf, yum are supported.*  
*Warning : Packages can have different names depending on the Linux distribution.*

### Power
The reboot step will reboot the VM/LXC.
```json
"steps": [
  {
    "type": "reboot"
  }
]
```
The start step will start the VM/LXC.
```json
"steps": [
  {
    "type": "start"
  }
]
```

The stop step will stop the VM/LXC.
```json
"steps": [
  {
    "type": "stop"
  }
]
```

### Replace in file
The replace-in-file step will replace a string in a file.
```json
"steps": [
  {
    "type": "replace-in-file",
    // replace in a single file
    "path": "/var/data/config/traefikv2/traefik.toml", // inside lxc/vm
    // or use a list of files
    "path": ["/var/data/config/traefikv2/traefik.toml", "/var/data/config/traefikv2/traefik2.toml"] // inside lxc/vm
    "search": "abc", // string to search
    "replace": "xyz", // string to replace
    "case_sensitive": true, // (optional) case sensitive? default: true
  }
]
```