1.0.0 Release IaaS
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Operation
|
||||
|
||||
## Migration backup
|
||||
|
||||
When the migration is decided, the manual backup after shutting all services down is necessary. Run the DB dump backup and kopia backup manually before reset the system.
|
||||
|
||||
## Windows
|
||||
|
||||
- Following the process step.
|
||||
- [01-windows.md](./01-windows.md)
|
||||
|
||||
## Certificates and wireguard
|
||||
|
||||
- Following the process two steps.
|
||||
- [02-certificates.md](./02-certificates.md)
|
||||
- [03-wireguard.md](./03-wireguard.md)
|
||||
|
||||
## console
|
||||
|
||||
### Data restore
|
||||
|
||||
Only when kopia repository exists.
|
||||
|
||||
```bash
|
||||
kopia repository connect --override-username="console" --override-hostname="console.ilnmors.internal"
|
||||
|
||||
kopia snapshot list --all
|
||||
|
||||
# mount volumes
|
||||
kopia mount $CONSOLE_SNAPSHOT_ID ~/workspace/homelab/volumes/console
|
||||
kopia mount $INFRA_SNAPSHOT_ID ~/workspace/homelab/volumes/infra
|
||||
kopia mount $APP_SNAPSHOT_ID ~/workspace/homelab/volumes/app
|
||||
|
||||
# Set initiating db dump files
|
||||
cp ~/workspace/homelab/volumes/infra/cluster/cluster.sql ~/workspace/homelab/config/services/containers/infra/postgresql/init/cluster.sql
|
||||
# ...
|
||||
```
|
||||
|
||||
### Provisioning
|
||||
|
||||
Ansible playbooks should be declarative. This won't contain complex branch logics \(Declarative over imperative\). Playbooks describes what should be there, not how to. The basic rule is manual destroy and auto reprovisioning.
|
||||
|
||||
#### vmm and fw
|
||||
|
||||
- Following the process step.
|
||||
- [04-hypervisor.md](./04-hypervisor.md)
|
||||
|
||||
```bash
|
||||
# Sign on vmm's host ssh keys and set libvirt environment # Ignore fingerprint just once
|
||||
cd ~/workspace/homelab/ansible
|
||||
|
||||
# Disconnect all internet connection except LAN between vmm
|
||||
ansible-playbook playbooks/vmm/site.yaml --tags "init" --ssh-common-args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
|
||||
# Create fw
|
||||
ansible-playbook playbooks/vmm/create_vm.yaml --tags "fw"
|
||||
# Make it sure disable other NIC (Wifi or etc) before run playbooks, and set gateway
|
||||
ansible-playbook playbooks/fw/site.yaml --tags "init" --ssh-common-args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null' # When nftables is changed, crowdsec handlers work. However, when it is in initiating, handler is failed. It is normal work.
|
||||
ansible-playbook playbooks/fw/site.yaml --tags "site"
|
||||
# Set vmm after air-gap
|
||||
ansible-playbook playbooks/vmm/site.yaml --tags "site"
|
||||
```
|
||||
|
||||
## External hardware on CLIENT
|
||||
|
||||
- Following the process two steps.
|
||||
- [05-hardwares.md](./05-hardwares.md)
|
||||
- [06-kopia.md](./06-kopia.md)
|
||||
|
||||
## Other vms
|
||||
|
||||
```bash
|
||||
# Kopia mount
|
||||
## Create Kopia password file
|
||||
touch /etc/secret/$CONSOLE_UID/kopia_password
|
||||
### Add kopia.user.console value from secrets.yaml
|
||||
## Connect repository
|
||||
KOPIA_PASSWORD="$(cat /etc/secrets/$CONSOLE_UID/kopia_password)" \
|
||||
/usr/bin/kopia repository connect server \
|
||||
--url=https://nas.ilnmors.internal:51515 \
|
||||
--override-username=console \
|
||||
--override-hostname=console.ilnmors.internal
|
||||
## mount kopia directories
|
||||
kopia mount $INFRA_SNAPSHOT_ID ~/workspace/data/volumes/infra
|
||||
kopia mount $APP_SNAPSHOT_ID ~/workspace/data/volumes/app
|
||||
## bringing sql dump files to init directory
|
||||
cp ~/workspace/data/volumes/infra/cluster/$FILE_NAME ~/workspace/config/services/containers/infra/postgresql/init/pg_cluster.sql
|
||||
## repeate for every service
|
||||
# Create vms (Possible tags = ["fw", "infra", "auth", "app"])
|
||||
ansible-playbook playbooks/vmm/create_vm.yaml --tags "$VM_NAME"
|
||||
# Sign on vms' host ssh keys # Ignore fingerprint just once
|
||||
ansible-playbook playbooks/$VM_NAME/site.yaml --tags "init" --ssh-common-args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
|
||||
ansible-playbook playbooks/$VM_NAME/site.yaml --tags "site"
|
||||
# app
|
||||
ansible-playbook playbooks/app/site.yaml --tags "init" --ssh-common-args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
|
||||
## Rsync app data
|
||||
rsync -avz --numeric-ids -e ssh --rsync-path="sudo rsync" ~/workspace/homelab/volumes/kopia/app/ app@app.ilnmors.internal:/home/app/data/
|
||||
# --numeric-ids: set owner and groups based on uid and gid
|
||||
# -e ssh --rsync-path="sudo rsync": run rsync as root permission, sudo password needed
|
||||
# Maybe rsync-path is run on background, it might not print prompt for password. Just enter the password
|
||||
ansible-playbook playbooks/app/site.yaml --tags "site"
|
||||
|
||||
# Create console snapshot
|
||||
kopia snapshot create ~/workspace
|
||||
```
|
||||
@@ -0,0 +1,255 @@
|
||||
# Windows prerequisite
|
||||
|
||||
## Installation
|
||||
|
||||
- Language: English (US)
|
||||
- Region: Korea
|
||||
- Keyboard: US
|
||||
- Internet connection
|
||||
- Account setting: DOS prompt (Shift + F10)
|
||||
```dos
|
||||
start ms-cxh:localonly
|
||||
```
|
||||
|
||||
## Initial configuration
|
||||
|
||||
### Time and language
|
||||
|
||||
- Preferred Languages: Add a Language: Korean
|
||||
- Don't set additional options
|
||||
- Typing: Advanced Keyboard setting: Override for default input method: Korean: Microsoft IME
|
||||
|
||||
### Set the password
|
||||
- Settings: Accounts: Sign-in options
|
||||
- Password
|
||||
- PIN
|
||||
- Windows Hello
|
||||
|
||||
### Disable unused functions
|
||||
|
||||
```PowerShell
|
||||
# Administrator privileges are required
|
||||
## Disable Recall
|
||||
DISM /Online /Disable-Feature /FeatureName:Recall /NoRestart
|
||||
|
||||
## Disable Copilot
|
||||
reg add "HKCU\Software\Policies\Microsoft\Windows\WindowsCopilot" /v TurnOffWindowsCopilot /t REG_DWORD /d 1 /f
|
||||
reg add "HKLM\Software\Policies\Microsoft\Windows\WindowsCopilot" /v TurnOffWindowsCopilot /t REG_DWORD /d 1 /f
|
||||
|
||||
## Delete programs
|
||||
Get-AppxPackage *bingnews* | Remove-AppxPackage
|
||||
Get-AppxPackage *bingweather* | Remove-AppxPackage
|
||||
Get-AppxPackage *gethelp* | Remove-AppxPackage
|
||||
Get-AppxPackage *tips* | Remove-AppxPackage
|
||||
Get-AppxPackage *feedbackhub* | Remove-AppxPackage
|
||||
|
||||
Get-AppxPackage *solitairecollection* | Remove-AppxPackage
|
||||
Get-AppxPackage *windowscommunicationsapps* | Remove-AppxPackage
|
||||
Get-AppxPackage *outlookforwindows* | Remove-AppxPackage
|
||||
Get-AppxPackage *people* | Remove-AppxPackage
|
||||
Get-AppxPackage *clipchamp* | Remove-AppxPackage
|
||||
|
||||
Get-AppxPackage *todos* | Remove-AppxPackage
|
||||
Get-AppxPackage *quickassist* | Remove-AppxPackage
|
||||
|
||||
Get-AppxPackage *windowsmaps* | Remove-AppxPackage
|
||||
|
||||
Get-AppxPackage *xbox* | Remove-AppxPackage
|
||||
Get-AppxPackage *gamingapp* | Remove-AppxPackage
|
||||
|
||||
## Disable telemetry
|
||||
Stop-Service DiagTrack
|
||||
Set-Service DiagTrack -StartupType Disable
|
||||
Stop-Service dmwappushservice
|
||||
Set-Service dmwappushservice -StartupType Disable
|
||||
|
||||
## Compact OS configuration
|
||||
compact /compactos:always
|
||||
```
|
||||
|
||||
### Delete programs
|
||||
|
||||
- Add or remove programs
|
||||
- Copilot
|
||||
- Family
|
||||
- Microsoft 365 (Other languages; except en-us, ko-kr)
|
||||
- Microsoft 365 copilot
|
||||
- Microsoft edge game assist
|
||||
- Microsoft Onedrive
|
||||
- Microsoft Teams
|
||||
- Speech pack
|
||||
|
||||
### Edge configuration
|
||||
- Settings: System and performance: System
|
||||
- Disable All options
|
||||
|
||||
### Explorer configuration
|
||||
|
||||
- Settings: Privacy
|
||||
- Disable All options
|
||||
|
||||
### Login on Microsoft account
|
||||
|
||||
sign in on app only
|
||||
- surface app
|
||||
- microsoft office 365
|
||||
|
||||
### Install Microsoft app
|
||||
|
||||
- Microsoft PC manager
|
||||
|
||||
### Firewall configuration
|
||||
|
||||
- WindowsDefender Firewall:Inbound Rules:
|
||||
- File and Printer Sharing (Echo Request - ICMPv4-In) - Profile: Private, Public
|
||||
- General: \[x\] Enable
|
||||
- Scope: 192.168.1.0/24, 192.168.10.0/24, 192.168.99.0/24
|
||||
- File and Printer Sharing (Echo Request - ICMPv6-In) - Profile: Private, Public
|
||||
- General: \[x\] Enable
|
||||
- Scope: fd00::/8
|
||||
- Apply
|
||||
|
||||
### NIC VLAN configuration
|
||||
|
||||
<!-- There is no physical managed switch, only hub. It is necessary to set so that filter vlan tag from the dummy hub -->
|
||||
- Device Manager:Network Adapters:Surface network adpaters
|
||||
- Advanced:Priority & VLAN: Priority & VLAN disable
|
||||
|
||||
## Set WSL2
|
||||
|
||||
### Create wsl config
|
||||
|
||||
- C:\Users\$USERNAME\.wslconfig
|
||||
```ini
|
||||
[wsl2]
|
||||
processors=4
|
||||
memory=4294967296
|
||||
swap=0
|
||||
defaultVhdSize=34359738368
|
||||
networkingMode=Mirrored
|
||||
```
|
||||
|
||||
### Installation
|
||||
|
||||
```PowerShell
|
||||
# Run with administrator's authority
|
||||
# Install wsl
|
||||
wsl --install -d Debian
|
||||
# Enter new UNIX username: debian
|
||||
# Enter new password: debian
|
||||
wsl --shutdown
|
||||
# --- Uninstall Debian ---
|
||||
# wsl --unregister Debian
|
||||
# wsl --uninstall
|
||||
# ------
|
||||
```
|
||||
|
||||
### WSL Debian configuration
|
||||
|
||||
```bash
|
||||
# PowerShell
|
||||
wsl -d Debian
|
||||
# bash
|
||||
## User create
|
||||
sudo groupadd -g 2000 svadmins
|
||||
sudo useradd -u 2999 -g svadmins -G sudo -c "Console Client" -m -d /home/console -s /bin/bash console
|
||||
sudo passwd console
|
||||
# New password: [password]
|
||||
exit
|
||||
|
||||
# PowerShell
|
||||
wsl --shutdown
|
||||
wsl -d Debian -u console
|
||||
|
||||
# bash
|
||||
## User delete and set wsl.conf
|
||||
sudo userdel -r debian
|
||||
echo '[user]' | sudo tee -a /etc/wsl.conf
|
||||
echo 'default=console' | sudo tee -a /etc/wsl.conf
|
||||
echo '[network]' | sudo tee -a /etc/wsl.conf
|
||||
echo 'generateHosts = false' | sudo tee -a /etc/wsl.conf
|
||||
exit
|
||||
|
||||
wsl --shutdown
|
||||
|
||||
wsl -d Debian
|
||||
|
||||
## package
|
||||
sudo apt update && sudo apt upgrade
|
||||
sudo apt install curl jq gnupg git fuse3
|
||||
sudo apt install ansible-core ansible-lint --no-install-recommends
|
||||
ansible-galaxy collection install community.libvirt
|
||||
ansible-galaxy collection install community.general
|
||||
ansible-galaxy collection install ansible.posix
|
||||
|
||||
## SOPS (arm64)
|
||||
## Check for latest release: Current version: 3.12.1
|
||||
## x86_64 url: https://github.com/getsops/sops/releases/download/v3.12.1/sops_3.12.1_amd64.deb
|
||||
|
||||
curl -L -o sops_3.12.1.deb https://github.com/getsops/sops/releases/download/v3.12.1/sops_3.12.1_arm64.deb
|
||||
sudo apt install ./sops_3.12.1.deb
|
||||
rm -rf ./sops_3.12.1.deb
|
||||
|
||||
mkdir ~/workspace
|
||||
```
|
||||
|
||||
### VS Code
|
||||
|
||||
#### VS Code Download
|
||||
- Download from https://code.visualstudio.com/Download#
|
||||
- `User Installer Arm64`
|
||||
|
||||
#### VS Code configuration
|
||||
|
||||
- WSL extension\(`Ctrl + shift + x`\)
|
||||
- Install `WSL` by Microsoft
|
||||
- Remote Explorer:Debian:Connect in Current Windows
|
||||
- `Ctrl + k` and `Ctrl + o`
|
||||
- Open folder: `/home/console/workspace`
|
||||
- `` Ctrl + shift + ` `` for Terminal
|
||||
- Extensions\(`Ctrl + shift + x`\)
|
||||
- Install `Ansible` by RedHat
|
||||
|
||||
### Playbooks
|
||||
|
||||
```bash
|
||||
# Ansible playbook
|
||||
|
||||
# Copy all files "workspace" directory to Debian via VS Code
|
||||
|
||||
cd ~/workspace/homelab/ansible
|
||||
ansible-playbook playbooks/console/site.yaml --tags "init"
|
||||
```
|
||||
|
||||
#### Terminal configuration
|
||||
|
||||
- Terminal: Setting: Startup
|
||||
- Default profile
|
||||
- Debian
|
||||
- Profile: Debian:
|
||||
- Icon
|
||||
- "ms-appx:///ProfileIcons/{61c54bbd-c2c6-5271-96e7-009a87ff44bf}.png"
|
||||
- Starting Directory
|
||||
- ~
|
||||
|
||||
## Secret management
|
||||
|
||||
### Flow
|
||||
|
||||
- Secret data
|
||||
- `secrets.yaml`
|
||||
- encrypted by sops with age-key
|
||||
- age-key
|
||||
- `age-key.gpg` and `ansible/group_vars/all.yaml`
|
||||
- encrypted by gpg and ansible vault with master key
|
||||
- Master key
|
||||
- The key which has above 40 characters containing upper and lower letters, numbers, and special letters
|
||||
- managed by physical media \(Mind, MDisc, paper\) as file, string, and QR
|
||||
- This value is never saved in server or console.
|
||||
- Root CA \(including ssh CA\) must not be deployed.
|
||||
- The tasks with root CA must be performed manually. The source of Trust is the most important in security.
|
||||
- Intermediate CA can be deployed.
|
||||
- Intermediate CA is operated as a live server.
|
||||
- Intermediate CA can be revoked by the root CA
|
||||
- Only encrypted intermediate CA's private key is deployed.
|
||||
- CA server decrypts intermediate CA's private key in memory at runtime.
|
||||
@@ -0,0 +1,169 @@
|
||||
# Certificates
|
||||
|
||||
Create and renew certificates are very important, and very barely executed. It is managed manually without ansible.
|
||||
|
||||
#### PKI CA signed offline
|
||||
|
||||
step-cli is installed by ansible playbook for console.
|
||||
|
||||
```bash
|
||||
# Generate CA key password
|
||||
openssl rand -base64 32 > /run/user/$UID/root_ca_password
|
||||
openssl rand -base64 32 > /run/user/$UID/intermediate_ca_password
|
||||
# Save the values in `secrets.yaml`
|
||||
|
||||
# Create CAs \(Key and cert)
|
||||
# Root CA
|
||||
step certificate create \
|
||||
"ilnmors.internal Root CA" /run/user/$UID/root_ca.crt /run/user/$UID/root_ca.key \
|
||||
--password-file /run/user/$UID/root_ca_password \
|
||||
--profile root-ca \
|
||||
--not-after 87600h
|
||||
# Save the key and crt files content in `secrets.yaml`
|
||||
|
||||
# Intermediate CA
|
||||
step certificate create \
|
||||
"ilnmors.internal Intermediate CA" /run/user/$UID/intermediate_ca.crt /run/user/$UID/intermediate_ca.key \
|
||||
--password-file /run/user/$UID/intermediate_ca_password \
|
||||
--profile intermediate-ca \
|
||||
--ca /run/user/$UID/root_ca.crt \
|
||||
--ca-key /run/user/$UID/root_ca.key \
|
||||
--ca-password-file /run/user/$UID/root_ca_password \
|
||||
--not-after 43800h
|
||||
# Save the key and crt files content in `secrets.yaml`
|
||||
|
||||
# fw
|
||||
|
||||
step certificate create \
|
||||
"crowdsec.ilnmors.internal" /run/user/$UID/crowdsec.crt /run/user/$UID/crowdsec.key \
|
||||
--profile leaf \
|
||||
--san crowdsec.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
step certificate create \
|
||||
"blocky.ilnmors.internal" /run/user/$UID/blocky.crt /run/user/$UID/blocky.key \
|
||||
--profile leaf \
|
||||
--san blocky.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
# infra
|
||||
|
||||
step certificate create \
|
||||
"postgresql.ilnmors.internal" /run/user/$UID/postgresql.crt /run/user/$UID/postgresql.key \
|
||||
--profile leaf \
|
||||
--san postgresql.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
step certificate create \
|
||||
"ldap.ilnmors.internal" /run/user/$UID/ldap.crt /run/user/$UID/ldap.key \
|
||||
--profile leaf \
|
||||
--san ldap.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
step certificate create \
|
||||
"prometheus.ilnmors.internal" /run/user/$UID/prometheus.crt /run/user/$UID/prometheus.key \
|
||||
--profile leaf \
|
||||
--san prometheus.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
step certificate create \
|
||||
"loki.ilnmors.internal" /run/user/$UID/loki.crt /run/user/$UID/loki.key \
|
||||
--profile leaf \
|
||||
--san loki.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
# DSM
|
||||
|
||||
step certificate create \
|
||||
"nas.ilnmors.internal" /run/user/$UID/nas.crt /run/user/$UID/nas.key \
|
||||
--profile leaf \
|
||||
--san nas.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password
|
||||
|
||||
## Recreate leaf certificates
|
||||
## update secrets.yaml
|
||||
step certificate create \
|
||||
"crowdsec.ilnmors.internal" /run/user/$UID/crowdsec.crt /run/user/$UID/crowdsec.key \
|
||||
--profile leaf \
|
||||
--san crowdsec.ilnmors.internal \
|
||||
--ca /run/user/$UID/intermediate_ca.crt \
|
||||
--ca-key /run/user/$UID/intermediate_ca.key \
|
||||
--ca-password-file /run/user/$UID/intermediate_ca_password \
|
||||
--not-after 21900h \
|
||||
--insecure --no-password -f
|
||||
# print
|
||||
cat /run/user/$UID/crowdsec.key
|
||||
cat /run/user/$UID/crowdsec.crt
|
||||
|
||||
# Verify
|
||||
step certificate verify /run/user/$UID/test.crt --roots /run/user/$UID/root_ca.crt
|
||||
# Inspect
|
||||
step certificate inspect /run/user/$UID/test.crt
|
||||
# validate date
|
||||
sudo step certificate inspect --format json /run/user/$UID/test.crt | jq '.validity.end'
|
||||
# margin date
|
||||
echo "$(( ($(date -d 2028-07-17T03:50:10Z +%s) - $(date +%s)) / 60 / 60 / 24 ))"
|
||||
|
||||
# Delete temporary files
|
||||
rm /run/user/$UID/root_ca*
|
||||
rm /run/user/$UID/intermediate_ca*
|
||||
rm /run/user/$UID/*.key
|
||||
rm /run/user/$UID/*.crt
|
||||
```
|
||||
|
||||
#### SSH CA
|
||||
|
||||
```bash
|
||||
# Generate SSH CA
|
||||
ssh-keygen -t ed25519 -f /run/user/$UID/id_local_ssh_ca -C "LOCAL_SSH_CA" -N ""
|
||||
# Save the key and crt files content in `secrets.yaml`
|
||||
echo @cert-authority *.ilnmors.internal "$(cat /run/user/$UID/id_local_ssh_ca.pub)" | sudo tee /etc/ssh/ssh_known_hosts >/dev/null && sudo chmod 644 /etc/ssh/ssh_known_hosts
|
||||
|
||||
# Signing HOST SSH crt by SSH CA key
|
||||
ssh-keygen -s /run/user/$UID/id_local_ssh_ca \
|
||||
-h \
|
||||
-I "vmm" \
|
||||
-n "vmm,vmm_init,vmm.ilnmors.internal,init.vmm.ilnmors.internal" \
|
||||
/run/user/$UID/id_vmm_ssh_host.pub
|
||||
# This process is automated by ansible
|
||||
|
||||
ssh-keygen -L -f /etc/ssh/ssh_host_ed25519_key-cert.pub
|
||||
|
||||
# Create SSH client key
|
||||
ssh-keygen -t ed25519 -f /etc/secrets/$UID/id_console -C "il@ilnmors.internal" -N ""
|
||||
|
||||
# Signing SSH client crt by SSH CA key
|
||||
ssh-keygen -s /run/user/$UID/id_local_ssh_ca \
|
||||
-I "console" \
|
||||
-n "vmm,fw,infra,auth,app" \
|
||||
/etc/secrets/$UID/id_console.pub
|
||||
# This process is automated by ansible
|
||||
```
|
||||
@@ -0,0 +1,23 @@
|
||||
# WireGuard key get
|
||||
|
||||
```bash
|
||||
wg genkey | tee /run/user/$UID/server_private.key | wg pubkey | tee /run/user/$UID/server_pub.key
|
||||
wg genkey | tee /run/user/$UID/client_private.key | wg pubkey | tee /run/user/$UID/client_pub.key
|
||||
wg genpsk | tee /run/user/$UID/preshared.key
|
||||
# Save the key files content in `secrets.yaml`
|
||||
|
||||
# Client wireguard setting
|
||||
|
||||
[Interface]
|
||||
PrivateKey = client_private.key
|
||||
Address = 192.168.99.20/32, fd00:99::20/128
|
||||
DNS = 192.168.10.2, fd00:10::2
|
||||
|
||||
[Peer]
|
||||
PublicKey = server_pub.key
|
||||
PresharedKey = preshared.key
|
||||
AllowedIPs = 192.168.0.0/16, fd00::/8
|
||||
Endpoint = vpn.ilnmors.com:11290
|
||||
PersistentKeepalive = 25
|
||||
|
||||
```
|
||||
@@ -0,0 +1,162 @@
|
||||
# Hypervisor \(vmm\)
|
||||
|
||||
Initiating hypervisor doesn't use ansible. Hypervisor is working on hardware itself, so there is a lot of possible variables like IOMMU id, MAC addresses, etc.
|
||||
|
||||
Hypervisor is initiated manually with the configuration files which are stored in USB or External HDD and, WAN connection is from ISP DHCP. All the files in `~/workspace/homelab/data/vmm_init/`
|
||||
|
||||
## Installing Debian on server
|
||||
|
||||
- BIOS:
|
||||
- Check BIOS configuration; IOMMU/VT-d
|
||||
- General:
|
||||
- Language: English - English
|
||||
- Location: Other > Asia > South Korea
|
||||
- Locale: United State - en_US.UTF-8
|
||||
- Keymap to use: American English
|
||||
- Network:
|
||||
- Auto Configuration: Using DHCP
|
||||
- Server:
|
||||
- Hostname: vmm
|
||||
- Domain: ilnmors.internal
|
||||
- User:
|
||||
- Root Password: \[blank\]
|
||||
- Full name for the new user: vmm
|
||||
- User Name: bootstrap
|
||||
- User Password: debian
|
||||
- Partition setting: manual
|
||||
- 512MiB - EFI system partition \(Booting flag: on\)
|
||||
- 1GiB - Ext4 Journaling \(Mount: /boot)
|
||||
- 800 GiB -LVM
|
||||
- 64GiB: vmm-root - Ext4 Journaling \(Mount: /\)
|
||||
- 700GiB: vmm-libvirt - Ext4 \(Mount: /var/lib/libvirt\)
|
||||
- Debian package manager setting
|
||||
- Scan extra installation media: no
|
||||
- Mirror country: South Korea
|
||||
- Archive mirror: deb.debian.org
|
||||
- Proxy: \[blank\]
|
||||
- Popularity-contest: no
|
||||
- Installing packages setting
|
||||
- \[\*\] SSH server
|
||||
- \[\*\] Standard system utilities
|
||||
|
||||
### Initial configuration
|
||||
|
||||
Hypervisor operates pure L2 switch for fw and it never can access WAN without fw after initial configuration. This means, there is an air-gap which means hypervisor cannot access to WAN for a while \(from end of initial setting to the beginning of fw setting\).
|
||||
|
||||
Hypervisor operates on hardware. Hardware information is always uncertain, and it is set only once. Managing this process as IaC is over engineering.
|
||||
|
||||
```bash
|
||||
# Mount USB on server
|
||||
lsblk -l
|
||||
# /dev/xxx # USB
|
||||
sudo mkdir /mnt/usb
|
||||
sudo mount /dev/xxx /mnt/usb
|
||||
|
||||
# Setting user and groups
|
||||
sudo groupadd svadmins -g 2000
|
||||
sudo useradd -u 2000 -g svadmins -G sudo -c "Hypervisor" -m -d /home/vmm -s /bin/bash vmm
|
||||
|
||||
# Installing packages
|
||||
sudo apt update && sudo apt upgrade
|
||||
|
||||
sudo apt install -y \
|
||||
acl curl jq crowdsec systemd-resolved \
|
||||
qemu-system-x86 ksmtuned libvirt-daemon-system virt-top \
|
||||
python3 python3-apt python3-libvirt python3-lxml
|
||||
|
||||
# Deploy ssh ca
|
||||
sudo cp /mnt/usb/vmm/ssh/local_ssh_ca.pub /etc/ssh/
|
||||
sudo chmod 644 /etc/ssh/local_ssh_ca.pub
|
||||
sudo cp /mnt/usb/vmm/ssh/sshd_config.d/*.conf /etc/ssh/sshd_conifg.d/
|
||||
sudo chmod 644 /etc/ssh/sshd_config.d/
|
||||
|
||||
# Deploy networkd configuration files
|
||||
sudo cp /mnt/usb/vmm/network/* /etc/systemd/network/
|
||||
|
||||
sudo chmod 644 /etc/systemd/network/*
|
||||
|
||||
sudo cp /mnt/usb/vmm/sysctl.d/bridge.conf /etc/sysctl.d/bridge.conf
|
||||
sudo chmod 644 /etc/sysctl.d/bridge.conf
|
||||
|
||||
# Check physical MAC address and modify .link file
|
||||
ip addr
|
||||
sudo nano /etc/systemd/network/eth0.link
|
||||
sudo nano /etc/systemd/network/eth1.link
|
||||
|
||||
sudo systemctl disable networking.service
|
||||
sudo systemctl enable systemd-networkd.service
|
||||
|
||||
# Deploy nftables config files
|
||||
sudo cp /mnt/usb/vmm/nftables.conf /etc/nftables.conf
|
||||
|
||||
sudo chmod 700 /etc/nftables.conf
|
||||
|
||||
# Fix grub
|
||||
sudo cp /mnt/usb/vmm/grub.d/iommu.cfg /etc/default/grub.d/iommu.cfg
|
||||
# GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
|
||||
## Intel CPU: intel_iommu=on, AMD CPU: amd_iommu=on
|
||||
sudo chmod 644 /etc/default/grub.d/iommu.cfg
|
||||
|
||||
# Check PCIE ID and IOMMU group
|
||||
lspci -nn | grep -i -e "SATA" -e "VGA"
|
||||
|
||||
# 00:02.0 VGA compatible controller [0300] ... [8086:46d4]
|
||||
# PCI address: 00:02.0 / Device ID 8086:46d4
|
||||
|
||||
# 04:00.0 SATA controller [0106] ... [1b21:1064]
|
||||
# PCI address: 04:00.00 / Device ID 1b21:1064
|
||||
|
||||
# Check the iommu group of devices
|
||||
## VGA
|
||||
readlink /sys/bus/pci/devices/0000\:02\:00.0/iommu_group
|
||||
# ../../../../kernel/iommu_groups/12
|
||||
ls /sys/kernel/iommu_groups/12/devices/
|
||||
# 02:00.00
|
||||
## SATA Controller
|
||||
readlink /sys/bus/pci/devices/0000\:04\:00.0/iommu_group
|
||||
# ../../../../kernel/iommu_groups/14
|
||||
ls /sys/kernel/iommu_groups/14/devices/
|
||||
# 04:00.00
|
||||
|
||||
# To passthrough device, it has to be only device in its IOMMU group or All devices in the same group should be passthroughed simultanaeously.
|
||||
|
||||
# Add modprobe.d/vfio.conf
|
||||
sudo cp /mnt/usb/vmm/modprobe.d/vfio.conf /etc/modprobe.d/vfio.conf
|
||||
sudo nano /etc/modprobe.d/vfio.conf
|
||||
# options vfio-pci ids=8086:46d4,1b21:1064
|
||||
# softdep i915 pre: vfio-pci
|
||||
# softdep ahci pre: vfio-pci
|
||||
sudo chmod 644 /etc/modprobe.d/vfio.conf
|
||||
|
||||
# Apply vfio configurations
|
||||
sudo update-grub
|
||||
sudo update-initramfs -u
|
||||
|
||||
# umount usb
|
||||
sudo umount -f -l /mnt/usb
|
||||
|
||||
# reboot system
|
||||
sudo reboot
|
||||
|
||||
# Connect console and vmm via lan cable, Set console's NIC as 192.168.1.11
|
||||
# delete default user
|
||||
# ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null vmm_init
|
||||
# login: vmm
|
||||
|
||||
# check vfio-pci
|
||||
lspci -nnk -d 8086:46d4
|
||||
lspci -nnk -d 1b21:1064
|
||||
# Kernel driver in user: vfio-pci
|
||||
|
||||
id vmm
|
||||
# check sudo group
|
||||
su - bootstrap
|
||||
sudo passwd vmm
|
||||
# enter new password
|
||||
exit
|
||||
|
||||
# delete default user
|
||||
sudo userdel -r bootstrap
|
||||
id bootstrap
|
||||
# id: 'bootstrap': no such user
|
||||
```
|
||||
@@ -0,0 +1,220 @@
|
||||
# Hardware
|
||||
|
||||
All hardware configuration is set after fw vm. The MAC address of hardware is reservated on kea-dhcp.
|
||||
|
||||
## Vlan switch
|
||||
|
||||
### Access VLAN switch
|
||||
|
||||
- http://switch.ilnmors.internal \(192.168.1.2, KEA-DHCP, Only IPv4 support\)
|
||||
- before set ipv6, use ip4 address instead of FQDN
|
||||
- id: admin, password: admin
|
||||
- new password: switch.password
|
||||
|
||||
### Set VLAN
|
||||
- VLAN:802.1Q VLAN
|
||||
- \[x\] Enable - Apply
|
||||
- VLAN client
|
||||
- id 1
|
||||
- name default > client
|
||||
- member \(Untagged\)
|
||||
- Port 1 \(Trunk, untagged\): Linux bridge is already process untagged packet as id 1
|
||||
- Port 3
|
||||
- Port 4
|
||||
- Port 5
|
||||
- Port 6
|
||||
- Port 7
|
||||
- Port 8
|
||||
- VLAN server
|
||||
- id 10
|
||||
- name server
|
||||
- member
|
||||
- Port 1 \(Trunk, tagged\)
|
||||
- VLAN user
|
||||
- id 20
|
||||
- name user
|
||||
- member
|
||||
- Port 1 \(Trunk, tagged\)
|
||||
- Port 2 \(Not a member of client vlan, untagged\)
|
||||
|
||||
- VLAN:802.1Q VLAN PVID setting
|
||||
- Port 2
|
||||
- PVID 20
|
||||
|
||||
### Verify VLAN configuration
|
||||
|
||||
- Manually set consol ip as user
|
||||
- Connect console to Port 2
|
||||
- Check internet connection
|
||||
|
||||
|
||||
## DSM \(DS124\)
|
||||
|
||||
- https://finds.synology.com/# \(192.168.1.11, KEA-DHCP\)
|
||||
- Install DSM
|
||||
|
||||
### Initial configuration
|
||||
- Device name: ilnmorsNAS
|
||||
- Administrator account: il
|
||||
- Password: dsm.il.password
|
||||
|
||||
- automatical update
|
||||
- synology account - skip, skip anyway
|
||||
- opt disagree
|
||||
|
||||
### Storage
|
||||
|
||||
- Storage Manager:Storage:Create:btrfs - Create
|
||||
|
||||
### IP address
|
||||
|
||||
Kea in fw already reserved DSM's IP. However it is necessary to set IP address statically for stable operation.
|
||||
|
||||
- Control Panel:Network:Network Interface:LAN
|
||||
- Edit:IPv4:Use manual configuration
|
||||
- Edit:IPv6:Use manual configuration
|
||||
|
||||
### Certificates
|
||||
|
||||
- Control Panel:Security:Certificate
|
||||
- Replace an existing certificate:synology
|
||||
- Description: ilnmors.internal
|
||||
- Private Key
|
||||
- Certificate
|
||||
- Intermediate certificate
|
||||
- Edit: For: Set as default certificate
|
||||
- Setting \(!CAUTION!\)
|
||||
- Even though you set the certificate as default, you have to set certificate for each services.
|
||||
- configure: service: certificate: nas.ilnmors.internal
|
||||
|
||||
## Authelia OIDC
|
||||
|
||||
- **!CAUTION!** It can be set after authelia is implemented
|
||||
- Following [here](../../../config/containers/auth/authelia/config/authelia.yaml.j2) for Authelia configuration
|
||||
- Control Panel:Domain/LDAP:SSO Client
|
||||
- Login Settings: \[x\] Select SSO by default on the login page
|
||||
- Services
|
||||
- \[x\] Enable OpenID Connect SSO service
|
||||
- OpenID Connect SSO Settings
|
||||
- Profile: OIDC
|
||||
- Account type: Domain/LDAP/local
|
||||
- Name: Authelia
|
||||
- Well-Known URL: https://authelia.ilnmors.com/.well-known/openid-configuration
|
||||
- Application ID: dsm \(what you designated\)
|
||||
- Application Secret: secret value
|
||||
- Redirect URI: https://nas.ilnmors.internal:5001
|
||||
- Authorization scope: openid profile groups email
|
||||
- Username claim: preferred_username
|
||||
- Match the user name \(ID\) in DSM and lldap id.
|
||||
|
||||
### Kopia in DSM
|
||||
|
||||
#### Upload Kopia repository to DSM
|
||||
|
||||
- Directory
|
||||
- Control Panel:shared folder: docker
|
||||
- Create: docker/kopia - permission: everyone rwx - inheretence to sub directories
|
||||
|
||||
- Container manager
|
||||
- Package Center:Conatiner manager:install
|
||||
|
||||
- Upload repository directory from console to DSM
|
||||
- docker/kopia/repository
|
||||
|
||||
- Add certificate - DSM reverse proxy cannot deal with gRPC
|
||||
- /docker/kopia/config/ssl/nas.key
|
||||
- /docker/kopia/config/ssl/nas.crt \(including intermediate crt\)
|
||||
|
||||
- container manager:images:import
|
||||
- kopia/kopia
|
||||
- tags: \{\{ version['packages']['kopia'] \}\}
|
||||
- run
|
||||
- image: kopia/kopia
|
||||
- containername: kopia-server
|
||||
- \[x\] Enable auto restart
|
||||
- port: 51515:51515
|
||||
- volume: /docker/kopia/config:/app/config:rw
|
||||
- volume: /docker/kopia/cache:/app/cache:rw
|
||||
- volume: /docker/kopia/logs:/app/logs:rw
|
||||
- volume: /docker/kopia/repository:/repository:rw
|
||||
- environment: KOPIA_PASSWORD=$KOPIA.REPOSITORY
|
||||
- command: server start --no-ui --tls-cert-file=/app/config/ssl/nas.crt --tls-key-file=/app/config/ssl/nas.key --address=0.0.0.0:51515 --log-level=info
|
||||
- action:Terminal:Create
|
||||
```bash
|
||||
kopia repository connect filesystem \
|
||||
--path=/repository \
|
||||
--override-username="il" \
|
||||
--override-hostname="nas.ilnmors.internal"
|
||||
```
|
||||
- action:restart
|
||||
|
||||
- Set firewall nftables
|
||||
- Remove kopia_tmp dir from console
|
||||
|
||||
### Connection from client
|
||||
|
||||
#### Structure
|
||||
|
||||
Repository directory - encrypted by server KOPIA_PASSWORD as master key of repository
|
||||
|
||||
Server manage ACL with user password, user's KOPIA_PASSWORD. When server verify user with their password, server works with its repository password.
|
||||
|
||||
Repository - \(Repository key; master key\) - Server - \(User key; access key\) - Client
|
||||
|
||||
- Client knows its access password as KOPIA_PASSWORD to access server. It doesn't know master key, server's KOPIA_PASSWORD. server will control repository by its KOPIA_PASSWORD. their name is the same but it is different.
|
||||
|
||||
#### Access
|
||||
|
||||
```bash
|
||||
# Console
|
||||
# you have to use `'` not `"`
|
||||
KOPIA_PASSWORD='$kopia.user.user_name' \
|
||||
/usr/bin/kopia repository connect server \
|
||||
--url=https://nas.ilnmors.internal:51515 \
|
||||
--override-username=console \
|
||||
--override-hostname=console.ilnmors.internal
|
||||
# This makes repository.config on ~/.config/kopia
|
||||
# verify with kopia server acl list command
|
||||
|
||||
# infra or app
|
||||
## /etc/secrets/$KOPIA_UID/kopia.env
|
||||
KOPIA_PASSWORD={{ hostvars['console']['kopia']['user'][node['name']] }}
|
||||
KOPIA_CONFIG_PATH=/etc/kopia/repository.config
|
||||
KOPIA_CACHE_DIRECTORY=/var/cache/kopia
|
||||
KOPIA_LOG_DIR=/var/cache/kopia/logs
|
||||
KOPIA_CHECK_FOR_UPDATES=false
|
||||
## .service file
|
||||
BindReadOnlyPaths=/path/to/backup
|
||||
# In root namescope, %u always bring 0
|
||||
BindPaths=/etc/kopia
|
||||
BindPaths=/etc/secrets/{{ kopia_uid }}
|
||||
BindPaths=/var/cache/kopia
|
||||
EnvironmentFile=/etc/secrets/{{ kopia_uid }}/kopia.env
|
||||
|
||||
ExecStartPre=/usr/bin/kopia repository connect server \
|
||||
--url=https://{{ infra_uri['kopia']['domain'] }}:{{ infra_uri['kopia']['ports']['https'] }} \
|
||||
--override-username={{ node['name'] }} \
|
||||
--override-hostname={{ node['name'] }}.ilnmors.internal
|
||||
|
||||
ExecStart=/usr/bin/kopia snapshot create \
|
||||
/path/to/backup
|
||||
```
|
||||
|
||||
### Check kopia snapshot
|
||||
|
||||
```bash
|
||||
# snapshot id check
|
||||
kopia snapshot list [--all]
|
||||
# Snapshot ID check
|
||||
kopia ls -l [-r: for recursive] $SNAPSHOT_ID
|
||||
kopia show -l $SNAPSHOT_ID/file/path
|
||||
# or
|
||||
kopia show -l $FILE_ID
|
||||
```
|
||||
|
||||
### Restore
|
||||
|
||||
```bash
|
||||
mkdir -p /mnt/kopia
|
||||
kopia mount [$SNAPSHOT_ID|all] kopia &
|
||||
```
|
||||
@@ -0,0 +1,248 @@
|
||||
# kopia
|
||||
|
||||
## Create repository
|
||||
|
||||
```bash
|
||||
# export KOPIA_PASSWORD="your_repository_password"
|
||||
# It exists by Docker environment, however when it is used by pure linux server this environemnt is needed.
|
||||
|
||||
# *! CAUTION !*
|
||||
# THIS PROCESS CONTAINING SECRET VALUES.
|
||||
# WHEN YOU TYPE THE COMMAND ON SHELL, YOU MUST USE [BLANK] BEFORE COMMAND
|
||||
# e.g.
|
||||
# shell@shell$ command (X)
|
||||
# shell@shell$ [BLANK]command (O)
|
||||
# BLANK prevent the command to save on .bash_history
|
||||
# After finish this process, use `history -c` and `clear` for just in case.
|
||||
|
||||
|
||||
# Kopia CLI Create repository
|
||||
mkdir -p /home/console/workspace/kopia_tmp/{repository,config,cache}
|
||||
|
||||
/usr/bin/kopia repository create filesystem \
|
||||
--password="kopia.repository" \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--path="/home/console/workspace/kopia_tmp/repository" \
|
||||
--description="ilnmors.internal kopia repository" \
|
||||
--create-only
|
||||
|
||||
# block hash: BLAKE2B-256-128
|
||||
# encryption: AES256-GCM-HMAC-SHA256
|
||||
# key derivation: scrypt-65536-8-1
|
||||
# splitter: DYNAMIC-4M-BUZHASH
|
||||
|
||||
# Kopia CLI Connect repository
|
||||
/usr/bin/kopia repository connect filesystem \
|
||||
--password="kopia.repository" \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--path="/home/console/workspace/kopia_tmp/repository" \
|
||||
--override-username="il" \
|
||||
--override-hostname="nas.ilnmors.internal" \
|
||||
--description="ilnmors.internal kopia repository"
|
||||
|
||||
# Connected to repository
|
||||
|
||||
# Start kopia server on pure linux server
|
||||
|
||||
# /usr/bin/kopia server start \
|
||||
# --password="kopia.repository" \
|
||||
# --config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
# --cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
# --no-ui \
|
||||
# --address=localhost:51515 \
|
||||
# --insecure \
|
||||
# --log-level=info &
|
||||
|
||||
# All after these processes reset history.
|
||||
clear
|
||||
history -c
|
||||
```
|
||||
|
||||
### Set users
|
||||
|
||||
- Container manager: container: kopia-server: Action: Open terminal: Create: bash
|
||||
|
||||
```bash
|
||||
# export KOPIA_PASSWORD="your_repository_password"
|
||||
# It exists by Docker environment, however when it is used by pure linux server this environemnt is needed.
|
||||
|
||||
# Create users
|
||||
## console@console.ilnmors.internal, infra@infra.ilnmors.internal, app@app.ilnmors.internal
|
||||
## Repository pasword is not required, because it is already connected
|
||||
/usr/bin/kopia server users add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user-password="$kopia.user.console" \
|
||||
console@console.ilnmors.internal
|
||||
|
||||
/usr/bin/kopia server users add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user-password="$kopia.user.infra" \
|
||||
infra@infra.ilnmors.internal
|
||||
|
||||
/usr/bin/kopia server users add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user-password="$kopia.user.app" \
|
||||
app@app.ilnmors.internal
|
||||
|
||||
## verify
|
||||
/usr/bin/kopia server users list \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache"
|
||||
|
||||
# ACL
|
||||
## Console - Run these commands on local container
|
||||
## snapshot, policy, user, acl, content
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="console@console.ilnmors.internal" \
|
||||
--target="type=snapshot" \
|
||||
--access=FULL
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="console@console.ilnmors.internal" \
|
||||
--target="type=policy" \
|
||||
--access=FULL
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="console@console.ilnmors.internal" \
|
||||
--target="type=user" \
|
||||
--access=FULL
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="console@console.ilnmors.internal" \
|
||||
--target="type=acl" \
|
||||
--access=FULL
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="console@console.ilnmors.internal" \
|
||||
--target="type=content" \
|
||||
--access=FULL
|
||||
|
||||
## app and infra - Run these command on console
|
||||
### Global pull to remove duplicates
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="infra@infra.ilnmors.internal" \
|
||||
--target="type=content" \
|
||||
--access=APPEND
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="app@app.ilnmors.internal" \
|
||||
--target="type=content" \
|
||||
--access=APPEND
|
||||
|
||||
### Snapshot
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="infra@infra.ilnmors.internal" \
|
||||
--target=type=snapshot,username=infra,hostname=infra.ilnmors.internal \
|
||||
--access=FULL
|
||||
|
||||
/usr/bin/kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="app@app.ilnmors.internal" \
|
||||
--target=type=snapshot,username=app,hostname=app.ilnmors.internal \
|
||||
--access=FULL
|
||||
|
||||
### Policy
|
||||
kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="infra@infra.ilnmors.internal" \
|
||||
--target="type=policy" \
|
||||
--access=READ
|
||||
|
||||
kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="infra@infra.ilnmors.internal" \
|
||||
--target="type=policy,username=infra,hostname=infra.ilnmors.internal" \
|
||||
--access=FULL
|
||||
|
||||
kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="app@app.ilnmors.internal" \
|
||||
--target="type=policy" \
|
||||
--access=READ
|
||||
|
||||
kopia server acl add \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--user="app@app.ilnmors.internal" \
|
||||
--target="type=policy,username=app,hostname=app.ilnmors.internal" \
|
||||
--access=FULL
|
||||
```
|
||||
|
||||
### Set policy
|
||||
|
||||
- Container manager: container: kopia-server: Action: Open terminal: Create: bash
|
||||
|
||||
```bash
|
||||
# compression
|
||||
/usr/bin/kopia policy set \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--global \
|
||||
--compression=zstd
|
||||
|
||||
# infra for db dump sql
|
||||
kopia policy set infra@infra.ilnmors.internal \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--compression=zstd \
|
||||
--keep-latest=10 \
|
||||
--keep-hourly=0 \
|
||||
--keep-daily=14 \
|
||||
--keep-weekly=4 \
|
||||
--keep-monthly=6 \
|
||||
--keep-annual=1
|
||||
|
||||
# app for media, stateful data
|
||||
kopia policy set app@app.ilnmors.internal \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
--compression=none \
|
||||
--keep-latest=10 \
|
||||
--keep-hourly=0 \
|
||||
--keep-daily=14 \
|
||||
--keep-weekly=4 \
|
||||
--keep-monthly=6 \
|
||||
--keep-annual=1
|
||||
|
||||
# verify
|
||||
kopia server acl list \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
|
||||
kopia policy list \
|
||||
--config-file="/home/console/workspace/kopia_tmp/config/repository.config" \
|
||||
--cache-directory="/home/console/workspace/kopia_tmp/cache" \
|
||||
```
|
||||
|
||||
## Remove repository from Console
|
||||
|
||||
```bash
|
||||
# After uploading repository to DSM
|
||||
rm -rf /home/console/workspace/kopia_tmp
|
||||
history -c
|
||||
```
|
||||
@@ -0,0 +1,71 @@
|
||||
# Git configuration
|
||||
|
||||
## Local git
|
||||
|
||||
```bash
|
||||
# git config
|
||||
git config --global user.name "il"
|
||||
git config --global user.email "il@ilnmors.internal"
|
||||
git config --global init.defaultBranch main # Set default branch name as main
|
||||
|
||||
# Git repository path
|
||||
# $PROJECT_REPOSITORY/.git
|
||||
|
||||
# Remote registration and push git
|
||||
cd ~/workspace/homelab
|
||||
|
||||
# Create .gitignore management for data directory
|
||||
echo data/bin/ | tee ./.gitignore
|
||||
echo data/volumes/ | tee -a ./.gitignore
|
||||
echo data/images/ | tee -a ./.gitignore
|
||||
echo docs/archives/textfiles/ | tee -a ./.gitignore
|
||||
|
||||
# Select files
|
||||
# When the set repository as first time
|
||||
git init
|
||||
|
||||
# To commit and tag, you should `git add`
|
||||
git add .
|
||||
# Check git changes
|
||||
git status
|
||||
git commit -m "1.0.0: Release IaaS baseline"
|
||||
# git commit -m "docs: update 07-git.md to add the way to manage git system"
|
||||
# Make current documents as snapshot
|
||||
git tag -a 1.0.0 -m "IaaS baseline"
|
||||
# Make special changes
|
||||
# In this homelab, [Infra_structure_change]:[Services_change]:[Documents_and_configuration_change]
|
||||
# Tagging and commit should be distinguished.
|
||||
# The change which affects system: tagging
|
||||
# The change which doesn't affect system: commit
|
||||
|
||||
# Commands
|
||||
git status # What files are changed
|
||||
git log # The version record
|
||||
git diff # What is changed after last commit
|
||||
git show $tag # Tag version and information check.
|
||||
git checkout $tag # rollback to tag version
|
||||
git branch $branch_name # Create branch
|
||||
git switch $branch_name # Switch branch
|
||||
git branch # list of branch
|
||||
git switch main # Switch to main branch
|
||||
git merge $branch_name # run at the main branch, merge.
|
||||
git stash # temporary save
|
||||
git stash pop # get temporary save
|
||||
|
||||
# After git switch
|
||||
git switch service
|
||||
git rebase --ignore-date main # set date as current time on main branch
|
||||
```
|
||||
|
||||
## Add Service with git
|
||||
|
||||
```bash
|
||||
# Example of establish gitea
|
||||
git branch caddy-app
|
||||
git switch caddy-app
|
||||
git commit -m "0.0.1-caddy-app: Start caddy-app branch"
|
||||
git tag -a 0.0.1-caddy-app -m "caddy-app: Start caddy-app branch"
|
||||
## After finishing gitea implement
|
||||
git switch main
|
||||
git merge caddy-app
|
||||
```
|
||||
Reference in New Issue
Block a user