il/ilnmors-homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

1.0.0 Release IaaS

Browse Source
This commit is contained in:
Il Lee
2026-03-15 04:41:02 +09:00
commit a7365da431
292 changed files with 36059 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

657
docs/archives/2025-12/03_common/03_01_debian_configuration.md Normal file
View File

@@ -0,0 +1,657 @@
Tags: #common, #configuration, #os
## Installation
### General setting
- Language: English - English
- Location: Other > Asia > South Korea
- Locale: United State - en_US.UTF-8
- Keymap to use: American English
### Network setting
#### Auto Configuration
- Auto Configuration: Using DHCP
#### Manual configuration
> `net` server should be configured manually - It has no private DHCP server and DNS server
- `cancel` auto configuration is needed
- IP address: 192.168.10.11/24
- Gateway: 192.168.10.1
- Name server address: 1.1.1.2
> After AdGuard home and BIND are set, change 192.168.10.11 on `/etc/resolv.conf`
#### Common setting
- Hostname: \[vmm, net, auth, dev, app\]
- Domain: ilnmors.internal
### User setting
- Root Password
- User name: \[vmm, net, auth, dev, app\]
- User account: \[vmm, net, auth, dev, app\]
- User Password
### Partition setting
- Partitioning method: Manual
#### Common
- 512MiB - EFI system partition (Boot flag: on)
- 1GiB - Ext4 Journaling (Mount: `/boot`)
#### Hypervisor
- 780GiB - Physical volume for LVM
- Configure the Logical Volume Manager
- Create volume group - vmm
- Create logical volume - vmm - \[root, swap, data\]
- Finish
- vmm-root: 64GiB - Ext Journaling (Mount: `/`)
- vmm-swap: 16GiB - SWAP area
- vmm-libvirt: 700GiB - Ext4 Journaling (Mount: `/var/lib/libvirt`)
#### VM servers
- 32GiB(net), 64GiB(auth), 256GiB(dev,app) - Ext4 Journaling(mount: /)
> Don't set swap partition, in qcow2 it is not effective but causes over head(CoW)
#### Application server
- Don't set HDD's partition
- Each HDD (2TB)
> btrfs supports RAID itself. After installing, set RAID10. These will be set with `fdisk`
### Debian package manager setting
- Scan extra installation media: no
- Mirror country: South Korea
- Archive mirror: deb.debian.org
- Proxy: \[blank\]
- Popularity-contest: no
### Installing packages setting
- \[\*\] SSH server
- \[\*\] Standard system utilities
## Environment configuration
### Package configuration
#### Common packages
```bash
apt update && apt upgrade
apt install sudo iptables-persistent crowdsec acl
```
- wait-for-it.sh
- Download it [here](https://github.com/vishnubob/wait-for-it/blob/master/wait-for-it.sh)
- ~/data/config/scripts
#### Hypervisor
```bash
apt install qemu-system-x86 ksmtuned libvirt-daemon-system virtinst virt-top ifupdown2 openvswitch-switch
# ksmtuned automatically adjusts ksm following its usage
# `cat /sys/kernel/mm/ksm/pages_sharing` shows the page shared by ksmtuned, result * 4KiB is sharing volume
```
#### VMs
```bash
apt install podman curl jq age
# app
apt install btrfs-progs
# SOPS package
curl -LO https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops
chmod +x /usr/local/bin/sops
# Check update this via Diun
```
### User configuration
```bash
usermod [user_name] -u [uid]
# Change default group as svadmins
groupmod [group_name] -g 2000 -n svadmins
chown -R [user_name]:svadmins /home/[user_name]
chmod 770 /home/[user_name]
usermod -aG sudo [user_name]
# Hypervisor
usermod -aG kvm vmm
usermod -aG libvirt vmm
# After user configuration, proceed all step as user with sudo
```
### Make directory structure
#### Common
- /etc/secrets (root:root 711)
- /etc/secrets/\$UID (\$UID:root 500, files: $UID:root 400)
#### Hypervisor
- (vmm:svadmins 700)
- ~/data/config/{scripts,server,services,vms}
- ~/data/config/vms/{networks,storages,dumps}
- /var/lib/libvirt/images
#### VMs
- (\[vms\]:svadmins 700)
- ~/data/{config,containers}
- ~/data/config/{containers,scripts,secrets,server,services}
- ~/data/containers/apps/{certs,etc.}
- ~/kopia
#### Application server
##### SSD
- (app:svadmins 700)
- ~/data/{config,containers}
- ~/data/config/{containers,scripts,secrets,server,services}
- ~/data/containers/app/{certs,etc.}
- ~/kopia
##### HDD
RAID10 HDDs mount on this directory. Following 08_01_app_vm before you set.
- (app:svadmins 700)
- ~/hdd/data/containers
- (app:svadmins 770)
- ~/hdd/backups
- The scrub timer systemd is required for its integrity.
### SSH configuration
- File:
- /etc/ssh/sshd_config
- ~/.ssh/authorized_keys
```ini
# /etc/ssh/sshd_config
# key generation
ssh-keygen -t ed25519 -f ~/.ssh/key_name -C "comment"
# ...
PermitRootLogin no
# ...
```
```bash
# Reload sshd
sudo systemctl restart sshd
# Deploy publicly key for ssh
mkdir ~/.ssh && chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys
# Paste public key from the client
```
> If the ICMP protocol doesn't work towards Windows, check Windows' firewall inbound `파일 및 프린터 공유(에코 요청 - ICMPv4-In)" and "파일 및 프린터 공유(에코 요청 - ICMPv6-In)` activatation
### Linger configuration
```bash
sudo loginctl enable-linger [user_name]
loginctl show-user $(whoami)
# Linger=yes
```
> Linger is necessary to execute services regardless their login session. If linger option were not activated, all the process would be terminated after logout.
### Network configuration
- File:
- /etc/hosts
- /etc/resolv.conf
- /etc/network/interfaces
#### Hostname
```ini
# /etc/hosts
[ip_address] [FQDN] [hostname]
```
#### DNS
- Only for `hypervisor` and `net` which are set manually.
```ini
# /etc/resolv.conf
# Before setting local DNS
nameserver 1.1.1.2
# After setting local DNS (net server)
# nameserver 192.168.10.11
```
#### Interface
- Only for `hypervisor` and `net` which are set manually
##### Hypervisor
```ini
# /etc/network/interfaces
# ifupdown2 and openvswitch-switch are required
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The openvswitch virtual bridge interfaces
auto ovsbr0
iface ovsbr0 inet manual
ovs_type OVSBridge
ovs_ports enp2s0
auto ovsbr1
iface ovsbr1 inet manual
ovs_type OVSBridge
ovs_ports enp3s0 vlan1 vlan10
# The primary network interfaces
auto enp2s0
iface enp2s0 inet manual
ovs_type OVSPort
ovs_bridge ovsbr0
auto enp3s0
iface enp3s0 inet manual
ovs_type OVSPort
ovs_bridge ovsbr1
ovs_options vlan_mode=native-untagged tag=1 trunks=10
# The vlan interfaces
auto vlan1
iface vlan1 inet static
ovs_type OVSIntPort
ovs_bridge ovsbr1
ovs_options tag=1
address 192.168.1.10/24
auto vlan10
iface vlan10 inet static
ovs_type OVSIntPort
ovs_bridge ovsbr1
ovs_options tag=10
address 192.168.10.10/24
gateway 192.168.10.1
# The primary network interfaces for IPv6
# iface enp2s0 inet6 auto
# iface enp3s0 inet6 auto
# openvswitch vlan options
# ovs_options tag=n : access mode
# ovs_options trunk=n : trunk mode
# ovs_options vlan_mode=native-untagged tag=n trunk=m: native untagged mode
```
> Having two IP addresses in a client can cause some problems. One of them is `Asymmetric routing`. When it gets the packet via vlan10 interface from vlan1 client, the server uses vlan1 interface, because client and server are in the same subnet (L2). It can cause problem, like disconnection usually. There is the solution to solve this.
>
> - Use `mangle` table in iptables, you can solve this, but it is complex to use.
> - Use `IP rule` command, it is easier to set.
##### net
```ini
# /etc/network/interfaces
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto enp1s0
iface enp1s0 inet static
address 192.168.10.11/24
gateway 192.168.10.1
# Don't set dns options. It will be set on /etc/resolv.conf
```
### Hypervisor
#### VFIO configuration
##### IOMMU setting
- File: /etc/default/grub
```ini
# /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
# Intel CPU: intel_iommu=on, AMD CPU: amd_iommu=on
```
##### Device setting
- File: /etc/modprobe.d/vfio.conf
```bash
# Check device
lspci -nn | grep -i sata
# 04:00.0 STAT controller [0106] ... [1b21:1064]
# PCI address: 04:00.00 / Device ID 1b21:1064
# IOMMU group checking
readlink /sys/bus/pci/devices/0000\:04\:00.0/iommu_group
# ../../../../kernel/iommu_groups/14
# The group of PCI device which is needed to passthrough is 14
ls /sys/kernel/iommu_groups/devices/14
# 04:00.00 # To passthrough device, it has to be only device in its IOMMU group
```
```ini
# /etc/modprobe.d/vfio.conf
options vfio-pci ids=1b21:1064
softdep ahci pre: vfio-pci
```
##### Apply all configuration
```bash
sudo update-grub
sudo update-initramfs -u
sudo reboot
# Check IOMMU enable message
dmesg | grep -e DMAR -e IOMMU
# Check the driver
lspci -nnk -d 1b21:1064
# Kernel driver in user: vfio-pci
```
### VM servers
#### Serial console setting
```bash
# It is necessary to connect via ssh to set serial setting
sudo systemctl enable getty@ttyS0
sudo systemctl start getty@ttyS0
```
#### Secret management
##### SOPS setting
- File:
- ~/data/config/secrets/age-key.gpg
- ~/data/config/secrets/.sops.yaml
- ~/data/config/secrets/.secret.yaml
- /etc/secrets/\$UID (\$UID:root 500)
```bash
# Generate the key for sops
age-keygen -o ~/data/config/secrets/age-key
# # created: 2025-10-17T13:30:00Z
# # public key: age1ql3z7h0cfscg......
# AGE-SECRET-KEY-1.....
# Public key is printed when key generated
gpg --symmetric age-key && rm age-key
> GPG password: password
nano ~/data/config/secrets/.sops.yaml && chmod 600 ~/data/config/secrets/.sops.yaml
```
```yaml
# ~/data/config/secrets/.sops.yaml
creation_rules:
- path_regex: \.secret\.yaml$
age: [public_key]
```
##### Create Secret file
```yaml
# ~/data/config/secrets/.secret.yaml
# Format of .secret.yaml
# app1.env:
1SECRET: '1secret'
2SECRET: '2secret'
app1.file: |
-----TEXT-AREA-----
contents of 3secret
-----END-AREA-----
# app2.env
3SECRET: '3secret'
4SECRET: '4secret'
# ...
```
```bash
sops --encrypt --in-place ~/data/config/secrets/.secret.yaml
```
##### Secret scripts
- File:
- ~/data/config/scripts/secrets/edit_secret.sh
- ~/data/config/scripts/secrets/extract_secret.sh
```bash
#!/bin/bash
# edit_secret.sh /path/of/secret
set -e
KEY_PATH="$HOME/data/config/secrets"
SECRET_FILE="$1"
usage() {
echo "Usage: $0 \"/path/of/secret/file\""
exit 1
}
if [ -z "$SECRET_FILE" -o ! -f "$SECRET_FILE" ]; then
echo "Error: Secret file path is needed"
usage
fi
if [ ! -f "$KEY_PATH/age-key.gpg" ]; then
echo "Error: There is no key file"
exit 1
fi
# Delete password file after script
cleanup() {
if [ -f "/run/user/$UID/age-key" ]; then
rm -f "/run/user/$UID/age-key"
fi
}
trap cleanup EXIT
echo -n "Enter GPG passphrase: "
read -s GPG_PASSPHRASE
echo
echo "$GPG_PASSPHRASE" | gpg --batch --yes --passphrase-fd 0 \
--output "/run/user/$UID/age-key" \
--decrypt "$KEY_PATH/age-key.gpg" && \
chmod 600 "/run/user/$UID/age-key"
if [ -z "/run/user/$UID/age-key" ]; then
echo "Error: Key file does not exist"
exit 1
fi
gpgconf --kill gpg-agent
SOPS_AGE_KEY="$(cat "/run/user/$UID/age-key")"
SOPS_AGE_KEY="$SOPS_AGE_KEY" sops "$SECRET_FILE"
```
```bash
#!/bin/bash
# extract_secret.sh /path/of/secret (-f|-e <value>)
set -e
KEY_PATH="$HOME/data/config/secrets"
SECRET_FILE=$1
# shift the $2 as $1 ($1 < $2)
shift
# usage() function
usage() {
echo "Usage: $0 \"/path/of/secret/file\" (-f|-e \"yaml section name\")" >&2
echo "-f <type name>: Print secret file" >&2
echo "-e <type name>: Print secret env file" >&2
exit 1
}
while getopts "f:e:" opt; do
case $opt in
f)
VALUE="$OPTARG"
TYPE="FILE"
;;
e)
VALUE="$OPTARG"
TYPE="ENV"
;;
\?) # unknown options
echo "Invalid option: -$OPTARG" >&2
usage
;;
:) # parameter required option
echo "Option -$OPTARG requires an argument." >&2
usage
;;
esac
done
# Get option and move to parameters - This has no functional thing, because it only use arguments with parameters
shift $((OPTIND - 1))
# Check necessary options
if [ ! -f "$SECRET_FILE" ]; then
echo "Error: secret file path is required" >&2
usage
fi
if [ -z "$TYPE" ]; then
echo "Error: -f or -e option requires" >&2
usage
fi
if [ ! -f "$KEY_PATH/age-key.gpg" ]; then
echo "Error: There is no key file" >&2
usage
fi
# Delete password file after script
cleanup() {
if [ -f "/run/user/$UID/age-key" ]; then
rm -f "/run/user/$UID/age-key"
fi
}
trap cleanup EXIT
echo -n "Enter GPG passphrase: " >&2
read -s GPG_PASSPHRASE
echo >&2
echo "$GPG_PASSPHRASE" | gpg --batch --yes --passphrase-fd 0 \
--output "/run/user/$UID/age-key" \
--decrypt "$KEY_PATH/age-key.gpg" && \
chmod 600 "/run/user/$UID/age-key"
if [ ! -f "/run/user/$UID/age-key" ]; then
echo "Error: Key file does not exist" >&2
exit 1
fi
gpgconf --kill gpg-agent
SOPS_AGE_KEY="$(cat "/run/user/$UID/age-key")"
if [ "$TYPE" == "FILE" ]; then
if RESULT=$(SOPS_AGE_KEY="$SOPS_AGE_KEY" sops --decrypt --extract "[\"$VALUE\"]" --output-type binary "$SECRET_FILE") ; then
echo -n "$RESULT"
exit 0
else
echo "Error: SOPS extract error" >&2
exit 1
fi
fi
if [ "$TYPE" == "ENV" ]; then
if RESULT=$(SOPS_AGE_KEY="$SOPS_AGE_KEY" sops --decrypt --extract "[\"$VALUE\"]" --output-type dotenv "$SECRET_FILE") ; then
echo -n "$RESULT"
exit 0
else
echo "Error: SOPS extract error" >&2
exit 1
fi
fi
```
##### Secret value management
- Using `extract_secret.sh`
- Inject secret value to `podman secret` or `/etc/secrets/$UID`
```bash
# /etc/secrets/$UID
# Before use sudo tee, make sure sudo doesn't need password.
# i.e. sudo ps -ef command execute before this command.
# Env file
extract_secret.sh ~/data/config/secrets/.secret.yaml -e "$value" > /run/user/$UID/tmp.env \
&& sudo mv /run/user/$UID/tmp.env /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chown $UID:root /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chmod 400 /etc/secrets/$UID/"$FILE_NAME"
# Normal file
extract_secret.sh ~/data/config/secrets/.secret.yaml -f "$value" > /run/user/$UID/tmp.env \
&& sudo mv /run/user/$UID/tmp.env /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chown $UID:root /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chmod 400 /etc/secrets/$UID/"$FILE_NAME"
# Podman secret
# Podman doesn't supports .env file parsing, you have to enroll all values
extract_secret.sh ~/data/config/secrets/.secret.yaml -f "$value" | podman secret create "[$FILE_NAME|$ENV_NAME]" -
```
> The reason why apply the secret value manually is its complexity. The `yaml` file can hold a lot of values in one section. It is hard to make the script to deal with a lot of exception and senario. The simple way is the best way.

273
docs/archives/2025-12/03_common/03_02_iptables.md Normal file
View File

@@ -0,0 +1,273 @@
Tags: #common, #configuration, #network, #security
## iptables
iptables is the firewall program to manage netfilter which is in the linux kernel. Basically, the iptables' settings are temporary(when you reboot the computer they disappear). So, you can use netfilter-persistent program to make settings permanent. iptables has 3 modes(tables), which are filter, nat, and mangle. And each table has their chain like input, forward or prerouting, etc. iptables' setting is temporary, when the machine reboot, all rules will be reset. After set the rules use `netfilter-persistent save` to make the rules permanently.
### tables
#### filter
filter is the basic and most important table in iptables. Its role is to simply judge, to ACCEPT or DROP the packets. When you use the iptables command without table option (-t), the filter table is default option. There are chians of filter table below.
- INPUT: check the packets in
- OUTPUT: check the packets out
- FORWARD: check the packets which are passing through
#### nat
nat is the table changing packets' address and port without changing contents of packets. There are chians of nat table below.
- PREROUTING: change the destination address or port right after packets arrived
- POSTROUTING: change the source address or port before packets depart
- OUTPUT: change the destination address or port of packets which produced by itself (This doesn't change the source IP; DNAT)
#### mangle
mangle is a special table to mark on the packets. It works on the every chain and it works on special purpose like asymmetric routing.
### grammar
#### Commands
- -A \[--append\]: create the new rules
- -C \[--check\]: check the packets
- -D \[--delete\]: delete the rules
- -F \[--flush\]: delete all rules from the chain
- -I \[--insert\]: Insert the new rules
- -L \[--list\]: print the rules
- -N \[--new\]: create the new chain
- -P \[--policy\]: change the default policy
- -R \[--replace\]: change the rules as a new rule
- -X \[--delete-chain\]: delete chain
- -Z \[--zero\]: reset the packet and byte counter value of all chain
#### match
- -s \[--source\]: designate source ip address or networks
- -d \[--destination\]: designate destination ip address or networks
- -p \[--protocol\]: match protocol(tcp/udp/icmp.. etc.)
- --dport: designate specific protocol number (When the protocol is already defined - tcp or udp)
- --syn: match syn packets. when starting new TCP connection, apply the rule.
- -i \[--in-interface\]: input interface
- -o \[--out-interface\]: output interface
- --comment: comment(max 256byte)
- -f \[--fragment\]
- -t \[--table\]: designate table set(default: filter)
- -j \[--jump\]: designate targets
- -m \[--match\]: match with specific module
- conntrack --ctstate: current linked connection
#### target
- ACCEPT: allow packets
- DROP: deny packets without response (Hide server existence)
- REJECT: deny packets with response (Show server existance)
- LOG: log the packets on syslog
- RETURN: stop current rules, and return to the previous chain
#### Command
```bash
iptables -L -v -n # Print all rules in filter table
iptables -L -v -n -t nat # Print all rules in nat table
```
## netfilter-persistent
### Save the rules
```bash
sudo netfilter-persistent save
```
- File:
- /etc/iptables/rules.v4
- /etc/iptables/rules.v6
### Reload the rules (manual)
```bash
# Edit the file
# Test
sudo bash -c 'iptables-restore --test < /etc/iptables/rules.v4'
# If there were no message, it would have no error
sudo netfilter-persistent start
# or
sudo netfilter-persistent reload
```
### Rule files
- File: /etc/iptables/rules.v4
#### Hypervisor (vmm)
```ini
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "allow established connection" -j ACCEPT
-A INPUT -i lo -m comment --comment "allow local connection" -j ACCEPT
-A INPUT -p icmp -m comment --comment "allow ICMP connection" -j ACCEPT
-A INPUT -s 192.168.1.11/32 -p tcp -m tcp --dport 22 -m comment --comment "allow emergemcy LAN console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.2/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.3/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.4/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 22 -m comment --comment "allow code-server ssh connection" -j ACCEPT
COMMIT
```
#### net
```ini
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "allow established connection" -j ACCEPT
-A INPUT -i lo -m comment --comment "allow local connection" -j ACCEPT
-A INPUT -p icmp -m comment --comment "allow ICMP connection" -j ACCEPT
-A INPUT -s 10.10.10.2/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.3/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.4/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 22 -m comment --comment "allow OPNsense ssh connection for ACME update" -j ACCEPT
-A INPUT -s 192.168.10.10/32 -p tcp -m tcp --dport 22 -m comment --comment "allow hypervisor ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 22 -m comment --comment "allow code-server ssh connection" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2053 -m conntrack --ctorigdstport 53 -m comment --comment "allow tcp DNS connection which is only prerouted from 53" -j ACCEPT
-A INPUT -p udp -m udp --dport 2053 -m conntrack --ctorigdstport 53 -m comment --comment "allow udp DNS connection which is only prerouted from 53" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2443 -m conntrack --ctorigdstport 443 -m comment --comment "allow tcp DoH(https) connection which is only prerouted from 443" -j ACCEPT
-A INPUT -p udp -m udp --dport 2443 -m conntrack --ctorigdstport 443 -m comment --comment "allow udp DoH(https) connection which is only prerouted from 443" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 2253 -m comment --comment "allow opnsense tcp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p udp -m udp --dport 2253 -m comment --comment "allow opnsense udp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.12/32 -p tcp -m tcp --dport 2253 -m comment --comment "allow auth tcp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.12/32 -p udp -m udp --dport 2253 -m comment --comment "allow auth udp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 2253 -m comment --comment "allow dev tcp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p udp -m udp --dport 2253 -m comment --comment "allow dev udp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.14/32 -p tcp -m tcp --dport 2253 -m comment --comment "allow app tcp nsupdate connection" -j ACCEPT
-A INPUT -s 192.168.10.14/32 -p udp -m udp --dport 2253 -m comment --comment "allow app udp nsupdate connection" -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --dport 53 -m comment --comment "allow and preroute tcp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A PREROUTING -p udp -m udp --dport 53 -m comment --comment "allow and preroute udp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A PREROUTING -p tcp -m tcp --dport 443 -m comment --comment "allow and preroute tcp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
-A PREROUTING -p udp -m udp --dport 443 -m comment --comment "allow and preroute udp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 53 -m comment --comment "NAT local tcp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A OUTPUT -d 127.0.0.1/32 -p udp -m udp --dport 53 -m comment --comment "NAT local udp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A OUTPUT -d 192.168.10.11/32 -p tcp -m tcp --dport 53 -m comment --comment "NAT local tcp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A OUTPUT -d 192.168.10.11/32 -p udp -m udp --dport 53 -m comment --comment "NAT local udp DNS connection 53 to 2053" -j REDIRECT --to-ports 2053
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p udp -m udp --dport 443 -m comment --comment "NAT local udp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 192.168.10.11/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 192.168.10.11/32 -p udp -m udp --dport 443 -m comment --comment "NAT local udp DoH(https) connection 443 to 2443" -j REDIRECT --to-ports 2443
COMMIT
```
#### auth
```ini
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [204:15800]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "allow established connection" -j ACCEPT
-A INPUT -i lo -m comment --comment "allow local connection" -j ACCEPT
-A INPUT -p icmp -m comment --comment "allow ICMP connection" -j ACCEPT
-A INPUT -s 10.10.10.2/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.3/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.4/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 22 -m comment --comment "allow OPNsense ssh connection for ACME update" -j ACCEPT
-A INPUT -s 192.168.10.10/32 -p tcp -m tcp --dport 22 -m comment --comment "allow hypervisor ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 22 -m comment --comment "allow code-server ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 9000 -m comment --comment "allow opnsense step-ca connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 9000 -m comment --comment "allow dev step-ca connection" -j ACCEPT
-A INPUT -s 192.168.10.14/32 -p tcp -m tcp --dport 9000 -m comment --comment "allow app step-ca connetcion" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2080 -m conntrack --ctorigdstport 80 -m comment --comment "allow tcp http connection which is only from 80" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2443 -m conntrack --ctorigdstport 443 -m comment --comment "allow tcp https connection which is only from 443" -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --dport 80 -m comment --comment "allow and preroute tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A PREROUTING -p tcp -m tcp --dport 443 -m comment --comment "allow and preroute tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 192.168.10.12/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 192.168.10.12/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 636 -m comment --comment "NAT local tcp ldaps connection 636 to 6360" -j REDIRECT --to-ports 6360
-A OUTPUT -d 192.168.10.12/32 -p tcp -m tcp --dport 636 -m comment --comment "NAT local tcp ldaps connection 636 to 6360" -j REDIRECT --to-ports 6360
COMMIT
```
#### dev
```ini
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "allow established connection" -j ACCEPT
-A INPUT -i lo -m comment --comment "allow local connection" -j ACCEPT
-A INPUT -p icmp -m comment --comment "allow ICMP connection" -j ACCEPT
-A INPUT -s 10.10.10.2/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.3/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.4/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 22 -m comment --comment "allow OPNsense ssh connection for ACME update" -j ACCEPT
-A INPUT -s 192.168.10.10/32 -p tcp -m tcp --dport 22 -m comment --comment "allow hypervisor ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.12/32 -p tcp -m tcp --dport 5432 -m comment --comment "allow auth postgresql connection" -j ACCEPT
-A INPUT -s 192.168.10.14/32 -p tcp -m tcp --dport 5432 -m comment --comment "allow app postgresql connection" -j ACCEPT
-A INPUT -s 192.168.10.12/32 -p tcp -m tcp --dport 2080 -m conntrack --ctorigdstport 80 -m comment --comment "allow tcp http connection which is only from 80 and main caddy" -j ACCEPT
-A INPUT -s 192.168.10.12/32 -p tcp -m tcp --dport 2443 -m conntrack --ctorigdstport 443 -m comment --comment "allow tcp https connection which is only from 443 and main caddy" -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --dport 80 -m comment --comment "allow and preroute tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A PREROUTING -p tcp -m tcp --dport 443 -m comment --comment "allow and preroute tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 192.168.10.13/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 192.168.10.13/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
COMMIT
```
#### app
```ini
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "allow established connection" -j ACCEPT
-A INPUT -i lo -m comment --comment "allow local connection" -j ACCEPT
-A INPUT -p icmp -m comment --comment "allow ICMP connection" -j ACCEPT
-A INPUT -s 10.10.10.2/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.3/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 10.10.10.4/32 -p tcp -m tcp --dport 22 -m comment --comment "allow vpn console ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.1/32 -p tcp -m tcp --dport 22 -m comment --comment "allow OPNsense ssh connection for ACME update" -j ACCEPT
-A INPUT -s 192.168.10.10/32 -p tcp -m tcp --dport 22 -m comment --comment "allow hypervisor ssh connection" -j ACCEPT
-A INPUT -s 192.168.10.13/32 -p tcp -m tcp --dport 22 -m comment --comment "allow code-server ssh connection" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2080 -m conntrack --ctorigdstport 80 -m comment --comment "allow tcp http connection which is only from 80" -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2443 -m conntrack --ctorigdstport 443 -m comment --comment "allow tcp https connection which is only from 443" -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --dport 80 -m comment --comment "allow and preroute tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A PREROUTING -p tcp -m tcp --dport 443 -m comment --comment "allow and preroute tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 192.168.10.14/32 -p tcp -m tcp --dport 80 -m comment --comment "NAT local tcp http connection 80 to 2080" -j REDIRECT --to-ports 2080
-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
-A OUTPUT -d 192.168.10.14/32 -p tcp -m tcp --dport 443 -m comment --comment "NAT local tcp https connection 443 to 2443" -j REDIRECT --to-ports 2443
COMMIT
```

306
docs/archives/2025-12/03_common/03_03_podman.md Normal file
View File

@@ -0,0 +1,306 @@
systTags: #common, #configuration, #virtualization, #container, #os
## podman
The container is one of virtual technology to run applications independently regardless of what the host system is. One of them, Docker uses daemon(dockerd) and docker socket(root authority), so all container can access hosts' root authority. It causes danger that hackers can take down root authority via docker containers. Additionally, docker daemon system makes it hard to combine each container to systemd, because all containers are in charge of dockerd.
Podman is a new technology to solve docker's problems. It implements daemonless, and rootless container environment. It reduces the danger of the hacker getting root authority via containers, and it is easy to combine each container to systemd.
### Configuration
- File: /etc/containers/registries.conf
```bash
# Check the linger
loginctl show-user $(whoami)
# Linger=yes
```
```ini
# /etc/containers/registries.conf
unqualified-search-registries = ["docker.io"]
```
### Networking
Podman uses a specific IP address and domain names to communicate with its host system using pasta. Pasta is a new default network mode in podman 4.0 which can allow communication with the host system directly. Therefore, it doesn't need a bridge or network host. `169.254.1.2` is 'link-local address' to communicate with host's system. The pasta conduct the SNAT. Therefore, even though the packet's original source IP were container's IP, when the packet passes the pasta its source IP changes as 127.0.0.1 or its own IP (localhost).
- `/etc/hosts` in container
- 169.254.1.2 host.containers.internal host.docker.internal
When you use the command below, you can add domain to the line.
- --host-name mydomain.internal:host-gateway
Additionally, rootless podman containers cannot bind host's privileged ports number(=<1024). Therefore, if container needed to use these ports, you would have to use iptables' nat table. The example of iptables' usage is [here](./03_02_iptables.md).
#### Bridge mode
Bridge mode create a separated virtual IP network from host's network. This mode supports simple DNS function between containers belonging to the same network. This mode basically uses SNAT(Source NAT) mutually; both inbound and outbond. It is because basically podman runs as rootless. So, the container can't distinguish where the packets come from, except from the container belonging to the same podman network. At the same time, client also can't distinguish where the packets come from. Because every packet seems like come from host server. It makes inspection hard for both of client and containers.
### ID mapping
Podman basically mapped container's root to host's executing user. Despite the root, podman uses subuid, subgid system. They are set on `/etc/subuid`, `/etc/subgid`.
- host(1000:1000) < container(0:0)
- host(100999:100999) < container(1000:1000) : subuid, subgid
When podman runs and executes commands with -u option, --userns=keep-id option, -uid, -gid option, it can adjust mapping.
- -u uid:gid: excute container with hosts's uid (bring the host's UID/GID towards container's `/etc/passwd` directly.)
- --userns=keep-id: mapping all container's file permission to host's file permission.
- --cap-add=DAC_READ_SEARCH option: without root authority, container can access every file regardless permission.
#### Mapping error
Some container doesn't execute a container with root permission. They execute the container with their specific uid (i.e. UID:53 - BIND). In this case, when the container runs with `-u` option or `--userns=keep-id` option can make mapping error very frequently.
- `-u` option
When the container runs with `-u` option, the entrypoint can't work properly in many cases because they were already set that runs entrypoint as specific uid. So, if `-u` option were set, then it would cause permission error.
- `--userns=keep-id` option
This option makes container's directory/file UID as the same as the host's directory/file UID. So, it turns off the UID/GID mapping itself. When some directory which has root authorization mapped with hosts' file it occurs UID mapping error.
#### Permission management
Use ACL packages, to give additional permission of directory. It can give the extra permission to subuid or host's uid.
```bash
# u:[subuid]:rwx
sudo getfacl /path/of/podman_directory
# `-d` option is to set permission for file or directory which are created automatically
# `-R` option is to set permission for file or directory which already existed
sudo setfacl -d -m u:[subuid]:rwx /path/of/podman_directory
sudo setfacl -d -m u:[hostuid]:rwx /path/of/podman_directory
```
### Usage of podman
#### Containerfile and build
Containerfile's format is compatible with dockerfile. Here is the example below. Containerfile can be built as podman image with `podman build` command.
```containerfile
FROM caddy:2.10.2-builder-alpine AS builder
RUN xcaddy build \
--with github.com/caddy-dns/rfc2136 \
--with github.com/hslatman/caddy-crowdsec-bouncer/crowdsec \
--with github.com/hslatman/caddy-crowdsec-bouncer/http \
FROM caddy:2.10.2
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
```
```bash
# Build container file as podman image
podman build -t caddy:2.10.2-main -f /path/of/containterfile-caddy-main . && podman image prune -f
# Delete source images
podman rmi caddy:2.10.2-builder-alpine
podman rmi caddy:2.10.2
```
#### Podman images
- `podman images`: Print list of all local podman images
- `podman image pull [image_name]`: Download podman images from repository
- `podman pull` is the same command
- `podman image prune`: Remove unused and untagged images
- `podman image rm [images]`: Remove local podman image
- `podman rmi` is the same command
#### Run and exec
podman ps \[--all\]: it shows podman container lists
- podman run
- --name: container's name
- --restart: restart mode - unless-stopped
- --add-host: additional host domain name on 169.254.1.2
- --cap-add: add some specific privileges without root authority
- -p host_ports:container_port
- -v host_path:container_path:permission(rw, ro, and when you use SELinux, you can use Z or z)
- -e environment_value
- -d: run background
- image_name
- podman exec -it \[container_name\] \[command\]
#### Pod
Pod makes each container which are in the same pod share some specific resources. The network(IP address), storage volumes. However, each container has their own file system, process, and resource limits. So, this is very useful to use various containers which has close relationship like application and Redis(cache db).
- podman pod create
- --name: pod's name
- -p: host_ports:pod_ports
- podman run
- ...
- --pod pod's name
> Don't use `-p` option. Pod already has `-p` option.
#### Container and file management
#### container
- check pure container
```bash
podman run --rm -it --entrypoint sh [image_name] --args
# or
podman run --rm -it [image_name] sh --args
```
#### file management
- Using `podman exec` to manage file
- Use ACL package `setfacl`
- `--cap-add=DAC_READ_SEARCH` option allows to read all file without permission to backup (for kopia)
### Quadlet and systemd
#### Register the secret on podman secret
- Using `edit_secret.sh` and `extract_secret.sh`
- Inject secret value to `podman secret` or `/etc/secrets/$UID`
```bash
# /etc/secrets/$UID
# Before use sudo tee, make sure sudo doesn't need password.
# i.e. sudo ps -ef command execute before this command.
# Env file
extract_secret.sh ~/data/config/secrets/.secret.yaml -e "$value" > /run/user/$UID/tmp.env \
&& sudo mv /run/user/$UID/tmp.env /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chown $UID:root /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chmod 400 /etc/secrets/$UID/"$FILE_NAME"
# Normal file
extract_secret.sh ~/data/config/secrets/.secret.yaml -f "$value" > /run/user/$UID/tmp.env \
&& sudo mv /run/user/$UID/tmp.env /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chown $UID:root /etc/secrets/$UID/"$FILE_NAME" \
&& sudo chmod 400 /etc/secrets/$UID/"$FILE_NAME"
# Podman secret
# Podman doesn't supports .env file parsing, you have to enroll all values
extract_secret.sh ~/data/config/secrets/.secret.yaml -f "$value" | podman secret create "[$FILE_NAME|$ENV_NAME]" -
```
> ` podman secret inspect --showsecret --format '{{.SecretData}}' $secret_name` shows the content of secret
#### Define quadlet file
- File:
- ~/data/config/containers/app/app.container
- ~/data/config/containers/app/app.pod
Quadlet is to define of specification as `.quadlet` or `.container`. Quadlet uses these file to make `.service` file to combine container to systemd. Here is the example of `.container` file below.
```ini
# app.container
[Quadlet]
# Don't make a dependencies
DefaultDependencies=false
[Unit]
Description=app
After=a.service
Wants=a.service
Requires=a.service
[Service]
ExecStartPre=%h/data/config/scripts/wait-for-it.sh -h 192.168.10.1 -p 8080 -t 20
[Container]
# Pod=app.pod
Image=localhost/app:1.0.0
ContainerName=app
PublishPort=2080:80/tcp
PublishPort=2443:443/tcp
AddHost=app.service.internal:host-gateway
Volume=%h/data/containers/app:/home/app:rw
Environment="ENV1=ENV1"
Secret=ENV_NAME,type=env
Secret=app.file,target=/path/of/secret/file/name
# podman run [options] [image] example --config exconfig
Exec=example --config exconfig
# If you want to change Entrypoint itself, use
Entrypoint=sh -c 'command'
# For Diun
Label=diun.enable=true
# For Diun to track repository new version
Label=diun.watch_repo=true
# For Diun, and it needs `diun.yml` configuration
Label=diun.regopt=container-source
[Install]
WantedBy=default.target
```
```ini
# app.pod
[Quadlet]
# Don't make a dependencies
DefaultDependencies=false
[Pod]
Name=app
PublishPort=2080:80/tcp
```
#### Create systemd `.service` file
```bash
# linger has to be activated
mkdir -p ~/.config/containers/systemd
ln -s ~/data/config/containers/app/app.container ~/.config/containers/systemd/app.container
# This command makes ~/.config/systemd/user/my-app.service
systemctl --user daemon-reload
```
#### Enable and start service
```bash
systemctl --user enable app.service
systemctl --user start app.service
```
---
## Following goal
### Health check
```ini
# i.e. caddy
# Podman [Container] section
[Container]
# Health check configuration
# Health check command
HealthcheckCommand=curl -f http://localhost/ || exit 1
# Health check interval
HealthcheckInterval=30s
# the time to wait for health check
HealthcheckTimeout=5s
# the number to try to health check
HealthcheckRetries=3
# the time to wait to start first health check
HealthcheckStartPeriod=15s
# override.conf [Service] section
[Service]
# Restart, if it is not healthy
Restart=on-failure
```

321
docs/archives/2025-12/03_common/03_04_crowdsec.md Normal file
View File

@@ -0,0 +1,321 @@
Tags: #common, #configuration, #network, #security
## CrowdSec
CrowdSec is the free, open-source IPS(Intrusion Prevention System). It has a distributed architecture. When an agent detects malicious IPs from log, it reports the information to LAPI. This information is shared to CrowdSec's central server anonymously and is spread across the world. Additionally, bouncer receives a blacklist from LAPI and when they approach the server, the bouncer blocks them.
### Collection
#### Parser
It is how to organize the raw logs to the parsed log that scenario can understand. It is works on agent, and parsed log is transferred to LAPI to decide.
#### Scenario
It is how to analyze the malicious attack from the parsed log from the Agent. The LAPI decides what to do for malicious attack, and transfers the result to Bouncer to block.
### Agent
Agent is the detector in each server. They analyze the logs. When they find malicious approaches, or abnormal and harmful actions, they report the information to LAPI(Local API). It analizes the log following `Parsers`.
### LAPI
LAPI server is a local central collector and reporter of malicious attack information. It decides what to apply for traffic following `Scenarios`. In this homelab it is located in OPNsense, because the center of gateway of home network is Firewall. When agent reports threats to the LAPI server, LAPI decides whether or not to block them, and reports to the central CrowdSec server. This information will be spread to all CrowdSec users in the world.
### Bouncer
When LAPI decides to block some IPs, they create a blacklist and give it to bouncer. Bouncer blocks and bans some IPs depending on LAPI's blacklist. `Caddy-auth (L7)` and `OPNsense (L4)` will be bouncer to ban. The most important thing is LAPI just decide what to ban, and Bouncer conducts ban practically.
## CrowdSec in OPNsense
OPNsense supports CrowdSec with community plugin. This is not a basic function so when you want to use it in OPNsense you should install the community plugin.
### Installation
- System:Firmware:Plugins
- \[\*\] Show community plugins
- `os-crowdsec`
### General configuration
Services:CrowdSec:Settings
- \[\*\] Enable Log Processor (IDS)
- \[\*\] Enable LAPI
- \[\*\] Enable Remediation Component (IPS)
- \[ \] Manual LAPI configuration
- LAPI listen address: \[opnsense IP: 192.168.10.1\]
- LAPI listen port: 8080
- \[\*\] Create blocklist rules
- `Apply`
> Enable Remediation Component (IPS) option means, Bouncer will be integrated with OPNsense's firewall rules
> Set LAPI configuration manually, `Manual LAPI configuration` is needed.
### Machines configuration
#### OPNsense console
```sh
# 8) shell
cscli machines add [server_name] -a -f -
# ---
# Machine 'server_name' successfully added to the local API.
# url: http://192.168.10.1:8080
# login: [server_name]
# password: (API key)
# ---
```
#### Each server
```ini
# /etc/crowdsec/local_api_credentials.yaml
url: http://192.168.10.1:8080
login: [server_name]
password: (API key)
# /etc/crowdsec/acquis.d/sshd.yaml
---
source: journalctl
journalctl_filter:
- "_SYSTEMD_UNIT=ssh.service" labels:
type: sshd
# origin value is syslog
---
```
```bash
sudo systemctl restart crowdsec
```
#### OPNsense web UI
- Services:CrowdSec:Machines
- checking the lists of server
- Main CLI commands of CrowdSec
```sh
# View active decisions(Ban list)
cscli decisions list
# View alerts
cscli alerts list
# Check connected machines(agents)
cscli machines list
```
---
## TLS on crowdsec communication
TLS can be applied in CrowdSec communication when internal PKI are set (BIND, Step-CA, ACME-Client in OPNsense). CrowdSec communication can contain sensitive information such as API key, it is recommended to set TLS.
### General configuration
- Services:CrowdSec:Settings
- \[\*\] Manual LAPI configuration
### DNS setting
### Add new domain in BIND
Following [here](../06_network/06_03_net_bind.md).
- net server
- file: ~/data/containers/bind/lib/db.ilnmors.internal
```text
...
crowdsec IN CNAME opnsense.ilnmors.internal.
...
```
### ACME setting
#### Use opnsense certificates
- Services:ACME Client:Certificates - Certificates
- Alt name: crowdsec.ilnmors.internal
#### Certificate and key file
- opnsense
- file:
- /var/etc/acme-client/cert-home/\[ramdom_string\]/opnsense.ilnmors.internal/
- fullchain.cer
- opnsense.ilnmors.internal.key
> There is `opnsense.ilnmors.internal.cer` file. However, when client verify the certificate, it verify the intermediate CA's certificate and root CA's certificate both. Therefore, in this case, use fullchain.cer
- /usr/local/etc/ssl/cert.pem
### Add TLS setting in LAPI configuration
- opnsense
- file:
- /usr/local/etc/crowdsec/config.yaml
- /usr/local/etc/crowdsec/local_api_credentials.yaml
- /usr/local/etc/crowdsec/bouncer/crowdsec-firewall-bouncer.yaml
```yaml
# config.yaml
api:
client:
# ...client configurations
server:
enable: true
listen_uri: 192.168.10.1:8080 # actual IP address is required. (Do not use FQDN in here, the service listener is binded on network interface)
# ... server configurations
tls:
cert_file: /var/etc/acme-client/cert-home/[random_string]/opnsense.ilnmors.internal/fullchain.cer
key_file: /var/etc/acme-client/cert-home/[ramdom_string]/opnsense.ilnmors.internal/opnsense.ilnmors.internal.key
# random string is generated by opnsense itself. In real environment, check it first.
```
```yaml
# local_api_credentials.yaml
...
url: https://crowdsec.ilnmors.internal:8080/
```
```yaml
# crowdsec-firewall-bouncer.yaml
...
api_url: https://crowdsec.ilnmors.internal:8080/
...
```
```sh
service crowdsec restart
```
#### CrowdSec LAPI restart setting
- opnsense
- file: /usr/local/etc/cron.d/crowdsec
```text
#minute hour mday month wday who command
0 3 * * * root /usr/local/libexec/crowdsec/upgrade-hub
30 3 * * * root /usr/sbin/service crowdsec reload # Add this line to reload every day.
```
### Each server configuration
#### Server's certificate trust
- each server
- file: /usr/local/share/ca-certificates/root_ca.crt
```bash
sudo update-ca-certificates
```
#### CrowdSec Agent setting
```ini
# /etc/crowdsec/local_api_credentials.yaml
url: https://crowdsec.ilnmors.internal:8080
login: [server_name]
password: (API key)
```
```bash
sudo systemctl restart crowdsec
```
---
## Crowdsec in Caddy-auth
Caddy supports bouncer. Also, it can be work as agent via auth server where Caddy-auth is located.
### Bouncer configuration
> Caddy has to contain local CA's root_ca.crt (Step-CA). - containerfile already includes root_ca.crt in container when it was built
#### OPNsense console
```sh
cscli bouncer add caddy-auth
> API key for 'caddy-auth':
> Secret_value
> Please keep this key since you will not be able to retrieve it!
>
>
cscli collections install crowdsecurity/caddy
```
#### Caddyfile
```ini
# ...
# Crowdsec bouncer setting
{
crowdsec {
# CrowdSec LAPI
api_url https://crowdsec.ilnmors.internal:8080
api_key "{env.CADDY_CROWDSEC_KEY}"
}
}
# ...
```
- `podman exec caddy-auth caddy reload --config /etc/caddy/Caddyfile`
### Agent configuration
#### auth sv
- File:
- /etc/crowdsec/acquis.yaml
- ~/data/container/caddy_auth/data/access.log
```yaml
# /etc/crowdsec/acquis.d/caddy-auth.yaml
filenames:
- /var/log/caddy.log
labels:
type: caddy
```
```ini
# Caddyfile
# ...
(crowdsec_log) {
log {
output file /data/access.log {
roll_size 10mb
roll_keep 5
}
}
}
# ...
caddy.ilnmors.com {
import crowdsec_log
route {
crowdsec
root * /usr/share/caddy
file_server
}
}
```
```bash
podman exec caddy-auth caddy reload --config /etc/caddy/Caddyfile
sudo mkdir /etc/crowdsec/acquis.d
sudo nano /etc/crowdsec/acquis.d/caddy-auth.yaml
ln -s /home/auth/data/containers/caddy-auth/data/access.log /var/log/caddy.log
# install collection(senario + parser) crowdsecurity/caddy
sudo cscli collections install crowdsecurity/caddy
sudo systemctl restart crowdsec
sudo cscli metrics
```
---
## PLAN
- distributed bouncer
- [x] Caddy bouncer
- [ ] dash board

107
docs/archives/2025-12/03_common/03_05_redis.md Normal file
View File

@@ -0,0 +1,107 @@
Tags: #os, #configuration, #virtualization, #container, #database, #cache
## Redis
Redis is the cache database. It doesn't use SQL, it is just key-value data storage.
### Redis in Pod
Redis is combined with master services in the pod. It allows master services and Redis can communication in localhost. It doesn't need TLS, password, or even ACL itself.
There is the list of services which use Redis in Pod.
#### Preperation
##### Create directory for container
```bash
mkdir -p ~/data/containers/app/redis
setfacl -m d:g::0 ~/data/containers/app/redis
setfacl -m d:o::0 ~/data/containers/app/redis
setfacl -m u:
$UID:rwx ~/data/containers/app/redis
setfacl -m u:100998:rwx ~/data/containers/app/redis
setfacl -d -m u:$UID:rwx ~/data/containers/app/redis
setfacl -d -m u:100998:rwx ~/data/containers/app/redis
```
>Redis container executes as 999:999(redis:redis) permission in container. It is mapped host's 100998. Therefore, directories have to have ACL via `setfacl`
### Podman Image
```bash
podman pull redis:8.2.2 # Do not use latest version to management
```
### Quadlet
- File:
- ~/data/config/containers/app/app.pod
- ~/data/config/containers/app/app-redis.container
- ~/data/config/containers/app/app.container
```ini
# ~/data/config/containers/app/app.pod
[Quadlet]
DefaultDependencies=false
[Pod]
PodName=app
# web port
PublishPort=9080:9000/tcp
# LDAP port
#PublishPort=[set_port]:3389
# Prometheus Port
#PublishPort=[set_port]:9300
```
```ini
# ~/data/config/containers/app/app-redis.container
[Quadlet]
DefaultDependencies=false
[Unit]
Description=app - redis
Before=app.service
[Container]
Pod=app.pod
Image=redis:8.2.2
ContainerName=app-redis
# Port 6379
Volume=%h/data/containers/app/redis:/data:rw
Environment="TZ=Asia/Seoul"
[Install]
WantedBy=default.target
```
```ini
# ~/data/config/containers/app/app.container
[Quadlet]
DefaultDependencies=false
[Unit]
Description=app
After=app-redis.service
Requires=app-redis.service
[Container]
Pod=app.pod
Image=app:version
Volume=%h/data/containers/app/app:/data:rw
Environment="TZ=Asia/Seoul"
[Install]
WantedBy=default.target
```

130
docs/archives/2025-12/03_common/03_06_btrfs.md Normal file
View File

@@ -0,0 +1,130 @@
Tags: #common, #configuration, #os, #filesystem
## BTRFS usage
### Setting
```bash
# All hdd needs partition, but it has no filesystem. To make a partition use `fdisk`.
sudo fdisk "$DIVICE_PATH"
> n # create the new parition
> 1 # Partition number
> Default # First Sector
> Default # Last Sectort
> w # write the new partition
# check btrfs-progs package
sudo apt list --installed | grep btrfs-progs
# btrfs-progs/stable,now 6.14-1 amd64 [installed]
sudo mkfs.btrfs -d raid10 -m raid10 -L hdd /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
# These are partition device files
```
### Snapshot
Usually, the Read-Only snapshot is used.
```bash
# Create snapshot_subvolume
sudo btrfs subvolume create /home/app/hdd/data
# Create snapshot_directory
mkdir /home/app/hdd/.snapshot
# Create snapshot
sudo btrfs subvolume snapshot -r /home/app/hdd/data /home/app/hdd/.snapshot/data_[date]
# Rollback (file)
cp /home/app/hdd/.snapshot/data_[date]/file /home/app/hdd/data/
# Roleback (volume)
# Current subvolume move
mv /home/app/hdd/data /home/app/hdd/data_fail
sudo btrfs subvolume snapshot /home/app/hdd/.snapshot/data_[date] /home/app/hdd/data
# If the data successfully recovered
sudo btrfs subvolume delete /home/app/hdd/data_fail
```
### replace HDD to new system
btrfs has its own volume manangement data and filesystem in hdd, as metadata. It has no dependency on specific OS or hardware but it can work on every linux system which supports btrfs.
```bash
# unmount filesystem
sudo umount /home/app/hdd
# turn off the system, and remove all hdd
# add all disk to new hardware(server), and turn on
# scan btrfs
sudo btrfs device scan
sudo btrfs filesystem show
# mount
sudo nano /etc/fstab
# LABEL=hdd /home/app/hdd btrfs defaults,compress=zstd,autodefrag 0 0
sudo mount -a
```
### add extra HDD
```bash
# Add HDD and check the device file
lsblk
# Add hdd to btrfs RAID
sudo btrfs device add /dev/xxx /dev/xxy /home/app/hdd
# Expand the volume
sudo btrfs balance start /home/app/hdd
```
### change HDD
#### btrfs replace
When sata slot is enough to connect new HDD and old HDD, you can use this way.
```bash
# check devid of old HDD
sudo btrfs device stats /home/app/hdd # you can check IO error
sudo btrfs filesystem show /home/app/hdd
# check the new disk's path
lsblk
# /dev/xxx
# Replace
sudo btrfs replace start [old HDD\'s devid] /dev/xxx /home/app/hdd
# check
sudo btrfs replace status /home/app/hdd
```
#### btrfs device add and delete
When sata slot is not enough to connect new HDD and old HDD simultaneously, you can use this way.
```bash
# Check the HDD which will change
sudo btrfs device stats /home/app/hdd # you can check IO error
sudo btrfs filesystem show /home/app/hdd # check disk's devid
# turn off the system and change the broken HDD to the new HDD
# Make sure the hardware supports Hot-Swap, if it didn't support, you would have to turn off your system fully when you are changing the HDD
# if the system couldn't mount automatically, mount it manually
# sudo mount -o degraded /dev/xxx /home/app/hdd # /dev/xxx is one of ordinary HDD
lsblk
# /dev/xxy # new HDD device file path
# Add new HDD
sudo btrfs device add /dev/xxy /home/app/hdd
# Delete broken HDD
sudo btrfs device delete missing /home/app/hdd
# To major balance
sudo btrfs balance start /home/app/hdd
```