il/ilnmors-homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

1.0.0 Release IaaS

Browse Source
This commit is contained in:
Il Lee
2026-03-15 04:41:02 +09:00
commit a7365da431
292 changed files with 36059 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/archives/2025-12/02_theory/02_01_dns.md
+101
View File
@@ -0,0 +1,101 @@
Tags: #theory, #network, #protocol
## DNS (Domain Name System)
In the beginning of the internet, there were a few hosts on networks. It was possible to manage all hosts on network via IP address or domain name in `/etc/hosts` file in each servers. However, it is hard for people to match and remember what IP addresses means. When the internet environment became bigger and bigger, the complex of route the target server would be harder. To solve this problem, the DNS emerged as a translator between IP address and domain name. In modern internet environment, DNS has hierarchy structure from root to TLD, TLD to authoritative server for efficiency.
### Structure of DNS
#### Communication
- DNS: 53 tcp/udp
DNS communication basically uses 53/udp port. However, in the modern internet environment; which means complex environment sometimes the size of packet is above 512 bytes. In this case, DNS uses 53/tcp too. The vulnerability of DNS is that all communication is on plain data. Everyone can conduct sniffing attack towards DNS packet.
- DoT (DNS over TLS): 853 tcp
DoT was developed to encrypt DNS query. DoT uses [TLS](./02_04_tls.md) to request query. This protocol uses TLS. Moreover, because of TLS, nobody can do sniffing attack towards DoT. However, it uses specific port 853. If ISP block the 853 or analyze 853 port, the pattern of usage will be analyzed or even you cannot use DoT itself. Additionally, there is also DNS over DTLS which uses 853 udp.
- DoH (DNS over HTTPS): 443 tcp/udp
DoH is very similar with DoT. It uses TLS, and it was developed to encrypt DNS query. there is just one difference. This uses https(443 tcp/udp) instead of 853 tcp. https is standard of web protocol, so it is hard to analyze someone sends DNS request or common web packets. It means, ISP or government cannot block 443 port itself or analyze the pattern of DNS query. Since 2022, there's the new standard DNS over HTTP/3 which uses 443 udp port.
- DNSSEC (DNS SECurity extensions)
Originally, client couldn't verify integrity of the response from DNS server. If malicious attacker could get authority of cache DNS server to change their records, all clients would get affected. (i.e. pharming attack). DNSSEC is a protocol to guarantee integrity of DNS record. DNSSEC protocol adds some records in zone, RRSIG(Resource Record Signature), DNSKEY, DS, NSEC, CDNSKEY, CDS. All resolver DNS verify integrity of their records to authoritative DNS with these records. This process is similar to PKI, the chain of trust.
- ECH (Encrypted Client Hello)
Basically, client hello packet has SNI (Server Name Indication). Even though all communication under TLS is encrypted, but to start session the packet has to contain the SNI to identify server. To encrypt this information, SNI the ESNI(Encrypted SNI) was developed in 2018 based on TLS 1.3. However, ESNI just could encrypt SNI information. Now, since 2020, the new standard ECH was developed to supersede ESNI. ECH not only encrypt SNI but also encrypt all client hello process. ECH is latest protocol, and it has a lot of dependency in DNS server, service server and client. When all of them supports ECH, then user can use ECH. Because when ECH encrypts client hello data client need the target server's public key (certificate), it has to look up from encrypted DNS (DoH or DoT).
#### Zone
DNS server has zones; Forward zone and Reverse zone.
- Forward zone
Forward zone has basically information of the pair of domain and IP address. The role of this zone is change domain name to IP address. The domains are managed by IANA, TLD is already reserved. (i.e. `.com`, `.org`, etc...) For private network, `.home.arpa` or `.internal` are reserved.
- Reverse zone
Reverse zone also has basically information of the pair of IP address and domain. The role of this zone is change IP address to domain name. To change domain to IP address it uses specific domain name. \[reversed_ip_address\].in_addr.arpa (i.e. 1.168.192.in-addr.arpa)
#### Records
Each zone has their record type. If zone were a kind of DB, record would be a data of DB. There is basic records type below.
- SOA type
Information of ZONE management. Every zone has this SOA type record.
- NS type
Designate authoritative name server of domain zone
- A type
Mapping domain to IPv4 address
- AAAA type
Mapping domain to IPv6 address
- PTR type
Mapping IP address to domain
- CNAME type
Mapping domain to domain. CNAME type is kind of alias of domain. It can't have IP address value. The query acts recursively, and it gets IP address at the end.
#### Key
There is the keys to control DNS records or zone, even DNS server itself.
- rndc key
This key is to control DNS server itself. When rndc key set on DNS server, client can control DNS server with this key like, reboot server, load or unload zone. rndc key is basically generated by `rndc-confgen` command and it is defined on `rndc.conf` and `named.conf`.
- tsig key
This key is to guarantee integrity when the server syncronize zone data between other servers (usually master-slave server). It is possible update records via this key depending on the setting. Therefore, tsig key is usually used for DDNS or DNS-01 challenge. The key is generated in the DNS server, and it defined in `named.conf`.
### DNS Server type
DNS server basically separated as authoritative DNS and recursive DNS.
#### Authoritative DNS
Authoritative DNS has literally authority of domain zone. It doesn't ask recursive queries towards other DNS server in case of the query that is in its authoritative zone. It is necessary to use DNS-01 challenge (ACME protocol).
#### Recursive DNS
Recursive DNS oppositely doesn't have authority of the records in its zone. When it gets query request, it ask recursive query towards authoritative DNS. It can store the information of records (cache) and give response towards client with the cache.
### Split Horizon DNS
Split Horizon DNS means getting different IP address depending on where the client exists. For instance, if there were the domain `example.com`. This domain has its own private IP address, simultaneously own public IP address (from NAT). When client request the query `example.com` in the private network, private DNS would respond its private IP address. However, when the client request the query in the WAN network, public DNS would respond its public address. Client can access `example.com` in both case, but the IP address which client respond are different. To use this protocol, the network route will be efficient because the packet doesn't have to go out to the WAN area in private network. Basically, it is implemented internal authoritative DNS and recursive DNS. Recursive DNS decides where to send the query based on domain.
### DDNS (Dynamic DNS)
Public IP address can be changed by ISP at any time. It is hard (or expensive) to get static public IP address by ISP. However, the service (server) always guarantee their availability regardless what is their IP. DDNS is basically the protocol to change A or AAAA (or CNAME) records in DNS as server's current IP. Server keeps checking their current public IP and when it changes the server send the request to change its A or AAAA records to public authoritative DNS server with authentication with API key or tsig key.
docs/archives/2025-12/02_theory/02_02_dhcp.md
+55
View File
@@ -0,0 +1,55 @@
Tags: #theory, #network, #protocol
## DHCP (Dynamic Host Configuration Protocol)
Before DHCP emerged, every client had to set their own static IP or using RARP(Reverse Address Resolution Protocol. They have critical problems.
- Static IP
- Each host has their own IP regardless they run or not. It cause lack of IP address.
- If administrator made configuration mistake, the network itself could stop. For instance, IP conflict, or subnet configuration error, etc.
- RARP
- RARP works on L2, it makes hard to implement on hardware.
- RARP server had to exist on every subnet(L2), it was inefficient.
To solve this problem, BOOTP was developed and it evolved as DHCP. DHCP works on L3, and like its name they lease IP dynamically. It allocates IP to hosts with 3 steps. Lease, renewal, and release.
### Lease
When DHCP server gets request from host to allocate IP, DHCP server choose one IP address from its subnet pool, and it responds IP/subnet and information to the hosts. It follows the process called DORA.
- DHCP Discover
The host which has no IP address broadcast discover packet including its MAC to local network. Only DHCP server responds this packet, the others discard the packets.
- DHCP Offer
When DHCP server gets discover packet, DHCP server broadcast Offer packet including host's MAC of which sends discovery packet. Only host which sends discovery packet, the others discard the packets.
- DHCP Request
When the host gets offer packet, it consider the network has DHCP server and broadcast request packet to the local network. More than one DHCP server can exist in one network, therefore host broadcast request packet.
- DHCP ACK
When DHCP server gets the Request packet, it searches for an IP address which can be allocated in its pool. When DHCP server finds an available IP address, it sends to ACK packet including IP address and subnet, and optional information(DNS, gateway, etc...) with reservation time.
### Renewal
The host allocated IP from DHCP try to renew its IP before it expires. There are two chances to renew reservation time. The first try to renew is when the reservation time remains half of them; T1. In this time, host uses unicast. If the first try failed, then it try one more time when 87.5% of the lease time has passed; T2. In this time, the host uses broadcast. All try fails, client gives up the leased IP and try the lease process again. DHCP server release the IP address from the host.
- DHCP Request
The host sends request packet to DHCP server as unicast.
- DHCP ACK
When DHCP server gets the request, it sends ACK packet to the host as unicast.
### Release
When the host doesn't use IP address anymore, DHCP server makes IP as available IP in its pool. Especially, client can send `DHCPRELEASE` to DHCP server explicitly when it doesn't need IP address.
### DHCP relay
Commonly, DHCP is located in router. Because router is the center of networks, and it takes charge of number of networks. However, DHCP server doesn't have to be located in router because of existence of DHCP relay. When router gets DHCP packets (DORA), router can relay the packets as unicast between host and DHCP server which are in different subnet.
docs/archives/2025-12/02_theory/02_03_pki.md
+103
View File
@@ -0,0 +1,103 @@
Tags: #theory, #network, #security
## PKI(Public Key Infrastructure)
PKI is defined on RFC4949 and RFC5280. PKI is neither one of the protocols nor algorithms, it is a huge infrastructure itself. It has various parts which are CA(Certification Authority), RA(Registration Authority), VA(Validation Authority) and Certificates.
### CA (Certification Authority)
CA is the most important part in PKI. It is the source of certification for the secure communication. CA has various roles, some can be delegated to the RA or VA. However, CA is essential in PKI. RA or VA are optional to support CA. The purposes of CA are as follows:
- Managing certification policy
- Issuing certificates
- Verifying certificates validity
- Managing CRL(Certification Revocation List)
- Cross-certification between root CAs
CAs often operate in a hierarchy. Each CA has their own role, and they are basically PAA(Policy Approving Authority), PCA(Policy Certification Authority), and CA(Certification Authority). Normally, PAA and PCA are usually combined as root CA, and CA is an intermediate CA.
#### root CA
Root CA is simply CA of CA. It is the source of trust chain. Only a root CA can authenticate itself alongside other root CAs(cross certification) or intermediate CAs below it. Theoretically, and precisely root CA can be divided into PAA and PCA, but practically they operate as one authority, root CA.
#### intermediate CA
Intermediate CA is the authority which issues certificates for end entities we can use (i.e. web servers). Their certificate is digitally signed (the hash value is signed by root CA's private key) by root CA, so every user can verify their trust with root CA's certificate (root CA's public key). The reason for using intermediate CA is for security. If CA's private key was violated, all chain of trust will be disrupted. When only root CA exists and its private key was violated, all secure communication will be threatened. To avoid and manage these threats root CA and intermediate CA are divided.
#### Structure of CA
The basic structure of CA is very simple. It only contains private key, certificate (which is designated by X.509), and provisioner. When CA gets a request of sign on services' certificates it operates as shown below.
- Provisioner checks CSR (Certificate Signing Request) is valid on its policy
- When CSR is valid, it checks the content of certificates(x.509) and its policy
- CSR and content of certificates are valid, CA signs on the certificate with its private key (sign on hash value with CA's private key)
After CA's signing, every client can check its validation with CA's certificates (CA's public key)
### RA (Registration Authority)
Due to CAs many roles, all processes above are made to be inefficient when conducted. Therefore, CA delegates its registration role to RA. RA seeks to examine CSR and certificates' content. When the content of certificate and CSR are valid under its policy, RA would send request to CA to sign.
### VA (Validation Authority)
Additionally, just as RA, CA can delegate its validation role to VA. Basically, VA manages certificates' validation based on CRL (Certificate Revocation List). However, in modern internet environment, it is inefficient to manage all invalid certificates based on a list. To solve this problem, OCSP (Online Certificate Status Protocol) was developed. However, OCSP also has its limitation, the advanced way is being suggested.
#### CRL (Certificate Revocation List)
Once the certificate gets signed from CA, it is always a valid cryptograph before it expires. However, in the real world some certificates have to be revoked in case CA's key is violated or the service no longer operates, etc. So, CA needs to manage these kinds of invalid certificates which doesn't expire, such as letting clients know that their certificates are not valid anymore. This is a concept of CRL. VA releases the CRL to clients, the clients download or update the list to judge the certificates' validation.
#### OCSP (Online Certificate Status Protocol)
CRL was very effective to revoke invalid certificates but when internet is growing, a massive of invalid certificates appears it shows its limitation. Because to check certificates validation, clients need to download and update CRL from CA or VA. There are tons of invalid certificates, and CRL's size become bigger and bigger. OCSP's concept is when client request to check certificates validation, OCSP server response it. To use OCSP, clients don't need to download full CRL anymore. It is very effective to solve CRL's problem. However, it has problems too. OCSP is defined on RFC6960. OCSP's problems are below.
- Clients should expose its identity(IP address) to check certificates validation.
- VA can be SPOF(Single Point of Failure).
- It is very hard to match time syncronization
- The more requests come to VA, the more burden is on VA.(DoS, DDoS problem)
#### OCSP stapling
OCSP's problem was that client should request to VA directly. So, when the certificate providers(like web server, db, etc.) request its validation and give the proof to client it, clients doesn't need to check the validation of certificate to VA anymore. It reduces VA's burden and time synchronization problems.
### Certificate
The form and content of certificate are standardized by X.509. X.509 format contains server's public key, domain(SAN), expired date, and sign of CA, and etc. When CA validate services or people who sent CSR(which contains information to create certificate with x.509 form), CA signs on their certificates (sign on hash value with CA's private key). Then, all clients can validate that certificates with CA's public key(CA's certificate) cryptographically. Ultimately, when clients trust CA, then they can trust services that have certificate signed by CA. However, even if the certificate were valid cryptographically, some certificates would not be valid. It is reason why CRL or OCSP is needed.
#### The way to issue certificate
##### ACME
Basically, certificate is formed by X.509. The services which want to get certificate send request(CSR) to CA. When the services were open internet, and it should get the certificate from public CA like let's encrypt on its public domain name. Public CA always sign on public domain which is open internet, and it is the role of public CA. The protocol to automate this process is ACME. Before ACME protocol, getting a certificate was manual. The person in charge should send CSR to CA, and CA checks the CSR and return the certificate ... All process was manual. ACME protocol made this process automatically. Simply, ACME protocol checks services authority on domain with various way, and issues certificate.
- http-01 challenge
The server creates certificate file for CA. CA accesses the service server's specific directory via 80/TCP. When CA checked the file, CA issues certificate.
- DNS-01 challenge
The server sends request to authoritative DNS server to add specific record with TSIG key(API key). When the specific record successfully added on DNS server and CA can check this, CA issues certificate. This challenge can verify full authority of domain including sub domain, so CA can issue wildcard certificate.
- TLS_ALPN-01 challenge
It uses TLS handshake process on 443/TCP. ALPN(Application Layer Protocol Negotiation) is the protocol to decide what protocol server and client will use. In this challenge, CA sends some token to service server. Service server creates temporary TLS certificates using `acme-tls/1` protocol with token from CA. When CA access to server, it asks `acme-tls/1` protocol to server. The server presents the temporary certificate, then CA issues certificate.
##### Based on identity
ACME is powerful protocol to automate issuing certificate. However, it is necessary to check ownership of domain to use ACME challenge (Usually, using http or https). However, It is possible to use TLS protocol without http or https. Originally, the process was also manual way called PKIX(Public-Key Infrastructure). It is so complex and slow, and it is impossible to use ACME. To automate this process, modern CA uses JWK(JSON Web Key) and JWT(JSON Web Token). The process is below.
- The administrator registers system's public key as JWK at CA.
> JWK is a format of specification of key. It shows key and its information as JSON format.
- The system sends the JWT to get certificate signed by its private key to CA.
> JWT is a specification of what client can do after connection or of proof this request is valid. In this process, JWT substitute CSR.
- CA verify JWT by pre-registered JWK. When JWT is valid, it issues X.509 certificate.
##### Usage of X.509 Certificates
Regardless the way to issue certificate, either ACME or the way based on its identity, the certificate which is already issued is always valid before it is revoked. You can use X.509 certificate from the way based on its identity for https, oppositely you can use X.509 certificate from ACME for other TLS protocols (like LDAPS, DB TLS communication, etc).
#### X.500 and LDAP(Lightweight Directory Access Protocol)
In the beginning of PKI, there was plan to make a server for all certificates. It is X.500. However, the protocol to implement X.500, DAP(Directory Access Protocol) was too complex and heavy to use internet environment. To make way easy and light, LDAP(Lightweight Directory Access Protocol) was developed to store certificates. However, there are a lot more efficient way to manage certificates appeared already. LDAP could not realize X.500, but it is utilized as centralized authentication system like [SSO](./02_05_sso.md), or OS account management.
docs/archives/2025-12/02_theory/02_04_tls.md
+92
View File
@@ -0,0 +1,92 @@
Tags: #theory, #network, #security, #protocol
## TLS/SSL
TLS(Transport Layer Security) is the protocol that encrypts communication with certificates under [PKI](./02_03_pki.md). Originally, the communication encryption protocol was suggested as SSL(Secure Socket Layer) in 1995 by Netscape. Before the emergence of TLS/SSL protocol, all communication in web(or database, some more protocols) used plain text, even it contained sensitive data like password. After 1996, the update of SSL protocol stopped at version 3.0, and it has its own vulnerability. The next version of SSL is TLS. Virtually, a lot of people treat SSL as the same as TLS.
### Encryption
Before talking about TLS, it is essential to understand what is encryption. Basically, encryption is to hide plain data using algorithm and key mathematically. Encryption can categorized by various criteria, but in this case focusing on key.
#### Symmetric key encryption
This way needs only one key to encrypt and decrypt. Just because using one key to encrypt and decrypt, it doesn't require complex calculation. This algorithm is fast, and easy to handle huge data. However, the key is only one so for security the key should be protected.
#### Asymmetric key encryption
This way needs two keys to encrypt and decrypt. It means the key for encryption and the key for decryption are different. This uses very complex algorithm to seperate keys. It makes this slow and hard to handle huge data. However, the keys are devided so, one key can be publically shared.
### Principle of TLS
The PKI is necessary to use TLS communication. TLS uses both of algorithm to encrypt the communication. When the communication were encrypted, the key should be existed. For this process(to generate key for encryption), server needs X.509 certificate. The certificate contains data of server's public key, domain, and extra informations. To start communication, the server and the client conduct specific process, called `TLS-handshake`. A client sends the request to start communication. The server accepts this request, and negotiates the protocol what they will use in communication. After negotiation, they start to process to generate symmetric key for communication. In this part, there are two protocol to generate symmetric key: RSA and Diffie-hellman.
#### RSA (Legacy way)
In this process server and client uses server's public key and private key. A client generates metadata (Pre-master secret) to generate symmetric key. Metadata is encrypted with server's public key and sent to the server. The server decrypts the data from a client with its private key. They uses this data to generate symmetric key. When this process ends, they share the same symmetric key to encrypt content of communication.
- Cons of RSA
Security strength of RSA depends on the server's private key. The server's private key was taken from whom have the record of past communication, all communication can be decrypted. Because session key itself (precisely, pre-master secret) is in the communication encrypted by server's public key. So, this way can't guarantee Forward Secrecy which means security of previous communication. Additionally, RSA needs high resource to encrypt and decrypt. Therefore, modern internet environment RSA way is not used frequently.
#### Diffie-Hellman & ECDHE(Modern standard)
On the other hand, TLS can use Diffie-Hellman algorithm. This process doesn't exchange any clue of session key. In this process, the server and clients generate secret values. And they make specific results from calculation with public parameters. The server's private key signs on its results just for prooving they are not altered and authenticated. When they exchange each other's results, both of them generate the same session key from results and their own secret value. All process is publically open, but except the server and client themselves, nobody can calculate the secret values and session key mathematically. After all process is done, finally the client combines all communication value between server. And it makes hash value of this, and encrypt the hash value with the session key. The client sends it to the server as finished message, and server verify this. This key is not permanent, it is temporary. There is no encryption way, so even if hacker could get the secret key of server, he wouldn't know what was the session key.
- Pros of DHE & ECDHE
On the contrary with RSA case, security strength of DHE & ECDHE doesn't depend on some specific key. Because when it generate session key, the server and client don't send any sensitive values (like pre-master secret) in this protocol. What they send to each other is open values. The server's public key is used for only signing. So, every session key can never be existed after each session; PFS (Perfect Forward Secrecy). Additionally, Elliptic curve way can provide the same strength with shorter key comparing RSA.
The detail of `TLS-handshake` process is below.
### Detail of TLS handshake
#### Start
- Client hello
The client sends the request to server with information including SNI (Server Name Indication), TLS version, cipher suite, and client random. SNI is the server's domain information to access. cipher suite is the protocol of encryption, and client random will be used to generate session key. Basically, the content of `client hello` isn't encrypted. It has a problem that ISP(Internet Service Provider) or government can conduct sniffing attack to the Client hello packet. To solve this problem, ECH (Encrypted Client Hello) was developed. However, ECH needs the support of DNS server, browser(host), and server itself. Today, a lot of servers and DNS servers don't support ECH, so it is hard to apply ECH in every environment.
- Server hello
When the server receives clients request, it designate how to communicate from the list of client hello. The server sends to the client with the information of what protocol, TLS version, cipher suite to use, and certificate, and server random. The way of encryption: RSA, DHE, ECDHE are including in the cipher suite.
#### RSA
When the server and client decide to use RSA way, server sends certificate including public key to the client.
- Client key exchange
The client verifies `Server hello` with server's certificate using client's trusted CA list. The certificate is valid then the client generates `Pre-master secret` and encrypts this value with server's public key to send to the server.
- Session key generation
The server recieves `pre-master secret` encrypted by its public key that only server (who knows the private key of server) can decrypt. When the server decrypts this value successfully, the server and client knows three values: `Client random`, `Server random`, and `Pre-master secret`. The server and client generate `Session key` from these values individually.
#### DHE or ECDHE
DHE and ECDHE follows the exactly same principle of Diffie-hellman algorithm. However, they have a difference of basement in mathematic. DHE is using discrete logarithm, but ECDHE is using elliptic curve discrete logrithm. This is a mathametical topic, so this is skipped. Just important thing is ECDHE is more efficient than RSA or DHE.
> Key length to guarantee the same level of security:
>
> - RSA/DHE: above 2048 bit
> - ECDHE: above 256 bit
- Server key exchange
The server generate the pair of temporary key based on its own principal. The server sends public temporary key to the client. The public temporary key is signed by server's private key.
- Client key exchange
The client verify server's temporary public key with server's public key. Simultaneously, client also generate the pair of temporary key based on its own principal. The client sends its public temporary key to the server.
- Session key generation
Both of them have their pair of temporary key, and other's temporary public key. They generate the same `Pre-master secret` from their temporary secret key and other's temporary public key. This process has no communication, they calculate the same `Pre-master secret` from themselves. When they have `Pre-master secret`, they generate session key from pre-master secret and client random and server random.
#### Finish
After all process is done, the server and client both sends the `Finished message` encrypted by session key. When they can decrypt this messages, the session key is generated properly. They can start communication securely with session key.
### TLS 1.3
Current standard TLS 1.3 changed handshake process as 1-RTT(1 Round Trip Time) and there are no longer available to use RSA way (Legacy).
docs/archives/2025-12/02_theory/02_05_sso.md
+131
View File
@@ -0,0 +1,131 @@
Tags: #theory, #network, #security, #protocol
## SSO (Single Sign On)
When someone wants to use some services, usually they have to identify themselves to the services. The service verifies who you are (authentication), and what you can do (authorization). Originally, each service has their own authentication and authorization system. However, in modern internet environment, a lot of services are organically connected to each other. It is common for one provider to operate various services. So, it is very inefficient and complex to operate authentication and authorization system like this way. To solve this problem, the concept of SSO which is to centralize all services' authentication and authorization system at once emerged.
### LDAP (Lightweight Directory Access Protocol)
To centralize authentication and authorization system, it is necessary to kind of central database of user list naturally. Fortunately, there was a very good and suitable protocol already; LDAP. LDAP was originally suggested to replace DAP (Directory Access Protocol) for X.500 in [PKI](./02_03_pki.md). Even though X.500 didn't materialized, the potentiality of LDAP was selected to materialize SSO. Because LDAP's structure which was designed for manage certificates itself allows to manage user and its authorization. In modern internet environment, many services don't usually use LDAP itself directly in SSO anymore, but it is still used as database of user and their authorization.
#### Structrue of LDAP
LLDAP will be used in this homelab. It is the easiest way to understand LDAP is comparing each part of LDAP with filesystem. Because LDAP itself is fundamentally the database which has the tree structure like filesystem.
##### DN (Distinguished Name)
DN is the unique path of some specific entry in tree. It is like an absolute path in filesystem.
- Example:
- `uid=admin,ou=people,dc=ilnmors,dc=internal`
- `uid=authelia,ou=people,dc=ilnmors,dc=internal`
##### Base DN (Base Distinguished Name)
Base DN is the root of the tree of LDAP. it is like root path of filesystem `/`. All the actions such as search or work are started from Base DN.
- LLDAP setting
- Environment.="LLDAP_LDAP_BASE_DN=dc=ilnmors,dc=internal"
- It means, the domain 'ilnmors.internal' will be the Base DN (the root of tree).
##### Components of DN
DN has components what has ordinary order. The order of these components are `cn(or uid)`, `ou`, `dc`. They are special attributes, which makes DN.
- cn (Common name): The name of object. It is like a file itself in filesystem.
- ou (Organizational Unit): The name of the container which contain entries. It is like a folder in filesystem.
- dc (domain component): The domain components, Usually, it devides the full domain `ilnmors.internal` as `dc=ilnmors`, `dc=internal`.
##### Object (or Entry)
Object is the real data item the DN defines, like file or folder itself in filesystem.
##### ObjectClass
This is the template or blueprint of object. ObjectClass defines what is this object; Either a person or a group, or an organization unit.
- What is this object; user (cn or uid) or group (ou).
- What is the attribute object must have, and might have.
- Example:
- `ObjectClass: person`: This must have `sn` and `cn` attributes.
- `ObjectClass: organizationalPerson`: This is inherited the `ObjectClass: person`, and it can have `mail` or `number` attributes.
- LLDAP uses standard ObjectClass: `person`, `groupOfNames`.
##### Attribute
Attribute is the value of object, like the content of file or folder in filesystem. It is saved as the pairs of key-value. This values are following the definition of ObjectClass.
- Example
- `uid=user1` has the value following `objectClass: person`, `objectClass: organizationalPerson`
- It must have `sn`, and `cn`
- It can have `mail`, or `number`
- It can have special attribute `memberOf: cn=admins,ou=admin_group, ...` (The attribute that shows the group the user belongs to)
### IdP (Identity Provider)
Many of modern services, SP (Service provider), use SAML (Security Assertion Markup Language) or OIDC (OAuth 2.0/OpenID Connect) protocol to implement SSO. Originally, each service had to send request of authentication to central server (which can be LDAP or else) individually to implement SSO. It means each service should protect the sensitive data itself, and some services which have vulnerability can threaten all system. This is why many of modern services use and support SAML or OIDC protocols. The IdP is needed to use these protocols. The IdP acts as the agent of all services to substitute authentication process on their behalf. Only IdP can access the real database, and all services trust IdP's authentication in this model.
#### SAML (Security Assertion Markup Language)
SAML protocol was developed in 2001 for SSO. This protocol works on XML (eXtensible Markup Language) format which can meet complex security requirements in enterprise environments. It has been used for a long time, it makes this protocol very stable. However, XML itself is a complex and heavy format. This fact makes the protocol complex and heavy to use in common and small environment. There is the process of SAML below.
- Start
User sends the request to the service. When SP recieves the request, it redirect the request to IdP.
- Authentication
IdP asks the login information to the user and it authenticate user from database (like LDAP). When authentication process succeed, IdP generates SAML assertion including user's identity based on XML with sign to ensure integrity. IdP sends the this assertion to SP via user's browser.
- Finish
SP receives the assertion from IdP via user's browser, it verify the assertion. When it is valid, they allow login.
#### OIDC (OpenID Connect) and OAuth 2.0
OIDC was developed in 2014, it is newer than SAML. This is an authentication layer on OAuth 2.0 protocol. Basically, OAuth 2.0 is for authorization, and OIDC is for authentication. OAuth 2.0 protocol works on JSON/REST format, especially JWT (JSON Web Token) which is lighter and simpler than XML. OIDC is latest standard of SSO, it supports social login, and friendly to API. These features makes this protocol use on small and personal environment easily. There is the process of OIDC below.
- Start
User sends the requset to the service (in OIDC, RP; Relying Party). When RP receives the request, it redirects to IdP (in OIDC, OP; OpenID Provider).
- Authentication
IdP (OP) asks login information, and simultaneously asks permission to provide the information to RP. After getting information and permission, IdP generates two tokens which are for different purposes. One is an ID token (JWT) which contains user identity for authentication, the other is an access token for authorization. IdP sends these token to RP. The differences between SAML are what data format does protocol use, and whether IdP gets permission or not.
- Finish
RP recieves the tokens from IdP, and it verify the tokens. When it is valid, they allow login.
#### Reverse proxy
When application doesn't support SSO, then you can use reverse proxy as the door of SSO using `Forward Authentication`. Usually, every web packet pass through reverse proxy in modern internet environment. Therefore, reverse proxy (i.e. Caddy) can intercepts the packets and force them to SSO from IdP or OP (or LDAP itself) before they reach to the application.
- Header based
When Authelia success to authenticate someone, Caddy sends specific header which contains user information like `X-Forwarded-User: A`. Application gets this header, it automatically allows login to A. However, it is needed the ID in the application manually.
> If hacker can access to application without reverse proxy, it can make X header to fake application. In this homelab, all access towards application will be limited by iptables from reverse proxy or sidecar reverse proxy.
- LDAP based
LLDAP will be a light and simple LDAP server. Authelia supports OIDC OP(idP) based on external LDAP server. LLDAP server will becom the external LDAP server for Authelia
---
## Example of Authelia flow
### Flow
- Define the user and group in LLDAP server
#### OIDC supported app
- At the service, choose login way as Authelia (OP)
- Login at the Authelia.
- Authelia access to LLDAP server to authenticate the user based on LDAP.
- Login succeed, Authelia generate token and Service turst OP's token, and allow login (authentication and authorization)
#### Non-OIDC supported app
- Foward_Auth function with Authelia.
docs/archives/2025-12/02_theory/02_06_email.md
+242
View File
@@ -0,0 +1,242 @@
Tags: #theory, #network, #protocol
## Email service
Email is the mail service online via the internet. ARPANET was developed in 1969, since then there has been many attempts to send messages via the internet. The mail which uses `@` character in 1971 and `SMTP(Simple Mail transfer Protocol)` was developed to standardize various ways to email.
### Component of Email service
#### Address
Basically, Email address has format like this. `local-parts@domain`. `local-parts` is identifier, and `domain` is service provider's domain. Following RFC 5321, `domain` doesn't distinguish it upper or lower case. `local-parts` must distinguish them, but practically they doesn't.
#### MUA (Mail User Agent)
MUA is the client of Email. The user can write Email, or read the Email which they got recieved. For instance, Outlook, Thunderbird, etc.
#### MTA (Mail Transfer Agent)
This is the essential part of Email service. MTA transpers the mail to other MTA or MDA. For instance, Postfix, sendmail, Exim, etc.
#### MDA (Mail Delivery Agent)
MDA recieves the mail from MTA, and it store the mail on receivers' mailbox. Sometimes, it is combined MTA or IMAP/POP3 servers. For instance, Dovecot LDA, Procmail, etc.
#### Flow of Email service
- User writes the mail on MUA.
- User sends the mail from MUA to MTA using SMTP submission protocol.
- MTA checks receiver's domain, and transfer the mail to other MTA which takes charge of that domain.
- MTA recieves the mail and sends receiver's MDA.
- The receiver's MUA access to the MDA such as IMAP or POP3 server. the receiver can check and read the email on their MUA.
## Protocols
### SMTP (Simple Mail Transfer Protocol)
SMTP is standard of email transfer protocol internet defined on RFC 5321. This protocol is used when MUA sends the mail to MTA, and MTA sends the mail other MTAs. This protocol takes charge of all process of transportation of the mails.
#### Detail of SMTP
##### Start
- Connection
The client and server make the connection via SMTP port (25/tcp).
##### Greeting
- `220` code
The server sends `220` code to the client, they are ready.
- `HELO` or `EHLO`
The client sends `HELO` or expand version of `HELO`; `EHLO` command to server to introduce itself.
##### Designate sender and recipient
Use the command below, they designate sender and recient.
- `MAIL FROM:<sender@domain>`
- `RCPT TO:<recipient@domain>`
- If there were various recipients, use this command as much as recipients number.
##### Transper the mail data
- `DATA` and `354`
The client sends `DATA` command to server. After the server responds with `354` code, client sends the data including mail header (From, To, Subject), and content of mail. The end of data is `.`.
##### End
- `QUIT`
The client sends `QUIT` command, the connection is terminated.
##### Ports
- `25/tcp`
Traditional SMTP's standard port. All content using `25/tcp` is not encrypted. Because of security and SPAM problems, a lot of ISP block the `25/tcp` port of common user.
- `587/tcp` (Submission)
The standard port of SMTP for encryption. Generally MUA sends the mail to MTA with this port. It is needed to use encrypted connection via `STARTTLS`
- `465/tcp` (SMTPS)
This port used to be used for TLS/SSL for SMTP. This is not standard, so it is recommended to use `587/tcp` port for TLS/SSL of SMTP. However, even now this is generally and commonly used.
##### Security
SMTP is very old protocol, and this protocol use plain data. It is recommended to use `STARTTLS` or `SMTPS` to encrypt data for security.
- `SMTPS`
It uses TLS/SSL from the beginning of connection via `465/tcp`
- `STARTTLS`
It uses TLS/SSL after beginning of connection via `587/tcp` as plain data, and start encryption with `STARTTLS` command.
##### Authentication
It is necessary to use users' identity like name and password to prevent anyone can sends malicious mail using server. SMTP uses SASL (Simple Authentication and Secuirty Layer) machanism to authenticate its users.
##### Relay
MTA has to send the mail to the other MTA for guarantee the mail can arrive the recipitent. MTA uses `relay` function for this. Make sure to allow this function for authenticated user or trusted network to prevent malicious usage.
### IMAP (Internet Message Access Protocol)
IMAP is the protocol to read and manage the mails from remote MDA (mail server). The difference between POP3 is that IMAP can manage the mail and its mailbox remotely even without download. It is defined on RFC 3501.
#### Detail of IMAP
IMAP is the protocol to have a communication with various commands while the connection is stable. The client sends specific `tag` in front of command, and the server responds with `tag` to process the actions.
##### Authentication
- `LOGIN` or `AUTHETICATE`
IMAP authenticate the user with `LOGIN` command with ID and password or `AUTHENTICATE` command with SASL.
##### Mailbox
- `LIST`
- `SELECT`
- `CREATE`
- `DELETE`
- `RENAME`
##### Mail
- `FETCH`
IMAP can take the mail list, the mail itself, or content of the mail, even the attachment in the mail.
##### Statement
- `STORE`
- `\Seen`
- `\Flagged`
- `\Answered`
- `\Deleted`
IMAP can set the status flag of mail with command flags.
##### Search
- `SEARCH`
IMAP can search the mail with various condition of the mail (Sender, title, contents, date, etc) from server.
##### Ports
IMAP strongly recommend to use TLS/SSL with `STARTTLS`. Even though the beginning of conversation is not encrypted, TLS/SSL is applied with the `STARTTLS` command.
- `143/tcp`
The basic IMAP port. It is mendetory to use `STARTTLS` to use IMAP with this port.
- `993/tcp` (IMAPS)
This port uses TLS/SSL in the beginning of communication. It is not a standard but it is generally and commonly use for security.
##### Synchronization
IMAP basically server's mail and mail list, so wherever you access the mail you can see the same condition and status of mailbox. When one mail is modified on one device it is applied all devices simultanaeously.
- `IDLE`
This command supports to maintain connection between server and client, when the new mail comes or the status is changed the client can get notification immediately.
### POP3 (Post Office Protocol version 3)
POP3 protocol is basically designed to download the mail on local client from remote mail server. It is defined on RFC 1939. The biggest difference between IMAP and POP3 is, POP3 basically delete the mail at the server after downloading.
#### Detail of POP3
##### Authorization
- `USER` and `PASS`
The client connect to server and it conduct authentication with `USER` and `PASS` command.
##### Transcation
- `STAT`
- `LIST`
- `RETR <msg_number>`
- `DELE <msg_number>`
- `RETR`
POP3 uses various commands to download or delete the mail. It checks the number of mail and size with `STAT`, downloads the mail with `RETR`, deletes the mail with `DELE`, and save the mail on client with `RETR`.
##### Update
When the client sends `QUIT` command, then server deletes the mails which have `DELE` marks from server and terminate the connection.
##### Ports
POP3 strongly recommend to use TLS/SSL with `STARTTLS`. Even though the beginning of conversation is not encrypted, TLS/SSL is applied with the `STARTTLS` command.
- `110/tcp`
The basic port of POP3. It is mendetory to use `STARTTLS` to use IMAP with this port.
- `995/tcp` (POP3S)
This port uses TLS/SSL in the beginning of communication. It is not a standard but it is generally and commonly use for security.
##### Simplity and locality
POP3 basically delete the mail from mail server, the mail is only on the local client. However, it doesn't require the complex features like IMAP, it can have simplity.
---
## local mail service in homelab
### SMTP server (MTA)
#### Postfix
Postfix will be used as MTA which takes charge of `@ilnmors.internal` domain. However, Postfix in this homelab will never open towards WAN environment. It works as local private MTA. The internal services (Gitea, OPNsense, Prometheus, etc) will sends the mail via `587/tcp` to Postfix. When it needs to send mail towards WAN, it will use `relayhost` function and external Email services such as Google or Naver, etc. `relayhost` makes postfix as one of a `client` not a `MTA`. It means, administrator never takes care about IP reputation or SPAM problems. WAN area's `MTA` function is delegated to public mail service providers.
### IMAP/POP3 server (MDA)
#### Dovecot
Dovecot will be used as IMAP server of local private MTA; Postfix. The user can use MUA (Thunderbird, Outlook, or mail application, even Roundcube webmail) to access the private mail `
`@ilnmors.internal` via Dovecot. The user will ues `993/tcp` to access Dovecot, and Postfix store the mails on Dovecot.
#### mbsync
mbsync will be used as IMAP client of public MTA; Google or Naver. This will fetch public mail `@external-domain.com` to local Postfix from public mail service provider, and eventually the user can access the mail on Dovecot. However, it is important not to delete the mails from public mail servers with proper configuration.
### MUA
#### SnappyMail web mail
This will be used as MUA server on `app` server to access all mails at the same space.