andersen/pxe_boot_alma
1
0 Fork 0
mirror of https://git.valherun.com/valherun/pxe_boot_alma.git synced 2026-10-15 19:42:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

refactor: move tree under homelab/, add alma10 hardening and kickstart docs

Browse Source 
- restructure boot/kickstarts/scripts/docs into homelab/
- docs: alma9->10 breaking changes, CIS hardening, kickstart changes
- add find_cron_entries.sh helper and codex running memory notes

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
andersenandClaude Fable 5 committed 2026-07-06 22:35:07 -04:00
1 parent c44e450fb4
commit 6d398b826a
1703 files changed
+713908 -90
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.gitignore
+24 -14
View File
@@ -1,23 +1,33 @@
# Local artifacts
# Editor / OS noise
*.log
*.tmp
*.swp
*.swo
# PXE/TFTP assets
tftpboot/
httpboot/
images/
isos/
/*.iso
/*.ISO
AlmaLinux-10.1-x86_64-minimal.iso
.DS_Store
.idea/
.vscode/
# Python
.venv/
__pycache__/
# Editor
.DS_Store
.idea/
.vscode/
# ── ISOs ─────────────────────────────────────────────────────────────────────
# Re-download with: curl -L https://repo.almalinux.org/...
*.iso
*.ISO
# ── Extracted install trees ───────────────────────────────────────────────────
# Populated by homelab/scripts/sync_alma_tree.sh from the ISO.
# Covers all AlmaLinux and CentOS versions (8.x, 9.x, 10.x, etc.)
work/KickStart/AlmaLinux-*/
work/KickStart/CentOS-*/
# ── PXE boot kernels and initrds ─────────────────────────────────────────────
# Re-extract from ISO: images/pxeboot/vmlinuz + initrd.img
# Rename convention: vmlinuz-AlmaLinux-10.1-x86_64-pxe, initrd-AlmaLinux-8.8-*-pxe.img, etc.
work/KickStart/tftpboot/vmlinuz-*
work/KickStart/tftpboot/initrd-*
# Homelab equivalents (same naming, different root)
homelab/tftpboot/vmlinuz-*
homelab/tftpboot/initrd-*
codex-running-memory.md
+82
View File
@@ -0,0 +1,82 @@
# Codex Running Memory
## Goal
Get the `probe` kickstart to a clean end state:
- boots from installed disk
- reachable by SSH
- expected users exist
- expected repos/packages and `cpanm` modules are installed
- expected services/timers/scripts are configured
- final proof run should preserve encrypted `/opt/iglass` if feasible
## Process Rules
- Read this file after any compaction before resuming work.
- Update this file after meaningful harness or repo changes.
- Use staged testing and isolate the first newly added failing script.
- Prefer local LLM first for narrow triage, then Claude, then Codex when needed.
## Current Test Model
- Baseline scripts:
- `P200A-diskpart`
- `S110A-baseline-users-ssh`
- Fixed probe MAC:
- `52:54:00:aa:bb:01`
- Snapshot tests should only promote a template after:
- stage apply rc=0
- `Stage apply finished` logged
- expected new script present in `stage-applied-scripts.txt`
- reboot happened on a new `boot_id`
- SSH returned on that new boot
## Important Harness State
- Snapshot payloads now support:
- `cumulative` when rebuilding an older parent from baseline
- `delta` when parent template is the immediately previous verified stage
- `probe_snapshot_stage.sh` now:
- uses a 5-minute boot window
- verifies reboot via changed `boot_id`
- `restart_probe_watcher.sh` now retries stale snapshot apply and boot triage instead of idling forever
## Verified Progress
- Baseline verified good manually
- Stage `1` verified good: `S000A-start`
- Stage `2` verified good: `S104P-confluent`
## Key Repo Fixes Already Made
- `S105A-extra-pkgs`
- Alma 10 `@Xfce` guarded
- VMware SDK `cd`/`Makefile.PL` bug fixed
- `S200A-systemctl`
- missing unit files no longer abort stage apply
- `S350A-chrony`
- `timedatectl`, `chronyc`, and `hwclock` failures are best-effort in snapshot replay
- `build_probe_stage_payload.sh`
- reconstructs `DISK` for snapshot replay
- `S400P-noc-iglass`
- `/home` handling no longer hard-fails on non-empty directory
- AWS placeholder symlink lines corrected
## Reset Point
- Transient harness logs, attempt artifacts, state files, and flags are being cleared for the next run.
- Next run should be treated as a fresh start from scratch.
## Watcher State
- Last updated: 2026-03-29 20:40:36
- Current mode: snapshot-stage
- Current stage: 7
- Total staged scripts: 30
- Baseline template: /var/lib/libvirt/images/probe-stage/probe-baseline-template.qcow2
- Latest verified stage: none
- Current suspect: S400P-noc-iglass
- Current phase: triage-requested
- Current detail: attempt=1 stage=7 suspect=S400P-noc-iglass outcome=snapshot stage failed rc=1
find_cron_entries.sh
+6
View File
@@ -0,0 +1,6 @@
#!/bin/bash
# Script to find cron jobs using grep across common locations
# Safe search for cron jobs
find /etc -type f -name "cron*" -o -name "*crontab" -o -name "*cronjob" | \
xargs grep -l 'monitor_probe_harness\|claude_harness_supervisor' 2>/dev/null || echo "No cron entries found"
README.md → homelab/README.md
View File
File renamed without changes.
boot/grub2/grub.cfg → homelab/boot/grub2/grub.cfg
View File
File renamed without changes.
boot/pxelinux.cfg/default → homelab/boot/pxelinux.cfg/default
View File
File renamed without changes.
dnsmasq/dnsmasq.conf.example → homelab/dnsmasq/dnsmasq.conf.example
View File
File renamed without changes.
homelab/docs/alma10-cis-hardening.md
+302
View File
@@ -0,0 +1,302 @@
# AlmaLinux 10 — CIS Level 1 Hardening Decisions
**Created:** 2026-03-27 **Updated:** 2026-03-28
**Reference:** CIS AlmaLinux OS 10 Benchmark (tracks RHEL 10)
**Script:** `work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S940A-cis-hardening`
---
## Summary of Decisions
| CIS Control | Description | Decision |
|---|---|---|
| 1.1.1 | Disable unused filesystem modules | **Applied** |
| 1.1.2 | /tmp as separate partition or tmpfs | **Deferred** — pending dev review |
| 1.6 | Disable core dumps | **Applied** |
| 3.3.1 | Disable IP forwarding | **Applied** (probes/servers are not routers) |
| 3.3.2 | Disable packet redirect sending | **Applied** |
| 3.3.3 | Refuse source-routed packets | **Applied** |
| 3.3.4 | Refuse ICMP redirects | **Applied** |
| 3.3.5 | Refuse secure ICMP redirects | **Applied** |
| 3.3.6 | Log martian packets | **Applied** |
| 3.3.7 | Ignore broadcast ICMP echo requests | **Applied** |
| 3.3.8 | Ignore bogus ICMP error responses | **Applied** |
| 3.3.9 | Enable TCP SYN cookies | **Applied** |
| 3.3.10 | Disable IPv6 router advertisements | **Applied** |
| 4.1.2 | auditd enabled at boot | **Applied** |
| 4.1.3 | Audit processes prior to auditd start | **Applied** (`audit=1 audit_backlog_limit=8192` kernel args) |
| 4.1.4 | Date/time change events | **Applied** |
| 4.1.5 | User/group identity changes | **Applied** |
| 4.1.6 | Network environment changes | **Applied** |
| 4.1.7 | MAC policy changes (SELinux) | **Applied** |
| 4.1.8 | Login/logout events | **Applied** |
| 4.1.9 | Session initiation | **Applied** |
| 4.1.10 | Permission modification events | **Applied** |
| 4.1.11 | Unauthorized file access attempts | **Applied** |
| 4.1.12 | Privileged command usage | **Applied** (generated dynamically via `find`) |
| 4.1.13 | File system mounts | **Applied** |
| 4.1.14 | File deletion by users | **Applied** |
| 4.1.15 | sudoers scope changes | **Applied** |
| 4.1.16 | sudo log monitoring | **Applied** |
| 4.1.17 | Kernel module load/unload | **Applied** |
| 4.1.18 | Audit rules immutable (`-e 2`) | **Applied** |
| 5.3.2 | sudo: require pty | **Applied** |
| 5.3.3 | sudo: log all usage | **Applied** |
| 5.4.1 | Password quality (pwquality) | **Applied** |
| 5.4.2 | Password aging | **Excluded** — no expiration policy in this environment |
| 5.4.3 | Account lockout (pam_faillock) | **Applied** (pre-configured; active after `authselect enable-feature with-faillock` post-IPA) |
| squashfs module blacklist | CIS recommends disabling | **Excluded** — containers (Docker/Podman) require squashfs |
| udf module blacklist | CIS recommends disabling | **Excluded** — not prohibited in this environment |
| AIDE cron | CIS recommends local AIDE cron | **Excluded** — managed centrally, not locally |
| authselect profile change | Required to activate faillock in PAM | **Excluded from kickstart** — IPA owns the authselect profile |
---
## Applied Controls
### 1.1.1 — Disable Unused Filesystem Modules
**File:** `/etc/modprobe.d/cis-fs-blacklist.conf`
```
install cramfs /bin/true
install freevxfs /bin/true
install hfs /bin/true
install hfsplus /bin/true
install jffs2 /bin/true
blacklist cramfs
blacklist freevxfs
blacklist hfs
blacklist hfsplus
blacklist jffs2
```
Both `install` (prevents modprobe load) and `blacklist` (prevents udev autoload) directives are used for belt-and-suspenders.
**Excluded from this list:**
- `squashfs` — Docker/Podman containers require it. Re-evaluate if container usage changes.
- `udf` — Not prohibited in this environment. Re-evaluate if needed.
### 1.6 — Core Dump Restrictions
Three layers applied:
**`/etc/security/limits.d/cis-coredump.conf`** (PAM limits):
```
* hard core 0
```
**`/etc/sysctl.d/20-cis-coredump.conf`** (kernel — disables setuid core dumps):
```
fs.suid_dumpable = 0
```
**`/etc/systemd/coredump.conf.d/cis.conf`** (systemd coredump handler):
```
[Coredump]
Storage=none
ProcessSizeMax=0
```
### 3.3 — Network Sysctl Hardening
**File:** `/etc/sysctl.d/30-cis-network.conf`
```ini
# 3.3.1 — disable IP forwarding
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0
# 3.3.2 — disable packet redirect sending
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
# 3.3.3 — refuse source-routed packets
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
# 3.3.4 — refuse ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
# 3.3.5 — refuse secure ICMP redirects
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
# 3.3.6 — log martian packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# 3.3.7 — ignore broadcast ICMP echo requests
net.ipv4.icmp_echo_ignore_broadcasts = 1
# 3.3.8 — ignore bogus ICMP error responses
net.ipv4.icmp_ignore_bogus_error_responses = 1
# 3.3.9 — enable TCP SYN cookies
net.ipv4.tcp_syncookies = 1
# 3.3.10 — disable IPv6 router advertisements
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
# Kernel ASLR
kernel.randomize_va_space = 2
```
**Note on `net.ipv4.ip_forward`:** The default is already `0` on RHEL 10 for non-router systems. This is explicitly set for audit compliance. If Docker/Podman networking requires it, Docker daemon config (`"ip-forward": true`) re-enables it automatically — but explicitly setting `0` here means Docker/Podman will need to override it at startup. Verify if container networking is affected.
### 4.x — Auditd Configuration
**Added 2026-03-28.**
#### Kernel boot flag (4.1.3)
```bash
grubby --update-kernel=ALL --args="audit=1 audit_backlog_limit=8192"
```
Ensures processes that start before auditd is running are still audited. `audit_backlog_limit=8192` prevents kernel audit buffer overflow during boot.
#### Rules file: `/etc/audit/rules.d/cis.rules`
Controls 4.1.4–4.1.17 are written as a static heredoc, with 4.1.12 (privileged commands) appended dynamically:
```bash
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null \
| awk '{print "-a always,exit -F path=" $1 " -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged"}' \
>> /etc/audit/rules.d/cis.rules
```
`-e 2` (immutable flag, 4.1.18) is appended last so it takes effect after all rules including the dynamic privileged-command list.
#### Network environment path note (4.1.6)
CIS references `/etc/sysconfig/network-scripts/` which does not exist in RHEL 10. Replaced with `/etc/NetworkManager/` which is the canonical location for NM keyfiles.
#### auditd service
```bash
systemctl enable auditd
```
`audit` package is a dependency of `audit-libs` which is pulled in transitively. No explicit `KS_Packages.cfg` entry required.
---
### 5.3 — sudo Hardening
**File:** `/etc/sudoers.d/cis-sudo` (mode `0440`)
```
# 5.3.2 — require pty (prevents background sudo abuse)
Defaults use_pty
# 5.3.3 — log all sudo usage
Defaults logfile=/var/log/sudo.log
# credential caching limit
Defaults timestamp_timeout=5
```
### 5.4.1 — Password Quality (pwquality)
**File:** `/etc/security/pwquality.conf.d/cis.conf`
```ini
# CIS 5.4.1 — password complexity requirements
minlen = 14
minclass = 4
dcredit = -1
ucredit = -1
ocredit = -1
lcredit = -1
maxrepeat = 3
maxsequence = 3
```
Applies to all interactive password changes. Does not affect pre-hashed passwords set in kickstart.
### 5.4.3 — Account Lockout (pam_faillock)
**File:** `/etc/security/faillock.conf`
```ini
# 5.4.3 — lock after 5 failures in a 15-minute window
deny = 5
fail_interval = 900
unlock_time = 900
even_deny_root
root_unlock_time = 60
```
**Important:** This configuration is pre-staged but **not active** until PAM is wired to use it. `ipa-client-install` owns the authselect profile. After IPA enrolment, enable faillock:
```bash
authselect enable-feature with-faillock
```
Do **not** run this in kickstart — IPA resets the authselect profile during enrolment.
---
## Deferred Controls
### 1.1.2 — /tmp as Separate Partition or tmpfs
**Status:** Deferred pending dev review.
**Reason:** `/var/tmp` may need `exec` permissions for some application stack components. Current kickstart uses `autopart --type=lvm` which places `/tmp` on the root LVM volume. Mounting `/tmp` as `tmpfs` or a separate partition requires validating that:
1. No application code writes large temporary files to `/tmp` that would exhaust a size-limited tmpfs
2. `/var/tmp` exec requirements are understood — CIS also recommends `noexec` on `/var/tmp`, which may conflict with application use
**Action:** Revisit with dev team. If approved, add to kickstart:
```
%pre
# ... partition layout
%end
```
or post-install:
```bash
echo "tmpfs /tmp tmpfs defaults,nodev,nosuid,noexec,size=2G 0 0" >> /etc/fstab
```
---
## Excluded Controls
### 5.4.2 — Password Aging
No password expiration policy for any accounts in this environment. `PASS_MAX_DAYS`, `PASS_MIN_DAYS`, `PASS_WARN_AGE` are left at system defaults. Service accounts and probe accounts do not rotate passwords on a schedule.
### squashfs Module Blacklist
CIS recommends blacklisting `squashfs`. Excluded because Docker and Podman use squashfs for container image layers. If container usage is removed from these hosts, re-evaluate.
### udf Module Blacklist
CIS recommends blacklisting `udf` (Universal Disk Format). Not prohibited in this environment and excluded from scope.
### AIDE Cron
AIDE is installed (`aide` package in `KS_Packages.cfg`) but cron-based integrity scanning is managed centrally by a dedicated machine — not configured locally on each probe/server. Local AIDE cron is not added.
### authselect Profile Change
`ipa-client-install` owns and resets the authselect profile. Any pre-kickstart authselect change would be overwritten during IPA enrolment. `faillock.conf` is pre-staged; `authselect enable-feature with-faillock` is a manual post-enrolment step.
---
## Kernel ASLR Note
`kernel.randomize_va_space = 2` (full ASLR) is set in `30-cis-network.conf`. This is the RHEL 10 default but is explicitly set for audit compliance. No known application compatibility issues with this value.
---
## Future Review Items
- **SELinux verification:** `audit2allow` outputs CIL format by default in RHEL 10 (changed from `.te`). Any local SELinux policy management scripts need to be updated. Verify in test VM.
- **Docker/container network compatibility with `ip_forward=0`:** If containers use bridged networking, Docker daemon re-enables `ip_forward` at startup. Confirm no conflict with the sysctl set above.
homelab/docs/alma10-kickstart-changes.md
+454
View File
@@ -0,0 +1,454 @@
# AlmaLinux 10 Kickstart — Changes & Known Issues
**Created:** 2026-03-27
**Scope:** Production kickstart build — `work/KickStart/cfg/AlmaLinux-10.1-x86_64/`
**Environment:** Monolith at `10.1.1.1` provides dnsmasq (DHCP+TFTP), Squid proxy (`:3128`), and HTTP for kickstart and repo. Probes/servers have no other network access during install.
---
## Architecture
- KickStart delivered via CGI: `http://10.1.1.1/KickStart/cfg/ks.cgi?DIST=AlmaLinux-10.1-x86_64&DISKTYPE=ssd&TYPE=probe|server`
- Install tree: DVD ISO (not minimal/lite) — required for `@Server`, `@Development Tools`, and other groups
- tftpboot root: `/home/kickstart/tftpboot` on monolith
- Post-install scripts: modular `ks.d/` (N/P/S series), executed in sort order
---
## Install Source Change: Minimal → DVD
Switched from `AlmaLinux-10.1-x86_64-minimal.iso` to `AlmaLinux-10.1-x86_64-dvd.iso`.
**Why:** The minimal ISO does not include package groups (`@Server`, `@Development Tools`, `@Security Tools`, etc.) or many individual packages listed in `KS_Packages.cfg`. Using it caused anaconda to abort with missing package errors and also caused GDM/initial-setup failures on the minimal environment.
---
## KS_Packages.cfg — Removed Packages
The following packages were removed from `KS_Packages.cfg` with justification:
| Package | Reason |
|---|---|
| `perl-core` | Meta-package removed in RHEL 10; individual `perl-*` packages are preferred |
| `dracut-fips` | Hard-removed in RHEL 10 — FIPS is boot-time only (`fips=1` kernel param). This project does not use FIPS. |
| `dracut-fips-aesni` | Same as above |
| `libXScrnSaver` | Merged into `libX11` in RHEL 10; no longer a separate package |
| `usbguard` | `usbguard-selinux` scriptlet crashes anaconda's RPM transaction during kickstart. Moved to `S930A-security` (installed via `dnf` in `%post`). |
| `perl-NetAddr-IP` | EPEL-only on RHEL 10; not available from DVD. Moved to `S105A-extra-pkgs` EPELPKGS. |
| `perl-Net-DNS` | EPEL-only on RHEL 10; not available from DVD. Moved to `S105A-extra-pkgs` EPELPKGS. |
### lshw — Confirmed Present in DVD
`lshw` is available in the AlmaLinux 10.1 DVD AppStream repo. An earlier filename-based check produced a false miss because the package version string starts with a letter (`lshw-B.02.20...`), defeating a naive `sed 's/-[0-9].*//'` version strip. Confirmed by test VM install — `lshw` installs cleanly.
---
## S850A-network — dhclient and ifcfg Removed in RHEL 10
**Changes made:**
1. Removed the `dhclient` block entirely — `dhclient` package is removed in RHEL 10. NetworkManager uses its built-in DHCP client (`dhcp=internal`).
2. Removed the `cd /etc/sysconfig/network-scripts` block — `ifcfg` file format is removed in RHEL 10. NetworkManager uses keyfiles at `/etc/NetworkManager/system-connections/`.
3. Replaced ifcfg DNS overrides with a Python `configparser` approach that modifies the NM keyfile directly:
```python
import configparser, os, sys, glob
condir, first = sys.argv[1], sys.argv[2]
candidates = (
glob.glob(os.path.join(condir, first + '.nmconnection')) +
glob.glob(os.path.join(condir, '*.nmconnection'))
)
for path in candidates:
cfg = configparser.RawConfigParser()
cfg.optionxform = str
cfg.read(path)
if not cfg.has_section('ipv4'):
continue
if cfg.get('ipv4', 'method', fallback='') not in ('auto', 'dhcp'):
continue
cfg.set('ipv4', 'dns', '192.168.194.102;192.168.226.102;')
cfg.set('ipv4', 'dns-search', 'iglass.net;')
cfg.set('ipv4', 'dns-priority', '-100')
with open(path, 'w') as fh:
cfg.write(fh)
os.chmod(path, 0o600)
break
```
---
## S200A-systemctl — GDM / initial-setup Guards
Added `|| true` and file existence checks around service disabling and GDM config modification, because the DVD minimal environment may or may not install GDM or initial-setup. Without these guards, the script errored out and halted kickstart.
```bash
for _svc in initial-setup-graphical.service initial-setup-reconfiguration.service \
initial-setup-text.service initial-setup.service; do
systemctl disable "$_svc" 2>/dev/null && echo "Disabled $_svc" || true
done
if [ -f /etc/gdm/custom.conf ]; then
perl -spi -e 's/\[daemon\]/\[daemon\]\nInitialSetupEnable=False/' /etc/gdm/custom.conf
fi
```
---
## S930A-security — usbguard Moved to Post-Install
**Problem:** `usbguard-selinux` contains a scriptlet that crashes the anaconda RPM transaction with:
```
Error in <unknown> scriptlet in rpm package usbguard-selinux
```
This does not happen in a running system — only during anaconda's install phase.
**Fix:** Removed `usbguard` from `KS_Packages.cfg`. `S930A-security` now installs it via `dnf` after boot:
```bash
dnf install -y usbguard 2>&1
if [ -e /bin/usbguard ]; then
/bin/usbguard generate-policy > /etc/usbguard/rules.conf
perl -spi -e 's/ImplicitPolicyTarget=\S+/ImplicitPolicyTarget=reject/' \
/etc/usbguard/usbguard-daemon.conf
systemctl disable usbguard.service
fi
```
---
## S940A-cis-hardening — New Script
Added CIS Level 1 hardening script. See [alma10-cis-hardening.md](alma10-cis-hardening.md) for full details on what was applied, deferred, and excluded.
---
## S105A-extra-pkgs — Known Alma 10 Compatibility Issues
This script has several items that need Alma 10 review before production use:
| Item | Issue | Action Needed |
|---|---|---|
| `dnf config-manager --set-enabled powertools` | `powertools` repo does not exist in RHEL 10 — superseded by `crb`. **Removed** (2026-03-27). CRB is already enabled by the lines that follow. | Done |
| MS repo URL (`rhel/9.0/prod.repo`) | Updated to `rhel/10/prod.repo` (2026-03-27). Package renamed: `mssql-tools` → `mssql-tools18`. `msodbcsql18` name unchanged. Confirmed present at `https://packages.microsoft.com/rhel/10/prod/`. | Done |
| `dnf -y module install nodejs:18/common` | DNF AppStream modules removed in RHEL 10; `nodejs` is a regular package. **Replaced** (2026-03-27) with `dnf -y install nodejs`. RHEL 10 AppStream ships nodejs 22. | Done |
| PowerShell `powershell-7.3.4-1.rh.x86_64.rpm` | RHEL 9 build retained; WS-Man support required. No RHEL 10 build available as of 2026-03-27. TODO comment added. | Verify RHEL 10 RPM when available at https://github.com/PowerShell/PowerShell/releases |
| `@Xfce` in `EPELPKGS` | Alma 10 snapshot stage `3` failed on 2026-03-29 with `Module or Group 'Xfce' is not available.` It is now optional in the Alma 10 script and skipped when unavailable. | Re-test stage `3` |
| `perl-NetAddr-IP`, `perl-Net-DNS` | Added to `EPELPKGS` (2026-03-27) — EPEL-only, removed from `KS_Packages.cfg`. | Done |
---
## Server RPM Dependency Coverage — Pending
Only the `probe` RPM was available for dependency analysis (2026-03-27). A `server` RPM exists but was not available in this repo at time of review. The following packages were moved from `PERLPKGS` (dnf) to `cpanm` because RHEL 10 has no RPM for them — they are retained in cpanm to cover potential server usage until a server RPM can be checked:
- `Crypt::SSLeay` — removed from RHEL 10 (superseded by `Net::SSLeay`); not used by probe but retained for server
- `HTML::Restrict` — no EPEL 10 package; used by probe and possibly server
- `Cache::FastMmap` — no EPEL 10 package; used by probe and possibly server
**Action:** When a server RPM is available, run the same `rpm -ql | xargs grep '^use '` analysis and cross-reference against the cpanm and dnf lists.
---
## EPEL Dependency Chain
The `S105A-extra-pkgs` script enables EPEL via `dnf -y install epel-release` before installing `EPELPKGS`. The following packages are EPEL-only and must remain in that section rather than `KS_Packages.cfg`:
- `perl-NetAddr-IP`
- `perl-Net-DNS`
These were confirmed EPEL-only by test VM install (2026-03-27): present in EPEL 10, not in DVD BaseOS or AppStream.
---
## net-snmp-perl Perl Dependencies
`net-snmp-perl` does **not** directly require `perl-NetAddr-IP` or `perl-Net-DNS`. They are likely pulled in by local monitoring Perl scripts (in the `iglass` application stack), not by RPM-level dependencies of `net-snmp-perl`. They must be installed; remove them only after confirming no monitoring scripts reference them.
---
## authselect / pam_faillock Post-IPA Note
`faillock.conf` is pre-configured by `S940A-cis-hardening`, but `pam_faillock` is not active until authselect enables it. This **must** run after IPA enrolment:
```bash
authselect enable-feature with-faillock
```
Do not run this in kickstart — `ipa-client-install` owns the authselect profile and will reset it.
---
## AIDE
AIDE is installed via `KS_Packages.cfg` but cron-based scanning is managed centrally (not locally). No local AIDE cron is configured in kickstart. A central machine runs AIDE against these hosts.
---
## Symlinks Restored (Windows Transit)
15 `V`/`P` variant scripts that are symlinks to their `A` counterparts were zeroed out during a Linux→Windows→Linux filesystem round-trip. All restored (2026-03-27/28):
| Symlink | Target |
|---|---|
| S105V-extra-pkgs | S105A-extra-pkgs |
| S150V-etc-mod | S150A-etc-mod |
| S200V-systemctl | S200A-systemctl |
| S350V-chrony | S350A-chrony |
| S570V-Postgres | S570A-Postgres |
| S650V-Crontabs | S650A-Crontabs |
| S701P-chroot-maps | S701A-chroot-maps |
| S701V-chroot-maps | S701A-chroot-maps |
| S800V-netsnmp | S800A-netsnmp |
| S856V-unbound | S856A-unbound |
| S870V-sshd | S870A-sshd |
| S875V-firewall | S875A-firewall |
| S960V-selinux | S960A-selinux |
| S980V-monitor | S980A-monitor |
| S990V-mailme | S990A-mailme |
| S104V-confluent | S104P-confluent |
| S400V-noc-iglass | S400P-noc-iglass |
| S700V-chroot-as400 | S700P-chroot-as400 |
| S900P-rmlogrotate | S900S-rmlogrotate |
| S900V-rmlogrotate | S900S-rmlogrotate |
| S910V-chmodHosts | S910P-chmodHosts |
---
## S000A-start — NIC Detection Fix
The original script hardcoded `eth0` as the fallback interface name. On RHEL 10 with predictable interface naming (`eno1`, `em1`, `p1p1`, etc.), this fails silently. Fixed to detect the first interface with a MAC address:
```bash
ST=$(ip -o link show | grep -v '\blo\b' | perl -ne 'if (/link\/ether\s+(\S+)/) { print "$1\n"; last }')
```
---
## S104P-confluent — Simplified: Standard RHEL 10 Repos Sufficient
**Status (2026-03-28):** Confluent publishes `librdkafka` for EL10 at `clients/rpm/centos/10/x86_64/`, but RHEL 10 ships `librdkafka` in AppStream and `librdkafka-devel` in CRB. The Confluent package (`librdkafka1`) conflicts with the AppStream package (`librdkafka`).
**Resolution:** S104P now enables CRB and installs from standard repos — no Confluent clients repo needed:
```bash
dnf config-manager --set-enabled crb
dnf install -y librdkafka librdkafka-devel
```
Verified on alma10-pkgtest (2026-03-28): `librdkafka-2.1.1-7.el10` from AppStream, `librdkafka-devel-2.1.1-7.el10` from CRB, `Net::Kafka` builds successfully.
The `confluent.repo` file (deployed via `etc.tgz`) still provides the `[Confluent]` platform repo (rpm/8.2) for potential future use, but `[Confluent-Clients]` was removed to avoid the package naming conflict.
---
## S105A-extra-pkgs — PERLPKGS and cpanm Changes
### PERLPKGS (dnf-installed Perl modules)
Packages removed (no RHEL 10 / EPEL 10 RPM available — moved to cpanm):
- `perl-Crypt-SSLeay` — superseded by `Net::SSLeay` in RHEL 10
- `perl-HTML-Restrict` — no EPEL 10 package
- `perl-Cache-FastMmap` — no EPEL 10 package
Package added:
- `perl-IPC-Run` — available in RHEL 10 AppStream
### EPELPKGS additions
Added to cover probe dependencies not in BaseOS/AppStream:
- `perl-NetAddr-IP`
- `perl-Net-DNS`
- `perl-Regexp-Common`
- `perl-WWW-Mechanize`
- `perl-Protocol-WebSocket`
- `perl-Crypt-Rijndael`
- `perl-Net-SSH-Perl`
### cpanm list changes
Removed (now covered by EPELPKGS above):
- `IO::Select` (stdlib)
- `HTML::Entities` (part of `HTML::Parser`, available as RPM)
- `Regexp::Common`
- `WWW::Mechanize`
- `Protocol::WebSocket`
- `Crypt::Rijndael`
Added (no RPM available, retained for probe and potential server use):
- `HTML::Restrict`
- `Cache::FastMmap`
- `Net::Address::IP::Local`
- `Net::Ping::External`
- `Net::SIP`
- `Net::SSH`
- `Net::Telnet`
### cpanm Smoke Test Results (2026-03-28, alma10-pkgtest)
22 modules tested via `cpanm -n` on AlmaLinux 10.1. Two failures:
| Module | Result | Action |
|---|---|---|
| `Crypt::SSLeay` | **Hard fail** — uses `SSLv2/SSLv3_client_method()` removed in OpenSSL 3.x | **Removed** from cpanm list. RHEL 10 ships OpenSSL 3.x; this module is incompatible and unmaintained (last release 2016). Use `Net::SSLeay` (available as `perl-Net-SSLeay` RPM) instead. |
| `Net::Kafka` | **Configure fail** — needs `librdkafka-devel` (`rdkafka.pc`) | **Resolved:** `librdkafka-devel` is in CRB, installed by S104P. `Net::Kafka` now builds cleanly. Conditional guard removed. |
All 22 modules install successfully once S104P has run (librdkafka-devel in place).
---
## S150A-etc-mod — systemctl --user Guards
`systemctl --user` commands for masking PulseAudio fail during kickstart post-install because no user session is running. Guarded with `2>/dev/null || true`:
```bash
systemctl --user mask pulseaudio 2>/dev/null || true
systemctl --user disable pulseaudio.socket 2>/dev/null || true
systemctl --user disable pulseaudio.service 2>/dev/null || true
```
---
## S400P-noc-iglass — init.d → systemd Migration
The `.init` copy loop was replaced with systemd unit file installation (2026-03-28). Key findings:
- `iFunctions.init` is a bash function library, not a service init script — it does not need to go in `/etc/init.d/`
- The iGlass RPM already ships 24 systemd unit files in `/opt/iglass/systemd/system/` (services + timers)
- `ctl-iglass.service` is the master controller that starts all configured iglass services via `properties.pl`
New block installs all `.service` and `.timer` files to `/etc/systemd/system/` and runs `daemon-reload`. Services are **not enabled** during kickstart — `ctl-iglass.service` requires probe configuration (customer/hostname) which is set post-install.
---
## S570A-Postgres — pgdg Repo and useradd Fix
Three changes:
1. **pgdg repo URL:** `EL-9-x86_64` → `EL-10-x86_64`
2. **Module disable:** `dnf -y module disable postgresql` commented out with TODO — DNF AppStream modules are removed in RHEL 10, this command would fail
3. **useradd flag:** `-n` (no user private group) changed to `-N` — the flag was renamed in RHEL 10
---
## S700P-chroot-as400 / S701A-chroot-maps — AlmaLinux 10 Version Check
Both scripts checked `^VERSION="9"` in `/etc/os-release`. Updated to accept RHEL 9 or 10:
```bash
if ! grep -qE '^VERSION="(9|10)' /etc/os-release; then
echo "This script is for AlmaLinux 9 or 10"
```
**Note:** `mkjail.pl` and the full chroot setup have not been tested on RHEL 10. Test on first physical build.
---
## S750A-serialLogin — /etc/securetty Guard
`/etc/securetty` was removed in RHEL 9.2+ and does not exist in RHEL 10. Guarded the append:
```bash
if [ -f /etc/securetty ] && ! grep -q "$TTYS" /etc/securetty; then
echo "$TTYS" >> /etc/securetty
fi
```
**Pending:** Verify that `rhgb quiet` is present in `GRUB_CMDLINE_LINUX` on RHEL 10 — the regex that removes it may not match the default RHEL 10 grub config. Check on first physical probe build.
---
## S900S-rmlogrotate — Timer-Based logrotate (RHEL 10)
In RHEL 10, logrotate runs via `logrotate.timer` (systemd timer), not just a cron job. Both are now disabled:
```bash
# Disable cron-based logrotate (RHEL 9 and earlier)
if [ -f /etc/cron.daily/logrotate ]; then
sed -i '2s/^/exit\n/' /etc/cron.daily/logrotate
fi
# Disable timer-based logrotate (RHEL 10+)
systemctl disable --now logrotate.timer 2>/dev/null || true
rm -f /etc/systemd/system/timers.target.wants/logrotate.timer
```
S900P and S900V are symlinks to S900S (not separate scripts).
---
## S920V-vm-cleanup — dhclient Block Removed
The script previously migrated `/etc/dhcp/dhclient-em1.conf` from the installer environment into the installed system. `dhclient` is removed in RHEL 10 — NetworkManager uses its built-in DHCP client. The block was removed:
```bash
# dhclient removed in RHEL 10 — dhclient.conf migration no longer needed
```
The interactive `read -p 'Customer:'` / `read -p 'Hostname/Probe:'` prompts are intentional — this script is run interactively during VM provisioning, not as part of the automated kickstart sequence.
---
## S980A-monitor — SSSD enumerate Removed
SSSD `enumerate = true` is not supported for AD or IdM providers in RHEL 10 (removed from those backends). The option was removed from the sssd.conf block. A comment was left explaining why:
```bash
# NOTE: 'enumerate = true' was removed in RHEL 10 for AD/IdM SSSD providers.
```
---
## S990A-mailme — NIC Detection and SSH Protocol Fix
Two changes:
1. **NIC detection:** Replaced hardcoded `eth0` with a fallback chain covering predictable interface names, then `ip link` as a last resort:
```bash
ETH=
for _candidate in eth0 em1 p1p1 eno1; do
if [ -e "/sys/class/net/$_candidate" ]; then
ETH="$_candidate"
break
fi
done
if [ -z "$ETH" ]; then
ETH=$(ip -o link show | awk -F': ' '!/lo/{print $2; exit}')
fi
```
2. **SSH config:** Removed deprecated `Protocol 2` directive from the inline SSH config block — `Protocol` is not a valid sshd_config option in OpenSSH 7.6+ and causes a parse error.
---
## S995A-firmware — Firmware Update Deferred
A TODO comment was added. Dell BIN format firmware update support on RHEL 10 needs verification before re-enabling. No functional change to the script.
---
## P200A-diskpart — NVMe and VirtIO Disk Support
**Problem:** The disk enumeration loop only scanned `sd*` and `hd*` device names. NVMe (`nvme*`) and VirtIO (`vd*`) disks were silently skipped, causing the script to log `"Error: No Disks Found!!!"` on any system without a SCSI/SATA disk.
**Fix 1 — disk enumeration (line 14):**
```bash
for DSK in $(cd /sys/block; ls -d sd* 2>/dev/null; ls -d hd* 2>/dev/null; ls -d nvme* 2>/dev/null; ls -d vd* 2>/dev/null); do
```
**Fix 2 — NVMe partition naming:** NVMe partitions use a `p` separator (`nvme0n1p1`) while SCSI/VirtIO do not (`sda1`, `vda1`). Added partition prefix variables after disk selection:
```bash
DEVAP=""; [[ "$DEVA" == nvme* ]] && DEVAP="p"
DEVBP=""; [[ "$DEVB" == nvme* ]] && DEVBP="p"
```
All `--onpart=` references updated to `${DEVA}${DEVAP}$PRT` and `${DEVB}${DEVBP}$PRT` (8 substitutions across EFI, MBR, and second-disk blocks).
---
## no.R010P-1passwd — /mnt/sysimage Path (Disabled)
This script is disabled (`no.` prefix). If ever re-enabled, note that Anaconda changed its chroot path from `/mnt/sysimage/` to `/mnt/sysroot/` in RHEL 9+. The script currently references `/mnt/sysimage/dev/shm/` which would not exist on RHEL 10.
homelab/docs/alma9-to-alma10-breaking-changes.md
+219
View File
@@ -0,0 +1,219 @@
# AlmaLinux 9 → AlmaLinux 10 Breaking Changes
**Generated:** 2026-03-24
**Scope:** Hard-removed features, non-overridable crypto changes, dropped packages — relevant to server/security/infrastructure workloads.
**Upstream:** AlmaLinux 10 tracks RHEL 10 (kernel 6.12, released May 2025).
---
## 1. Cryptography — Hard Removals (no workaround)
### SHA-1
| Scenario | RHEL 9 | RHEL 10 |
|---|---|---|
| SHA-1 in TLS | Disabled in DEFAULT; re-enable via `DEFAULT:SHA1` subpolicy | **Hard removed from TLS** — LEGACY policy does NOT re-enable it in TLS. Must override per-app via OpenSSL cipher string with `SECLEVEL=0`. |
| SHA-1 in NSS/GnuTLS (non-TLS) | Policy-gated | **Hard at library level** — NSS and GnuTLS reject SHA-1 regardless of crypto-policy setting |
| SHA-1 in OpenSSL (non-TLS) | Re-enable via `DEFAULT:SHA1` subpolicy | Soft — re-enable via LEGACY policy only |
| `DEFAULT:SHA1` subpolicy | Exists | **Hard removed** — the subpolicy definition is gone entirely |
### DSA
- **NSS**: DSA is **hard-compiled out** of the NSS library. No policy toggle. Affects anything using NSS (curl-NSS builds, certutil, etc.).
- **OpenSSL**: DSA available in LEGACY (soft).
- **GnuTLS**: Deprecated, hard removal expected in a future 10.x minor.
### Other Hard Crypto Removals
| Item | Notes |
|---|---|
| SEED algorithm | Hard-removed from NSS |
| SRP (Secure Remote Password) in TLS | Hard-removed from GnuTLS; was incompatible with TLS 1.3 + relied on CBC+SHA-1 |
| TLS HeartBeat extension | Hard-removed from OpenSSL (attack surface reduction post-Heartbleed) |
| MIT Kerberos RSA PKINIT method | Hard-removed from `krb5` as Marvin attack mitigation; must use ECDH-based PKINIT |
| `openssl-pkcs11` ENGINE bridge | Package removed; replaced by `pkcs11-provider` (new OpenSSL provider API) |
### FIPS Mode Operational Changes
- `/etc/system-fips` file removed as FIPS indicator.
- `fips-mode-setup` command removed. FIPS is **boot-time only** via `fips=1` kernel parameter. No runtime switching.
---
## 2. Cryptography — Soft Changes (re-enable via LEGACY policy or config)
| Item | How to re-enable |
|---|---|
| TLS 1.0 / TLS 1.1 | `update-crypto-policies --set LEGACY` |
| 3DES, RC4, DES | `update-crypto-policies --set LEGACY` |
| RSA key-exchange TLS cipher suites (non-PFS) | `update-crypto-policies --set LEGACY` |
| RSA keys < 2048 bits (≥ 1024 allowed) | `update-crypto-policies --set LEGACY` |
| SHA-1 (OpenSSL, non-TLS) | `update-crypto-policies --set LEGACY` |
---
## 3. Removed Packages
### Networking
| Package | Replacement |
|---|---|
| `dhclient` / `dhcp` client | NetworkManager built-in (`dhcp=internal` in connection profile) |
| `teamd` + `libteam` (network teaming) | Bonding (`nmcli bond`) |
| `NetworkManager-initscripts-updown` | N/A (depends on removed `network-scripts`) |
| `ifcfg` file format | Keyfile format (`/etc/NetworkManager/system-connections/`) |
| `iptables`, `ip6tables`, `arptables`, `ebtables` userspace tools | `nftables` / `nft` |
> **Note:** iptables kernel modules (`xt_*`, `ip_tables`, `ip6_tables`) are **not shipped in the kernel**. Only `nf_tables` is present. Docker/Moby iptables mode will fail.
### Security / Auth
| Package | Replacement |
|---|---|
| `pam_console` | `systemd-logind` (already the default in RHEL 9) |
| `pam_ssh_agent_auth` | No in-distro replacement |
| `sssd-files` (SSSD files provider) | SSSD `proxy` provider for local-user smart card / session recording paths |
| `libsss_simpleifp.so` / `libsss_simpleifp` | Removed (deprecated in RHEL 9) |
| `fips-mode-setup` command | `fips=1` kernel parameter at install |
### SSSD Config Options Removed
- `reconnection_retries`
- `ad_allow_remote_domain_local_groups` (deprecated in RHEL 9.6)
- `enumeration` feature for AD/IdM providers
### Identity Management
- **IdM NIS functionality** removed from Identity Management stack entirely.
### Cockpit
- `cockpit-composer` replaced by `cockpit-image-builder` plugin.
---
## 4. Kernel Changes
### cgroups v1 — Hard Removed
- cgroups v1 is **gone from the RHEL 10 kernel** (6.12). No boot parameter re-enables it.
- `systemd.unified_cgroup_hierarchy=0` has no effect — v1 hierarchy does not exist.
- `/sys/fs/cgroup/memory`, `/sys/fs/cgroup/cpuset`, etc. will not exist.
- **Impact:** Any container runtime, application, or systemd drop-in that explicitly requests cgroup v1 will break. Container runtimes (Docker, containerd, Podman) must be configured for cgroup v2 only.
### iptables Kernel Modules — Hard Removed
- `xt_*`, `ip_tables`, `ip6_tables`, `arptable_filter`, `ebtables` modules are not included in the kernel.
- Docker's default `iptables` networking mode fails at startup. Must configure Docker with `"iptables": false` and manage firewall rules via nftables, or use a Docker version with native nftables support.
### Other Kernel Notes
- Frame pointers enabled by default in kernel and userspace (improves eBPF/perf tracing, ~1% overhead).
- No documented removals from the audit subsystem or seccomp BPF interface.
---
## 5. Authentication / PAM
| Change | Type |
|---|---|
| `pam_console` removed | Hard — use `systemd-logind` |
| `pam_ssh_agent_auth` removed | Hard — no in-distro replacement |
| SSSD files provider removed | Hard — reconfigure to proxy provider |
| `pam_tally`/`pam_tally2` | Already gone in RHEL 8/9; use `pam_faillock` (unchanged) |
| `authconfig` | Already gone in RHEL 8/9; use `authselect` (unchanged) |
---
## 6. Systemd
| Change | Type |
|---|---|
| cgroups v1 | Hard — systemd 256+ refuses to run under cgroup v1; v1 absent from kernel |
| SysV init script compatibility | Still present in 10.0; hard removal flagged as upcoming in systemd 259+ |
---
## 7. SELinux
- Policy updated to version 3.8.
- `audit2allow` outputs CIL format by default (instead of `.te` files). Update any scripts that parse `audit2allow` output.
- No policy module hard-removals documented for 10.0, but boolean/module names may differ after the version bump — verify with `semodule -l` post-migration.
---
## 8. Quick Reference: Hard vs Soft
| Item | Hard | Soft |
|---|---|---|
| SHA-1 in TLS | ✓ | |
| SHA-1 via NSS/GnuTLS (any context) | ✓ | |
| `DEFAULT:SHA1` subpolicy | ✓ | |
| DSA in NSS | ✓ | |
| SEED in NSS | ✓ | |
| SRP in GnuTLS | ✓ | |
| TLS HeartBeat (OpenSSL) | ✓ | |
| MIT Kerberos RSA PKINIT | ✓ | |
| `openssl-pkcs11` ENGINE | ✓ (replaced by `pkcs11-provider`) | |
| `fips-mode-setup` command | ✓ | |
| cgroups v1 | ✓ | |
| iptables kernel modules | ✓ | |
| `dhclient` package | ✓ | |
| Network teaming (`teamd`) | ✓ | |
| `ifcfg` file format | ✓ | |
| SSSD files provider | ✓ | |
| `pam_console` | ✓ | |
| `pam_ssh_agent_auth` | ✓ | |
| IdM NIS | ✓ | |
| TLS 1.0 / 1.1 | | ✓ LEGACY policy |
| 3DES / RC4 / DES | | ✓ LEGACY policy |
| RSA key-exchange TLS suites | | ✓ LEGACY policy |
| SHA-1 via OpenSSL (non-TLS) | | ✓ LEGACY policy |
| RSA keys 1024–2047 bits | | ✓ LEGACY policy |
| DSA via OpenSSL | | ✓ LEGACY policy |
---
## 9. Application-Specific Clarifications
### net-snmp / SNMPv3 SHA-1 Authentication
**Not broken.** SNMPv3 USM uses SHA-1 as an **HMAC** for message authentication, not as a signature algorithm. The RHEL 10 DEFAULT crypto policy (and the `NO-SHA1` module) explicitly restricts SHA-1 only in *signatures and certificate validation* — HMAC-SHA1 remains permitted. From Red Hat's own documentation:
> "The NO-SHA1 module disables the SHA-1 hash function only in signatures and not elsewhere. In particular, the NO-SHA1 module still allows the use of SHA-1 with HMAC. This is because HMAC security properties do not rely on the collision resistance of the corresponding hash function."
Additionally, the system-wide crypto policy framework only governs TLS, IPsec, DNSSEC, and Kerberos — not SNMPv3 USM, which is an application-layer protocol. Net-snmp links against OpenSSL (not NSS), and OpenSSL 3.5 on RHEL 10 does not block HMAC-SHA1 at the API level.
SHA-256 auth is available as `SHA-256` in `snmpd.conf` `createUser` directives and via `snmpwalk -a SHA-256`. Net-snmp 5.9+ (shipped in RHEL 9/10) includes full RFC 7860 support: SHA-224, SHA-256, SHA-384, SHA-512.
**Exception:** Under FIPS mode, SHA-1 HMAC is also disallowed. Outside FIPS, SNMPv3 SHA-1 auth works normally on RHEL 10.
### FreeIPA / Red Hat IdM
Standard FreeIPA deployments are largely unaffected by RHEL 10 changes. Detailed breakdown:
| Change | FreeIPA Impact |
|---|---|
| MIT Kerberos RSA PKINIT removed | **None** — FreeIPA smart card PKINIT uses DH-based key agreement; RSA-envelope mode was not used |
| DSA removed from NSS | **None** — Dogtag CA uses RSA/ECDSA; DSA CA keys were never a supported install config |
| IdM NIS functionality removed | **None** for standard deployments — only affects sites actively serving ypserv-style NIS queries from FreeIPA (deprecated since RHEL 8.3). The `cn=compat` LDAP tree (legacy LDAP clients) is **not** removed. |
| SSSD files provider removed | **None** — FreeIPA clients use the `ipa` SSSD provider, not `files` |
| `pam_ssh_agent_auth` removed | **None** — not part of any FreeIPA authentication flow |
| TLS RSA key exchange removed from DEFAULT | **None** — FreeIPA and Dogtag already use ECDHE/DHE |
| RC4/3DES removed from Kerberos defaults | **Handled automatically** — FreeIPA 4.12.2 removes these enctypes itself on upgrade |
**One genuine operational change (not crypto):** 389-ds is migrating from BerkeleyDB to LMDB in RHEL 10 (`libdb` is removed). `ipa-server-upgrade` handles the database migration automatically, but it should be tested in a lab before upgrading a production IPA server. Expect longer upgrade time proportional to directory size.
---
## Sources
- [RHEL 10.0 Release Notes — Removed features](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/10.0_release_notes/removed-features)
- [RHEL 10.0 Release Notes — Deprecated features](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/10.0_release_notes/deprecated-features)
- [Considerations in adopting RHEL 10 — Security](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/considerations_in_adopting_rhel_10/security)
- [RHEL 10 System-wide cryptographic policies](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/security_hardening/using-system-wide-cryptographic-policies)
- [AlmaLinux 10.0 Release Notes](https://wiki.almalinux.org/release-notes/10.0.html)
- [Considerations in adopting RHEL 10 — Identity Management](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/considerations_in_adopting_rhel_10/identity-management)
- [NO-SHA1 subpolicy — RHEL crypto policies](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/10/html/security_hardening/using-system-wide-cryptographic-policies)
- [RFC 7860 — HMAC-SHA-2 Authentication Protocols in USM for SNMPv3](https://datatracker.ietf.org/doc/html/rfc7630)
- [FreeIPA 4.12.2 Release Notes](https://www.freeipa.org/release-notes/4-12-2.html)
docs/replication.md → homelab/docs/replication.md
View File
File renamed without changes.
docs/virtual-lab.md → homelab/docs/virtual-lab.md
View File
File renamed without changes.
kickstarts/alma10-auto.ks → homelab/kickstarts/alma10-auto.ks
View File
File renamed without changes.
scripts/post-install-bootstrap.sh → homelab/scripts/post-install-bootstrap.sh
View File
File renamed without changes.
homelab/scripts/sync_alma_tree.sh
Executable
+101
View File
@@ -0,0 +1,101 @@
#!/bin/bash
# sync_alma_tree.sh — Mount an AlmaLinux ISO and sync its contents into the
# KickStart HTTP tree, with optional tftpboot kernel/initrd extraction.
#
# Usage:
# sync_alma_tree.sh --iso FILE --http-root DIR [options]
#
# Options:
# --iso FILE Path to the AlmaLinux (or CentOS) ISO
# --http-root DIR Destination HTTP-served install tree
# --mount-dir DIR Temp mount point (default: auto via mktemp)
# --tftpboot-dir DIR If given, copy vmlinuz + initrd from images/pxeboot/
# into this dir using the versioned naming convention:
# vmlinuz-<LABEL>-pxe
# initrd-<LABEL>-pxe.img
# where LABEL is derived from --iso filename
# (e.g. AlmaLinux-10.1-x86_64 from AlmaLinux-10.1-x86_64-dvd.iso)
# -h, --help Show this help
set -euo pipefail
ISO_PATH=""
HTTP_ROOT=""
MOUNT_DIR=""
TFTPBOOT_DIR=""
usage() {
sed -n '/^# Usage:/,/^[^#]/{ /^[^#]/d; s/^# \{0,3\}//; p }' "$0"
}
while [[ $# -gt 0 ]]; do
case "$1" in
--iso) ISO_PATH="$2"; shift 2 ;;
--http-root) HTTP_ROOT="$2"; shift 2 ;;
--mount-dir) MOUNT_DIR="$2"; shift 2 ;;
--tftpboot-dir) TFTPBOOT_DIR="$2"; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) echo "Unknown option: $1" >&2; usage >&2; exit 1 ;;
esac
done
if [[ -z "$ISO_PATH" || -z "$HTTP_ROOT" ]]; then
usage >&2
exit 1
fi
if [[ ! -f "$ISO_PATH" ]]; then
echo "ISO not found: $ISO_PATH" >&2
exit 1
fi
if [[ $EUID -ne 0 ]]; then
echo "Re-running with sudo..."
exec sudo "$0" "$@"
fi
if [[ -z "$MOUNT_DIR" ]]; then
MOUNT_DIR="$(mktemp -d /tmp/alma-iso.XXXXXX)"
fi
cleanup() {
if mountpoint -q "$MOUNT_DIR" 2>/dev/null; then
umount "$MOUNT_DIR"
fi
rmdir "$MOUNT_DIR" 2>/dev/null || true
}
trap cleanup EXIT
mount -o loop,ro "$ISO_PATH" "$MOUNT_DIR"
# ── HTTP install tree ─────────────────────────────────────────────────────────
mkdir -p "$HTTP_ROOT"
echo "Syncing install tree to $HTTP_ROOT ..."
rsync -aH --delete --info=progress2 "$MOUNT_DIR"/ "$HTTP_ROOT"/
echo "Install tree synced."
# ── tftpboot kernel + initrd extraction ──────────────────────────────────────
if [[ -n "$TFTPBOOT_DIR" ]]; then
PXEBOOT="$MOUNT_DIR/images/pxeboot"
if [[ ! -f "$PXEBOOT/vmlinuz" || ! -f "$PXEBOOT/initrd.img" ]]; then
echo "WARNING: images/pxeboot/vmlinuz or initrd.img not found in ISO — skipping tftpboot extraction." >&2
else
# Derive label from ISO filename: strip path, strip extension(s)
# AlmaLinux-10.1-x86_64-dvd.iso → AlmaLinux-10.1-x86_64
# AlmaLinux-9.2-x86_64-minimal.iso → AlmaLinux-9.2-x86_64
BASENAME="$(basename "$ISO_PATH")"
LABEL="${BASENAME%.iso}"
LABEL="${LABEL%.ISO}"
# Strip trailing -dvd, -minimal, -boot, -netinstall suffixes
LABEL="$(echo "$LABEL" | sed -E 's/-(dvd|minimal|boot|netinstall)$//')"
VMLINUZ_DEST="$TFTPBOOT_DIR/vmlinuz-${LABEL}-pxe"
INITRD_DEST="$TFTPBOOT_DIR/initrd-${LABEL}-pxe.img"
mkdir -p "$TFTPBOOT_DIR"
echo "Extracting tftpboot files with label: $LABEL"
cp -v "$PXEBOOT/vmlinuz" "$VMLINUZ_DEST"
cp -v "$PXEBOOT/initrd.img" "$INITRD_DEST"
echo "tftpboot extraction complete."
fi
fi
scripts/cline-presets/local-mistral-openai.json
+8
View File
@@ -0,0 +1,8 @@
{
"apiProvider": "openai",
"openAiBaseUrl": "http://127.0.0.1:11434/v1",
"openAiApiKey": "local-llm",
"openAiModelId": "Ministral-3-3B-Instruct-2512-Q4_K_M.gguf",
"apiModelId": "Ministral-3-3B-Instruct-2512-Q4_K_M.gguf",
"welcomeViewCompleted": true
}
scripts/cline-presets/remote-qwen-openai.json
+8
View File
@@ -0,0 +1,8 @@
{
"apiProvider": "openai",
"openAiBaseUrl": "http://192.168.144.120:8080/v1",
"openAiApiKey": "remote-llm",
"openAiModelId": "qwen2.5-coder-14b-instruct-q4_k_m.gguf",
"apiModelId": "qwen2.5-coder-14b-instruct-q4_k_m.gguf",
"welcomeViewCompleted": true
}
scripts/set_cline_provider.sh
Executable
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
DB=/home/valherun/.config/Code/User/globalStorage/state.vscdb
KEY=saoudrizwan.claude-dev
profile="${1:-local-mistral}"
case "$profile" in
local|local-mistral|mistral)
preset="$ROOT/scripts/cline-presets/local-mistral-openai.json"
;;
remote|remote-qwen|qwen)
preset="$ROOT/scripts/cline-presets/remote-qwen-openai.json"
;;
*)
echo "usage: $0 {local-mistral|remote-qwen}" >&2
exit 1
;;
esac
python3 - "$DB" "$KEY" "$preset" <<'PY'
import json
import sqlite3
import sys
db_path, key, preset_path = sys.argv[1:4]
with open(preset_path, "r", encoding="utf-8") as f:
preset = json.load(f)
conn = sqlite3.connect(db_path)
cur = conn.cursor()
row = cur.execute("select value from ItemTable where key=?", (key,)).fetchone()
state = {}
if row and row[0]:
try:
state = json.loads(row[0])
except Exception:
state = {}
state.update(preset)
value = json.dumps(state, separators=(",", ":"))
cur.execute(
"insert into ItemTable(key, value) values(?, ?) on conflict(key) do update set value=excluded.value",
(key, value),
)
conn.commit()
conn.close()
print(value)
PY
scripts/sync_alma_tree.sh
-76
View File
@@ -1,76 +0,0 @@
#!/bin/bash
set -euo pipefail
ISO_PATH=""
HTTP_ROOT=""
MOUNT_DIR=""
usage() {
cat <<'EOF'
Usage: sync_alma_tree.sh --iso FILE --http-root DIR [--mount-dir DIR]
Mount an AlmaLinux ISO read-only and copy its contents into an HTTP-served
install tree for PXE/Kickstart use.
EOF
}
while [[ $# -gt 0 ]]; do
case "$1" in
--iso)
ISO_PATH="$2"
shift 2
;;
--http-root)
HTTP_ROOT="$2"
shift 2
;;
--mount-dir)
MOUNT_DIR="$2"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
echo "Unknown option: $1" >&2
usage >&2
exit 1
;;
esac
done
if [[ -z "$ISO_PATH" || -z "$HTTP_ROOT" ]]; then
usage >&2
exit 1
fi
if [[ ! -f "$ISO_PATH" ]]; then
echo "ISO not found: $ISO_PATH" >&2
exit 1
fi
if [[ $EUID -ne 0 ]]; then
echo "Re-running with sudo..."
exec sudo "$0" "$@"
fi
if [[ -z "$MOUNT_DIR" ]]; then
MOUNT_DIR="$(mktemp -d /tmp/alma-iso.XXXXXX)"
fi
cleanup() {
if mountpoint -q "$MOUNT_DIR"; then
umount "$MOUNT_DIR"
fi
rmdir "$MOUNT_DIR" 2>/dev/null || true
}
trap cleanup EXIT
mkdir -p "$HTTP_ROOT"
mount -o loop,ro "$ISO_PATH" "$MOUNT_DIR"
rsync -aH --delete "$MOUNT_DIR"/ "$HTTP_ROOT"/
echo "Synced AlmaLinux tree to $HTTP_ROOT"
tools/build_probe_stage_payload.sh
Executable
+145
View File
@@ -0,0 +1,145 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
CFG_DIR="$ROOT/work/KickStart/cfg"
KS_DIR="$CFG_DIR/AlmaLinux-10.1-x86_64/ks.d"
STAG_FILE="$CFG_DIR/stag.d/52:54:00:aa:bb:01.cfg"
KSCONFIG="$CFG_DIR/AlmaLinux-10.1-x86_64/ks.config"
REMOTE=[email protected]
stage="${1:?usage: $0 <stage>}"
out="${2:-/dev/stdout}"
payload_mode="${3:-cumulative}"
if [[ "$stage" == "baseline" ]]; then
echo "snapshot stage payload is only valid for numeric stages" >&2
exit 1
fi
case "$payload_mode" in
cumulative)
mapfile -t EXTRA_SCRIPTS < <("$ROOT/tools/run_probe_stage.sh" extras "$stage")
;;
delta)
mapfile -t EXTRA_SCRIPTS < <("$ROOT/tools/run_probe_stage.sh" extras "$stage" | tail -n 1)
;;
*)
echo "invalid payload mode: $payload_mode" >&2
exit 1
;;
esac
if [[ "${#EXTRA_SCRIPTS[@]}" -eq 0 ]]; then
echo "no extra scripts for stage=$stage mode=$payload_mode" >&2
exit 1
fi
source "$KSCONFIG"
source "$STAG_FILE"
DIST="${DIST:-AlmaLinux-10.1-x86_64}"
PTYPE="${TYPE:-probe}"
DISKTYPE="${DISKTYPE:-ssd}"
TZ="${TZ:-America/New_York}"
KSRV="${KSRV:-monolith.iglass.net}"
SSHHOST="${SSHHOST:-$KSRV}"
SSHPORT="${SSHPORT:-22}"
IG_HOME="${IG_HOME:-/opt/iglass}"
IG_ETC="${IG_ETC:-/etc/iglass}"
NOCPASS="${NOCPASS:-}"
IGRELEASE="$(ssh -o BatchMode=yes "$REMOTE" 'cd /home/KickStart/iglass-releases && ls -v probe*rpm | tail -1' 2>/dev/null || true)"
tmp_out="$(mktemp)"
trap 'rm -f "$tmp_out"' EXIT
{
cat <<EOF
#!/usr/bin/env bash
set -euo pipefail
# Generated by tools/build_probe_stage_payload.sh on $(date '+%F %T %Z')
# Stage: $stage
# Payload mode: $payload_mode
MAC=52:54:00:aa:bb:01
DIST='$DIST'
PTYPE='$PTYPE'
DISKTYPE='$DISKTYPE'
TZ='$TZ'
KSRV='$KSRV'
SSHHOST='$SSHHOST'
SSHPORT='$SSHPORT'
IG_HOME='$IG_HOME'
IG_ETC='$IG_ETC'
IGRELEASE='$IGRELEASE'
NOCPASS='$NOCPASS'
LOG="/root/ks/stage-apply-stage${stage}.log"
LOGGER='logger -t KickStart[\$\$] -p local3.info -s'
CURRENT_SCRIPT='bootstrap'
mkdir -p /root/ks /root/kickstart-debug
touch "\$LOG"
trap 'rc=\$?; echo "-- Failed in \$CURRENT_SCRIPT rc=\$rc" | tee -a "\$LOG" | sh -c "\$LOGGER"; exit \$rc' ERR
echo "Stage apply start stage=$stage" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "DIST=\$DIST" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "PTYPE=\$PTYPE" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "DISKTYPE=\$DISKTYPE" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "KSRV=\$KSRV" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "SSHHOST=\$SSHHOST" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "IG_HOME=\$IG_HOME" | tee -a "\$LOG" | sh -c "\$LOGGER"
echo "IG_ETC=\$IG_ETC" | tee -a "\$LOG" | sh -c "\$LOGGER"
# Reconstruct the primary install disk for snapshot-stage payloads. Full kickstart
# establishes this during partitioning, but replayed staged scripts still reference DISK.
DISK=''
root_source=\$(findmnt -n -o SOURCE / 2>/dev/null || true)
if [[ -n "\$root_source" ]]; then
root_pkname=\$(lsblk -no PKNAME "\$root_source" 2>/dev/null | head -n1 || true)
if [[ -n "\$root_pkname" ]]; then
DISK="\$root_pkname"
elif [[ "\$root_source" =~ ^/dev/([a-zA-Z0-9._-]+)p?[0-9]+\$ ]]; then
DISK="\${BASH_REMATCH[1]}"
fi
fi
if [[ -z "\$DISK" ]]; then
DISK=\$(lsblk -dn -o NAME,TYPE | awk '\$2==\"disk\" {print \$1; exit}')
fi
echo "DISK=\${DISK:-unknown}" | tee -a "\$LOG" | sh -c "\$LOGGER"
EOF
for script in "${EXTRA_SCRIPTS[@]}"; do
printf '# Source: %s sha256=%s\n' \
"$KS_DIR/$script" \
"$(sha256sum "$KS_DIR/$script" | awk '{print $1}')"
done
echo
cat "$KS_DIR/functions"
echo
for script in "${EXTRA_SCRIPTS[@]}"; do
cat <<EOF
CURRENT_SCRIPT='$script'
echo '-- Starting $script' | tee -a "\$LOG" | sh -c "\$LOGGER"
EOF
perl -pe 's/\#LOG/2\>\&1 \| tee \-a \$LOG \| sh \-c "\$LOGGER"/g' "$KS_DIR/$script"
cat <<EOF
echo '-- Done with $script' | tee -a "\$LOG" | sh -c "\$LOGGER"
EOF
done
cat <<EOF
echo "Stage apply finished stage=$stage" | tee -a "\$LOG" | sh -c "\$LOGGER"
: > /root/kickstart-debug/stage-${stage}-scripts.txt
EOF
for script in "${EXTRA_SCRIPTS[@]}"; do
printf "printf '%%s\\n' '%s' >> /root/kickstart-debug/stage-%s-scripts.txt\n" \
"$script" "$stage"
done
cat <<EOF
touch /root/kickstart-debug/stage-${stage}-complete
EOF
} > "$tmp_out"
install -m 0755 "$tmp_out" "$out"
tools/claude_harness_supervisor.sh
Executable
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
WATCH_ROOT=/tmp/probe-stage-watch
SUP_ROOT="$WATCH_ROOT/supervisor"
LOCK_FILE="$SUP_ROOT/claude.lock"
RUN_LOG="$SUP_ROOT/claude-runs.log"
PROMPT_FILE="$SUP_ROOT/claude-prompt.txt"
LAST_MSG_FILE="$SUP_ROOT/claude-last-message.txt"
STDOUT_FILE="$SUP_ROOT/claude-stdout.txt"
LATEST_FILE="$WATCH_ROOT/latest-summary.txt"
STATE_FILE="$WATCH_ROOT/state.env"
FAIL_LOG="$WATCH_ROOT/failures.md"
MEMORY_FILE="$ROOT/codex-running-memory.md"
TRIAGE_FILE="$WATCH_ROOT/needs-triage.env"
CLAUDE_BIN="${CLAUDE_BIN:-/home/valherun/.local/bin/claude}"
export PATH="/home/valherun/.local/bin:$PATH"
mkdir -p "$SUP_ROOT"
timestamp() {
date '+%F %T'
}
latest_attempt_dir() {
find "$WATCH_ROOT" -maxdepth 1 -type d -name '20*-attempt-*' | sort | tail -n 1
}
print_triage_status() {
if [[ -f "$TRIAGE_FILE" ]]; then
cat "$TRIAGE_FILE"
return
fi
if [[ -f "$STATE_FILE" ]]; then
# shellcheck disable=SC1090
source "$STATE_FILE"
fi
echo "no active triage request"
echo "watcher_phase='${CURRENT_PHASE:-unknown}'"
echo "watcher_detail='${CURRENT_DETAIL:-}'"
}
{
flock -n 9 || exit 0
attempt_dir="$(latest_attempt_dir || true)"
{
echo "You are a failure triage and repair assistant for a PXE/kickstart harness."
echo "Diagnose likely root cause from the latest failure."
echo "When the cause is clear and the fix is small and safe, edit files directly."
echo "Prefer the smallest safe mitigation."
echo "Update $MEMORY_FILE with a short bullet when you learn or fix something important."
echo "Do not remove scripts permanently from the repo."
echo
echo "Time: $(timestamp)"
echo
echo "Watcher summary:"
[[ -f "$LATEST_FILE" ]] && cat "$LATEST_FILE"
echo
echo "Watcher state:"
[[ -f "$STATE_FILE" ]] && cat "$STATE_FILE"
echo
echo "Triage request:"
print_triage_status
echo
echo "Failure ledger tail:"
[[ -f "$FAIL_LOG" ]] && tail -n 40 "$FAIL_LOG"
echo
echo "Running memory tail:"
[[ -f "$MEMORY_FILE" ]] && tail -n 80 "$MEMORY_FILE"
echo
echo "Latest attempt dir: ${attempt_dir:-missing}"
if [[ -n "${attempt_dir:-}" ]]; then
echo
echo "Latest attempt log:"
sed -n '1,240p' "$attempt_dir/attempt.log" 2>/dev/null || true
echo
echo "Latest attempt files:"
find "$attempt_dir" -maxdepth 1 -type f | sort
echo
[[ -f "$attempt_dir/stage-meta.txt" ]] && { echo "stage-meta:"; cat "$attempt_dir/stage-meta.txt"; echo; }
[[ -f "$attempt_dir/included-scripts.txt" ]] && { echo "included-scripts:"; cat "$attempt_dir/included-scripts.txt"; echo; }
[[ -f "$attempt_dir/boot.log" ]] && { echo "boot.log tail:"; tail -n 200 "$attempt_dir/boot.log"; echo; }
[[ -f "$attempt_dir/ks-post.log" ]] && { echo "ks-post.log tail:"; tail -n 200 "$attempt_dir/ks-post.log"; echo; }
fi
echo
echo "Reply with concise findings ordered by confidence. Include: likely cause, most suspicious files/lines, what you changed if you edited anything, minimal safe mitigation, and whether to retry immediately or wait for a repo fix."
} > "$PROMPT_FILE"
echo "[$(timestamp)] claude-supervisor start" >> "$RUN_LOG"
if "$CLAUDE_BIN" -p --dangerously-skip-permissions \
--allowedTools 'Read,Edit,Bash(grep:*),Bash(sed:*),Bash(cat:*),Bash(head:*),Bash(tail:*),Bash(rg:*),Bash(ls:*),Bash(git status:*),Bash(git diff:*)' \
< "$PROMPT_FILE" > "$STDOUT_FILE" 2>&1; then
cp "$STDOUT_FILE" "$LAST_MSG_FILE"
status="ok"
else
status="failed"
fi
{
echo "[$(timestamp)] claude-supervisor status=$status"
echo "--- last message ---"
cat "$LAST_MSG_FILE" 2>/dev/null || true
echo
} >> "$RUN_LOG"
} 9>"$LOCK_FILE"
tools/codex_harness_supervisor.sh
Executable
+213
View File
@@ -0,0 +1,213 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
WATCH_ROOT=/tmp/probe-stage-watch
SUP_ROOT="$WATCH_ROOT/supervisor"
LOCK_FILE="$SUP_ROOT/lock"
RUN_LOG="$SUP_ROOT/runs.log"
PROMPT_FILE="$SUP_ROOT/prompt.txt"
LAST_MSG_FILE="$SUP_ROOT/last-message.txt"
STDOUT_FILE="$SUP_ROOT/codex-stdout.txt"
LATEST_FILE="$WATCH_ROOT/latest-summary.txt"
STATE_FILE="$WATCH_ROOT/state.env"
FAIL_LOG="$WATCH_ROOT/failures.md"
MEMORY_FILE="$ROOT/codex-running-memory.md"
TRIAGE_FILE="$WATCH_ROOT/needs-triage.env"
mkdir -p "$SUP_ROOT"
timestamp() {
date '+%F %T'
}
latest_attempt_dir() {
find "$WATCH_ROOT" -maxdepth 1 -type d -name '20*-attempt-*' | sort | tail -n 1
}
payload_source_lines() {
local attempt_dir="$1"
local manifest="$attempt_dir/payload-manifest.txt"
local stage_apply="$attempt_dir/stage-apply.sh"
if [[ -f "$manifest" ]]; then
grep '^# Source: ' "$manifest" && return 0
fi
if [[ -f "$stage_apply" ]]; then
grep '^# Source: ' "$stage_apply" && return 0
fi
return 1
}
print_payload_drift() {
local attempt_dir="$1"
local manifest="$attempt_dir/payload-manifest.txt"
local stage_apply="$attempt_dir/stage-apply.sh"
local found=0
local line_count=0
local source_mode="missing"
if [[ -f "$manifest" ]] && grep -q '^# Source: ' "$manifest"; then
source_mode="manifest"
elif [[ -f "$stage_apply" ]] && grep -q '^# Source: ' "$stage_apply"; then
source_mode="stage-apply"
else
echo "payload-source-drift: unavailable (no source hashes in payload-manifest.txt or stage-apply.sh)"
return 0
fi
while IFS= read -r line; do
[[ "$line" =~ ^#\ Source:\ (.+)\ sha256=([0-9a-f]{64})$ ]] || continue
line_count=$((line_count + 1))
local source_path="${BASH_REMATCH[1]}"
local payload_sha="${BASH_REMATCH[2]}"
local current_sha=""
if [[ -f "$source_path" ]]; then
current_sha="$(sha256sum "$source_path" | awk '{print $1}')"
if [[ "$current_sha" != "$payload_sha" ]]; then
if [[ "$found" == "0" ]]; then
echo "payload-source-drift:"
found=1
fi
echo "STALE payload script: $source_path payload_sha=$payload_sha current_sha=$current_sha"
fi
fi
done < <(payload_source_lines "$attempt_dir" || true)
if [[ "$line_count" == "0" ]]; then
echo "payload-source-drift: unavailable (source hash parsing failed)"
elif [[ "$found" == "0" && "$source_mode" == "manifest" ]]; then
echo "payload-source-drift: none"
elif [[ "$found" == "0" ]]; then
echo "payload-source-drift: none (recovered from stage-apply.sh; payload-manifest.txt lacked source hashes)"
fi
}
print_triage_status() {
if [[ -f "$TRIAGE_FILE" ]]; then
cat "$TRIAGE_FILE"
return
fi
if [[ -f "$STATE_FILE" ]]; then
# shellcheck disable=SC1090
source "$STATE_FILE"
fi
echo "no active triage request"
echo "watcher_phase='${CURRENT_PHASE:-unknown}'"
echo "watcher_detail='${CURRENT_DETAIL:-}'"
}
{
flock -n 9 || exit 0
attempt_dir="$(latest_attempt_dir || true)"
{
echo "# Codex Harness Supervisor"
echo
echo "Time: $(timestamp)"
echo "Repo: $ROOT"
echo
echo "Your job:"
echo "- Inspect the current harness and latest attempt artifacts."
echo "- If the harness itself is wrong or stale, patch the harness."
echo "- If the current stage exposed a kickstart issue, patch the repo or harness as needed."
echo "- Update $MEMORY_FILE with a short new finding if you learn something important."
echo "- Do not remove scripts permanently from the repo. Temporary staged exclusion is allowed only via harness/stage config."
echo "- Keep changes small and targeted."
echo
echo "Current watcher summary:"
if [[ -f "$LATEST_FILE" ]]; then
cat "$LATEST_FILE"
else
echo "latest-summary.txt missing"
fi
echo
echo "Current watcher state:"
if [[ -f "$STATE_FILE" ]]; then
cat "$STATE_FILE"
else
echo "state.env missing"
fi
echo
echo "Triage request:"
print_triage_status
echo
echo "Failure ledger tail:"
if [[ -f "$FAIL_LOG" ]]; then
tail -n 40 "$FAIL_LOG"
else
echo "failures.md missing"
fi
echo
echo "Running memory tail:"
if [[ -f "$MEMORY_FILE" ]]; then
tail -n 80 "$MEMORY_FILE"
else
echo "codex-running-memory.md missing"
fi
echo
echo "Latest attempt dir: ${attempt_dir:-missing}"
if [[ -n "${attempt_dir:-}" ]]; then
echo
echo "Latest attempt log:"
sed -n '1,240p' "$attempt_dir/attempt.log" 2>/dev/null || true
echo
echo "Latest attempt files:"
find "$attempt_dir" -maxdepth 1 -type f | sort
echo
if [[ -f "$attempt_dir/boot.log" ]]; then
echo "boot.log tail:"
tail -n 200 "$attempt_dir/boot.log"
echo
fi
if [[ -f "$attempt_dir/ks-post.log" ]]; then
echo "ks-post.log tail:"
tail -n 200 "$attempt_dir/ks-post.log"
echo
fi
if [[ -f "$attempt_dir/stage-apply.log" ]]; then
echo "stage-apply.log tail:"
tail -n 200 "$attempt_dir/stage-apply.log"
echo
fi
if [[ -f "$attempt_dir/stage-apply.stdout" ]]; then
echo "stage-apply.stdout tail:"
tail -n 200 "$attempt_dir/stage-apply.stdout"
echo
fi
if [[ -f "$attempt_dir/payload-manifest.txt" ]]; then
echo "payload-manifest:"
cat "$attempt_dir/payload-manifest.txt"
echo
print_payload_drift "$attempt_dir"
echo
fi
fi
echo
echo "Please make any needed repo changes directly, then end with a concise summary."
} > "$PROMPT_FILE"
echo "[$(timestamp)] supervisor start" >> "$RUN_LOG"
if codex exec \
--dangerously-bypass-approvals-and-sandbox \
-C "$ROOT" \
-o "$LAST_MSG_FILE" \
- < "$PROMPT_FILE" > "$STDOUT_FILE" 2>&1; then
status="ok"
else
status="failed"
fi
{
echo "[$(timestamp)] supervisor status=$status"
echo "--- last message ---"
cat "$LAST_MSG_FILE" 2>/dev/null || true
echo
} >> "$RUN_LOG"
} 9>"$LOCK_FILE"
tools/llm_failure_router.sh
Executable
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
WATCH_ROOT=/tmp/probe-stage-watch
TRIAGE_FILE="$WATCH_ROOT/needs-triage.env"
RUN_LOG="$WATCH_ROOT/supervisor/router.log"
STATUS_FILE="$WATCH_ROOT/supervisor/router-status.env"
ENABLE_CODEX_FALLBACK="${ENABLE_CODEX_FALLBACK:-0}"
mkdir -p "$WATCH_ROOT/supervisor"
timestamp() {
date '+%F %T'
}
if [[ ! -f "$TRIAGE_FILE" ]]; then
exit 0
fi
# shellcheck disable=SC1090
source "$TRIAGE_FILE"
outcome="${TRIAGE_OUTCOME:-unknown}"
mode="${TRIAGE_MODE:-unknown}"
stage="${TRIAGE_STAGE:-unknown}"
suspect="${TRIAGE_SUSPECT:-unknown}"
attempt="${TRIAGE_ATTEMPT:-unknown}"
run_codex=0
run_claude=0
route_reason=""
case "$outcome" in
*install\ timeout*|*template\ capture*|*snapshot\ stage\ apply\ failed*)
run_claude=1
route_reason="claude-first-harness-or-apply-failure"
;;
*snapshot\ stage\ failed*)
run_claude=1
route_reason="claude-first-snapshot-stage-regression"
;;
*boot\ failed/no\ ssh*|*snapshot\ boot\ failed/no\ ssh*|*snapshot\ reboot\ failed/no\ ssh*)
run_claude=1
route_reason="claude-first-boot-or-stage-regression"
;;
*)
run_claude=1
route_reason="claude-first-default"
;;
esac
if [[ "$ENABLE_CODEX_FALLBACK" == "1" ]]; then
case "$outcome" in
*watcher\ missing/stale\ state*|*watcher\ running/state\ stale*|*template\ capture*|*snapshot\ stage\ apply\ failed*)
run_codex=1
route_reason="${route_reason}+codex-fallback"
;;
esac
fi
{
echo "ROUTED_AT='$(timestamp)'"
echo "TRIAGE_STAGE='${stage}'"
echo "TRIAGE_ATTEMPT='${attempt}'"
echo "TRIAGE_SUSPECT='${suspect}'"
echo "TRIAGE_OUTCOME='${outcome}'"
echo "TRIAGE_MODE='${mode}'"
echo "ROUTE_REASON='${route_reason}'"
echo "RUN_CODEX='${run_codex}'"
echo "RUN_CLAUDE='${run_claude}'"
} > "$STATUS_FILE"
echo "[$(timestamp)] route stage=$stage attempt=$attempt suspect=$suspect outcome=$outcome reason=$route_reason codex=$run_codex claude=$run_claude" >> "$RUN_LOG"
if [[ "$run_claude" == "1" ]]; then
"$ROOT/tools/claude_harness_supervisor.sh" || true
fi
if [[ "$run_codex" == "1" ]]; then
"$ROOT/tools/codex_harness_supervisor.sh" || true
fi
tools/monitor_probe_harness.sh
Executable
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
WATCH_ROOT=/tmp/probe-stage-watch
LOG_FILE="$WATCH_ROOT/cron-monitor.log"
STATE_FILE="$WATCH_ROOT/state.env"
LATEST_FILE="$WATCH_ROOT/latest-summary.txt"
TRIAGE_FILE="$WATCH_ROOT/needs-triage.env"
FAIL_LOG="$WATCH_ROOT/failures.md"
RESTART_HELPER="$ROOT/tools/restart_probe_watcher.sh"
TRIAGE_STALE_AFTER_SEC="${TRIAGE_STALE_AFTER_SEC:-600}"
mkdir -p "$WATCH_ROOT"
timestamp() {
date '+%F %T'
}
triage_age_seconds() {
if [[ ! -f "$TRIAGE_FILE" ]]; then
return 1
fi
local triage_epoch
triage_epoch="$(stat -c %Y "$TRIAGE_FILE" 2>/dev/null || echo 0)"
[[ "$triage_epoch" -gt 0 ]] || return 1
echo "$(( $(date +%s) - triage_epoch ))"
}
clear_obsolete_watcher_triage() {
[[ -f "$TRIAGE_FILE" ]] || return 0
# shellcheck disable=SC1090
source "$TRIAGE_FILE"
case "${TRIAGE_OUTCOME:-}" in
watcher\ missing/stale\ state|watcher\ running/state\ stale)
rm -f "$TRIAGE_FILE"
;;
esac
}
request_monitor_triage() {
local outcome="$1"
if [[ -f "$TRIAGE_FILE" ]]; then
return 0
fi
local latest_failure_stage="unknown"
local latest_failure_suspect="unknown"
local latest_failure_attempt="unknown"
if [[ -f "$FAIL_LOG" ]]; then
latest_failure_stage="$(tail -n 1 "$FAIL_LOG" | awk -F'|' '{gsub(/^ +| +$/, "", $4); print $4}')"
latest_failure_attempt="$(tail -n 1 "$FAIL_LOG" | awk -F'|' '{gsub(/^ +| +$/, "", $3); print $3}')"
latest_failure_suspect="$(tail -n 1 "$FAIL_LOG" | awk -F'|' '{gsub(/^ +| +$/, "", $5); print $5}')"
fi
if [[ -f "$STATE_FILE" ]]; then
# shellcheck disable=SC1090
source "$STATE_FILE"
fi
cat > "$TRIAGE_FILE" <<EOF
TRIAGE_REQUESTED_AT='$(timestamp)'
TRIAGE_STAGE='${CURRENT_STAGE:-$latest_failure_stage}'
TRIAGE_MODE='${CURRENT_MODE:-unknown}'
TRIAGE_ATTEMPT='${latest_failure_attempt}'
TRIAGE_SUSPECT='${CURRENT_SUSPECT:-$latest_failure_suspect}'
TRIAGE_OUTCOME='$outcome'
TRIAGE_ATTEMPT_DIR=''
EOF
}
watcher_ps="$(ps -ef | grep '[p]robe_stage_watch.sh' || true)"
if [[ -n "$watcher_ps" ]]; then
watcher_status="running"
else
watcher_status="missing"
if [[ -x "$RESTART_HELPER" ]]; then
"$RESTART_HELPER" >> "$LOG_FILE" 2>&1 || true
watcher_ps="$(ps -ef | grep '[p]robe_stage_watch.sh' || true)"
if [[ -n "$watcher_ps" ]]; then
watcher_status="running"
fi
fi
fi
state_age="unknown"
triage_age="unknown"
if [[ -f "$STATE_FILE" ]]; then
now_epoch="$(date +%s)"
state_epoch="$(stat -c %Y "$STATE_FILE" 2>/dev/null || echo 0)"
if [[ "$state_epoch" -gt 0 ]]; then
state_age="$(( now_epoch - state_epoch ))s"
fi
fi
if [[ -f "$TRIAGE_FILE" ]]; then
triage_age="$(triage_age_seconds || true)"
[[ -n "${triage_age:-}" ]] || triage_age="unknown"
fi
{
echo "[$(timestamp)] watcher=$watcher_status state_age=$state_age"
if [[ -f "$LATEST_FILE" ]]; then
sed 's/^/ /' "$LATEST_FILE"
else
echo " latest-summary=missing"
fi
if [[ "$watcher_status" == "missing" ]]; then
echo " action-needed=watcher-not-running"
elif [[ "$state_age" != "unknown" ]] && [[ "${state_age%s}" -gt 180 ]]; then
echo " action-needed=state-stale"
fi
if [[ -f "$TRIAGE_FILE" ]]; then
echo " action-needed=llm-triage"
sed 's/^/ triage: /' "$TRIAGE_FILE"
echo " triage_age=$triage_age"
fi
} >> "$LOG_FILE"
if [[ "$watcher_status" == "missing" ]]; then
request_monitor_triage "watcher missing/stale state"
elif [[ "$state_age" != "unknown" ]] && [[ "${state_age%s}" -gt 180 ]]; then
request_monitor_triage "watcher running/state stale"
else
clear_obsolete_watcher_triage
fi
if [[ -f "$TRIAGE_FILE" && -x "$ROOT/tools/llm_failure_router.sh" ]]; then
"$ROOT/tools/llm_failure_router.sh" >> "$LOG_FILE" 2>&1 || true
fi
if ! ps -ef | grep -q '[p]robe_stage_watch.sh'; then
if [[ -x "$RESTART_HELPER" ]]; then
"$RESTART_HELPER" >> "$LOG_FILE" 2>&1 || true
fi
fi
tools/probe_capture_baseline.sh
Executable
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
VM=probe
DISK=/var/lib/libvirt/images/probe.qcow2
ARTIFACT_ROOT=/tmp/probe-stage-watch
TEMPLATE_DIR=/var/lib/libvirt/images/probe-stage
META="$ARTIFACT_ROOT/baseline-template.env"
CURRENT_IP_FILE="$ARTIFACT_ROOT/baseline-source-ip.txt"
SUDO="${SUDO:-sudo}"
LIBVIRT_URI="${LIBVIRT_URI:-qemu:///system}"
VIRSH=( "$SUDO" virsh --connect "$LIBVIRT_URI" )
template_label="${1:-baseline}"
wait_secs="${2:-120}"
safe_label="$(printf '%s' "$template_label" | tr -cs 'A-Za-z0-9._-' '-')"
TEMPLATE="$TEMPLATE_DIR/probe-${safe_label}-template.qcow2"
mkdir -p "$ARTIFACT_ROOT"
"$SUDO" mkdir -p "$TEMPLATE_DIR"
current_ip() {
local ip=""
ip=$("${VIRSH[@]}" domifaddr "$VM" --source arp 2>/dev/null | awk '/ipv4/ {sub(/\/.*$/, "", $4); print $4; exit}')
if [[ -z "$ip" ]]; then
ip=$(ip neigh show dev virbr1 2>/dev/null | awk '$0 ~ /52:54:00:aa:bb:01/ {print $1; exit}')
fi
echo "$ip"
}
state=$("${VIRSH[@]}" domstate "$VM" 2>/dev/null || echo missing)
ip="$(current_ip || true)"
if [[ -n "${ip:-}" ]]; then
printf '%s\n' "$ip" > "$CURRENT_IP_FILE"
fi
if [[ "$state" != "shut off" ]]; then
if [[ -n "${ip:-}" ]]; then
ssh -o BatchMode=yes -o StrictHostKeyChecking=no root@"$ip" 'systemctl poweroff' >/dev/null 2>&1 || true
fi
"${VIRSH[@]}" shutdown "$VM" >/dev/null 2>&1 || true
for _ in $(seq 1 "$wait_secs"); do
state="$("${VIRSH[@]}" domstate "$VM" 2>/dev/null || echo missing)"
[[ "$state" == "shut off" ]] && break
sleep 1
done
fi
state="$("${VIRSH[@]}" domstate "$VM" 2>/dev/null || echo missing)"
if [[ "$state" != "shut off" ]]; then
echo "baseline source VM did not shut off cleanly" >&2
exit 1
fi
tmp_template="${TEMPLATE}.tmp"
"$SUDO" rm -f "$tmp_template"
"$SUDO" qemu-img convert -O qcow2 "$DISK" "$tmp_template"
"$SUDO" mv -f "$tmp_template" "$TEMPLATE"
{
echo "BASELINE_TEMPLATE='$TEMPLATE'"
echo "BASELINE_LABEL='$template_label'"
echo "BASELINE_CREATED='$(date '+%F %T')'"
echo "BASELINE_SOURCE_DISK='$DISK'"
echo "BASELINE_SOURCE_IP='${ip:-}'"
echo "BASELINE_VM='$VM'"
echo "BASELINE_SIZE_BYTES='$("$SUDO" stat -c %s "$TEMPLATE")'"
} > "$META"
printf '%s\n' "$TEMPLATE"
tools/probe_snapshot_stage.sh
Executable
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
VM=probe
VM_MAC=52:54:00:aa:bb:01
DISK=/var/lib/libvirt/images/probe.qcow2
ARTIFACT_ROOT=/tmp/probe-stage-watch
TEMPLATE_META="$ARTIFACT_ROOT/baseline-template.env"
PAYLOAD="$ARTIFACT_ROOT/.stage-apply.sh"
ATTEMPT_DIR="${2:?usage: $0 <stage> <attempt-dir>}"
stage="${1:?usage: $0 <stage> <attempt-dir>}"
VERIFICATION_ENV="$ATTEMPT_DIR/verification.env"
SUDO="${SUDO:-sudo}"
LIBVIRT_URI="${LIBVIRT_URI:-qemu:///system}"
BOOT_CHECKS="${BOOT_CHECKS:-30}"
BOOT_SLEEP_SEC="${BOOT_SLEEP_SEC:-10}"
REBOOT_START_CHECKS="${REBOOT_START_CHECKS:-24}"
REBOOT_START_SLEEP_SEC="${REBOOT_START_SLEEP_SEC:-5}"
VIRSH=( "$SUDO" virsh --connect "$LIBVIRT_URI" )
VIRT_INSTALL=( "$SUDO" virt-install --connect "$LIBVIRT_URI" )
SSH_OPTS=(
-o BatchMode=yes
-o ConnectTimeout=10
-o StrictHostKeyChecking=no
-o UserKnownHostsFile=/dev/null
-o GlobalKnownHostsFile=/dev/null
-o LogLevel=ERROR
)
CURRENT_STEP="init"
ORIGINAL_BOOT_ID=""
REBOOT_BOOT_ID=""
REBOOT_SYSTEM_STATE=""
PAYLOAD_MODE="cumulative"
determine_payload_mode() {
if [[ "${BASELINE_LABEL:-}" =~ ^stage-([0-9]+)$ ]]; then
local parent_stage="${BASH_REMATCH[1]}"
if [[ "$stage" =~ ^[0-9]+$ ]] && [[ "$parent_stage" -eq $(( stage - 1 )) ]]; then
echo "delta"
return 0
fi
fi
echo "cumulative"
}
expected_script_count() {
if [[ "$PAYLOAD_MODE" == "delta" ]]; then
echo 1
else
"$ROOT/tools/run_probe_stage.sh" included "$stage" | wc -l | awk '{print $1}'
fi
}
expected_new_script() {
"$ROOT/tools/run_probe_stage.sh" "$stage" | awk -F= '/^new_script=/{print $2; exit}'
}
if [[ "$stage" == "baseline" ]]; then
echo "snapshot stage mode requires numeric stage" >&2
exit 1
fi
if [[ ! -f "$TEMPLATE_META" ]]; then
echo "missing baseline template metadata: $TEMPLATE_META" >&2
exit 1
fi
source "$TEMPLATE_META"
PAYLOAD_MODE="$(determine_payload_mode)"
# Preserve stdout for the final reboot IP while appending setup failures to attempt.log.
exec 3>&1
exec 2>>"$ATTEMPT_DIR/attempt.log"
trap 'rc=$?; if [[ "$rc" -ne 0 ]]; then echo "snapshot-step-failed step=$CURRENT_STEP rc=$rc" >> "$ATTEMPT_DIR/attempt.log"; fi' EXIT
cat > "$VERIFICATION_ENV" <<EOF
STAGE='$stage'
VERIFY_APPLY_RC=''
VERIFY_STAGE_FINISHED='0'
VERIFY_EXPECTED_SCRIPT_COUNT='$(expected_script_count)'
VERIFY_EXPECTED_NEW_SCRIPT='$(expected_new_script)'
VERIFY_PAYLOAD_MODE='$PAYLOAD_MODE'
VERIFY_STAGE_SCRIPT_COUNT=''
VERIFY_EXPECTED_NEW_SCRIPT_PRESENT='0'
VERIFY_REBOOT_STARTED='0'
VERIFY_BOOT_ID_CHANGED='0'
VERIFY_REBOOT_SSH='0'
VERIFY_REBOOT_SYSTEM_STATE=''
VERIFY_REBOOT_IP=''
VERIFIED_GOOD='0'
EOF
current_ip() {
local ip=""
ip=$("${VIRSH[@]}" domifaddr "$VM" --source arp 2>/dev/null | awk '/ipv4/ {sub(/\/.*$/, "", $4); print $4; exit}')
if [[ -z "$ip" ]]; then
ip=$(ip neigh show dev virbr1 2>/dev/null | awk -v mac="$VM_MAC" '$0 ~ mac {print $1; exit}')
fi
echo "$ip"
}
boot_import_vm() {
CURRENT_STEP="clone-baseline"
"${VIRSH[@]}" destroy "$VM" >/dev/null 2>&1 || true
"${VIRSH[@]}" undefine "$VM" --remove-all-storage --nvram >/dev/null 2>&1 || true
"$SUDO" rm -f "$DISK"
"$SUDO" qemu-img create -f qcow2 -F qcow2 -b "$BASELINE_TEMPLATE" "$DISK" >/dev/null
CURRENT_STEP="virt-install-import"
"${VIRT_INSTALL[@]}" \
--name "$VM" \
--memory 4096 \
--vcpus 2 \
--disk "$DISK",format=qcow2 \
--network network=pxe-lab,model=virtio,mac="$VM_MAC" \
--os-variant almalinux10 \
--graphics vnc,listen=127.0.0.1 \
--machine q35 \
--features smm.state=on \
--boot loader=/usr/share/OVMF/OVMF_CODE_4M.secboot.fd,loader.readonly=yes,loader.type=pflash,loader.secure=yes,nvram.template=/usr/share/OVMF/OVMF_VARS_4M.ms.fd,hd \
--import \
--noautoconsole
}
CURRENT_STEP="boot-import"
boot_import_vm >> "$ATTEMPT_DIR/attempt.log"
boot_ip=""
CURRENT_STEP="wait-for-boot-ssh"
for tick in $(seq 0 $(( BOOT_CHECKS - 1 ))); do
ip="$(current_ip || true)"
echo "snapshot-boot-check=$tick ip=${ip:-none}" | tee -a "$ATTEMPT_DIR/attempt.log"
if [[ -n "${ip:-}" ]]; then
if ssh "${SSH_OPTS[@]}" -o ConnectTimeout=5 root@"$ip" \
'echo boot-ok; hostname; systemctl is-system-running || true' \
> "$ATTEMPT_DIR/snapshot-boot-${tick}.ssh" 2>&1; then
boot_ip="$ip"
break
fi
fi
sleep "$BOOT_SLEEP_SEC"
done
if [[ -z "$boot_ip" ]]; then
echo "snapshot clone never reached ssh" >&2
exit 20
fi
CURRENT_STEP="capture-initial-boot-id"
ORIGINAL_BOOT_ID="$(ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'cat /proc/sys/kernel/random/boot_id 2>/dev/null || true' 2>/dev/null || true)"
printf 'initial_boot_id=%s\n' "${ORIGINAL_BOOT_ID:-unknown}" >> "$ATTEMPT_DIR/attempt.log"
CURRENT_STEP="build-payload"
"$ROOT/tools/build_probe_stage_payload.sh" "$stage" "$PAYLOAD" "$PAYLOAD_MODE"
CURRENT_STEP="write-payload-artifacts"
cp "$PAYLOAD" "$ATTEMPT_DIR/stage-apply.sh"
sha256sum "$PAYLOAD" > "$ATTEMPT_DIR/stage-apply.sha256"
{
echo "stage=$stage"
echo "payload_mode=$PAYLOAD_MODE"
echo "generated_at=$(date '+%F %T %Z')"
echo "payload_sha256=$(awk '{print $1}' "$ATTEMPT_DIR/stage-apply.sha256")"
awk '
/^# Generated by / || /^# Stage: / || /^# Source: / { print }
' "$PAYLOAD"
} > "$ATTEMPT_DIR/payload-manifest.txt"
cat "$ATTEMPT_DIR/payload-manifest.txt" >> "$ATTEMPT_DIR/attempt.log"
CURRENT_STEP="copy-payload-to-guest"
scp "${SSH_OPTS[@]}" "$PAYLOAD" root@"$boot_ip":/root/stage-apply.sh >/dev/null
CURRENT_STEP="launch-stage-apply"
# Run apply in the background so SSH doesn't block indefinitely; capture exit code to a
# sentinel file so we can distinguish "still running" from "failed".
ssh "${SSH_OPTS[@]}" -o ConnectTimeout=10 root@"$boot_ip" \
'mkdir -p /root/ks; nohup bash -c "bash /root/stage-apply.sh; echo \$? > /root/ks/stage-apply-rc" \
>/root/ks/stage-apply.out 2>&1 &' \
>> "$ATTEMPT_DIR/attempt.log" 2>&1
CURRENT_STEP="poll-stage-apply"
apply_rc=""
# Poll every 30 s for up to 30 min (60 checks).
for tick in $(seq 0 59); do
sleep 30
sentinel=$(ssh "${SSH_OPTS[@]}" -o ConnectTimeout=10 root@"$boot_ip" \
'cat /root/ks/stage-apply-rc 2>/dev/null' 2>/dev/null || true)
echo "apply-poll=$tick sentinel=${sentinel:-pending}" | tee -a "$ATTEMPT_DIR/attempt.log"
if [[ -n "$sentinel" ]]; then
apply_rc="$sentinel"
break
fi
done
if [[ -z "$apply_rc" ]]; then
echo "apply-poll: timed out, script still active on guest" | tee -a "$ATTEMPT_DIR/attempt.log"
exit 23
fi
CURRENT_STEP="collect-stage-stdout"
ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'cat /root/ks/stage-apply.out 2>/dev/null || true' \
> "$ATTEMPT_DIR/stage-apply.stdout" 2>/dev/null || true
if [[ "$apply_rc" != "0" ]]; then
CURRENT_STEP="collect-stage-log-after-failure"
ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'cat /root/ks/stage-apply-stage'"$stage"'.log 2>/dev/null || true' \
> "$ATTEMPT_DIR/stage-apply.log" 2>/dev/null || true
exit 21
fi
CURRENT_STEP="collect-stage-log"
ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'cat /root/ks/stage-apply-stage'"$stage"'.log 2>/dev/null || true' \
> "$ATTEMPT_DIR/stage-apply.log" 2>/dev/null || true
CURRENT_STEP="collect-stage-script-list"
ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'cat /root/kickstart-debug/stage-'"$stage"'-scripts.txt 2>/dev/null || true' \
> "$ATTEMPT_DIR/stage-applied-scripts.txt" 2>/dev/null || true
stage_finished=0
if grep -q "Stage apply finished stage=$stage" "$ATTEMPT_DIR/stage-apply.log" 2>/dev/null; then
stage_finished=1
fi
stage_script_count="$(wc -l < "$ATTEMPT_DIR/stage-applied-scripts.txt" 2>/dev/null || echo 0)"
expected_new="$(expected_new_script)"
new_script_present=0
if [[ -n "${expected_new:-}" ]] && grep -Fxq "$expected_new" "$ATTEMPT_DIR/stage-applied-scripts.txt" 2>/dev/null; then
new_script_present=1
fi
CURRENT_STEP="request-guest-reboot"
ssh "${SSH_OPTS[@]}" root@"$boot_ip" \
'nohup bash -c "sleep 2; systemctl reboot || reboot || reboot -f" >/dev/null 2>&1 &' \
>/dev/null 2>&1 || true
CURRENT_STEP="wait-for-reboot-start"
reboot_started=0
for tick in $(seq 0 $(( REBOOT_START_CHECKS - 1 ))); do
if ! ssh "${SSH_OPTS[@]}" -o ConnectTimeout=5 root@"$boot_ip" \
'cat /proc/sys/kernel/random/boot_id >/dev/null 2>&1' \
>/dev/null 2>&1; then
reboot_started=1
echo "snapshot-reboot-started-check=$tick state=down" | tee -a "$ATTEMPT_DIR/attempt.log"
break
fi
current_boot_id="$(ssh "${SSH_OPTS[@]}" -o ConnectTimeout=5 root@"$boot_ip" \
'cat /proc/sys/kernel/random/boot_id 2>/dev/null || true' 2>/dev/null || true)"
echo "snapshot-reboot-started-check=$tick state=up boot_id=${current_boot_id:-unknown}" | tee -a "$ATTEMPT_DIR/attempt.log"
if [[ -n "$ORIGINAL_BOOT_ID" && -n "$current_boot_id" && "$current_boot_id" != "$ORIGINAL_BOOT_ID" ]]; then
reboot_started=1
echo "snapshot-reboot-started-check=$tick state=boot-id-changed" | tee -a "$ATTEMPT_DIR/attempt.log"
break
fi
sleep "$REBOOT_START_SLEEP_SEC"
done
if [[ "$reboot_started" -ne 1 ]]; then
echo "snapshot reboot never actually started" >&2
exit 24
fi
perl -0pi -e "s/VERIFY_REBOOT_STARTED='0'/VERIFY_REBOOT_STARTED='1'/" "$VERIFICATION_ENV"
reboot_ip=""
CURRENT_STEP="wait-for-reboot-ssh"
for tick in $(seq 0 $(( BOOT_CHECKS - 1 ))); do
ip="$(current_ip || true)"
echo "snapshot-reboot-check=$tick ip=${ip:-none}" | tee -a "$ATTEMPT_DIR/attempt.log"
if [[ -n "${ip:-}" ]]; then
if ssh "${SSH_OPTS[@]}" -o ConnectTimeout=5 root@"$ip" \
'echo reboot-ok; cat /proc/sys/kernel/random/boot_id; hostname; systemctl is-system-running || true' \
> "$ATTEMPT_DIR/snapshot-reboot-${tick}.ssh" 2>&1; then
current_boot_id="$(sed -n '2p' "$ATTEMPT_DIR/snapshot-reboot-${tick}.ssh" | tr -d '\r')"
if [[ -z "$ORIGINAL_BOOT_ID" || -z "$current_boot_id" || "$current_boot_id" != "$ORIGINAL_BOOT_ID" ]]; then
reboot_ip="$ip"
REBOOT_BOOT_ID="$current_boot_id"
REBOOT_SYSTEM_STATE="$(sed -n '4p' "$ATTEMPT_DIR/snapshot-reboot-${tick}.ssh" | tr -d '\r')"
break
fi
echo "snapshot-reboot-check=$tick boot_id_unchanged" | tee -a "$ATTEMPT_DIR/attempt.log"
fi
fi
sleep "$BOOT_SLEEP_SEC"
done
if [[ -z "$reboot_ip" ]]; then
echo "snapshot stage reboot did not return to ssh" >&2
exit 22
fi
cat > "$VERIFICATION_ENV" <<EOF
STAGE='$stage'
VERIFY_APPLY_RC='$apply_rc'
VERIFY_STAGE_FINISHED='$stage_finished'
VERIFY_EXPECTED_SCRIPT_COUNT='$(expected_script_count)'
VERIFY_EXPECTED_NEW_SCRIPT='$(expected_new_script)'
VERIFY_PAYLOAD_MODE='$PAYLOAD_MODE'
VERIFY_STAGE_SCRIPT_COUNT='$stage_script_count'
VERIFY_EXPECTED_NEW_SCRIPT_PRESENT='$new_script_present'
VERIFY_REBOOT_STARTED='1'
VERIFY_BOOT_ID_CHANGED='$([[ -n "$ORIGINAL_BOOT_ID" && -n "$REBOOT_BOOT_ID" && "$REBOOT_BOOT_ID" != "$ORIGINAL_BOOT_ID" ]] && echo 1 || echo 0)'
VERIFY_REBOOT_SSH='1'
VERIFY_REBOOT_SYSTEM_STATE='${REBOOT_SYSTEM_STATE:-unknown}'
VERIFY_REBOOT_IP='$reboot_ip'
VERIFIED_GOOD='1'
EOF
CURRENT_STEP="complete"
printf '%s\n' "$reboot_ip" >&3
tools/probe_stage_watch.sh
Executable
+525
View File
@@ -0,0 +1,525 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
VM=probe
VM_MAC=52:54:00:aa:bb:01
DISK=/var/lib/libvirt/images/probe.qcow2
ARTIFACT_ROOT=/tmp/probe-stage-watch
MEMORY_FILE="$ROOT/codex-running-memory.md"
LOCK_FILE="$ARTIFACT_ROOT/probe-stage-watch.lock"
STAGE="${1:-baseline}"
MAX_ATTEMPTS="${2:-100}"
INSTALL_CHECKS="${INSTALL_CHECKS:-90}"
INSTALL_SLEEP_SEC="${INSTALL_SLEEP_SEC:-30}"
BOOT_CHECKS="${BOOT_CHECKS:-30}"
BOOT_SLEEP_SEC="${BOOT_SLEEP_SEC:-10}"
FAIL_LOG="$ARTIFACT_ROOT/failures.md"
STATE_FILE="$ARTIFACT_ROOT/state.env"
LATEST_FILE="$ARTIFACT_ROOT/latest-summary.txt"
CURRENT_STAGE_META_FILE="$ARTIFACT_ROOT/current-stage-meta.txt"
CURRENT_STAGE_SCRIPTS_FILE="$ARTIFACT_ROOT/current-stage-scripts.txt"
TEMPLATE_META_FILE="$ARTIFACT_ROOT/baseline-template.env"
TRIAGE_FILE="$ARTIFACT_ROOT/needs-triage.env"
TOTAL_STAGES=0
MODE=""
CURRENT_SUSPECT="baseline"
LATEST_VERIFIED_STAGE=""
BASELINE_TEMPLATE=""
AUTO_TRIAGE_ON_FAILURE="${AUTO_TRIAGE_ON_FAILURE:-1}"
SUDO="${SUDO:-sudo}"
LIBVIRT_URI="${LIBVIRT_URI:-qemu:///system}"
VIRSH=( "$SUDO" virsh --connect "$LIBVIRT_URI" )
VIRT_INSTALL=( "$SUDO" virt-install --connect "$LIBVIRT_URI" )
mkdir -p "$ARTIFACT_ROOT"
exec 9>"$LOCK_FILE"
if ! flock -n 9; then
echo "[$(date '+%F %T')] watcher already running; lock held at $LOCK_FILE"
exit 0
fi
if [[ ! -f "$FAIL_LOG" ]]; then
cat > "$FAIL_LOG" <<'EOF'
# Probe Stage Failures
| Time | Attempt | Stage | Suspect Script | Outcome | Artifacts |
|---|---:|---|---|---|---|
EOF
fi
log() {
echo "[$(date '+%F %T')] $*"
}
capture_promoted_template() {
local label="$1"
local attempt="$2"
local attempt_dir="$3"
record_state "template-capture" "attempt=$attempt label=$label"
if ! "$ROOT/tools/probe_capture_baseline.sh" "$label" > "$attempt_dir/template-path.txt"; then
log "attempt=$attempt template capture failed label=$label" | tee -a "$attempt_dir/attempt.log"
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "template capture failed label=$label" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "template capture failed label=$label"
return 1
fi
update_mode
log "attempt=$attempt captured template label=$label path=${BASELINE_TEMPLATE:-unknown}" | tee -a "$attempt_dir/attempt.log"
return 0
}
stage_to_count() {
if [[ "$1" == "baseline" ]]; then
echo 0
else
echo "$1"
fi
}
next_stage() {
local current="$1"
if [[ "$current" == "baseline" ]]; then
echo 1
else
echo $(( current + 1 ))
fi
}
refresh_total_stages() {
TOTAL_STAGES="$("$ROOT/tools/run_probe_stage.sh" total)"
}
refresh_template_state() {
BASELINE_TEMPLATE=""
if [[ -f "$TEMPLATE_META_FILE" ]]; then
# shellcheck disable=SC1090
source "$TEMPLATE_META_FILE"
fi
}
update_mode() {
refresh_total_stages
refresh_template_state
if [[ "$STAGE" == "baseline" || -z "${BASELINE_TEMPLATE:-}" ]]; then
MODE="kickstart-baseline"
else
MODE="snapshot-stage"
fi
}
record_state() {
local phase="${1:-idle}"
local detail="${2:-}"
touch "$MEMORY_FILE"
cat > "$STATE_FILE" <<EOF
CURRENT_MODE='$MODE'
CURRENT_STAGE='$STAGE'
TOTAL_STAGES='$TOTAL_STAGES'
BASELINE_TEMPLATE='${BASELINE_TEMPLATE:-}'
LATEST_VERIFIED_STAGE='${LATEST_VERIFIED_STAGE:-}'
CURRENT_SUSPECT='${CURRENT_SUSPECT:-}'
LAST_UPDATED='$(date '+%F %T')'
CURRENT_PHASE='$phase'
CURRENT_DETAIL='$detail'
EOF
{
echo
echo '## Watcher State'
echo
echo "- Last updated: $(date '+%F %T')"
echo "- Current mode: $MODE"
echo "- Current stage: $STAGE"
echo "- Total staged scripts: $TOTAL_STAGES"
echo "- Baseline template: ${BASELINE_TEMPLATE:-none}"
echo "- Latest verified stage: ${LATEST_VERIFIED_STAGE:-none}"
echo "- Current suspect: ${CURRENT_SUSPECT:-baseline}"
echo "- Current phase: $phase"
echo "- Current detail: $detail"
} > "$ARTIFACT_ROOT/.state-snippet"
awk '
BEGIN {printed=0}
/^## Watcher State$/ {exit}
{print}
' "$MEMORY_FILE" 2>/dev/null > "$ARTIFACT_ROOT/.memory-base" || true
if [[ -s "$ARTIFACT_ROOT/.memory-base" ]]; then
cat "$ARTIFACT_ROOT/.memory-base" "$ARTIFACT_ROOT/.state-snippet" > "$MEMORY_FILE"
fi
{
echo "time=$(date '+%F %T')"
echo "mode=$MODE"
echo "stage=$STAGE"
echo "suspect=${CURRENT_SUSPECT:-baseline}"
echo "phase=$phase"
echo "detail=$detail"
} > "$LATEST_FILE"
}
append_failure() {
local attempt="$1"
local stage="$2"
local suspect="$3"
local outcome="$4"
local attempt_dir="$5"
printf '| %s | %s | %s | %s | %s | %s |\n' \
"$(date '+%F %T')" "$attempt" "$stage" "${suspect:-unknown}" "$outcome" "$attempt_dir" >> "$FAIL_LOG"
}
switch_to_hd_only() {
"${VIRSH[@]}" dumpxml "$VM" > /tmp/${VM}.xml
perl -0pi -e 's/\n\s*<boot dev=\x27network\x27\/>//g' /tmp/${VM}.xml
if ! grep -q "<boot dev='hd'/>" /tmp/${VM}.xml; then
perl -0pi -e 's#(<os[^>]*>\s*)#$1\n <boot dev=\x27hd\x27/>#s' /tmp/${VM}.xml
fi
"${VIRSH[@]}" define /tmp/${VM}.xml >/dev/null
}
capture_frame() {
local out="$1"
"${VIRSH[@]}" screenshot "$VM" "$out" >/dev/null 2>&1 || true
}
current_ip() {
local ip=""
ip=$("${VIRSH[@]}" domifaddr "$VM" --source arp 2>/dev/null | awk '/ipv4/ {sub(/\/.*$/, "", $4); print $4; exit}')
if [[ -z "$ip" ]]; then
ip=$(ip neigh show dev virbr1 2>/dev/null | awk -v mac="$VM_MAC" '$0 ~ mac {print $1; exit}')
fi
echo "$ip"
}
dump_http_tail() {
ssh -o BatchMode=yes [email protected] 'tail -n 80 /var/log/httpd/access_log' > "$1" 2>/dev/null || true
}
dump_domain_state() {
{
"${VIRSH[@]}" domstate "$VM" 2>/dev/null || true
echo
"${VIRSH[@]}" list --all 2>/dev/null || true
echo
"${VIRSH[@]}" domifaddr "$VM" --source arp 2>/dev/null || true
} > "$1"
}
payload_source_lines() {
local attempt_dir="$1"
local manifest="$attempt_dir/payload-manifest.txt"
local stage_apply="$attempt_dir/stage-apply.sh"
if [[ -f "$manifest" ]] && grep -q '^# Source: ' "$manifest"; then
grep '^# Source: ' "$manifest"
return 0
fi
if [[ -f "$stage_apply" ]] && grep -q '^# Source: ' "$stage_apply"; then
grep '^# Source: ' "$stage_apply"
return 0
fi
return 1
}
snapshot_payload_is_stale() {
local attempt_dir="$1"
local line source_path payload_sha current_sha
while IFS= read -r line; do
[[ "$line" =~ ^#\ Source:\ (.+)\ sha256=([0-9a-f]{64})$ ]] || continue
source_path="${BASH_REMATCH[1]}"
payload_sha="${BASH_REMATCH[2]}"
if [[ -f "$source_path" ]]; then
current_sha="$(sha256sum "$source_path" | awk '{print $1}')"
if [[ "$current_sha" != "$payload_sha" ]]; then
return 0
fi
fi
done < <(payload_source_lines "$attempt_dir" || true)
return 1
}
request_triage() {
local attempt="$1"
local attempt_dir="$2"
local outcome="$3"
cat > "$TRIAGE_FILE" <<EOF
TRIAGE_REQUESTED_AT='$(date '+%F %T')'
TRIAGE_STAGE='$STAGE'
TRIAGE_MODE='$MODE'
TRIAGE_ATTEMPT='$attempt'
TRIAGE_SUSPECT='$CURRENT_SUSPECT'
TRIAGE_OUTCOME='$outcome'
TRIAGE_ATTEMPT_DIR='$attempt_dir'
EOF
record_state "triage-requested" "attempt=$attempt stage=$STAGE suspect=$CURRENT_SUSPECT outcome=$outcome"
log "attempt=$attempt triage requested for stage=$STAGE suspect=$CURRENT_SUSPECT outcome=$outcome" | tee -a "$attempt_dir/attempt.log"
if [[ "$AUTO_TRIAGE_ON_FAILURE" == "1" && -x "$ROOT/tools/llm_failure_router.sh" ]]; then
"$ROOT/tools/llm_failure_router.sh" >> "$attempt_dir/attempt.log" 2>&1 || true
fi
}
clear_triage_if_matches_stage() {
[[ -f "$TRIAGE_FILE" ]] || return 0
local triage_stage="" triage_mode="" triage_suspect=""
# shellcheck disable=SC1090
source "$TRIAGE_FILE"
triage_stage="${TRIAGE_STAGE:-}"
triage_mode="${TRIAGE_MODE:-}"
triage_suspect="${TRIAGE_SUSPECT:-}"
if [[ "$triage_stage" == "$STAGE" && "$triage_mode" == "$MODE" && "$triage_suspect" == "$CURRENT_SUSPECT" ]]; then
rm -f "$TRIAGE_FILE"
fi
}
write_stage_files() {
local attempt_dir="$1"
local stage_meta="$("$ROOT/tools/run_probe_stage.sh" "$STAGE")"
local included_scripts="$("$ROOT/tools/run_probe_stage.sh" included "$STAGE")"
printf '%s\n' "$stage_meta" | tee -a "$attempt_dir/attempt.log"
printf '%s\n' "$stage_meta" > "$attempt_dir/stage-meta.txt"
printf '%s\n' "$stage_meta" > "$CURRENT_STAGE_META_FILE"
{
echo "# Included scripts for stage=$STAGE"
echo "# Generated: $(date '+%F %T')"
echo
printf '%s\n' "$included_scripts"
} > "$attempt_dir/included-scripts.txt"
cp "$attempt_dir/included-scripts.txt" "$CURRENT_STAGE_SCRIPTS_FILE"
CURRENT_SUSPECT="$(printf '%s\n' "$stage_meta" | awk -F= '/^new_script=/{print $2; exit}')"
if [[ -z "${CURRENT_SUSPECT:-}" ]]; then
CURRENT_SUSPECT="baseline"
fi
}
run_kickstart_attempt() {
local attempt="$1"
local attempt_dir="$2"
local state ip minute tick ssh_target
local install_done=0
local boot_ip=""
local last_seen_ip=""
"${VIRSH[@]}" destroy "$VM" >/dev/null 2>&1 || true
"${VIRSH[@]}" undefine "$VM" --remove-all-storage --nvram >/dev/null 2>&1 || true
rm -f "$DISK"
"${VIRT_INSTALL[@]}" \
--name "$VM" \
--memory 4096 \
--vcpus 2 \
--disk "$DISK",size=80,format=qcow2 \
--network network=pxe-lab,model=virtio,mac="$VM_MAC" \
--os-variant almalinux10 \
--graphics vnc,listen=127.0.0.1 \
--machine q35 \
--features smm.state=on \
--boot loader=/usr/share/OVMF/OVMF_CODE_4M.secboot.fd,loader.readonly=yes,loader.type=pflash,loader.secure=yes,nvram.template=/usr/share/OVMF/OVMF_VARS_4M.ms.fd,network,hd \
--pxe \
--noautoconsole | tee -a "$attempt_dir/attempt.log"
for minute in $(seq 0 $(( INSTALL_CHECKS - 1 ))); do
ts=$(printf '%02d' "$minute")
capture_frame "$attempt_dir/install-${ts}.png"
dump_domain_state "$attempt_dir/install-${ts}.state"
dump_http_tail "$attempt_dir/install-${ts}.http"
state=$("${VIRSH[@]}" domstate "$VM" 2>/dev/null || echo missing)
ip=$(current_ip || true)
record_state "install" "attempt=$attempt check=$minute state=$state ip=${ip:-none}"
log "attempt=$attempt install-check=$minute state=$state ip=${ip:-none}" | tee -a "$attempt_dir/attempt.log"
if [[ "$state" == "shut off" ]]; then
install_done=1
break
fi
sleep "$INSTALL_SLEEP_SEC"
done
if [[ "$install_done" -ne 1 ]]; then
record_state "install-timeout" "attempt=$attempt checks=$INSTALL_CHECKS interval=${INSTALL_SLEEP_SEC}s"
log "attempt=$attempt install never shut down within bootstrap window (${INSTALL_CHECKS} checks x ${INSTALL_SLEEP_SEC}s)" | tee -a "$attempt_dir/attempt.log"
"${VIRSH[@]}" destroy "$VM" >/dev/null 2>&1 || true
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "install timeout/no shutdown" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "install timeout/no shutdown"
return 1
fi
log "attempt=$attempt installer shutdown detected; switching to disk boot" | tee -a "$attempt_dir/attempt.log"
record_state "disk-boot" "attempt=$attempt switching-to-hd"
switch_to_hd_only
"${VIRSH[@]}" start "$VM" >/dev/null
for tick in $(seq 0 $(( BOOT_CHECKS - 1 ))); do
ts=$(printf '%02d' "$tick")
capture_frame "$attempt_dir/boot-${ts}.png"
dump_domain_state "$attempt_dir/boot-${ts}.state"
ip=$(current_ip || true)
if [[ -n "${ip:-}" ]]; then
last_seen_ip="$ip"
fi
ssh_target="${ip:-$last_seen_ip}"
record_state "boot" "attempt=$attempt check=$tick ip=${ip:-none} target=${ssh_target:-none}"
log "attempt=$attempt boot-check=$tick ip=${ip:-none} target=${ssh_target:-none}" | tee -a "$attempt_dir/attempt.log"
if [[ -n "${ssh_target:-}" ]]; then
if ssh \
-o BatchMode=yes \
-o ConnectTimeout=5 \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o GlobalKnownHostsFile=/dev/null \
-o LogLevel=ERROR \
root@"$ssh_target" \
'echo ssh-ok; hostname; systemctl is-system-running || true' \
> "$attempt_dir/boot-${ts}.ssh" 2>&1; then
boot_ip="$ssh_target"
break
fi
fi
sleep "$BOOT_SLEEP_SEC"
done
if [[ -z "$boot_ip" ]]; then
log "attempt=$attempt first boot failed; collecting cold artifacts" | tee -a "$attempt_dir/attempt.log"
record_state "boot-failed" "attempt=$attempt last_ip=${last_seen_ip:-none} checks=$BOOT_CHECKS interval=${BOOT_SLEEP_SEC}s"
"${VIRSH[@]}" destroy "$VM" >/dev/null 2>&1 || true
sudo virt-cat -a "$DISK" -m /dev/sda5:/ /var/log/boot.log > "$attempt_dir/boot.log" 2>/dev/null || true
sudo virt-cat -a "$DISK" -m /dev/sda5:/ /root/ks/ks-post-chrt-custom-$(date +%F).log > "$attempt_dir/ks-post.log" 2>/dev/null || true
sudo virt-ls -a "$DISK" -m /dev/sda1:/ -R /EFI > "$attempt_dir/efi-tree.txt" 2>/dev/null || true
sudo virt-ls -a "$DISK" -m /dev/sda2:/ -R /loader > "$attempt_dir/loader-tree.txt" 2>/dev/null || true
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "boot failed/no ssh" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "boot failed/no ssh"
return 1
fi
log "attempt=$attempt SUCCESS ip=$boot_ip" | tee -a "$attempt_dir/attempt.log"
record_state "boot-success" "attempt=$attempt ip=$boot_ip"
return 0
}
run_snapshot_attempt() {
local attempt="$1"
local attempt_dir="$2"
local ip=""
local rc=0
local outcome=""
local verification_file="$attempt_dir/verification.env"
record_state "snapshot-stage" "attempt=$attempt stage=$STAGE"
if ip="$("$ROOT/tools/probe_snapshot_stage.sh" "$STAGE" "$attempt_dir")"; then
if [[ ! -f "$verification_file" ]]; then
outcome="snapshot verification missing"
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "$outcome" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "$outcome"
return 1
fi
# shellcheck disable=SC1090
source "$verification_file"
if [[ "${VERIFIED_GOOD:-0}" != "1" \
|| "${VERIFY_APPLY_RC:-}" != "0" \
|| "${VERIFY_STAGE_FINISHED:-0}" != "1" \
|| "${VERIFY_EXPECTED_NEW_SCRIPT_PRESENT:-0}" != "1" \
|| "${VERIFY_REBOOT_SSH:-0}" != "1" \
|| "${VERIFY_BOOT_ID_CHANGED:-0}" != "1" ]]; then
outcome="snapshot verification gate failed"
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "$outcome" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "$outcome"
return 1
fi
log "attempt=$attempt snapshot SUCCESS ip=$ip" | tee -a "$attempt_dir/attempt.log"
record_state "snapshot-success" "attempt=$attempt ip=$ip"
return 0
else
rc=$?
record_state "snapshot-failed" "attempt=$attempt rc=$rc"
case "$rc" in
20) outcome="snapshot boot failed/no ssh" ;;
21)
if snapshot_payload_is_stale "$attempt_dir"; then
outcome="snapshot stage apply failed (stale payload drift)"
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "$outcome" "$attempt_dir"
clear_triage_if_matches_stage
record_state "snapshot-stale" "attempt=$attempt stage=$STAGE suspect=$CURRENT_SUSPECT"
log "attempt=$attempt snapshot payload stale relative to repo; retrying stage=$STAGE" | tee -a "$attempt_dir/attempt.log"
return 10
fi
outcome="snapshot stage apply failed"
;;
22) outcome="snapshot reboot failed/no ssh" ;;
23)
outcome="snapshot stage apply still running"
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "$outcome" "$attempt_dir"
record_state "snapshot-apply-pending" "attempt=$attempt stage=$STAGE suspect=$CURRENT_SUSPECT"
log "attempt=$attempt stage=$STAGE apply still running on guest; supervisor will check" | \
tee -a "$attempt_dir/attempt.log"
request_triage "$attempt" "$attempt_dir" "$outcome"
return 1
;;
*) outcome="snapshot stage failed rc=$rc" ;;
esac
append_failure "$attempt" "$STAGE" "$CURRENT_SUSPECT" "$outcome" "$attempt_dir"
request_triage "$attempt" "$attempt_dir" "$outcome"
return 1
fi
}
for attempt in $(seq 1 "$MAX_ATTEMPTS"); do
update_mode
record_state "attempt-start" "attempt=$attempt"
attempt_dir="$ARTIFACT_ROOT/$(date '+%Y%m%d-%H%M%S')-attempt-${attempt}"
mkdir -p "$attempt_dir"
log "attempt=$attempt mode=$MODE stage=$STAGE start" | tee -a "$attempt_dir/attempt.log"
write_stage_files "$attempt_dir"
record_state "attempt-prep" "attempt=$attempt mode=$MODE stage=$STAGE suspect=$CURRENT_SUSPECT"
if [[ "$MODE" == "kickstart-baseline" ]]; then
if ! run_kickstart_attempt "$attempt" "$attempt_dir"; then
exit 2
fi
if [[ "$STAGE" == "baseline" ]]; then
if ! capture_promoted_template "baseline" "$attempt" "$attempt_dir"; then
exit 2
fi
fi
else
if run_snapshot_attempt "$attempt" "$attempt_dir"; then
rc=0
else
rc=$?
fi
if [[ "$rc" -ne 0 ]]; then
if [[ "$rc" == "10" ]]; then
continue
fi
exit 2
fi
fi
log "attempt=$attempt stage=$STAGE verified good" | tee -a "$attempt_dir/attempt.log"
clear_triage_if_matches_stage
LATEST_VERIFIED_STAGE="$STAGE"
record_state "stage-good" "attempt=$attempt stage=$STAGE"
if [[ "$MODE" == "snapshot-stage" ]]; then
if ! capture_promoted_template "stage-${STAGE}" "$attempt" "$attempt_dir"; then
exit 2
fi
fi
current_count="$(stage_to_count "$STAGE")"
if [[ "$current_count" -ge "$TOTAL_STAGES" ]]; then
log "full staged suite passed through stage=$STAGE" | tee -a "$attempt_dir/attempt.log"
record_state "full-pass" "attempt=$attempt stage=$STAGE"
exit 0
fi
STAGE="$(next_stage "$STAGE")"
update_mode
log "promoting to next stage=$STAGE mode=$MODE" | tee -a "$attempt_dir/attempt.log"
record_state "promote" "next-stage=$STAGE next-mode=$MODE"
done
exit 1
tools/restart_probe_watcher.sh
Executable
+173
View File
@@ -0,0 +1,173 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
WATCH_ROOT=/tmp/probe-stage-watch
STATE_FILE="$WATCH_ROOT/state.env"
TRIAGE_FILE="$WATCH_ROOT/needs-triage.env"
RESTART_LOG="$WATCH_ROOT/restart.log"
NOHUP_LOG="$WATCH_ROOT/auto-restart.nohup"
LOCK_FILE="$WATCH_ROOT/probe-stage-watch.lock"
ACTIVE_STATE_MAX_AGE="${ACTIVE_STATE_MAX_AGE:-180}"
TRIAGE_RETRY_AFTER_SEC="${TRIAGE_RETRY_AFTER_SEC:-600}"
LIBVIRT_URI="${LIBVIRT_URI:-qemu:///system}"
timestamp() {
date '+%F %T'
}
state_age_seconds() {
if [[ ! -f "$STATE_FILE" ]]; then
return 1
fi
local state_epoch
state_epoch="$(stat -c %Y "$STATE_FILE" 2>/dev/null || echo 0)"
[[ "$state_epoch" =~ ^[0-9]+$ ]] || return 1
[[ "$state_epoch" -gt 0 ]] || return 1
echo "$(( $(date +%s) - state_epoch ))"
}
state_phase_is_active() {
case "${1:-unknown}" in
attempt-start|attempt-prep|install|disk-boot|boot|baseline-capture|snapshot-stage|promote)
return 0
;;
*)
return 1
;;
esac
}
triage_age_seconds() {
if [[ ! -f "$TRIAGE_FILE" ]]; then
return 1
fi
local triage_epoch
triage_epoch="$(stat -c %Y "$TRIAGE_FILE" 2>/dev/null || echo 0)"
[[ "$triage_epoch" =~ ^[0-9]+$ ]] || return 1
[[ "$triage_epoch" -gt 0 ]] || return 1
echo "$(( $(date +%s) - triage_epoch ))"
}
watcher_process_running() {
ps -ef | grep -q '[p]robe_stage_watch.sh'
}
payload_source_lines() {
local attempt_dir="$1"
local manifest="$attempt_dir/payload-manifest.txt"
local stage_apply="$attempt_dir/stage-apply.sh"
if [[ -f "$manifest" ]] && grep -q '^# Source: ' "$manifest"; then
grep '^# Source: ' "$manifest"
return 0
fi
if [[ -f "$stage_apply" ]] && grep -q '^# Source: ' "$stage_apply"; then
grep '^# Source: ' "$stage_apply"
return 0
fi
return 1
}
triage_payload_is_stale() {
local attempt_dir="$1"
local line source_path payload_sha current_sha
[[ -d "$attempt_dir" ]] || return 1
while IFS= read -r line; do
[[ "$line" =~ ^#\ Source:\ (.+)\ sha256=([0-9a-f]{64})$ ]] || continue
source_path="${BASH_REMATCH[1]}"
payload_sha="${BASH_REMATCH[2]}"
if [[ -f "$source_path" ]]; then
current_sha="$(sha256sum "$source_path" | awk '{print $1}')"
if [[ "$current_sha" != "$payload_sha" ]]; then
return 0
fi
fi
done < <(payload_source_lines "$attempt_dir" || true)
return 1
}
mkdir -p "$WATCH_ROOT"
exec 9>"$LOCK_FILE"
if ! flock -n 9; then
echo "[$(timestamp)] watcher already running (lock held)" >> "$RESTART_LOG"
exit 0
fi
flock -u 9
if watcher_process_running; then
echo "[$(timestamp)] watcher already running (ps)" >> "$RESTART_LOG"
exit 0
fi
stage="baseline"
phase="unknown"
mode="unknown"
suspect="baseline"
if [[ -f "$STATE_FILE" ]]; then
# shellcheck disable=SC1090
source "$STATE_FILE"
stage="${CURRENT_STAGE:-baseline}"
phase="${CURRENT_PHASE:-unknown}"
mode="${CURRENT_MODE:-unknown}"
suspect="${CURRENT_SUSPECT:-baseline}"
fi
if [[ "$phase" == "full-pass" ]]; then
echo "[$(timestamp)] restart skipped phase=$phase" >> "$RESTART_LOG"
exit 0
fi
if [[ -f "$TRIAGE_FILE" ]]; then
# shellcheck disable=SC1090
source "$TRIAGE_FILE"
triage_age="$(triage_age_seconds || true)"
if triage_payload_is_stale "${TRIAGE_ATTEMPT_DIR:-}"; then
echo "[$(timestamp)] clearing stale triage for attempt_dir=${TRIAGE_ATTEMPT_DIR:-unknown}" >> "$RESTART_LOG"
rm -f "$TRIAGE_FILE"
elif [[ "$phase" == "snapshot-stale" \
&& "${TRIAGE_STAGE:-}" == "$stage" \
&& "${TRIAGE_MODE:-}" == "$mode" \
&& "${TRIAGE_SUSPECT:-}" == "$suspect" ]]; then
echo "[$(timestamp)] clearing obsolete triage superseded by state phase=$phase stage=$stage suspect=$suspect" >> "$RESTART_LOG"
rm -f "$TRIAGE_FILE"
else
case "${TRIAGE_OUTCOME:-}" in
watcher\ missing/stale\ state|watcher\ running/state\ stale)
rm -f "$TRIAGE_FILE"
;;
snapshot\ stage\ apply\ failed|snapshot\ boot\ failed/no\ ssh)
# No active watcher (verified above). If the attempt has no source-hash
# lines, staleness cannot be determined — clear triage so a post-fix
# repo state can resume. When source lines DO exist, triage_payload_is_stale
# already cleared the file above on any drift. If the triage has sat for
# too long with no watcher, force a fresh retry rather than idling forever.
if ! payload_source_lines "${TRIAGE_ATTEMPT_DIR:-}" &>/dev/null; then
echo "[$(timestamp)] clearing unverifiable snapshot triage outcome=${TRIAGE_OUTCOME:-unknown} attempt_dir=${TRIAGE_ATTEMPT_DIR:-unknown}" >> "$RESTART_LOG"
rm -f "$TRIAGE_FILE"
elif [[ -n "${triage_age:-}" && "$triage_age" -ge "$TRIAGE_RETRY_AFTER_SEC" ]]; then
echo "[$(timestamp)] clearing aged snapshot triage outcome=${TRIAGE_OUTCOME:-unknown} after ${triage_age}s attempt_dir=${TRIAGE_ATTEMPT_DIR:-unknown}" >> "$RESTART_LOG"
rm -f "$TRIAGE_FILE"
else
echo "[$(timestamp)] restart skipped triage_outcome=${TRIAGE_OUTCOME:-unknown}" >> "$RESTART_LOG"
exit 0
fi
;;
*)
echo "[$(timestamp)] restart skipped triage_outcome=${TRIAGE_OUTCOME:-unknown}" >> "$RESTART_LOG"
exit 0
;;
esac
fi
fi
nohup env LIBVIRT_DEFAULT_URI="$LIBVIRT_URI" LIBVIRT_URI="$LIBVIRT_URI" \
"$ROOT/tools/probe_stage_watch.sh" "$stage" >> "$NOHUP_LOG" 2>&1 &
echo "[$(timestamp)] watcher restarted stage=$stage phase=$phase pid=$!" >> "$RESTART_LOG"
tools/run_probe_stage.sh
Executable
+145
View File
@@ -0,0 +1,145 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=/usr/local/valherun/pxe_boot_alma
CFG_DIR="$ROOT/work/KickStart/cfg"
KS_DIR="$CFG_DIR/AlmaLinux-10.1-x86_64/ks.d"
STAG_DIR="$CFG_DIR/stag.d"
REMOTE=[email protected]
REMOTE_CFG=/home/KickStart/cfg
VM_MAC=52:54:00:aa:bb:01
MAC_FILE_RAW=${VM_MAC}.cfg
SKIP_FILE="${PROBE_STAGE_SKIP_FILE:-$ROOT/tools/probe-stage-skip.txt}"
BASE_SCRIPTS=(
P200A-diskpart
S110A-baseline-users-ssh
)
ALLOW_BASE="$(IFS=,; echo "${BASE_SCRIPTS[*]}")"
stage="${1:-baseline}"
emit_included_scripts() {
local include_count="$1"
printf '%s\n' "${BASE_SCRIPTS[@]}"
if [[ "$include_count" -gt 0 ]]; then
printf '%s\n' "${REAL_SCRIPTS[@]:0:include_count}"
fi
}
emit_extra_scripts() {
local include_count="$1"
if [[ "$include_count" -gt 0 ]]; then
printf '%s\n' "${REAL_SCRIPTS[@]:0:include_count}"
fi
}
SKIP_PATTERNS=()
if [[ -f "$SKIP_FILE" ]]; then
mapfile -t SKIP_PATTERNS < <(grep -Ev '^\s*(#|$)' "$SKIP_FILE" || true)
fi
mapfile -t REAL_SCRIPTS < <(
cd "$KS_DIR"
for script in S*; do
[[ "$script" =~ ^S[0-9]+[APS]- ]] || continue
skip=0
for base_script in "${BASE_SCRIPTS[@]}"; do
if [[ "$script" == "$base_script" ]]; then
skip=1
break
fi
done
if [[ "$skip" -eq 0 && "${#SKIP_PATTERNS[@]}" -gt 0 ]]; then
for skip_pattern in "${SKIP_PATTERNS[@]}"; do
if [[ "$script" == "$skip_pattern" ]]; then
skip=1
break
fi
done
fi
[[ "$skip" -eq 0 ]] && printf '%s\n' "$script"
done | sort
)
case "$stage" in
total)
echo "${#REAL_SCRIPTS[@]}"
exit 0
;;
list)
printf '%s\n' "${REAL_SCRIPTS[@]}"
exit 0
;;
extras)
include_stage="${2:-baseline}"
case "$include_stage" in
baseline)
include_count=0
;;
''|*[!0-9]*)
echo "usage: $0 extras baseline|<count>" >&2
exit 1
;;
*)
include_count="$include_stage"
;;
esac
emit_extra_scripts "$include_count"
exit 0
;;
included)
include_stage="${2:-baseline}"
case "$include_stage" in
baseline)
include_count=0
;;
''|*[!0-9]*)
echo "usage: $0 included baseline|<count>" >&2
exit 1
;;
*)
include_count="$include_stage"
;;
esac
emit_included_scripts "$include_count"
exit 0
;;
baseline)
count=0
;;
''|*[!0-9]*)
echo "usage: $0 baseline|<count>" >&2
exit 1
;;
*)
count="$stage"
;;
esac
allowlist="$ALLOW_BASE"
new_script=""
if [[ "$count" -gt 0 ]]; then
extras=("${REAL_SCRIPTS[@]:0:count}")
if [[ "${#extras[@]}" -gt 0 ]]; then
allowlist="${allowlist},$(IFS=,; echo "${extras[*]}")"
new_script="${extras[-1]}"
fi
fi
mkdir -p "$STAG_DIR"
cat > "$STAG_DIR/$MAC_FILE_RAW" <<EOF
DIST='AlmaLinux-10.1-x86_64'
TYPE='probe'
DISKTYPE='ssd'
KS_ALLOWLIST='${allowlist}'
EOF
ssh -q "$REMOTE" "mkdir -p '$REMOTE_CFG/stag.d' && cat > '$REMOTE_CFG/stag.d/$MAC_FILE_RAW'" < "$STAG_DIR/$MAC_FILE_RAW"
echo "stage=$stage"
echo "mac=$VM_MAC"
echo "allowlist=$allowlist"
echo "new_script=${new_script:-baseline}"
echo "real_script_count=${#REAL_SCRIPTS[@]}"
echo "skip_file=$SKIP_FILE"
echo "remote_stag=$REMOTE_CFG/stag.d/$MAC_FILE_RAW"
work/KickStart/HEADER.html
+5
View File
@@ -0,0 +1,5 @@
<pre>
This box is the property of iGLASS NETWORKS.
Access by non-iGLASS NETWORKS employees is strictly forbidden.
</pre>
work/KickStart/cfg/AlmaLinux-10.1-x86_64/KS_Packages-VM.cfg
View File
Whitespace-only changes.
work/KickStart/cfg/AlmaLinux-10.1-x86_64/KS_Packages-probe.cfg
+1
View File
@@ -0,0 +1 @@
NetworkManager-config-server
work/KickStart/cfg/AlmaLinux-10.1-x86_64/KS_Packages-server.cfg
+1
View File
@@ -0,0 +1 @@
NetworkManager-config-server
work/KickStart/cfg/AlmaLinux-10.1-x86_64/KS_Packages.cfg
+152
View File
@@ -0,0 +1,152 @@
@Server
@Development Tools
@Security Tools
@Container Management
@Console Internet Tools
@Headless Management
@Legacy UNIX Compatibility
@System Tools
kexec-tools
policycoreutils-restorecond
ruby-libs
libcmpiCppImpl0
cim-schema
libwsman1
sblim-sfcb
sblim-sfcc
libnet
nmap
dos2unix
openssl-devel
perl-Scalar-List-Utils
perl-ExtUtils-MakeMaker
perl-ExtUtils-CBuilder
perl-URI
perl-IO-Socket-SSL
perl-CGI
perl-FCGI
perl-Digest-MD5
perl-threads-shared
perl-threads
perl-CPAN
perl-YAML
perl-Test-Simple
perl-LWP-Protocol-https
perl-Mozilla-CA
perl-HTTP-Cookies
perl-HTTP-Message
perl-HTTP-Date
perl-HTTP-Negotiate
perl-Sys-Syslog
perl-File-Listing
perl-Compress-Raw-Bzip2
perl-Compress-Raw-Zlib
perl-IO-Compress
perl-LWP-MediaTypes
perl-Net-HTTP
perl-WWW-RobotRules
perl-libwww-perl
perl-Module-CoreList
perl-Locale-Maketext
perl-Module-Metadata
perl-Module-Build
perl-Module-Load
perl-Module-Load-Conditional
perl-IPC-Cmd
perl-Params-Check
perl-CPAN-Meta-YAML
perl-Text-Soundex
perl-JSON
perl-XML-Simple
perl-XML-LibXML
perl-XML-SAX
perl-Digest-HMAC
perl-MailTools
perl-Socket
perl-Socket6
perl-GSSAPI
perl-Authen-SASL
perl-Storable
perl-DBI
unixODBC
uuid
unbound
libuuid-devel
samba
samba-client
samba-winbind
samba-winbind-clients
lshw
traceroute
gmp-devel
net-snmp
net-snmp-utils
net-snmp-perl
net-snmp-devel
ipmitool
aide
libxml2
-initial-setup
-initial-setup-gui
-gnome-initial-setup
tmpwatch
mariadb
net-tools
ipa-client
krb5-workstation
krb5-devel
libreswan
-fprintd
-fprintd-pam
-gdm-plugin-fingerprint
libicu
perl-App-cpanminus
libXaw
libxslt
xdg-utils
alsa-lib
libaio
tcpdump
liberation-fonts
liberation-mono-fonts
liberation-sans-fonts
liberation-fonts-common
liberation-narrow-fonts
liberation-serif-fonts
tmux
clang
clang-devel
clang-tools-extra
cmake-filesystem
libicu-devel
llvm
llvm-devel
jq
exchange-bmc-os-info
-PackageKit-command-not-found
-cockpit
-abrt
-abrt-addon-ccpp
-abrt-addon-kerneloops
-abrt-addon-pstoreoops
-abrt-addon-python
-abrt-addon-vmcore
-abrt-addon-xorg
-abrt-cli
-abrt-console-notification
-abrt-dbus
-abrt-desktop
-abrt-gui
-abrt-gui-libs
-abrt-libs
-abrt-python
-abrt-retrace-client
-abrt-tui
-mlocate
yum-utils
lz4
ipcalc
setools-console
glibc-langpack-en
s-nail
postfix
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.config
+16
View File
@@ -0,0 +1,16 @@
# DISKTYPE = sd, hd, or hdc
DISKTYPE=
INTERACTIVE=n
TZ=America/New_York
# python -c "import crypt,getpass; print crypt.crypt(getpass.getpass(), \"\$6\$$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 16 | head -n 1)\")"
MD5PASS='$6$SUoLupc1M1hqE63G$QeJmvis441Nwruw88Q0/8a2Q1V0nQA.qE/We7VN3IfDWaSym3.VjUGLQ664UFlZJiHIWlU8KgGLNY31pbIfaK/'
# grub2-mkpasswd-pbkdf2
GRUB2=grub.pbkdf2.sha512.10000.E3FA14561B8830F9034B3D3F259658BA1D00EAFFBC505C2C9DC31C585765DB0834776E4E29BF7DE5C8CE209A90B1348E8BDEA9CB589E361EE26998546CDCFC2B.D5ED4872522909DF2A2D18E48CDC5DA47D5E6660276C44D62066072F1A3AA7165B28471FBE97E4B70083AF1A65341653D0E184D532E958FE06FD198863C8D1B0
NOCPASS='$6$eZmWKt/UxYZVe.Jb$XwwjHjXvBhDxrEiWNQ6yCUw90och0V1SpRnasYKvPf/fOfHPQcJauSl1vo5p3gXTinSV..WJljtfIVucM.Ogi0'
FIREWALL=n
SELINUX=enforcing
REBOOT=y
PRE=y
POST=y
POSTNR=y
FOO=BAR
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/.P200A-diskpart.swn
BIN
View File
Binary file not shown.
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/HEADER.html
+31
View File
@@ -0,0 +1,31 @@
<pre>
This box is the property of iGLASS NETWORKS.
Access by non-iGLASS NETWORKS employees is strictly forbidden.
iGlass Network KickStart config files:
Naming syntax = SDDDT-discription
S, Stage:
P Pre
S Post with chroot
N Post no chroot
DDD Order
T, Type
A All
P Probe +A
D DS +A
S Server +A
G Plug -A
V Virtual/Remote -A
For Virtual/Remote:
obelisk$ sudo ssh -R localhost:8080:localhost:80 noc@vmhost
obelisk$ sudo ssh -R localhost:8081:localhost:22 noc@vmhost
wget -q -O - http://localhost:8080/KickStart/cfg/ks.cgi?TYPE=VM
</pre>
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/N000A-start
+14
View File
@@ -0,0 +1,14 @@
echo "Trying to start up sshd in post NO chroot" #LOG
mkdir -p /var/empty/sshd
grep sshd /mnt/sysimage/etc/passwd >> /etc/passwd
grep root /mnt/sysimage/etc/shadow >> /etc/shadow
grep sshd /mnt/sysimage/etc/shadow >> /etc/shadow
ln -s /mnt/sysimage/etc/ssh/ /etc/
ln -s /usr/bin/bash /bin/bash
ln -s /mnt/sysimage/var/log/ /var/log
ln -s /mnt/sysimage/var/run/utmp /var/run/
/mnt/sysimage/usr/sbin/sshd -4 -p 222 -o "PermitRootLogin yes" -o "UsePAM no" -o "UseDNS no" 2>&1 #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/N100A-cpcfg
+3
View File
@@ -0,0 +1,3 @@
mkdir /mnt/sysimage/root/ks
cp -af /tmp/ks* /mnt/sysimage/root/ks/
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/N999D-sleep
+11
View File
@@ -0,0 +1,11 @@
SLEEP='5d'
echo "Setting hostname..." #LOG
if [ -e "/etc/sysconfig/network" ]; then
. /etc/sysconfig/network
hostname $HOSTNAME #LOG
fi
echo "Waiting $SLEEP for whatever..." #LOG
bash -c "sleep $SLEEP"
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/P000A-start
+7
View File
@@ -0,0 +1,7 @@
# wget -q -O - http://$KSRV/KickStart/cfg/ksid.py | python #LOG
# This is just for debuging very early stuff (disk partitioning)
# Disable as its an open root login
echo "Starting sshd" #LOG
/sbin/sshd -f /etc/ssh/sshd_config.anaconda
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/P100A-vncdisplay
+11
View File
@@ -0,0 +1,11 @@
echo "Use local display or VNC connect back."
read -t 5 -p "--Continue with local display? [n] " VNC
if [ "$VNC" = "" -o "${VNC:0:1}" = "n" -o "${VNC:0:1}" = "N" ]; then
$LOGGER Starting VNC connect back
echo "vnc --host $KSRV --port 5500 --password=kickMe" > /tmp/vnc.cfg
else
$LOGGER Using local dislpay
echo "# VNC bypassed using local display " > /tmp/vnc.cfg
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/P200A-diskpart
+182
View File
@@ -0,0 +1,182 @@
KSPART=/tmp/ks-diskpart.cfg
BOOT=1024 # MB
EFI=600
#OPTIG=250 # MB
#OPTIG=61440 # MB = 57G
# Different sizes below
OPTIG=51200 # MB
SMALL=
DEVA=
DEVB=
for DSK in $(cd /sys/block; ls -d sd* 2>/dev/null; ls -d hd* 2>/dev/null; ls -d nvme* 2>/dev/null; ls -d vd* 2>/dev/null); do
RMBL=$(cat /sys/block/$DSK/removable)
DSIZE=$(( $(blockdev --getsize64 /dev/$DSK) / 1000000 )) # MB
MODEL=$(cat /sys/block/$DSK/device/model)
if [ $RMBL -eq 1 ]; then
echo "Disk $DSK is removable, skipping." #LOG
elif [ -z "$DSIZE" ]; then
echo "Error: Size for disk $DSK could not be found, skipping." #LOG
elif [ -z "$DEVA" ]; then
if [ $DSIZE -lt 70000 -a "$PTYPE" != "server" ]; then
echo "Warning: Size for disk $DSK ($DSIZE) is to small but ok!" #LOG
SMALL=1
OPTIG=5120 # MB
elif [ $DSIZE -gt 400000 -a "$PTYPE" != "server" ]; then
echo "Big disk 2: Size for disk $DSK ($DSIZE)" #LOG
OPTIG=256000 # MB
elif [ $DSIZE -gt 200000 -a "$PTYPE" != "server" ]; then
echo "Big disk 1: Size for disk $DSK ($DSIZE)" #LOG
OPTIG=102400 # MB
fi
DEVA=$DSK
elif [ -z "$DEVB" ]; then
DEVB=$DSK
else
echo "Skipping disk $DSK ($MODEL) of size $DSIZE because already have $DEVA and $DEVB" #LOG
fi
#This doesn't appear to be installed yet. Added later
#tune4fs -c 0 -i 0 /dev/sda4 #LOG
done
if [ -z "$DEVA" ]; then
echo "Error: No Disks Found!!!" #LOG
else
echo "Partitioning first disk $DEVA ($MODEL) of size $DSIZE swap at $SWAP" #LOG
# NVMe devices use 'p' before the partition number (e.g. nvme0n1p1 not nvme0n11)
DEVAP=""; [[ "$DEVA" == nvme* ]] && DEVAP="p"
DEVBP=""; [[ "$DEVB" == nvme* ]] && DEVBP="p"
dd if=/dev/zero of=/dev/$DEVA bs=512 count=1
SWAP=$(( ($(cat /proc/meminfo | awk '{ if ($1 ~ /^MemTotal/) print $2}') / 1024) + 100 )) # MB
PRT=1;
START=1;
if [ -d "/sys/firmware/efi" ]; then
parted -s /dev/$DEVA mklabel gpt
STOP=$EFI
echo "Writing /boot/efi on ${DEVA}${DEVAP}$PRT" #LOG
parted -a optimal -s /dev/$DEVA mkpart '"EFI system partition"' $START $STOP #LOG
parted /dev/$DEVA set 1 esp on #LOG
parted /dev/$DEVA set 1 boot on #LOG
#sfdisk --part-type /dev/$DEVA $PRT 1 #LOG
echo "part /boot/efi --fstype=efi --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
PRT=$(( $PRT + 1 ))
START=$STOP; STOP=$(( $START + $BOOT ))
parted -a optimal -s /dev/$DEVA mkpart boot $START $STOP #LOG
echo "Writing $KSPART for KS pre-install formating default xfs" #LOG
echo "part /boot --fstype=xfs --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
if [ "$PTYPE" != "server" ]; then
START=$STOP; STOP=$(( $START + $OPTIG ))
parted -a optimal -s /dev/$DEVA mkpart iglass $START $STOP #LOG
PRT=$(( $PRT + 1 ))
#sfdisk --part-type /dev/$DEVA $PRT da #LOG
echo "# /opt/iglass will be ${DEVA}${DEVAP}$PRT" >> $KSPART
fi
#if [ -n "$SMALL" -a -n "$DEVB" ]; then
# echo "Swap on second disk as first is small" #LOG
#else
START=$STOP; STOP=$(( $START + $SWAP ))
parted -a optimal -s /dev/$DEVA mkpart swap linux-swap $START $STOP #LOG
PRT=$(( $PRT + 1 ))
#sfdisk --part-type /dev/$DEVA $PRT 82 #LOG
echo "part swap --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
#fi
START=$STOP; STOP=$DSIZE
parted -a optimal -s /dev/$DEVA mkpart root $START 100% #LOG
PRT=$(( $PRT + 1 ))
echo "part / --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
parted -s /dev/$DEVA unit mb print free #LOG
else
parted -s /dev/$DEVA mklabel msdos
STOP=$BOOT
parted -a optimal -s /dev/$DEVA mkpart primary $START $STOP #LOG
PRT=1;
echo "Writing $KSPART for KS pre-install formating default xfs" #LOG
echo "part /boot --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
if [ "$PTYPE" != "server" ]; then
START=$STOP; STOP=$(( $START + $OPTIG ))
parted -a optimal -s /dev/$DEVA mkpart primary $START $STOP #LOG
PRT=$(( $PRT + 1 ))
sfdisk --change-id /dev/$DEVA $PRT da #LOG
echo "# /opt/iglass will be ${DEVA}${DEVAP}$PRT" >> $KSPART
fi
if [ -n "$SMALL" -a -n "$DEVB" ]; then
echo "Swap on second disk as first is small" #LOG
else
START=$STOP; STOP=$(( $START + $SWAP ))
parted -a optimal -s /dev/$DEVA mkpart primary linux-swap $START $STOP #LOG
PRT=$(( $PRT + 1 ))
echo "part swap --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
fi
START=$STOP; STOP=$DSIZE
parted -a optimal -s /dev/$DEVA -- mkpart primary $START -1 #LOG
PRT=$(( $PRT + 1 ))
echo "part / --onpart=${DEVA}${DEVAP}$PRT" >> $KSPART
parted -s /dev/$DEVA unit mb print free #LOG
fi
if [ -n "$DEVB" ]; then
echo "Partitioning additional disk $DEVB ($MODEL) of size $DSIZE" #LOG
if [ -b /dev/$DEVB ]; then
dd if=/dev/zero of=/dev/$DEVB bs=512 count=1
parted -s /dev/$DEVB mklabel gpt
START=1;
PRT=1;
if [ -n "$SMALL" ]; then
echo "Partitioning swap on /dev/$DEVB:" #LOG
STOP=$(( $START + $SWAP ))
parted -a optimal -s /dev/$DEVB mkpart swap linux-swap $START $STOP #LOG
START=$STOP
echo "part swap --onpart=${DEVB}${DEVBP}$PRT" >> $KSPART
PRT=$(( $PRT + 1 ))
fi
echo "Partitioning /export on /dev/$DEVB:" #LOG
parted -a optimal -s /dev/$DEVB mkpart export $START 100% #LOG
echo "part /export --onpart=${DEVB}${DEVBP}$PRT" >> $KSPART
fi
echo "## START $KSPART ##" #LOG
cat $KSPART #LOG
echo "## END $KSPART ##" #LOG
fi
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S000A-start
+30
View File
@@ -0,0 +1,30 @@
# Fire up syslog and send everything to obelisk first so logging will work.
if [ -x /usr/sbin/dmidecode ]; then
ST=`dmidecode -s system-serial-number`
else
ST=$(ip -o link show | grep -v '\blo\b' | perl -ne 'if (/link\/ether\s+(\S+)/) { print "$1\n"; last }')
fi
cp -a /etc/resolv.conf /etc/resolv.conf.orig
SYSCONF=/dev/shm/rsyslog.conf
cp /etc/rsyslog.conf $SYSCONF
echo -e "*.* \t\t\t @$KSRV" >> $SYSCONF
ps -efw |grep syslog #LOG
pkill rsyslogd #LOG
/sbin/rsyslogd -f $SYSCONF -i /dev/shm/kslog.pid #LOG
echo "Just attempted rsyslog restart in post chroot" #LOG
# Start up sshd
echo "Trying to start up sshd in post chroot" #LOG
/usr/bin/ssh-keygen -A #LOG
if [ ! -e /dev/pts ]; then
/bin/mount /dev/pts #LOG
fi
/usr/sbin/sshd -4 -o "PermitRootLogin yes" -o "UseDNS no" #LOG
#echo 'proxy=http://10.1.1.1:3128' >> /etc/dnf/dnf.conf
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S104P-confluent
+13
View File
@@ -0,0 +1,13 @@
# Install librdkafka for Kafka integration (Net::Kafka Perl module)
#
# RHEL 10 ships librdkafka in standard repos — no Confluent repo needed:
# librdkafka — AppStream (runtime, 2.1.1)
# librdkafka-devel — CRB (headers for Net::Kafka build, 2.1.1)
#
# Confluent platform repo (rpm/8.2) is still deployed via etc.tgz for
# potential future use but is NOT required for librdkafka on RHEL 10.
# Enable CRB (required for librdkafka-devel)
dnf config-manager --set-enabled crb #LOG
dnf install -y librdkafka librdkafka-devel #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S104V-confluent
Symlink
+1
View File
@@ -0,0 +1 @@
S104P-confluent
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S105A-extra-pkgs
+150
View File
@@ -0,0 +1,150 @@
echo "Enabling EPEL repo" #LOG
dnf -y config-manager --set-enabled crb #LOG
dnf -y install epel-release #LOG
crb enable #LOG
# Perl are in InstallPerlmods
EPELPKGS='freetds freetds-devel freetds-libs perl-NetAddr-IP perl-Net-DNS perl-Regexp-Common perl-WWW-Mechanize perl-Protocol-WebSocket perl-Crypt-Rijndael perl-Net-SSH-Perl'
dnf -y install $EPELPKGS #LOG
if dnf -q group list --available hidden ids | grep -Fqx 'xfce'; then
dnf -y install '@xfce' #LOG
else
echo "Skipping optional Xfce group; not available on $DIST" #LOG
fi
echo "Installing Perl Modules" #LOG
#export ALL_PROXY=http://10.1.1.1:3128
PERLPKGS='perl-PerlIO-gzip perl-Net-SNMP perl-URI-Encode perl-Net-Netmask perl-Proc-ProcessTable perl-Path-Class perl-IO-Socket-INET6 perl-Expect perl-LDAP perl-Text-CSV perl-Crypt-CBC perl-IO-FDPass perl-DBD-MariaDB perl-IPC-Run'
dnf -y install $PERLPKGS #LOG
export PERL_MM_USE_DEFAULT=1
# WWW::[email protected] ? mcpani ?
## 2026-03-06 - Chris A - Removed Net::SNMP \
cpanm -n \
Net::Syslog \
Date::Handler \
Thread::Queue::Any \
Parse::RPN \
HTML::Query \
HTML::Restrict \
Net::SMTP_auth \
Mail::POP3Client \
Mail::IMAPClient \
Cache::FastMmap \
UUID \
Net::INET6Glue \
Net::Address::IP::Local \
Net::Ping::External \
Net::SIP \
Net::SSH \
Net::Telnet \
Net::Azure::StorageClient \
MongoDB \
Asterisk::AMI \
XML::LibXML::PrettyPrint \
Forks::Queue #LOG
# Net::Kafka requires librdkafka-devel (installed by S104P from CRB).
cpanm -n Net::Kafka #LOG
echo "Installing Chrome" #LOG
rpm --import https://dl.google.com/linux/linux_signing_key.pub #LOG
dnf -y install https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm #LOG
echo "Installing MS SQL tools and PowerShell" #LOG
export ACCEPT_EULA=Y
curl -fsSL -o /etc/yum.repos.d/microsoft.repo https://packages.microsoft.com/config/rhel/10/prod.repo #LOG
# Note: package renamed mssql-tools → mssql-tools18 in RHEL 10 Microsoft repo
dnf -y install --assumeyes mssql-tools18 msodbcsql18 #LOG
# PowerShell is not in the RHEL 10 MS repo — install from GitHub release.
# TODO: Verify RHEL 10 compatible RPM when available.
# Ref: https://github.com/PowerShell/PowerShell/releases
dnf -y install --assumeyes https://github.com/PowerShell/PowerShell/releases/download/v7.3.4/powershell-7.3.4-1.rh.x86_64.rpm #LOG
# DNF AppStream modules removed in RHEL 10 — nodejs is a regular package.
# RHEL 10 AppStream ships nodejs 22 by default.
echo "Installing nodejs" #LOG
dnf -y install nodejs #LOG
echo "Updating all rpms" #LOG
dnf -y update #LOG
TMP=$(tmpdir)
if [ $? -eq 0 -a -d $TMP ]; then
cd $TMP
echo "Getting Amozon awscli bundle" #LOG
AWSCLI=awscli-exe-linux-x86_64.zip
wget -nv http://$KSRV/KickStart/cfg/$DIST/share/pkgs/$AWSCLI #LOG
if [ -s $AWSCLI ]; then
unzip -q $AWSCLI #LOG
./aws/install #LOG
/usr/local/bin/aws --version #LOG
else
echo "Not able to get Amozon awscli bundle" #LOG
fi
#NODEV="v18.15.0"
#echo "Getting node $NODEV bundle" #LOG
#wget -nv http://$KSRV/KickStart/cfg/$DIST/share/pkgs/node-${NODEV}-linux-x64.tar.xz #LOG
#(echo -n "I start in "; pwd) #LOG
#ls -l node-${NODEV}-linux-x64.tar.xz #LOG
#if [ -s node-${NODEV}-linux-x64.tar.xz ]; then
# cd "/usr/local" #LOG
# (echo -n "Now I am in "; pwd) #LOG
# tar xJf $TMP/node-${NODEV}-linux-x64.tar.xz #LOG
# ln -s node-${NODEV}-linux-x64 node #LOG
# cd $TMP
#else
# echo "Not able to get node $NODEV bundle" #LOG
#fi
VMWARE="VMware-vSphere-Perl-SDK-7.0.0-17698549.x86_64.tar.gz"
echo "Getting $VMWARE bundle" #LOG
wget -nv http://$KSRV/KickStart/cfg/$DIST/share/pkgs/${VMWARE} #LOG
if [ -s $VMWARE ]; then
tar xf $VMWARE #LOG
# Makefile.PL lives inside the extracted distrib dir, not at $TMP root
cd vmware-vsphere-cli-distrib
perl Makefile.PL #LOG
make -s #LOG
make -s install #LOG
cd "$TMP"
else
echo "Not able to get $VMWARE bundle" #LOG
fi
cd /
# rm -rf $TMP
else
echo "Could not create src dir $TMP" #LOG
fi
echo "Updating Net::SNMP USM.pm for more recent auth and priv protocols" #LOG
cp -a /usr/share/perl5/vendor_perl/Net/SNMP/Security/USM.pm /usr/share/perl5/vendor_perl/Net/SNMP/Security/USM.pm.orig
#cp -a /usr/local/share/perl5/5.32/Net/SNMP/Security/USM.pm /usr/local/share/perl5/5.32/Net/SNMP/Security/USM.pm.orig
wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/pkgs/USM.pm > /usr/share/perl5/vendor_perl/Net/SNMP/Security/USM.pm #LOG
#cp /usr/share/perl5/vendor_perl/Net/SNMP/Security/USM.pm /usr/local/share/perl5/5.32/Net/SNMP/Security/USM.pm #LOG
if [ ! -e /etc/ld.so.conf.d/usr-local.conf ]; then
echo "/usr/local/lib" > /etc/ld.so.conf.d/usr-local.conf
ldconfig #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S105V-extra-pkgs
Symlink
+1
View File
@@ -0,0 +1 @@
S105A-extra-pkgs
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S110A-baseline-users-ssh
+29
View File
@@ -0,0 +1,29 @@
# Minimal stage-0 customization for staged bring-up.
echo "Creating baseline noc account and SSH access" #LOG
getent group noc >/dev/null 2>&1 || groupadd -g 1108 noc #LOG
id noc >/dev/null 2>&1 || \
useradd -c "NOC Account" -d /home/noc -u 1108 -g 1108 -s /bin/bash -p "$NOCPASS" noc #LOG
for _user in root noc; do
_home=/root
[ "$_user" = "noc" ] && _home=/home/noc
install -d -m 700 -o "$_user" -g "$_user" "$_home/.ssh" #LOG
wget -q -O "$_home/.ssh/authorized_keys" \
"http://$KSRV/KickStart/cfg/$DIST/share/ssh/authorized_keys" #LOG
chown "$_user":"$_user" "$_home/.ssh/authorized_keys" #LOG
chmod 600 "$_home/.ssh/authorized_keys" #LOG
done
cat > /etc/ssh/sshd_config.d/10-stage-baseline.conf <<'EOF' #LOG
PubkeyAuthentication yes
PasswordAuthentication yes
PermitRootLogin no
Match Address 10.1.1.1,10.1.1.254
PermitRootLogin yes
EOF
restorecon -RF /root/.ssh /home/noc/.ssh /etc/ssh/sshd_config.d 2>/dev/null || true #LOG
systemctl enable sshd.service #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S150A-etc-mod
+65
View File
@@ -0,0 +1,65 @@
# description: Change /etc/issue & skel
if [ "$PTYPE" == "server" ]; then
echo "Creating Server configs and setup scripts" #LOG
echo "### Copying server /etc files" #LOG
cd / && wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/etc-srv.tgz | tar xvzf - --no-same-owner --no-same-permissions #LOG
cd / && wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/etc.tgz | tar xvzf - --no-same-owner --no-same-permissions ./etc/firewalld/ #LOG
restorecon -RF /etc/firewalld 2>/dev/null || true #LOG
echo "### Installing root SSH test access key" #LOG
install -d -m 700 /root/.ssh #LOG
wget -q -O /root/.ssh/authorized_keys http://$KSRV/KickStart/cfg/$DIST/share/ssh/authorized_keys #LOG
chmod 600 /root/.ssh/authorized_keys #LOG
restorecon -RF /root/.ssh #LOG
echo "Setting sebool use_nfs_home_dirs=1" #LOG
setsebool -P use_nfs_home_dirs=1 #LOG
mkdir /customers
else
echo "### Modifing /etc/issue" #LOG
echo > /etc/issue
echo "This box is the property of iGLASS NETWORKS." >> /etc/issue
echo "Access by non-iGLASS NETWORKS employees is strictly forbidden." >> /etc/issue
echo >> /etc/issue
ln -f /etc/issue /etc/issue.net
ln -f /bin/bash /bin/rbash
rm -f /etc/profile.d/colorls.*
echo "### Copying /etc files" #LOG
cd / && wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/etc.tgz | tar xvzf - --no-same-owner --no-same-permissions #LOG
restorecon -RF /etc 2>/dev/null || true #LOG
cp -vf /etc/skel/.bashrc /root/.bashrc
echo "### Installing root SSH test access key" #LOG
install -d -m 700 /root/.ssh #LOG
wget -q -O /root/.ssh/authorized_keys http://$KSRV/KickStart/cfg/$DIST/share/ssh/authorized_keys #LOG
chmod 600 /root/.ssh/authorized_keys #LOG
restorecon -RF /root/.ssh #LOG
fi
#echo -e 'DESKTOP="GNOME"\nDISPLAYMANAGER="GNOME"' >> /etc/sysconfig/desktop
#echo "autospawn = no" >> /etc/pulse/client.conf
#mkdir -p /root/.config/systemd/user/
#ln -s /dev/null /root/.config/systemd/user/pulseaudio.socket
#ln -s /dev/null /root/.config/systemd/user/pulseaudio.service
systemctl --user mask pulseaudio 2>/dev/null || true #LOG
systemctl --user disable pulseaudio.socket 2>/dev/null || true #LOG
systemctl --user disable pulseaudio.service 2>/dev/null || true #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S150V-etc-mod
Symlink
+1
View File
@@ -0,0 +1 @@
S150A-etc-mod
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S200A-systemctl
+84
View File
@@ -0,0 +1,84 @@
# description: Turn off unwanted services
echo "### Locking down services" #LOG
has_unit_file() {
systemctl list-unit-files --full --all --no-legend 2>/dev/null | awk '{print $1}' | grep -Fxq "$1"
}
disable_if_present() {
if has_unit_file "$1"; then
systemctl disable "$1" #LOG
else
echo "Skipping absent unit $1" #LOG
fi
}
stop_disable_if_present() {
if has_unit_file "$1"; then
systemctl --no-reload stop "$1" #LOG
systemctl --no-reload disable "$1" #LOG
else
echo "Skipping absent unit $1" #LOG
fi
}
mask_if_present() {
if has_unit_file "$1"; then
systemctl mask "$1" #LOG
else
echo "Skipping absent unit $1" #LOG
fi
}
systemctl enable postfix
for _svc in initial-setup-graphical.service initial-setup-reconfiguration.service \
initial-setup-text.service initial-setup.service; do
systemctl disable "$_svc" 2>/dev/null && echo "Disabled $_svc" #LOG || true
done
# GDM may not be installed on DVD minimal-environment; guard before editing
if [ -f /etc/gdm/custom.conf ]; then
perl -spi -e 's/\[daemon\]/\[daemon\]\nInitialSetupEnable=False/' /etc/gdm/custom.conf #LOG
fi
# Don't start X
systemctl set-default multi-user.target #LOG
disable_if_present avahi-daemon.socket
disable_if_present avahi-daemon.service
if [ "$PTYPE" != "server" ]; then
disable_if_present crond.service
fi
disable_if_present cups.service
#ln -s /dev/null /etc/systemd/system/nfs.target
disable_if_present nfs.target
stop_disable_if_present rpcbind.socket
disable_if_present rpcbind.service
mask_if_present rpcbind.service
disable_if_present libvirtd.service
disable_if_present ksmtuned.service
disable_if_present ksm.service
#echo 2 >/sys/kernel/mm/ksm/run
disable_if_present bluetooth.service
systemctl enable snmpd.service #LOG
systemctl enable snmptrapd.service #LOG
echo 'OPTIONS="-LS w6"' >> /etc/sysconfig/snmpd
echo 'OPTIONS="-OQ -t -LS w3"' >> /etc/sysconfig/snmptrapd
disable_if_present dnf-makecache.timer
#echo "Setting python3 as default" #LOG
#alternatives --set python /usr/bin/python3 #LOG
echo "install libiscsi /bin/true" >> /etc/modprobe.d/disable-libiscsi.conf
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S200V-systemctl
Symlink
+1
View File
@@ -0,0 +1 @@
S200A-systemctl
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S350A-chrony
+26
View File
@@ -0,0 +1,26 @@
# description: Sync box time
if [ -f "/usr/share/zoneinfo/$TZ" ]; then
echo "Setting Time Zone = $TZ" #LOG
else
export TZ="America/New_York" #LOG
echo "No Time Zone set. Using $TZ" #LOG
fi
rm -f /etc/localtime #LOG
ln -s "/usr/share/zoneinfo/$TZ" /etc/localtime #LOG
if command -v timedatectl >/dev/null 2>&1; then
timedatectl set-timezone "$TZ" || echo "timedatectl timezone update unavailable during stage apply" #LOG
fi
perl -spi -e 's/^(pool.*)$/# $1\nserver ntp.iglass.net iburst/' /etc/chrony.conf
systemctl enable chronyd.service #LOG
systemctl start chronyd.service || echo "chronyd start unavailable during stage apply" #LOG
chronyc -a makestep || echo "chronyc makestep unavailable during stage apply" #LOG
/sbin/hwclock --systohc --utc || echo "hwclock sync unavailable during stage apply" #LOG
if command -v timedatectl >/dev/null 2>&1; then
timedatectl set-local-rtc 0 || echo "timedatectl local-rtc update unavailable during stage apply" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S350V-chrony
Symlink
+1
View File
@@ -0,0 +1 @@
S350A-chrony
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S400P-noc-iglass
+330
View File
@@ -0,0 +1,330 @@
# description: Setup noc account & iglass dir
###
# Setup noc Account
###
CPP1=/dev/shm/pkey.txt
#IDENTITY="$IG_HOME/conf/tunnel/probe-id"
NOCHOME=/home/noc
echo "NOCHOME=$NOCHOME" #LOG
if [ "$PTYPE" = "VM" ]; then
CRYPTEDDEV=
elif [ -b /dev/mapper/vga-iGlassOpt ]; then
CRYPTEDDEV="/dev/mapper/vga-iGlassOpt"
elif [ -b "/dev/$DISK" ] && [ "$(sfdisk --part-type /dev/$DISK 2 2>/dev/null)" = "da" ]; then
CRYPTEDDEV="/dev/${DISK}2"
elif [ -b "/dev/$DISK" ] && parted -s /dev/$DISK print list 2>/dev/null | grep -qE "^\s+3\s.+iglass"; then
CRYPTEDDEV="/dev/${DISK}3"
else
CRYPTEDDEV=
fi
DECRYPTEDMAP="crypt-iGlassOpt"
common_opt() {
#mkdir -vp /home #LOG
if [ -L /home ]; then
rm -f /home #LOG
elif [ -d /home ] && [ -z "$(ls -A /home 2>/dev/null)" ]; then
rmdir /home #LOG
elif [ -d /home ]; then
echo "Warning /home not empty, leaving existing directory in place" #LOG
fi
mkdir $IG_HOME/home #LOG
if [ ! -e /home ]; then
ln -s $IG_HOME/home /home #LOG
fi
/usr/sbin/groupadd -g 106 noc #LOG
/usr/sbin/useradd -c "NOC Account" -d $NOCHOME -u 1108 -g 106 -s /bin/bash -p $NOCPASS noc #LOG
mkdir $IG_HOME/bin #LOG
mkdir $IG_HOME/lib #LOG
mkdir $IG_HOME/conf #LOG
mkdir $IG_HOME/sync #LOG
mkdir $IG_HOME/data #LOG
mkdir $IG_HOME/data/ps #LOG
mkdir $IG_HOME/sync/backup #LOG
mkdir -p $IG_HOME/sync/logs/messages #LOG
mkdir $IG_HOME/sync/logs/iglass #LOG
mkdir $IG_HOME/sync/logs/customer #LOG
install -d -m 755 "$IG_HOME/conf" "$IG_HOME/conf/db" #LOG
wget -q -O "$IG_HOME/conf/billing.properties" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/billing.properties #LOG
wget -q -O "$IG_HOME/conf/cpeInfo.properties" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/cpeInfo.properties #LOG
wget -q -O "$IG_HOME/conf/events.conf" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/events.conf #LOG
wget -q -O "$IG_HOME/conf/oui.txt" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/oui.txt #LOG
wget -q -O "$IG_HOME/conf/poller.properties" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/poller.properties #LOG
wget -q -O "$IG_HOME/conf/db/pg_service.conf" http://$KSRV/KickStart/cfg/$DIST/share/iglass/conf/db/pg_service.conf #LOG
#wget -q -O - http://$KSRV/KickStart/iglass-releases/$IGRELEASE | /bin/tar xvzf - #LOG
rpm --import http://$KSRV/KickStart/iglass-releases/RPM-GPG-KEY-pmanager #LOG
rpm -ivh --nodeps http://$KSRV/KickStart/iglass-releases/$IGRELEASE #LOG
#cat >> /etc/yum.repos.d/iGlass.repo <<EOF
#[iGlass-Networks]
#name=iGlass Networks
#baseurl=https://repo.iglass.net/
#enabled=0
#gpgcheck = 0
#gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-iGlass-release
#sslverify = 1
#sslcacert = /opt/iglass/conf/pki/iGlassCA.crt
#sslclientkey = /opt/iglass/conf/pki/probe.key
#sslclientcert = /opt/iglass/conf/pki/probe.crt
#EOF
/bin/chown -RL noc:noc $IG_HOME #LOG
/bin/chown -R 0.0 $IG_HOME/bin #LOG
/bin/chown -R 0.0 $IG_HOME/lib #LOG
/bin/chown -R 0.0 $IG_HOME/sql #LOG
/bin/chown -R 0.0 $IG_HOME/home #LOG
/bin/chmod 775 $IG_HOME #LOG
/bin/chmod g+s $IG_HOME #LOG
chmod 775 $IG_HOME/data $IG_HOME/sync #LOG
chmod g+s $IG_HOME/data $IG_HOME/sync #LOG
#chmod 770 $IG_HOME/conf/pki #LOG
#chmod 600 $IG_HOME/conf/pki/iGlassCA.crt #LOG
install -d -m 700 -o noc -g noc $IG_HOME/.aws
install -d -m 600 -o noc -g noc /dev/null $IG_HOME/.aws/credentials.stub
ln -sf /dev/null $IG_HOME/.aws/config #LOG
ln -sf /dev/null $IG_HOME/.aws/credentials #LOG
# Install iGlass systemd unit files (RPM ships them in /opt/iglass/systemd/system/)
if [ -d "$IG_HOME/systemd/system" ]; then
echo "Installing iGlass systemd unit files" #LOG
for f in "$IG_HOME"/systemd/system/*.service "$IG_HOME"/systemd/system/*.timer; do
[ -f "$f" ] || continue
/bin/cp -f "$f" /etc/systemd/system/ #LOG
done
systemctl daemon-reload #LOG
else
echo "WARNING: No iGlass systemd unit files found at $IG_HOME/systemd/system" #LOG
fi
#/bin/mkdir $IG_HOME/conf/tunnel #LOG
#echo "Creating root probe key" #LOG
ST=`/usr/sbin/dmidecode -s system-serial-number | sed 's/\s*//g'`
#ssh-keygen -t rsa -N "" -C "tunnel@$ST" -f $IDENTITY #LOG
#/bin/chown -R 0.0 $IG_HOME/conf/tunnel/ #LOG
restorecon -R $IG_HOME #LOG
chcon unconfined_u:object_r:home_root_t:s0 $IG_HOME/home #LOG
#echo "Creating tunnel Account" #LOG
#/usr/sbin/groupadd -g 107 tun #LOG
#/usr/sbin/useradd -c "iGlass Tunnel Account" -d /home/tunnel -o -u 0 -g 107 -s /bin/rbash tunnelXX #LOG
#if [ "$PTYPE" = "ds" -o "$PTYPE" = "VM" ]; then
# echo "OPTIONS=\"-q -x -y\"" > "$IG_ETC/tunnel.options"
#else
# echo "OPTIONS=\"-q -x -R -p\"" > "$IG_ETC/tunnel.options"
#fi
#echo "216.134.202.53" > "$IG_ETC/key_host"
echo "Setting up noc and iglass directory structure" #LOG
mkdir $NOCHOME/.ssh #LOG
mkdir $NOCHOME/.gnupg #LOG
mkdir $NOCHOME/.aws #LOG
mkdir $NOCHOME/.aws/credentials.d #LOG
#mkdir $NOCHOME/.postgresql #LOG
wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/noc/pgpass > $NOCHOME/.pgpass
chmod 600 $NOCHOME/.pgpass #LOG
cp --preserve=mode,timestamps $NOCHOME/.pgpass /root/ #LOG
chmod 755 $NOCHOME #LOG
chmod g+s $NOCHOME #LOG
mkdir $NOCHOME/tmp #LOG
echo "Fixing ssh for noc" #LOG
# Do authorized_keys via ipa now
# wget -nv -nH -r -P$NOCHOME/.ssh --cut-dirs 6 --follow-tags a -R "*.html*" -np http://$KSRV/KickStart/cfg/$DIST/share/noc/ssh/
wget -q -O "$NOCHOME/.gnupg/gpg.conf" http://$KSRV/KickStart/cfg/$DIST/share/noc/gnupg/gpg.conf #LOG
wget -q -O "$NOCHOME/.gnupg/pubring.gpg" http://$KSRV/KickStart/cfg/$DIST/share/noc/gnupg/pubring.gpg #LOG
wget -q -O "$NOCHOME/.gnupg/secring.gpg" http://$KSRV/KickStart/cfg/$DIST/share/noc/gnupg/secring.gpg #LOG
wget -q -O "$NOCHOME/.gnupg/random_seed" http://$KSRV/KickStart/cfg/$DIST/share/noc/gnupg/random_seed #LOG
wget -q -O "$NOCHOME/.gnupg/trustdb.gpg" http://$KSRV/KickStart/cfg/$DIST/share/noc/gnupg/trustdb.gpg #LOG
wget -q -O "$NOCHOME/.aws/config" http://$KSRV/KickStart/cfg/$DIST/share/noc/aws/config #LOG
wget -q -O "$NOCHOME/.aws/credentials" http://$KSRV/KickStart/cfg/$DIST/share/noc/aws/credentials #LOG
#mkdir -p $NOCHOME/.config/systemd/user/
## systemctl --user mask pulseaudio.socket
## systemctl --user mask pulseaudio.service
#ln -s /dev/null $NOCHOME/.config/systemd/user/pulseaudio.socket
#ln -s /dev/null $NOCHOME/.config/systemd/user/pulseaudio.service
#ln -s /dev/null $NOCHOME/.config/systemd/user/grub-boot-success.service
#ln -s /dev/null $NOCHOME/.config/systemd/user/grub-boot-success.timer
sudo -iu noc systemctl --user mask pulseaudio #LOG
sudo -iu noc systemctl --user disable pulseaudio.socket #LOG
sudo -iu noc systemctl --user disable pulseaudio.service #LOG
chown -Rh noc.noc $NOCHOME #LOG
chmod 0700 $NOCHOME/.ssh #LOG
#chmod 644 $NOCHOME/.ssh/known_hosts* #LOG
#chmod 600 $NOCHOME/.ssh/authorized_keys* #LOG
chmod 0700 $NOCHOME/.gnupg #LOG
chmod 600 $NOCHOME/.gnupg/* #LOG
chown -Rh noc.noc $NOCHOME/.aws #LOG
chmod 0700 $NOCHOME/.aws #LOG
chmod 600 $NOCHOME/.aws/* #LOG
ln -s /home/noc/.aws /root/ #LOG
if [ -d "/etc/sudoers.d/" ]; then
cat >> /etc/sudoers.d/noc-default <<EOF
Cmnd_Alias NOC = /sbin/ipa-client-install "", /etc/iglass/mount-iglass.sh
noc ALL=(root) NOC
Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
EOF
else
echo "Where is /etc/sudoers.d/ ?" #LOG
fi
chmod 440 /etc/sudoers.d/noc-default
restorecon -F /etc/sudoers.d/noc-default
}
if [ -e "$CRYPTEDDEV" ]; then
echo "Setting up crypted iglass partition on $CRYPTEDDEV" #LOG
#echo -n `wget -q -O - --no-check-certificate https://$KSRV/cgi-bin/keygen.pl` > $CPP1
#echo -n $(od -An -x /dev/urandom | head -1) > $CPP1
openssl rand -base64 30 | head -c30 > $CPP1
if [ -z "$CPP1" ]; then
echo "WARNING! Could not generate crypt key" #LOG
else
/sbin/swapoff -av #LOG
#sfdisk --change-id /dev/sda 2 da
sed -ie "/iGlassOpt/d" /etc/fstab #LOG
if [ "$DISKTYPE" != "ssd" ]; then
# These are slow and I don't think valid for SSDs
echo "Randomizing drive before encrypting" #LOG
#dd if=/dev/urandom of=$CRYPTEDDEV bs=1M #LOG
openssl enc -aes-256-ctr -pass pass:"$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64)" -nosalt < /dev/zero | dd of=$CRYPTEDDEV #LOG
# openssl rand can't do bigger than 754G
# openssl rand $(</proc/partitions awk '$4=="sda2" {print $3*1024}') >/dev/sda
# shred -v -n 1 $CRYPTEDDEV #LOG
fi
echo "Trying to load kernel mods" #LOG
modprobe -v dm_mod #LOG
modprobe -v dm_crypt #LOG
#modprobe -v cryptoloop #LOG
echo "Running luksFormat" #LOG
echo "cryptsetup -q luksFormat $CRYPTEDDEV $CPP1"
cryptsetup -q luksFormat $CRYPTEDDEV $CPP1 #LOG
echo "Running luksOpen" #LOG
cryptsetup --key-file $CPP1 open --type luks $CRYPTEDDEV $DECRYPTEDMAP #LOG
if [ -e /dev/mapper/$DECRYPTEDMAP ]; then
#mkfs.ext3 /dev/mapper/$DECRYPTEDMAP #LOG
mkfs.ext4 /dev/mapper/$DECRYPTEDMAP #LOG
cryptsetup -v luksDump $CRYPTEDDEV #LOG
echo "Creating encrypted IG_HOME($IG_HOME) and noc account " #LOG
mkdir -vp $IG_HOME #LOG
mkdir -vp $IG_ETC #LOG
chown 0.0 $IG_ETC #LOG
chmod 755 $IG_ETC #LOG
mount -v /dev/mapper/$DECRYPTEDMAP $IG_HOME #LOG
if [ ! -d /export ]; then
ONEDISK=1 # This is so I don't move pg here
echo "Looks like just 1 disk, no /export. Creating directory." #LOG
mkdir /export #LOG
fi
if [ "$PTYPE" != "ds" ] && [ "$PTYPE" != "plug" ]; then
echo "Not a DS probe, setting up crypttab"
#cp -af $CPP1 $IG_ETC/igpp #LOG
#chown 0.0 $IG_ETC/igpp #LOG
#chmod 600 $IG_ETC/igpp #LOG
#echo "$DECRYPTEDMAP $CRYPTEDDEV $IG_ETC/igpp" >> /etc/crypttab
#echo "$DECRYPTEDMAP UUID=$(blkid -o value -s UUID $CRYPTEDDEV) none noauto" >> /etc/crypttab
echo "$DECRYPTEDMAP $CRYPTEDDEV none noauto" >> /etc/crypttab
#echo "/dev/mapper/$DECRYPTEDMAP $IG_HOME ext3 defaults 1 2" >> /etc/fstab
echo "/dev/mapper/$DECRYPTEDMAP $IG_HOME ext4 rw,nosuid,dev,exec,noauto,nouser,async" >> /etc/fstab
fi
common_opt
fi
fi
elif [ "$PTYPE" = "VM" ]; then
echo "No partition to crypt, VM" #LOD
echo "Creating non-encrypted IG_HOME($IG_HOME) and noc account " #LOG
mkdir -vp $IG_HOME #LOG
mkdir -vp $IG_ETC #LOG
chown 0.0 $IG_ETC #LOG
chmod 700 $IG_ETC #LOG
common_opt
elif [ -b /dev/vda ] || [ -d /sys/block/vda ] || systemd-detect-virt --quiet; then
echo "No partition to crypt, virtualized test system" #LOG
echo "Creating non-encrypted IG_HOME($IG_HOME) and noc account " #LOG
mkdir -vp $IG_HOME #LOG
mkdir -vp $IG_ETC #LOG
chown 0.0 $IG_ETC #LOG
chmod 700 $IG_ETC #LOG
common_opt
else
echo "ERROR: No partition to encrypt" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S400S-noc-iglass
+31
View File
@@ -0,0 +1,31 @@
# description: Setup noc account & iglass dir
###
# Setup noc Account
###
echo "Creating noc Account (IG_HOME = $IG_HOME)" #LOG
mkdir -vp /home #LOG
/usr/sbin/groupadd -g 106 noc #LOG
/usr/sbin/useradd -c "NOC Account" -d /home/noc -u 1108 -g 106 -s /bin/bash -p $NOCPASS noc #LOG
echo "Setting up noc and iglass directory structure" #LOG
/bin/mkdir /home/noc/.ssh #LOG
/bin/chmod 755 /home/noc #LOG
/bin/chmod g+s /home/noc #LOG
/bin/mkdir /home/noc/tmp #LOG
/bin/mkdir $IG_HOME #LOG
echo "Fixing ssh for noc" #LOG
/bin/cp /mnt/tmp/custom/ssh/* /home/noc/.ssh/ #LOG
/bin/chown -R noc.noc /home/noc #LOG
/bin/chmod 0700 /home/noc/.ssh #LOG
/bin/chmod 644 /home/noc/.ssh/known_hosts* #LOG
/bin/chmod 600 /home/noc/.ssh/authorized_keys* #LOG
/usr/bin/perl -spi -e 's/^Defaults\s+requiretty/Defaults \!requiretty/' /etc/sudoers #LOG
echo '%eye ALL= /bin/su - noc' >> /etc/sudoers
echo '%eye ALL= /bin/su - tomcat' >> /etc/sudoers
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S400V-noc-iglass
Symlink
+1
View File
@@ -0,0 +1 @@
S400P-noc-iglass
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S570A-Postgres
+182
View File
@@ -0,0 +1,182 @@
# description: Setup Postgres
###
# Postgres Setup
###
#
echo "Setting up and installing PostgreSQL" #LOG
if rpm -q postgresql14-server >/dev/null; then
echo "PostgreSQL already installed" #LOG
else
dnf -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-10-x86_64/pgdg-redhat-repo-latest.noarch.rpm #LOG
dnf -y install postgresql14 postgresql14-devel postgresql14-server postgresql14-contrib postgresql14-libs #LOG
fi
[ -z "$IG_HOME" ] && export IG_HOME="/opt/iglass"
[ -z "$DIST" ] && export DIST="AlmaLinux-10.1-x86_64"
PGHOME=$(ls -vd /usr/pgsql* | tail -1)
if [ -d "$PGHOME" ]; then
PGDATA=/opt/iglass/pgdata/data${PGHOME/#\/usr\/pgsql}/
echo "Setting up Postgres from $PGHOME [$PGDATA]" #LOG
ln -sf $IG_HOME/conf/db/pg_service.conf /etc/sysconfig/pgsql/pg_service.conf #LOG
ln -sf $IG_HOME/conf/db/pg_service.conf /etc/pg_service.conf #LOG
mkdir /etc/pgsql #LOG
ln -s /opt/iglass/conf/db/pg_service.conf /etc/pgsql/ #LOG
POSTID=$(id -u postgres 2>/dev/null)
if [ -z "$POSTID" ]; then
echo "Creating postgres account" #LOG
/usr/sbin/groupadd -g 26 postgres
/usr/sbin/useradd -N -r -c "Postgres Account" -d /home/postgres -m -u 26 -g 26 -G 106 -s /bin/bash postgres #LOG
else
echo "Postgres account already exists, setting home." #LOG
/usr/sbin/usermod -d /home/postgres postgres #LOG
if [ ! -d /home/postgres ]; then
mkdir /home/postgres
fi
fi
if [ -d "/home/KickStart/cfg/$DIST/share/pgsql" ]; then
WGET="cat /home/KickStart/cfg/$DIST/share/pgsql"
. "/home/KickStart/cfg/$DIST/ks.d/functions"
else
WGET="wget -qO - http://$KSRV/KickStart/cfg/$DIST/share/pgsql"
fi
echo "WGET=$WGET" #LOG
if [ -d "$PGDATA" ]; then
echo "$PGDATA already exists" #LOG
exit
fi
mkdir -p $PGDATA #LOG
chown -R postgres:postgres $PGDATA #LOG
chown -R postgres:postgres /home/postgres #LOG
#setsebool -PV postgresql_can_rsync on #LOG
# data dir must be empty
export LANG="en_US.UTF-8"
echo "Running initdb with LANG=$LANG" #LOG
sudo -u postgres $PGHOME/bin/initdb --locale="$LANG" -D $PGDATA #LOG
mkdir -p $PGDATA/iglass #LOG
chown -R postgres:postgres $PGDATA #LOG
chmod 0700 $PGDATA #LOG
TMP=$(tmpdir)
if [ $? -eq 0 ]; then
for _pgfile in 01iglass.conf pg_hba.conf; do
wget -q -O "$TMP/$_pgfile" "http://$KSRV/KickStart/cfg/$DIST/share/pgsql/iglass/$_pgfile" #LOG
done
chmod 600 $TMP/*
/bin/cp -f $TMP/* $PGDATA/iglass/
/bin/rm -rf $TMP
else
echo ERROR: Could not create tmp directory for postgres data/iglass configs
fi
rm -f $PGDATA/pg_hba.conf
echo "include = 'iglass/01iglass.conf'" >> $PGDATA/postgresql.conf
echo "hba_file = '${PGDATA}/iglass/pg_hba.conf'" >> $PGDATA/postgresql.conf
echo "Generating DH parameters" #LOG
openssl dhparam -out $PGDATA/iglass/dhparams.pem 2048 2>/dev/null #LOG
TMP=$(tmpdir)
if [ $? -eq 0 ]; then
echo "Running postgres finish script" #LOG
$WGET/pg-openssl.cnf > $TMP/pg-openssl.cnf
chown -R postgres:postgres $TMP
cd $TMP
#(echo export PGPASS=$($WGET/pg-openssl.pw); echo export OPENSSL_CONF=$TMP/pg-openssl.cnf; $WGET/finish.sh) | sudo -u postgres -s
cat <<EOF | sudo -u postgres -s
export PGPASS=$($WGET/pg-openssl.pw);
export OPENSSL_CONF=$TMP/pg-openssl.cnf;
PGDATA=$PGDATA
openssl req -new -text -out cert.req
openssl rsa -in privkey.pem -out cert.pem -passin env:PGPASS
openssl req -sha256 -x509 -days 365 -in cert.req -text -key cert.pem -out cert.cert
cp cert.pem $PGDATA/iglass/server.key
cp cert.cert $PGDATA/iglass/server.crt
chmod 600 $PGDATA/iglass/server.*
rm -f cert.* *.pem
EOF
cd /
/bin/rm -rf $TMP
else
echo ERROR: Could not create tmp dir for postgres finish.sh
fi
chown -R postgres:postgres $PGDATA #LOG
cd /
# if [ -f "$IG_HOME/bin/pgMemTweak" ]; then
# $IG_HOME/bin/pgMemTweak >> /etc/sysctl.conf
# SHMALL=`$IG_HOME/bin/pgMemTweak shmall`
# if [ "$SHMALL" != "" ]; then
# echo "$SHMALL" > /proc/sys/kernel/shmall
#
# SHMMAX=`$IG_HOME/bin/pgMemTweak shmmax`
# if [ "$SHMMAX" != "" ]; then
# echo "$SHMMAX" > /proc/sys/kernel/shmmax
# fi
# fi
# else
# echo "iGlass software must be installed first for pgMemTweak"
# fi
# sysctl -w $(grep kernel.shmmax /etc/sysctl.conf | sed 's/\s*//g') #LOG
# sysctl -w $(grep kernel.shmall /etc/sysctl.conf | sed 's/\s*//g') #LOG
PGSRV=$(basename $(ls /usr/lib/systemd/system/postgresql* | tail -1))
if [ -z "$PGSRV" ]; then
echo "No postgres.service found! Can't start it." #LOG
else
echo "Starting postgres to create iglass user" #LOG
#perl -spi -e "s|^(Environment=PGDATA=)(.+)|#\$1\$2\n\$1$PGDATA|" "/usr/lib/systemd/system/$PGSRV"
mkdir /etc/systemd/system/${PGSRV}.d/ #LOG
echo -e "[Service]\nEnvironment=PGDATA=$PGDATA\n" > /etc/systemd/system/${PGSRV}.d/pgdata.conf
systemctl enable $PGSRV #LOG
# This doesn't work with systemd in chroot
#systemctl start $PGSRV #LOG
su - postgres -c "$PGHOME/bin/pg_ctl -D $PGDATA -l logfile start" #LOG
sleep 5
$WGET/dbmod.sql | sudo -u postgres /usr/bin/psql -U postgres -d template1 #LOG
su - postgres -c "$PGHOME/bin/pg_ctl -D $PGDATA -l logfile stop" #LOG
fi
/sbin/restorecon -r $PGDATA
else
echo " Error: Postgres does not look like its installed" #LOG
fi
echo "Installing DBD::Pg Perl Module" #LOG
PGHOME=$(ls -dtr /usr/pgsql* | tail -1)
if [ -n "$PGHOME" -a -d "$PGHOME" ]; then
echo "Setting POSTGRES_HOME=$PGHOME" #LOG
export POSTGRES_HOME=$PGHOME
export POSTGRES_LIB="$POSTGRES_HOME/lib -lssl -lcrypto"
cpanm -n DBD::Pg #LOG
else
echo "Could not find psql version" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S570V-Postgres
Symlink
+1
View File
@@ -0,0 +1 @@
S570A-Postgres
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S650A-Crontabs
+25
View File
@@ -0,0 +1,25 @@
# description: Install crontabs
###
# Crontab Entries
###
WGET="wget -qO - http://$KSRV/KickStart/cfg/$DIST/share/crontab"
if [ "$PTYPE" != "server" ]; then
echo "Installing crontabs" #LOG
$WGET/root.crontab | /usr/bin/crontab -u root - #LOG
$WGET/postgres.crontab | /usr/bin/crontab -u postgres - #LOG
$WGET/noc.crontab | /usr/bin/crontab -u noc - #LOG
else
echo "Installing server crontabs" #LOG
WGET="${WGET}-server"
$WGET/root.crontab | /usr/bin/crontab -u root - #LOG
$WGET/postgres.crontab | /usr/bin/crontab -u postgres - #LOG
$WGET/noc.crontab | /usr/bin/crontab -u noc - #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S650V-Crontabs
Symlink
+1
View File
@@ -0,0 +1 @@
S650A-Crontabs
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S700P-chroot-as400
+169
View File
@@ -0,0 +1,169 @@
#!/bin/bash
# Description: Setup chroot & iglass (formally as400) account
# Default user/group to lock down to
CRU='iglass'
#CRUID='1109'
CRUID='2222'
CRG='iglass'
CRGID='222'
umask 0022
# NEXTUID=$(cat /etc/passwd | grep -v nfsnobody | cut -d: -f 3 | sort -g | tail -1)
if [ -n "$IG_HOME" ]; then
PATH="$PATH:$IG_HOME/bin"
fi
while getopts "u:i:g:k:d" optionName; do
case "$optionName" in
u) CRU="$OPTARG";;
i) CRUID="$OPTARG";;
g) CRG="$OPTARG";;
k) CRGID="$OPTARG";;
d) DEBUG=1;;
esac
done
if [ -z "$IG_HOME" ]; then
PATH="$PATH:/opt/iglass/bin"
echo Looks like you are not running from KickStart?
echo
echo "Usage: $0 [-u <user>] [-i <uid>] [-g <grp>] [-k <grpid>]"
echo
echo "Default: $0 -u $CRU -i $CRUID -g $CRG -k $CRGID"
echo
read -t 60 -p "Continue? [no]" OK
if shopt -q nocasematch; then
NC=1
fi
shopt -s nocasematch
if [ "$OK" != "y" -a "$OK" != "yes" ]; then
echo bye bye
exit 1
fi
if [ -n "$NC" ]; then
shopt -s nocasematch
fi
fi
MKJAIL=$(which mkjail.pl)
ME=`whoami`
if ! grep -qE '^VERSION="(9|10)' /etc/os-release; then
echo "This script is for AlmaLinux 9 or 10" #LOG
elif [ -z "$MKJAIL" -o ! -e "$MKJAIL" ]; then
echo "No valid mkjail found!" #LOG
elif [ $ME != 'root' ]; then
echo "Please run as root!";
else
CRDIR=/home/chroot-$CRU
mkdir -vp $CRDIR/home/$CRU #LOG
echo "Creating $CRU($CRG) Account" #LOG
groupadd -g $CRGID $CRG #LOG
#useradd -c "$CRU restricted account" -d $CRDIR/$CRU -m -u $CRUID -g $CRGID -s /usr/bin/rssh $CRU #LOG
useradd -c "$CRU restricted account" -M -d /home/./$CRU -u $CRUID -g $CRGID -s /sbin/nologin $CRU #LOG
chown $CRU:$CRG $CRDIR/home/$CRU
chmod 750 $CRDIR/home/$CRU
cat /dev/null > $CRDIR/home/$CRU/.bash_profile
cat /dev/null > $CRDIR/home/$CRU/.bash_logout
cat /dev/null > $CRDIR/home/$CRU/.bashrc
chown root.root $CRDIR/home/$CRU/.bash_profile
chown root.root $CRDIR/home/$CRU/.bash_logout
chown root.root $CRDIR/home/$CRU/.bashrc
mkdir $CRDIR/home/$CRU/.ssh
chmod 700 $CRDIR/home/$CRU/.ssh
touch $CRDIR/home/$CRU/.ssh/authorized_keys
chmod 600 $CRDIR/home/$CRU/.ssh/authorized_keys
chown -R $CRU.$CRG $CRDIR/home/$CRU/.ssh
mkdir -p $CRDIR/home/$CRU/.config/systemd/user/
ln -s /dev/null $CRDIR/home/.config/systemd/user/pulseaudio.socket
ln -s /dev/null $CRDIR/home/.config/systemd/user/pulseaudio.service
chown -R $CRU.$CRG $CRDIR/home/.config
mkdir $CRDIR/home/$CRU/work
chown -R noc.$CRG $CRDIR/home/$CRU/work
chmod 770 $CRDIR/home/$CRU/work
cd /home
ln -s chroot-$CRU/home/$CRU .
if grep -Eq 'userlist_deny=NO' /etc/vsftpd/user_list; then
echo "vsftpd user_list already modified"
else
cp -af /etc/vsftpd/user_list /etc/vsftpd/user_list.orig
echo "# userlist_deny=NO So only allow users in this file" > /etc/vsftpd/user_list
fi
echo "$CRU" >> /etc/vsftpd/user_list
if ! grep -Eq '^cmds_denied=' /etc/vsftpd/vsftpd.conf; then
cat >> /etc/vsftpd/vsftpd.conf <<EOF
userlist_deny=NO
reverse_lookup_enable=NO
syslog_enable=YES
chroot_local_user=YES
passwd_chroot_enable=YES
max_clients=5
cmds_denied=mkdir,rmdir,rename
EOF
else
echo "vsftpd conf already modified"
fi
$MKJAIL $CRU #LOG
usermod -a -G $CRU noc #LOG
groupadd -g 2002 sftpusers
usermod -G sftpusers $CRU #LOG
if ! grep -Eq '^Subsystem\s+sftp\s+internal-sftp' /etc/ssh/sshd_config; then
echo "Configuring sshd for internal-sftp"
perl -spi -e 's/^(Subsystem.+)$/#$1\nSubsystem sftp internal\-sftp \-u 0022/' /etc/ssh/sshd_config
else
echo "sshd already configured for internal-sftp"
fi
if ! grep -Eq '^Match\s+Group\s+sftpusers' /etc/ssh/sshd_config; then
echo "Configuring sshd for sftpusers group"
echo -e "Match Group sftpusers\n\tChrootDirectory /home/chroot-%u" >> /etc/ssh/sshd_config
#echo -e "Match Group sftpusers\n\tChrootDirectory /home/chroot-%u\n\tForceCommand internal-sftp" >> /etc/ssh/sshd_config
else
echo "sshd already configured for sftpusers group"
fi
cd $CRDIR/bin
if [ ! -e rbash ]; then
mv -f bash rbash #LOG
fi
rm -f bash #LOG
#usermod -s /bin/rbash $CRU #LOG
if ! grep -q rbash /etc/shells; then
echo "/bin/rbash" >> /etc/shells
echo "/sbin/nologin" >> /etc/shells
fi
#setsebool -P ftp_home_dir=1
#setsebool -P ftpd_full_access on
setsebool -P ftpd_connect_all_unreserved 1
if [ -z "$IG_HOME" ]; then
echo
echo "REMINDER: Set password with 'passwd $CRU'"
echo
fi
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S700V-chroot-as400
Symlink
+1
View File
@@ -0,0 +1 @@
S700P-chroot-as400
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S701A-chroot-maps
+168
View File
@@ -0,0 +1,168 @@
#!/bin/bash
# Description: Setup chroot & iglass (formally as400) account
# Default user/group to lock down to
CRU='maps'
CRUID='2228'
CRG='maps'
CRGID='228'
umask 0022
# NEXTUID=$(cat /etc/passwd | grep -v nfsnobody | cut -d: -f 3 | sort -g | tail -1)
if [ -n "$IG_HOME" ]; then
PATH="$PATH:$IG_HOME/bin"
fi
while getopts "u:i:g:k:d" optionName; do
case "$optionName" in
u) CRU="$OPTARG";;
i) CRUID="$OPTARG";;
g) CRG="$OPTARG";;
k) CRGID="$OPTARG";;
d) DEBUG=1;;
esac
done
if [ -z "$IG_HOME" ]; then
PATH="$PATH:/opt/iglass/bin"
echo Looks like you are not running from KickStart?
echo
echo "Usage: $0 [-u <user>] [-i <uid>] [-g <grp>] [-k <grpid>]"
echo
echo "Default: $0 -u $CRU -i $CRUID -g $CRG -k $CRGID"
echo
read -t 60 -p "Continue? [no]" OK
if shopt -q nocasematch; then
NC=1
fi
shopt -s nocasematch
if [ "$OK" != "y" -a "$OK" != "yes" ]; then
echo bye bye
exit 1
fi
if [ -n "$NC" ]; then
shopt -s nocasematch
fi
fi
MKJAIL=$(which mkjail.pl)
ME=`whoami`
if ! grep -qE '^VERSION="(9|10)' /etc/os-release; then
echo "This script is for AlmaLinux 9 or 10" #LOG
elif [ -z "$MKJAIL" -o ! -e "$MKJAIL" ]; then
echo "No valid mkjail found!" #LOG
elif [ $ME != 'root' ]; then
echo "Please run as root!";
else
CRDIR=/home/chroot-$CRU
mkdir -vp $CRDIR/home/$CRU #LOG
echo "Creating $CRU($CRG) Account" #LOG
groupadd -g $CRGID $CRG #LOG
#useradd -c "$CRU restricted account" -d $CRDIR/$CRU -m -u $CRUID -g $CRGID -s /usr/bin/rssh $CRU #LOG
useradd -c "$CRU restricted account" -M -d /home/./$CRU -u $CRUID -g $CRGID -s /sbin/nologin $CRU #LOG
chown $CRU:$CRG $CRDIR/home/$CRU
chmod 750 $CRDIR/home/$CRU
cat /dev/null > $CRDIR/home/$CRU/.bash_profile
cat /dev/null > $CRDIR/home/$CRU/.bash_logout
cat /dev/null > $CRDIR/home/$CRU/.bashrc
chown root.root $CRDIR/home/$CRU/.bash_profile
chown root.root $CRDIR/home/$CRU/.bash_logout
chown root.root $CRDIR/home/$CRU/.bashrc
mkdir $CRDIR/home/$CRU/.ssh
chmod 700 $CRDIR/home/$CRU/.ssh
touch $CRDIR/home/$CRU/.ssh/authorized_keys
chmod 600 $CRDIR/home/$CRU/.ssh/authorized_keys
chown -R $CRU.$CRG $CRDIR/home/$CRU/.ssh
mkdir -p $CRDIR/home/$CRU/.config/systemd/user/
ln -s /dev/null $CRDIR/home/.config/systemd/user/pulseaudio.socket
ln -s /dev/null $CRDIR/home/.config/systemd/user/pulseaudio.service
chown -R $CRU.$CRG $CRDIR/home/.config
mkdir $CRDIR/home/$CRU/work
chown -R noc.$CRG $CRDIR/home/$CRU/work
chmod 770 $CRDIR/home/$CRU/work
cd /home
ln -s chroot-$CRU/home/$CRU .
if grep -Eq 'userlist_deny=NO' /etc/vsftpd/user_list; then
echo "vsftpd user_list already modified"
else
cp -af /etc/vsftpd/user_list /etc/vsftpd/user_list.orig
echo "# userlist_deny=NO So only allow users in this file" > /etc/vsftpd/user_list
fi
echo "$CRU" >> /etc/vsftpd/user_list
if ! grep -Eq '^cmds_denied=' /etc/vsftpd/vsftpd.conf; then
cat >> /etc/vsftpd/vsftpd.conf <<EOF
userlist_deny=NO
reverse_lookup_enable=NO
syslog_enable=YES
chroot_local_user=YES
passwd_chroot_enable=YES
max_clients=5
cmds_denied=mkdir,rmdir,rename
EOF
else
echo "vsftpd conf already modified"
fi
$MKJAIL $CRU #LOG
usermod -a -G $CRU noc #LOG
groupadd -g 2002 sftpusers
usermod -G sftpusers $CRU #LOG
if ! grep -Eq '^Subsystem\s+sftp\s+internal-sftp' /etc/ssh/sshd_config; then
echo "Configuring sshd for internal-sftp"
perl -spi -e 's/^(Subsystem.+)$/#$1\nSubsystem sftp internal\-sftp \-u 0022/' /etc/ssh/sshd_config
else
echo "sshd already configured for internal-sftp"
fi
if ! grep -Eq '^Match\s+Group\s+sftpusers' /etc/ssh/sshd_config; then
echo "Configuring sshd for sftpusers group"
echo -e "Match Group sftpusers\n\tChrootDirectory /home/chroot-%u" >> /etc/ssh/sshd_config
#echo -e "Match Group sftpusers\n\tChrootDirectory /home/chroot-%u\n\tForceCommand internal-sftp" >> /etc/ssh/sshd_config
else
echo "sshd already configured for sftpusers group"
fi
cd $CRDIR/bin
if [ ! -e rbash ]; then
mv -f bash rbash #LOG
fi
rm -f bash #LOG
#usermod -s /bin/rbash $CRU #LOG
if ! grep -q rbash /etc/shells; then
echo "/bin/rbash" >> /etc/shells
echo "/sbin/nologin" >> /etc/shells
fi
#setsebool -P ftp_home_dir=1
#setsebool -P ftpd_full_access on
setsebool -P ftpd_connect_all_unreserved 1
if [ -z "$IG_HOME" ]; then
echo
echo "REMINDER: Set password with 'passwd $CRU'"
echo
fi
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S701P-chroot-maps
Symlink
+1
View File
@@ -0,0 +1 @@
S701A-chroot-maps
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S701V-chroot-maps
Symlink
+1
View File
@@ -0,0 +1 @@
S701A-chroot-maps
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S750A-serialLogin
+45
View File
@@ -0,0 +1,45 @@
# description: Setup serial console login
echo "# /proc/tty/driver/serial:" #LOG
cat /proc/tty/driver/serial #LOG
TTYS=$(perl -ne 'if (/^(\d):.+uart:16550A/) { print "ttyS$1\n"; last }' /proc/tty/driver/serial)
if [ -e "/dev/$TTYS" ]; then
echo "Modifing grub for sol with $TTYS" #LOG
perl -spi -e 's/^(GRUB_TIMEOUT=)\d+$/${1}2/' /etc/default/grub
echo "Adding console for BMC" #LOG
# The elevator is the I/O scheduler, nothing to do with the console login but since we're changing it here...
#TTYS=$TTYS perl -spi -e 's/^(GRUB_CMDLINE_LINUX=.+) rhgb quiet(.*)$/$1 elevator=noop console=tty0 console=$ENV{TTYS},115200$2/' /etc/default/grub
# TODO: RHEL 10 may not include 'rhgb quiet' in GRUB_CMDLINE_LINUX by default.
# If the pattern doesn't match, console= params are silently not added.
# Verify grub defaults on first RHEL 10 physical probe build and adjust regex if needed.
TTYS=$TTYS perl -spi -e 's/^(GRUB_CMDLINE_LINUX=.+) rhgb quiet(.*)$/$1 console=tty0 console=$ENV{TTYS},115200$2/' /etc/default/grub
if [ -d /sys/firmware/efi ]; then
echo "UEFI" #LOG
if [ -d /boot/efi/EFI/almalinux ]; then
grub2-mkconfig -o /boot/efi/EFI/almalinux/grub.cfg #LOG
else
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg #LOG
fi
else
echo "BIOS" #LOG
grub2-mkconfig -o /boot/grub2/grub.cfg #LOG
fi
systemctl start serial-getty@$TTYS.service
systemctl enable serial-getty@$TTYS.service
# TODO: /etc/securetty was removed in RHEL 9.2+ and does not exist in RHEL 10.
# PAM no longer uses it. This block is a no-op on RHEL 10 but harmless if guarded.
if [ -f /etc/securetty ] && ! grep -q "$TTYS" /etc/securetty; then
echo "$TTYS" >> /etc/securetty
fi
else
echo "No console tty found" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S800A-netsnmp
+20
View File
@@ -0,0 +1,20 @@
# description: Configure box snmp
echo "Adding snmp configuration" #LOG
echo "Extracting MIBs" #LOG
rm -f /usr/share/snmp/mibs/* #LOG
mkdir -p /usr/share/snmp/mibs #LOG
wget -q -O - http://$KSRV/KickStart/cfg/$DIST/share/mibs.tgz | tar xzf - -C /usr/share/snmp/mibs #LOG
echo "Setting MIBs ownership to noc" #LOG
chown -R 1108:106 /usr/share/snmp/mibs #LOG
restorecon -R /usr/share/snmp/mibs #LOG
chown 1108:106 /etc/snmp/snmpd.conf #LOG
ln -s /etc/sysconfig/snmpd /etc/snmp/snmpd.options
ln -s /etc/sysconfig/snmptrapd /etc/snmp/snmptrapd.options
restorecon -R /etc/snmp
restorecon -R /etc/sysconfig
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S800V-netsnmp
Symlink
+1
View File
@@ -0,0 +1 @@
S800A-netsnmp
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S820A-OpenManage
+103
View File
@@ -0,0 +1,103 @@
# description: Install Dell OpenManage
if systemd-detect-virt --quiet; then
echo "Skipping Dell OpenManage on virtualized test system" #LOG
else
echo "Installing Dell OpenManage" #LOG
wget -q -O - https://linux.dell.com/repo/hardware/dsu/bootstrap.cgi | bash #LOG
dnf -y install dell-system-update #LOG
dnf -y install srvadmin-all #LOG
dnf -y install srvadmin-selinux #LOG
dnf -y install srvadmin-idracadm7 #LOG
echo "Done installing Dell OpenManage Services" #LOG
# srvadmin-services.sh start
export PATH=$PATH:/opt/dell/srvadmin/bin:/opt/dell/srvadmin/sbin
systemctl disable dsm_om_connsvc #LOG
systemctl enable dsm_sa_datamgrd.service #LOG
systemctl enable dsm_sa_eventmgrd.service #LOG
systemctl enable dsm_sa_snmpd.service #LOG
echo "BIOS boot setup before:" #LOG
ipmitool chassis bootdev disk options=persistent #LOG
#BOOTINFO=/dev/shm/boot.info
#omreport chassis biossetup display=shortnames </dev/null | perl -ne 'if (/^BootSeq\s*$/) {$p=1;next} elsif (/^s*$/) {$p=0} if ($p) {print};' | tee $BOOTINFO #LOG
#
#if ! grep -q Error $BOOTINFO; then
#
# HDD0=$(grep HardDisk $BOOTINFO | head -1 | awk '{ print $4 }')
# NIC0=$(grep NIC $BOOTINFO | head -1 | awk '{ print $4 }')
# OPT0=$(grep Optical $BOOTINFO | head -1 | awk '{ print $4 }')
#
# # Drives seemed to be swapped during kickstart?
# if [ -n "$HDD0" ]; then
# if [ -n "$NIC0" ]; then
# if [ -n "$OPT0" ]; then
# omconfig chassis biossetup attribute=BootSeq sequence=$HDD0,$OPT0,$NIC0 </dev/null #LOG
# echo "Set BIOS disk order to $HDD0,$OPT0,$NIC0" #LOG
# else
# omconfig chassis biossetup attribute=BootSeq sequence=$HDD0,$NIC0 </dev/null #LOG
# echo "Set BIOS disk order to $HDD0,$NIC0" #LOG
# fi
# else
# if [ -n "$OPT0" ]; then
# omconfig chassis biossetup attribute=BootSeq sequence=$HDD0,$OPT0 </dev/null #LOG
# echo "Set BIOS disk order to $HDD0,$OPT0 [No PXE NIC found]" #LOG
# else
# omconfig chassis biossetup attribute=BootSeq sequence=$HDD0 </dev/null #LOG
# echo "Set BIOS disk order to $HDD0 [No PXE NIC found]" #LOG
# fi
# fi
# else
# omconfig chassis biossetup attribute=BootSeq sequence=HardDisk.List.1-1 </dev/null #LOG
# echo "Guessing! Set BIOS disk order to HardDisk.List.1-1 [No HDD found]" #LOG
# fi
#
#else
# echo "Skipped OpenManage boot order setup due to previous error" #LOG
#
#fi
echo "Setting iDRAC ssh access:" #LOG
racadm set iDRAC.IPv4.Enable Enabled #LOG
racadm set iDRAC.Nic.Enable 1 #LOG
racadm set iDRAC.NIC.Selection LOM1 #LOG
racadm set iDRAC.SSH.Enable 1 #LOG
echo "Setting Power Recovery:" #LOG
#omconfig chassis biossetup attribute=AcPwrRcvry setting=On #LOG
#omconfig chassis biossetup attribute=AcPwrRcvryDelay setting=Immediate #LOG
ipmitool chassis policy always-on #LOG
#echo "Setting CPU Virtualization Support" #LOG
#omconfig chassis biossetup attribute=cpuvt setting=enabled #LOG
#echo "Setting redirect of bios/bootup to com2/ttyS1" #LOG
#omconfig chassis biossetup attribute=SerialComm setting=OnConRedirCom2 #LOG
echo "Enabling IPMI Over LAN" #LOG
#omconfig chassis remoteaccess config=nic enableipmi=true #LOG
ipmitool sol set enabled true 1 #LOG
echo "Setting BMC Remote Access IP Address" #LOG
#omconfig chassis remoteaccess config=nic ipaddress=10.1.1.99 subnet=255.255.255.0 ipsource=static gateway=10.1.1.1 #LOG
ipmitool lan set 1 ipsrc static #LOG
ipmitool lan set 1 ipaddr 10.1.1.99 #LOG
ipmitool lan set 1 netmask 255.255.255.0 #LOG
ipmitool lan set 1 defgw ipaddr 10.1.1.1 #LOG
echo "Setting BMC Remote Access User" #LOG
#omconfig chassis remoteaccess config=user id=2 newpw=nocnoc confirmnewpw=nocnoc #LOG
ipmitool user set password 2 nocnoc #LOG
echo "BMC remote access settings" #LOG
#omreport chassis remoteaccess |grep -A20 "Remote Access Device" #LOG
ipmitool lan print 1 #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S850A-network
+92
View File
@@ -0,0 +1,92 @@
# description: Setup network for local DHCP
# AlmaLinux 10: dhclient removed; ifcfg format removed.
# DNS overrides now use NetworkManager keyfile (via Python configparser).
# NTP is handled by S350A-chrony.
echo "Network configuration." #LOG
export ST=`/usr/sbin/dmidecode -s system-serial-number`
echo "Service Code = $ST" #LOG
if [ "$PTYPE" == "server" ]; then
hostnamectl set-hostname server${ST}.iglass.net
echo "server$ST.iglass.net" > /etc/hostname
else
hostnamectl set-hostname iprobe${ST}.iglass.net
echo "iprobe$ST.iglass.net" > /etc/hostname
fi
echo "Disabling ipv6" #LOG
cat >> /etc/sysctl.d/10-ipv6.conf <<EOF
# Turn off IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
EOF
sysctl -p /etc/sysctl.d/10-ipv6.conf #LOG
echo "Disabling link-local autoconf" #LOG
cat >> /etc/sysctl.d/11-autoconf.conf <<EOF
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.default.autoconf = 0
EOF
sysctl -p /etc/sysctl.d/11-autoconf.conf #LOG
FIRST=
for I in $(cd /sys/class/net/; ls -d eth* em* eno* p* enp* 2>/dev/null); do
[ "$I" = "lo" ] && continue
[ -z "$FIRST" ] && export FIRST=$I
echo "Found interface $I" #LOG
done
if [ "$PTYPE" = "server" ]; then
echo "Not configuring probe DNS overrides on server type" #LOG
elif [ -n "$FIRST" ]; then
echo "Configuring NM DNS overrides for probe on $FIRST" #LOG
# AlmaLinux 10: dhclient is removed. Set DNS via NetworkManager keyfile.
# The installer creates connection profiles in /etc/NetworkManager/system-connections/.
# We use Python configparser to inject iglass DNS ahead of DHCP-provided DNS.
NM_CON_DIR=/etc/NetworkManager/system-connections
python3 - "$NM_CON_DIR" "$FIRST" <<'PYEOF'
import configparser, os, sys, glob
condir, first = sys.argv[1], sys.argv[2]
# Prefer a profile matching the interface name; fall back to any ethernet profile
candidates = (
glob.glob(os.path.join(condir, first + '.nmconnection')) +
glob.glob(os.path.join(condir, '*.nmconnection'))
)
for path in candidates:
cfg = configparser.RawConfigParser()
cfg.optionxform = str # preserve case
cfg.read(path)
if not cfg.has_section('ipv4'):
continue
if cfg.get('ipv4', 'method', fallback='') not in ('auto', 'dhcp'):
continue
# Prepend iglass DNS; negative priority means ours ranks above DHCP-provided DNS
cfg.set('ipv4', 'dns', '192.168.194.102;192.168.226.102;')
cfg.set('ipv4', 'dns-search', 'iglass.net;')
cfg.set('ipv4', 'dns-priority', '-100')
with open(path, 'w') as fh:
cfg.write(fh)
os.chmod(path, 0o600)
print(f"Updated NM profile: {path}")
break
else:
print(f"WARNING: No DHCP ethernet NM profile found under {condir}", file=sys.stderr)
PYEOF
else
echo "WARNING: No network interfaces found" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S856A-unbound
+13
View File
@@ -0,0 +1,13 @@
echo "Setting up unbound" #LOG
echo "Telling NetworkManager not to override DNS" #LOG
perl -spi -e 's/^(\[main\].*)$/$1\ndns=none/' /etc/NetworkManager/NetworkManager.conf #LOG
echo "Making sure selinux perms are restored" #LOG
restorecon -FR /etc/systemd
echo "reloading/restarting NetworkManager and Unbound" #LOG
systemctl daemon-reload
systemctl enable --now unbound.service #LOG
systemctl restart NetworkManager
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S856V-unbound
Symlink
+1
View File
@@ -0,0 +1 @@
S856A-unbound
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S870A-sshd
+26
View File
@@ -0,0 +1,26 @@
# This is a bug when IPv6 is turned off X forwarding will not try IPv4 localhost if no IPv6 localhost
#perl -spi -e 's/(#?AddressFamily\s+.+)$/$1\nAddressFamily inet/' /etc/ssh/sshd_config
#perl -spi -e 's/^(#MaxSessions 10.*)/$1\nMaxSessions 20/' /etc/ssh/sshd_config
echo "MaxSessions 20" >> /etc/ssh/sshd_config.d/90-authnz.conf
echo "" >> /etc/ssh/sshd_config.d/90-authnz.conf
#perl -spi -e 's/^\s*PermitRootLogin\s+yes/PermitRootLogin no/' /etc/ssh/sshd_config
echo "PermitRootLogin no" >> /etc/ssh/sshd_config.d/90-authnz.conf
echo "PubkeyAuthentication yes" >> /etc/ssh/sshd_config.d/90-authnz.conf
cat >> /etc/ssh/sshd_config.d/90-authnz.conf <<'EOF'
Match Address 10.1.1.1,10.1.1.254
PermitRootLogin prohibit-password
EOF
# sshd -T |grep -iE 'cipher|mac|algorithm'
# sshd -t # test config
echo 'KexAlgorithms=curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512' > /etc/ssh/sshd_config.d/30-crypto-policy-override.conf
#echo '[email protected],[email protected],aes256-ctr,[email protected],aes128-ctr' >> /etc/ssh/sshd_config.d/30-crypto-policy-override.conf
echo '[email protected],aes256-ctr,[email protected],aes128-ctr' >> /etc/ssh/sshd_config.d/30-crypto-policy-override.conf
restorecon -F /etc/ssh/sshd_config.d/*
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S870V-sshd
Symlink
+1
View File
@@ -0,0 +1 @@
S870A-sshd
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S875A-firewall
+14
View File
@@ -0,0 +1,14 @@
systemctl enable firewalld.service #LOG
cat >> /etc/sysctl.d/15-fw-console.conf <<EOF
# Turn off below warn(4) to not log firewall messages to console
kernel.printk = 4 4 1 7
EOF
##perl -spi -e 's/^DefaultZone=.+$/DefaultZone=external/' /etc/firewalld/firewalld.conf
perl -spi -e 's/^DefaultZone=.+$/DefaultZone=block/' /etc/firewalld/firewalld.conf
##perl -spi -e 's/^LogDenied=.+$/LogDenied=all/' /etc/firewalld/firewalld.conf
perl -spi -e 's/^LogDenied=.+$/LogDenied=unicast/' /etc/firewalld/firewalld.conf
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S875V-firewall
Symlink
+1
View File
@@ -0,0 +1 @@
S875A-firewall
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S900P-rmlogrotate
Symlink
+1
View File
@@ -0,0 +1 @@
S900S-rmlogrotate
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S900S-rmlogrotate
+16
View File
@@ -0,0 +1,16 @@
echo "remove logrotate" #LOG
# Disable cron-based logrotate (RHEL 9 and earlier)
if [ -f /etc/cron.daily/logrotate ]; then
sed -i '2s/^/exit\n/' /etc/cron.daily/logrotate #LOG
fi
# Disable timer-based logrotate (RHEL 10+)
systemctl disable --now logrotate.timer 2>/dev/null || true #LOG
rm -f /etc/systemd/system/timers.target.wants/logrotate.timer #LOG
if [ "$PTYPE" != "server" ]; then
touch /var/log/iglass.log
touch /var/log/customer.log
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S900V-rmlogrotate
Symlink
+1
View File
@@ -0,0 +1 @@
S900S-rmlogrotate
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S910P-chmodHosts
+4
View File
@@ -0,0 +1,4 @@
echo "Allow hosts" #LOG
chgrp 106 /etc/hosts #LOG
chmod g+w /etc/hosts #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S910V-chmodHosts
Symlink
+1
View File
@@ -0,0 +1 @@
S910P-chmodHosts
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S920V-vm-cleanup
+58
View File
@@ -0,0 +1,58 @@
TMPCRON=/dev/shm/tmp.root.tab
/usr/bin/crontab -u root -l | grep -vE hwclock >> $TMPCRON
/usr/bin/crontab -u root $TMPCRON #LOG
rm -f $TMPCRON #LOG
# dhclient removed in RHEL 10 — dhclient.conf migration no longer needed
mv -f /etc/rsyslog.d/50-events.conf.on /etc/rsyslog.d/50-events.conf
rm -rf /etc/systemd/system/syslog.service.d/
systemctl daemon-reload #LOG
systemctl restart rsyslog #LOG
# Quiet VM-only boot noise from hardware-specific services/configs that are not
# relevant to lab PXE validation.
mkdir -p /etc/modules-load.d
ln -sfn /dev/null /etc/modules-load.d/fwupd-msr.conf #LOG
systemctl disable systemd-network-generator.service 2>/dev/null || true #LOG
systemctl disable NetworkManager-wait-online.service 2>/dev/null || true #LOG
if [ -t 0 ]; then
read -r -p 'Customer: ' CUST
read -r -p 'Hostname/Probe: ' HOST
else
CUST=${VM_CUST:-}
HOST=${VM_HOST:-}
echo "No interactive tty available; skipping Customer/Hostname prompts" #LOG
fi
if [ -n "$CUST" ]; then
echo "Setting customer to $CUST"
date +"%s:$CUST:" > /opt/iglass/conf/.probe-config
else
echo "No customer supplied"
fi
if [ -n "$HOST" ]; then
if ! [[ "$HOST" =~ ".iglass.net"$ ]]; then
HOST="$HOST.iglass.net"
fi
echo "Setting hostname to $HOST"
hostnamectl set-hostname $HOST
else
echo "No host supplied"
fi
rm -f /etc/firewalld/zones/ks.xml #LOG
systemctl restart firewalld.service #LOG
systemctl restart rsyslog.service #LOG
echo "CHECK firewalld config!" #LOG
[ -e "/etc/firewalld/zones/public.xml" ] && echo "public zone exists!" #LOG
#echo -n "etho0: "; grep ZONE /etc/sysconfig/network-scripts/ifcfg-eth0
echo "firewall-cmd --get-active-zones:" #LOG
firewall-cmd --get-active-zones #LOG
echo
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S930A-security
+15
View File
@@ -0,0 +1,15 @@
# usbguard-selinux scriptlet crashes the anaconda RPM transaction on AlmaLinux 10.
# Install here post-boot instead of via %packages.
echo "Installing usbguard post-boot" #LOG
dnf install -y usbguard 2>&1 #LOG
if [ -e /bin/usbguard ]; then
echo "Generating usbguard rules but leaving disabled" #LOG
/bin/usbguard generate-policy > /etc/usbguard/rules.conf
perl -spi -e 's/ImplicitPolicyTarget=\S+/ImplicitPolicyTarget=reject/' /etc/usbguard/usbguard-daemon.conf
systemctl disable usbguard.service #LOG
else
echo "WARNING: usbguard install failed, skipping setup" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S940A-cis-hardening
+283
View File
@@ -0,0 +1,283 @@
# description: CIS Benchmark Level 1 hardening for AlmaLinux 10
# Ref: CIS AlmaLinux OS 10 Benchmark (tracks RHEL 10)
#
# Scope of this script:
# 1.1.1 Disable unused filesystem kernel modules
# 1.6 Disable core dumps
# 3.3 Network sysctl hardening
# 4.x Auditd — audit rules and kernel boot flag
# 5.3 sudo hardening
# 5.4.1 Password quality (pwquality)
# 5.4.3 Account lockout (pam_faillock settings)
#
# Intentionally deferred / excluded:
# 1.1.2 /tmp as separate partition or tmpfs
# — deferred pending dev review of /var/tmp exec requirements
# 5.4.2 Password aging
# — no expiration policy for any accounts in this environment
# authselect profile change
# — ipa-client-install owns the authselect profile; run
# `authselect enable-feature with-faillock` after IPA enrolment
# squashfs module blacklist
# — left enabled; containers (Docker/Podman) require it
# udf module blacklist
# — left enabled; not prohibited in this environment
# AIDE cron
# — managed centrally, not locally
echo "### CIS Level 1 hardening" #LOG
# ── 1.1.1 Disable unused filesystem modules ───────────────────────────────────
echo "CIS 1.1.1: Blacklisting unused filesystem modules" #LOG
cat > /etc/modprobe.d/cis-fs-blacklist.conf <<EOF
# CIS 1.1.1 — disable unused legacy filesystem drivers
install cramfs /bin/true
install freevxfs /bin/true
install hfs /bin/true
install hfsplus /bin/true
install jffs2 /bin/true
blacklist cramfs
blacklist freevxfs
blacklist hfs
blacklist hfsplus
blacklist jffs2
EOF
# ── 1.6 Core dump restrictions ────────────────────────────────────────────────
echo "CIS 1.6: Disabling core dumps" #LOG
cat > /etc/security/limits.d/cis-coredump.conf <<EOF
# CIS 1.6.1
* hard core 0
EOF
cat > /etc/sysctl.d/20-cis-coredump.conf <<EOF
# CIS 1.6.1 — disable setuid core dumps
fs.suid_dumpable = 0
EOF
mkdir -p /etc/systemd/coredump.conf.d
cat > /etc/systemd/coredump.conf.d/cis.conf <<EOF
[Coredump]
Storage=none
ProcessSizeMax=0
EOF
# ── 3.3 Network kernel parameter hardening ────────────────────────────────────
echo "CIS 3.3: Applying network sysctl hardening" #LOG
cat > /etc/sysctl.d/30-cis-network.conf <<EOF
# CIS 3.3.1 — disable IP forwarding (probes/servers are not routers)
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0
# CIS 3.3.2 — disable packet redirect sending
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
# CIS 3.3.3 — refuse source-routed packets
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
# CIS 3.3.4 — refuse ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
# CIS 3.3.5 — refuse secure ICMP redirects
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
# CIS 3.3.6 — log martian packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# CIS 3.3.7 — ignore broadcast ICMP echo requests
net.ipv4.icmp_echo_ignore_broadcasts = 1
# CIS 3.3.8 — ignore bogus ICMP error responses
net.ipv4.icmp_ignore_bogus_error_responses = 1
# CIS 3.3.9 — enable TCP SYN cookies
net.ipv4.tcp_syncookies = 1
# CIS 3.3.10 — disable IPv6 router advertisements
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
# Kernel ASLR
kernel.randomize_va_space = 2
EOF
sysctl -p /etc/sysctl.d/30-cis-network.conf #LOG
# ── 5.3 sudo hardening ────────────────────────────────────────────────────────
echo "CIS 5.3: sudo hardening" #LOG
cat > /etc/sudoers.d/cis-sudo <<EOF
# CIS 5.3.2 — require pty (prevents background sudo abuse)
Defaults use_pty
# CIS 5.3.3 — log all sudo usage
Defaults logfile=/var/log/sudo.log
# Limit credential caching to 5 minutes
Defaults timestamp_timeout=5
EOF
chmod 440 /etc/sudoers.d/cis-sudo
# ── 5.4.1 Password quality ────────────────────────────────────────────────────
echo "CIS 5.4.1: pwquality settings" #LOG
cat > /etc/security/pwquality.conf.d/cis.conf <<EOF
# CIS 5.4.1 — password complexity requirements
minlen = 14
minclass = 4
dcredit = -1
ucredit = -1
ocredit = -1
lcredit = -1
maxrepeat = 3
maxsequence = 3
EOF
# ── 5.4.3 Account lockout via pam_faillock ────────────────────────────────────
echo "CIS 5.4.3: pam_faillock configuration" #LOG
# Write faillock settings — these take effect once authselect enables the
# with-faillock feature. Run after IPA enrolment:
# authselect enable-feature with-faillock
cat > /etc/security/faillock.conf <<EOF
# CIS 5.4.3 — lock after 5 failures in a 15-minute window
deny = 5
fail_interval = 900
unlock_time = 900
even_deny_root
root_unlock_time = 60
EOF
# ── 4.x Audit daemon configuration ───────────────────────────────────────────
echo "CIS 4.x: Configuring auditd" #LOG
VM_DMI_SIGNATURE="$(
cat /sys/class/dmi/id/sys_vendor /sys/class/dmi/id/product_name /sys/class/dmi/id/product_version 2>/dev/null \
| tr '\n' ' '
)"
VM_KS_HINT=0
if [ "$PTYPE" = "VM" ] || [ -b /dev/vda ] || [ -d /sys/block/vda ]; then
VM_KS_HINT=1
fi
if [ "$VM_KS_HINT" -eq 1 ] || systemd-detect-virt --quiet || echo "$VM_DMI_SIGNATURE" | grep -Eiq 'qemu|kvm|vmware|virtualbox|standard pc'; then
AUDITD_VM_SKIP=1
echo "Skipping boot-time auditd enablement on virtualized test system" #LOG
mkdir -p /etc/systemd/system/dbus-broker.service.d
cat > /etc/systemd/system/dbus-broker.service.d/10-no-audit.conf <<'EOF'
[Service]
ExecStart=
ExecStart=/usr/bin/dbus-broker-launch --scope system
EOF
else
AUDITD_VM_SKIP=0
# 4.1.3 — Ensure auditing for processes that start prior to auditd
grubby --update-kernel=ALL --args="audit=1 audit_backlog_limit=8192" #LOG
fi
# 4.1.4–4.1.17 — Static audit rules
mkdir -p /etc/audit/rules.d
cat > /etc/audit/rules.d/cis.rules <<'AUDITRULES'
## CIS AlmaLinux 10 Level 1 — Section 4 audit rules
## Generated by kickstart S940A-cis-hardening
# 4.1.4 — Date/time changes
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change
-a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change
-a always,exit -F arch=b64 -S clock_settime -k time-change
-a always,exit -F arch=b32 -S clock_settime -k time-change
-w /etc/localtime -p wa -k time-change
# 4.1.5 — User/group identity changes
-w /etc/group -p wa -k identity
-w /etc/passwd -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/security/opasswd -p wa -k identity
# 4.1.6 — Network environment changes
-a always,exit -F arch=b64 -S sethostname -S setdomainname -k system-locale
-a always,exit -F arch=b32 -S sethostname -S setdomainname -k system-locale
-w /etc/issue -p wa -k system-locale
-w /etc/issue.net -p wa -k system-locale
-w /etc/hosts -p wa -k system-locale
-w /etc/NetworkManager/ -p wa -k system-locale
# 4.1.7 — MAC policy (SELinux)
-w /etc/selinux/ -p wa -k MAC-policy
-w /usr/share/selinux/ -p wa -k MAC-policy
# 4.1.8 — Login/logout events
-w /var/log/faillog -p wa -k logins
-w /var/log/lastlog -p wa -k logins
-w /var/log/tallylog -p wa -k logins
# 4.1.9 — Session initiation
-w /var/run/utmp -p wa -k session
-w /var/log/wtmp -p wa -k logins
-w /var/log/btmp -p wa -k logins
# 4.1.10 — Permission modification events
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chown -S fchown -S fchownat -S lchown -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chown -S fchown -S fchownat -S lchown -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F auid!=4294967295 -k perm_mod
# 4.1.11 — Unauthorized file access attempts
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
# 4.1.13 — File system mounts
-a always,exit -F arch=b64 -S mount -F auid>=1000 -F auid!=4294967295 -k mounts
-a always,exit -F arch=b32 -S mount -F auid>=1000 -F auid!=4294967295 -k mounts
# 4.1.14 — File deletion by users
-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete
-a always,exit -F arch=b32 -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete
# 4.1.15 — sudoers scope changes
-w /etc/sudoers -p wa -k scope
-w /etc/sudoers.d/ -p wa -k scope
# 4.1.16 — sudo log
-w /var/log/sudo.log -p wa -k actions
# 4.1.17 — Kernel module loading/unloading
-w /sbin/insmod -p x -k modules
-w /sbin/rmmod -p x -k modules
-w /sbin/modprobe -p x -k modules
-a always,exit -F arch=b64 -S init_module -S delete_module -k modules
-a always,exit -F arch=b32 -S init_module -S delete_module -k modules
AUDITRULES
# 4.1.12 — Privileged command auditing (generated dynamically)
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null \
| awk '{print "-a always,exit -F path=" $1 " -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged"}' \
>> /etc/audit/rules.d/cis.rules #LOG
# 4.1.18 — Make rules immutable (requires reboot to change)
echo "-e 2" >> /etc/audit/rules.d/cis.rules
# 4.1.2 — Enable auditd service
if [ "$AUDITD_VM_SKIP" -eq 0 ]; then
systemctl enable auditd #LOG
else
systemctl disable auditd 2>/dev/null || true #LOG
systemctl daemon-reload #LOG
fi
echo "CIS hardening complete" #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S960A-selinux
+16
View File
@@ -0,0 +1,16 @@
# Selinux relabel the filesystem
#
echo "Setting up local rules for selinux" #LOG
wget -qO /root/local.avc http://$KSRV/KickStart/cfg/$DIST/share/selinux/local.avc
echo 'cd /root/; audit2allow -M local -i local.avc; semodule -i local.pp; cd -' > /root/sefix.sh
sh /root/sefix.sh
echo "/usr/share/snmp/mibs/.index" >> /etc/selinux/restorecond.conf
# Do this before aide
echo "Restoring any incorrect context labels" #LOG
#touch /.autorelabel
fixfiles -F restore / #LOG
#restorecon -r / #LOG
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S960V-selinux
Symlink
+1
View File
@@ -0,0 +1 @@
S960A-selinux
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S980A-monitor
+19
View File
@@ -0,0 +1,19 @@
if [ -x /sbin/aide ]; then
echo "Creating initial aide reference db" #LOG
/usr/sbin/aide --init #LOG
if [ -e /var/lib/aide/aide.db.new.gz ]; then
mv -f /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz #LOG
elif [ -e /var/lib/aide/aide.db.new ]; then
mv -f /var/lib/aide/aide.db.new /var/lib/aide/aide.db #LOG
else
echo "WARNING: AIDE did not create a reference database" #LOG
fi
else
echo "No aide package found" #LOG
fi
# NOTE: 'enumerate = true' was removed in RHEL 10 for AD/IdM SSSD providers.
# Setting it would produce a warning and be ignored. Removed 2026-03-27.
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S980V-monitor
Symlink
+1
View File
@@ -0,0 +1 @@
S980A-monitor
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S990A-mailme
+293
View File
@@ -0,0 +1,293 @@
# Tell me when and where I'm done
# Copy these to mem for mailem and not cat them via the script to be on the safe side.
TMP=$(tmpdir)
if [ $? -eq 0 ]; then
SSHDIR="$TMP/ssh"
mkdir $SSHDIR
chmod -R 700 $TMP #LOG
ME='[email protected]'
HOME=/root
SAVEDIR="/home/KickStart/ks-logs"
# IDENTITY from SXXXP-noc-iglass
MAIL_FILE="kickstart.info"
SSHID="obelisk_rsa"
SSHCFG="$SSHDIR/ssh.cfg"
wget -nv -r -P$TMP/ssh -l 1 -nd --follow-tags a -A "$SSHID*" http://$KSRV/KickStart/ks/probe/ #LOG
chmod 600 $SSHDIR/$SSHID
chmod 644 $SSHDIR/${SSHID}.pub
if [ -z "$SSHPORT" ]; then
echo "WARNING: No ssh options defined" #LOG
SSHPORT=22
fi
echo "Host *
User ks
IdentityFile $SSHDIR/$SSHID
StrictHostKeyChecking no
UserKnownHostsFile $TMP/ssh/known_hosts
CheckHostIP no
VerifyHostKeyDNS no
BatchMode yes
Port $SSHPORT
ForwardX11 no" > $SSHCFG
chmod 664 $SSHCFG
SSHOPT="-F $SSHCFG"
if ! getent ahostsv4 "$SSHHOST" >/dev/null 2>&1; then
echo "WARNING: SSHHOST $SSHHOST is not resolvable, skipping remote notification steps" #LOG
exit 0
fi
#ST=`/usr/sbin/dmidecode |perl -ne 'if (/Serial Number:\s*(.+)/) { print "$1\n"; last;}'`
#MD=`/usr/sbin/dmidecode |perl -ne 'if (/Product Name:\s*(.+)/) { print "$1\n"; last;}'`
ST=`/usr/sbin/dmidecode -s system-serial-number | sed 's/\s*//g'`
MD=`/usr/sbin/dmidecode -s system-product-name`
#FP=`/usr/bin/ssh-keygen -l -f $IDENTITY 2> /dev/null | awk '{ print $2 }'`
#if [ -z "$ST" -a $(uname -r | grep -c amz) -eq 1 ]; then
# wget -q -O- http://s3.amazonaws.com/ec2metadata/ec2-metadata > /usr/local/bin/ec2-metadata
# chmod 755 /usr/local/bin/ec2-metadata
# /usr/local/bin/ec2-metadata --instance-id
if [ "$PTYPE" = "VM" ]; then
if [ -z "$ST" ]; then
ST=$(wget -q -O- http://169.254.169.254/latest/meta-data/instance-id)
fi
if [ -z "$MD" ]; then
MD=$(wget -q -O- http://169.254.169.254/latest/meta-data/instance-type)
fi
fi
KEY="NA"
# CRYPTEDDEV should be previously setup if kickstarted
if [ -e "$CRYPTEDDEV" ]; then
KEY=`cryptsetup luksUUID $CRYPTEDDEV 2> /dev/null`
fi
# Detect primary NIC — try legacy names first, fall back to first non-loopback
ETH=
for _candidate in eth0 em1 p1p1 eno1; do
if [ -e "/sys/class/net/$_candidate" ]; then
ETH="$_candidate"
break
fi
done
if [ -z "$ETH" ]; then
ETH=$(ip -o link show | awk -F': ' '!/lo/{print $2; exit}')
fi
if [ -z "$ETH" ]; then
echo "ERROR: No primary network interface found" #LOG
fi
MAC=$(ip -o addr show $ETH | perl -ne 'if (/link\/ether\s+(\S+)/) { print "$1\n"; }')
if [ -n "$ST" ]; then
SAVEDIR="$SAVEDIR/$ST"
elif [ -n "$MAC" ]; then
ST="$MAC"
SAVEDIR="$SAVEDIR/$MAC"
else
SAVEDIR="$SAVEDIR/`date +%s`"
fi
ssh $SSHOPT $SSHHOST mkdir $SAVEDIR #LOG
ssh $SSHOPT $SSHHOST chmod 700 $SAVEDIR/ #LOG
if [ -e "$CPP1" ]; then
OP=`cat $CPP1`
fi
echo "Creating email to $ME" #LOG
echo "Date: `date`" > $TMP/$MAIL_FILE
echo "Product Name: $MD" >> $TMP/$MAIL_FILE
echo "Service Tag: $ST" >> $TMP/$MAIL_FILE
echo "Distribution: $DIST" >> $TMP/$MAIL_FILE
echo "iGlass Version: "$(echo $IGRELEASE | awk -F . '{print $1}') >> $TMP/$MAIL_FILE
echo "Type: $PTYPE" >> $TMP/$MAIL_FILE
ip -4 -o addr show | perl -ne 'if (/\d+:\s+(\S+)\s+inet\s+([\d\.]+)/ && ($1 !~ /^lo/)) { print "Address $1: $2\n"; }' >> $TMP/$MAIL_FILE
echo "Install Time: `uptime | perl -ne 'if (/^\s*\S+\s+up\s+([^,]+),/) { print $1; }'`" >> $TMP/$MAIL_FILE
cat $TMP/$MAIL_FILE #LOG
#echo "Opt Pass: $OP" >> $TMP/$MAIL_FILE
if [ "$PTYPE" != "server" -a "$PTYPE" != "VM" ]; then
if [ -n "$OP" ]; then
echo "Opt Pass: OK" >> $TMP/$MAIL_FILE
else
echo " !!!!!!!!!!!" >> $TMP/$MAIL_FILE
echo "Opt Pass: !!! BAD !!!" >> $TMP/$MAIL_FILE
echo " !!!!!!!!!!!" >> $TMP/$MAIL_FILE
fi
fi
echo "Host Key: $KEY" >> $TMP/$MAIL_FILE
if [ -n "$IDENTITY" ] && [ -e "${IDENTITY}.pub" ]; then
scp $SSHOPT ${IDENTITY}.pub $SSHHOST:$SAVEDIR/probe-id.pub #LOG
else
echo "WARNING: No probe identity public key available to upload" #LOG
fi
#echo "Public key: Attached" >> $TMP/$MAIL_FILE
# ATTACH="-a $SAVEDIR/probe-id.pub"
echo >> $TMP/$MAIL_FILE
echo "Total Memory: `perl -ne 'if (/^MemTotal:\s+(.+)$/) { print $1; }' /proc/meminfo`" >> $TMP/$MAIL_FILE
lshw -C memory | perl -ne 'if (/\*\-bank/) { $ok=1; print; } elsif ( $ok && (/description|slot|size/) ) { print; }' >> $TMP/$MAIL_FILE
echo "Disk Info:" >> $TMP/$MAIL_FILE
lshw -C disk | perl -ne 'if (/\*\-|description|product|vendor|serial|size/) { print; }' >> $TMP/$MAIL_FILE
echo >> $TMP/$MAIL_FILE
echo "SCSI Info [`perl -ne 'if (/^\s+\d+\s+\d+\s+(sd\w+)/) { push @s, $1 } END { print join(" ",@s) }' /proc/diskstats`]:" >> $TMP/$MAIL_FILE
cat /proc/scsi/scsi >> $TMP/$MAIL_FILE
echo >> $TMP/$MAIL_FILE
echo "Network Interfaces:" >> $TMP/$MAIL_FILE
ip -o -f link addr show |grep ether >> $TMP/$MAIL_FILE
echo >> $TMP/$MAIL_FILE
echo "Bios Setup:" >> $TMP/$MAIL_FILE
# /opt/dell/srvadmin/bin/omreport chassis biossetup |grep -E 'Boot Sequence|Virtualization|Power Recovery' 2>&1 >> $TMP/$MAIL_FILE
ipmitool chassis bootparam get 5 | grep Boot\ Device 2>&1 >> $TMP/$MAIL_FILE
# dmidecode -t processor | grep VME
echo >> $TMP/$MAIL_FILE
#/opt/dell/srvadmin/bin/omreport chassis remoteaccess |grep -A20 "Remote Access Device" >> $TMP/$MAIL_FILE
#/opt/dell/srvadmin/bin/omreport chassis remoteaccess 2>&1 >> $TMP/$MAIL_FILE
ipmitool lan print 1 |grep -A8 "IP Address Source" 2>&1 >> $TMP/$MAIL_FILE
scp $SSHOPT $TMP/$MAIL_FILE $SSHHOST:$SAVEDIR/ #LOG
for to in $ME; do
echo "Sending email to $to" #LOG
#ssh $SSHOPT $SSHHOST "mutt -s 'KickStart $MD $ST done' $ATTACH $to < $SAVEDIR/$MAIL_FILE" #LOG
ssh $SSHOPT $SSHHOST "mail -s 'KickStart $MD $ST done' $ATTACH $to < $SAVEDIR/$MAIL_FILE" #LOG
done
echo "Mailed setup to $ME" #LOG
scp $SSHOPT /root/*.cfg /root/*log $SSHHOST:$SAVEDIR/ #LOG
if [ -a /var/lib/aide/aide.db.gz ]; then
# Copy over base aide file integrity db from S980A-aide
scp $SSHOPT /var/lib/aide/aide.db.gz $SSHHOST:$SAVEDIR/ #LOG
fi
if [ "$PTYPE" != "server" ]; then
if [ -n "$IDENTITY" ] && [ -e "${IDENTITY}.pub" ]; then
PK=`cat ${IDENTITY}.pub | perl -ne 'if (/^(.+)\s+\S+\@\S+$/) { print "$1\n"; } else { print; }'`
else
PK="NA"
fi
WEEK=604800
WEEK2=1209600
WEEK4=2419200
EPOCH='1969-12-31 19:00:00-05'
THERE=`ssh $SSHOPT $SSHHOST "PGSERVICE=iglass2 psql -U noc -Aqtc \"SELECT service_tag FROM probes_auth WHERE service_tag = '$ST' LIMIT 1\""`
if [ "$ST" = "$THERE" ]; then
DATA="(hostkey, optpass, optexpire, expire, publickey)"
if [ "$PTYPE" = "ds" ]; then
VALUES="('$KEY', '$OP', (now() + '$WEEK'), (now() + '$WEEK4'), '$PK')"
echo "Updating probe auth: ('$KEY', <OP>, (now() + '$WEEK'), (now() + '$WEEK4'), <PK>) FOR '$ST'" #LOG
else
VALUES="('$KEY', '$OP', '$EPOCH', '$EPOCH', '$PK')"
echo "Updating probe auth: ('$KEY', <OP>, $EPOCH, $EPOCH, <PK>) FOR '$ST'" #LOG
fi
QUERY="UPDATE probes_auth SET $DATA = $VALUES WHERE service_tag = '$ST'"
#elif [ "$PTYPE" = "ds" -o "$PTYPE" = "VM" ]; then
elif false; then
NEWDEV=`ssh $SSHOPT $SSHHOST PGSERVICE=iglass2 psql -U noc -Aqtc \"SELECT tun_r_device FROM probes_auth WHERE tun_r_device like \'tun%\' ORDER BY probes_authid\" | perl -ne 'if (/^tun(\d+)$/) { if ( !defined $i ) { $i = $1; } elsif ( $1 > $i ) { $i = $1 } } END { if (defined $i) { print "\x27tun",($i+1),"\x27\n"; } else { print "NULL\n"; } }'`
NEWADDR=$(ssh $SSHOPT $SSHHOST PGSERVICE=iglass2 psql -U noc -Aqtc \"SELECT tun_l_addr FROM probes_auth WHERE tun_l_addr IS NOT NULL ORDER BY probes_authid\" | perl -ne 'if (/^(\d+\.\d+\.\d+\.)(\d+)$/) { if ( !defined $i ) { $i = $2; } elsif ( $2 > $i ) { $i = $2 } $j=$1 } END { if (defined $i) { print "\x27$j",($i+1),"\x27\n"; } else { print "NULL\n"; } }')
NEWRADDR=$(echo $NEWADDR | perl -ne 'if (/(\d+\.\d+\.\d+\.)(\d+)/) { print $1.(100+$2) } else { print }')
echo "Found next tunnel device to be $NEWDEV and addr $NEWADDR" #LOG
DATA="hostkey, service_tag, optpass, optexpire, expire, tun_r_device, tun_l_addr, tun_r_addr, publickey"
#VALUES="'$KEY', '$ST', '$OP', (now() + '$WEEK'), (now() + '$WEEK4'), $NEWDEV, $NEWADDR, '$NEWRADDR', '$PK'"
VALUES="'$KEY', '$ST', '$OP', '1969-12-31 19:00:00-05', '1969-12-31 19:00:00-05', $NEWDEV, $NEWADDR, '$NEWRADDR', '$PK'"
if [ "$PTYPE" = "VM" ]; then
RADDR=$(wget -qO - https://noc.iglass.net/login/whoami.cgi)
DATA="$DATA, r_addr"
VALUES="$VALUES, '$RADDR'"
fi
QUERY="INSERT INTO probes_auth ($DATA) VALUES ($VALUES)"
TUNCFG="$IG_HOME/conf/tunnel/tunnel.cfg"
echo "Creating $TUNCFG" #LOG
echo "HOME_ADDR=216.134.202.55" > $TUNCFG
echo "TUN_REMOTE_DEVICE=$NEWDEV" >> $TUNCFG
echo "TUN_REMOTE_ADDR=$NEWRADDR" >> $TUNCFG
echo "TUN_LOCAL_ADDR=$NEWADDR" >> $TUNCFG
echo "TUN_NETMASK=255.255.255.0" >> $TUNCFG
echo "LOCAL_ROUTES=\"192.168.175.0/24 192.168.176.0/24 192.168.177.0/24 192.168.185.112/28\"" >> $TUNCFG
echo "Adding DS probe auth: ('$KEY', '$ST', 'OP', (now() + '$WEEK'), (now() + '$WEEK4'), $NEWDEV, $NEWADDR, $NEWRADDR, 'PK')" #LOG
else
DATA="(hostkey, service_tag, optpass, optexpire, expire, tun_r_device, publickey)"
VALUES="('$KEY', '$ST', '$OP', '$EPOCH', '$EPOCH', 'NA', '$PK')"
QUERY="INSERT INTO probes_auth $DATA VALUES $VALUES"
echo "Adding probe auth: ('$KEY', '$ST', 'OP', 'PK')" #LOG
fi
fi
ssh $SSHOPT $SSHHOST PGSERVICE=iglass2 psql -U noc -c \"$QUERY\" #LOG
# SHM=/dev/shm
# # This relies on P010A-1passwd and R010P-1passwd as well
# # Think this would be better in probe-config
# if [ -f $SHM/user.1pw -a -f $SHM/pw.1pw ]; then
# OPUSER=$(cat $SHM/user.1pw)
# OPMPW=$(cat $SHM/pw.1pw)
#
# OPDOMAIN="iglassnetworks"
# OPVAULT="Private"
# OPTITLE="iprobe [$ST]"
#
# echo "For 1passord to work $(ssh $SSHOPT $SSHHOST id -un) must sign in first with: op signin $OPDOMAIN.1password.com $OPUSER <secretkey> [<masterpassword>] --output=raw" #LOG
#
#
# echo -n "Creating 1password entry for $OPTITLE: " #LOG
# # ssh -F /dev/shm/tmp.qCxU3EDdeR/ssh/ssh.cfg kserver.iglass.net eval ; PWENCODED=$(echo {"notesPlain":"Service Tag: HPYWHL2","sections":[],"passwordHistory":[],"password":"1w8ipN7zGAT6/Bc9uM2gdwiyVD2cpV"} | op encode); op create item Password --title="iprobe [HPYWHL2]" --vault=Private --tags="probe passphrases"; op signout OP_SESSION_iglassnetworks"
#
# #eval "\
# echo "\
# ssh $SSHOPT $SSHHOST eval \$(op signin $OPDOMAIN $OPMPW); \
# PWENCODED=\$(echo '{\"notesPlain\":\"Service Tag: $ST\",\"sections\":[],\"passwordHistory\":[],\"password\":\"$OP\"}' | op encode); \
# op create item Password \$PWENCODED --title=\"$OPTITLE\" --vault=$OPVAULT --tags=\"probe passphrases\"; \
# op signout OP_SESSION_$OPDOMAIN\"" #LOG
#
# # It will created duplicates with different uuids
# #if op get item \"$OPTITLE\" --vault=Private &> /dev/null; then echo 1password entry created; else echo Could not create 1password entry; fi \"" #LOG
#
#
# else
# echo ERROR: Master password for 1password not found #LOG
# fi
rm -rf $TMP
else
echo ERROR: Could not create tmp dir for mailme probe setup #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S990V-mailme
Symlink
+1
View File
@@ -0,0 +1 @@
S990A-mailme
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S995A-firmware
+82
View File
@@ -0,0 +1,82 @@
cd /root
VM_DMI_SIGNATURE="$(
cat /sys/class/dmi/id/sys_vendor /sys/class/dmi/id/product_name /sys/class/dmi/id/product_version 2>/dev/null \
| tr '\n' ' '
)"
VM_KS_HINT=0
if [ "$PTYPE" = "VM" ] || [ -b /dev/vda ] || [ -d /sys/block/vda ]; then
VM_KS_HINT=1
fi
if [ "$VM_KS_HINT" -eq 1 ] || systemd-detect-virt --quiet || echo "$VM_DMI_SIGNATURE" | grep -Eiq 'qemu|kvm|vmware|virtualbox|standard pc'; then
echo "Applying late VM boot fixes and skipping firmware lookup on virtualized test system" #LOG
mkdir -p /etc/systemd/system/dbus-broker.service.d
cat > /etc/systemd/system/dbus-broker.service.d/10-no-audit.conf <<'EOF'
[Service]
ExecStart=
ExecStart=/usr/bin/dbus-broker-launch --scope system
EOF
mkdir -p /etc/modules-load.d
ln -sfn /dev/null /etc/modules-load.d/fwupd-msr.conf #LOG
systemctl disable auditd.service audit-rules.service 2>/dev/null || true #LOG
systemctl mask auditd.service audit-rules.service 2>/dev/null || true #LOG
systemctl disable systemd-network-generator.service 2>/dev/null || true #LOG
systemctl disable NetworkManager-wait-online.service 2>/dev/null || true #LOG
rm -f /etc/systemd/system/multi-user.target.wants/auditd.service #LOG
mkdir -p /root/kickstart-debug
cat > /root/kickstart-debug/vm-boot-fixes.txt <<EOF
late_vm_boot_fixes=1
date=$(date -Is)
vm_dmi_signature=$VM_DMI_SIGNATURE
EOF
systemctl daemon-reload #LOG
else
MD=$(/usr/sbin/dmidecode -s system-product-name | sed -e 's/ /_/g')
wget -nv "http://$KSRV/KickStart/cfg/$DIST/share/firmware/BIOS-$MD.BIN" #LOG
wget -nv "http://$KSRV/KickStart/cfg/$DIST/share/firmware/RAC-$MD.BIN" #LOG
# TODO: Firmware update support is currently disabled (wrapped in 'if false').
# When re-enabling: verify Dell DSU / iDRAC firmware BIN format is still supported
# on RHEL 10, update paths if needed, and test on physical hardware before enabling.
if false; then
echo skip
if [ -e "BIOS-$MD.BIN" ]; then
if sh BIOS-$MD.BIN -qc #LOG
then
echo "Attempting bios update" #LOG
sh "BIOS-$MD.BIN" -q #LOG
else
echo "Not attempting bios update" #LOG
fi
rm -f "BIOS-$MD.BIN" #LOG
else
echo "No bios update for $MD found." #LOG
fi
if [ -e "RAC-$MD.BIN" ]; then
if sh RAC-$MD.BIN -qc #LOG
then
echo "Attempting RAC update" #LOG
sh "RAC-$MD.BIN" -q #LOG
ipmitool mc reset cold #LOG
else
echo "Not attempting RAC update" #LOG
fi
rm -f "RAC-$MD.BIN" #LOG
else
echo "No RAC update for $MD found." #LOG
fi
else
echo "Firmware uploaded"
fi
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/S999A-last
+8
View File
@@ -0,0 +1,8 @@
# This messes up DNS in containers and already set with authselct
#echo "Setting dns to resolver" #LOG
#rm -f /etc/resolv.conf
#ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf #LOG
#echo "Removing dnf kickstart proxy" #LOG
#perl -spi -e 's/^(proxy=.+)$/# $1/' /etc/dnf/dnf.conf
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/functions
+18
View File
@@ -0,0 +1,18 @@
tmpdir() {
local RETVAL=1
local TMP=
if [ -d /dev/shm ]; then
TMP=$(mktemp -d -p /dev/shm)
else
TMP=$(mktemp -d -p /tmp)
fi
if [ -n "$TMP" -a -d "$TMP" ]; then
echo $TMP
RETVAL=0
fi
return $RETVAL
}
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/no.P010A-1passwd
+35
View File
@@ -0,0 +1,35 @@
exec < /dev/tty6 > /dev/tty6 2> /dev/tty6
chvt 6
[email protected]
echo
echo "Please enter 1password credentials for saving /opt/iglass key."
echo
read -p "Email address [$OPUSER]: " TMPU
[ -n "$TMPU" ] && OPUSER="$TMPU"
#while [ -z "$OPMPW" -a "$OPMPW" -ne "$OPMPW2"]; do
while [ -z "$OPMPW" ]; do
#read -s -p "Master password for $OPUSER: " OPMPW
#read -s -p "Re-enter Master password: " OPMPW2
read -p "Master password for $OPUSER: " OPMPW
echo
# Add verify and get key here to know if login is correct?
clear
done
SHM=/dev/shm
if [ -d $SHM ]; then
echo -n "$OPUSER" > $SHM/user.1pw
echo -n "$OPMPW" > $SHM/pw.1pw
$LOGGER "Master password for 1password user $OPUSER saved"
else
$LOGGER "Could not save master password for 1password user $OPUSER"
fi
chvt 1
exec < /dev/tty1 > /dev/tty1 2> /dev/tty1
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/no.R010P-1passwd
+12
View File
@@ -0,0 +1,12 @@
echo "Moving 1password files" #LOG
if [ -d /mnt/sysimage/dev/shm/ ]; then
mv -f /dev/shm/user.1pw /mnt/sysimage/dev/shm/ #LOG
mv -f /dev/shm/pw.1pw /mnt/sysimage/dev/shm/ #LOG
if [ ! -f /mnt/sysimage/dev/shm/user.1pw ]; then
echo "Could not move 1password" #LOG
fi
else
echo "No chroot /dev/shm found" #LOG
fi
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/no.S450A-swap
+6
View File
@@ -0,0 +1,6 @@
echo "Reducing swapiness" #LOG
cat >> /etc/sysctl.d/20-swappiness.conf <<EOF
# Reduce swapiness
vm.swappiness=1
vm.vfs_cache_pressure=50
EOF
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/no.S500A-elevator
+26
View File
@@ -0,0 +1,26 @@
if [ "$PTYPE" = "VM" ]; then
SCHEDULER="mq-deadline"
# WWID=$(udevadm info --attribute-walk --name=/dev/nvme0n1 | grep wwid)
echo "VM disks: $(cd /sys/block; ls)"
else
SCHEDULER="none"
WWID=$(udevadm info --attribute-walk --name=/dev/sda | grep wwid)
fi
if [ -n "$WWID" ]; then
echo "Setting disk $WWID scheduler to none" #LOG
cat >> /etc/udev/rules.d/99-scheduler.rules <<EOF
ACTION=="add|change", SUBSYSTEM=="block", $WWID, ATTR{queue/scheduler}="$SCHEDULER"
EOF
udevadm control --reload-rules #LOG
udevadm trigger --type=devices --action=change #LOG
else
echo "No WWID found on /dev/sda for scheduler" #LOG
fi
echo "Scheduler set to: $(cat /sys/block/sda/queue/scheduler)"
work/KickStart/cfg/AlmaLinux-10.1-x86_64/ks.d/no.S855A-resolved
+37
View File
@@ -0,0 +1,37 @@
echo "Setting up systemd-resolved" #LOG
mkdir /etc/systemd/resolved.conf.d
cat >> /etc/systemd/resolved.conf.d/10-local.conf <<EOF
[Resolve]
DNS=192.168.194.102 192.168.226.102
FallbackDNS=192.168.176.2
Domains=iglass.net
DNSStubListener=true
LLMNR=false
MulticastDNS=false
DNSSEC=false
EOF
#mv -f /etc/resolv.conf /etc/resolv.conf.old #LOG
# This is done at the very end in S998A-last
#ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf #LOG
mkdir /etc/systemd/system/systemd-resolved.service.d #LOG
cat >> /etc/systemd/system/systemd-resolved.service.d/restart.conf <<EOF
[Service]
RestartSec=200ms
EOF
restorecon -FR /etc/systemd
perl -spi -e 's/^(\[main\].*)$/$1\ndns=none/' /etc/NetworkManager/NetworkManager.conf #LOG
perl -spi -e 's/^(hosts:\s+files)\s+(.+)$/$1 resolve \[!UNAVAIL=return\] $2/' /etc/authselect/user-nsswitch.conf
authselect apply-changes
# systemctl daemon-reload
systemctl enable systemd-resolved.service #LOG
# systemctl restart systemd-resolved.service #LOG
Loaded 100 of 1703 files, more files were not shown because too many files have changed in this diff. Show more