homelab-admin/homelab-public-context
1
0
Fork 0
Code Activity

runtime: publish FORUM-260923-RCA-DEEP-AUDIT8D42R1

Browse Source
This commit is contained in:
homelab-runtime-publisher
2026-09-23 20:57:08 +00:00
parent 03539a35ae
commit 56a1d887de
4 changed files with 931 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
runtime/history/FORUM-260923-RCA-DEEP-AUDIT8D42R1.json
+38
View File
File diff suppressed because one or more lines are too long
runtime/history/FORUM-260923-RCA-DEEP-AUDIT8D42R1.txt
+448
View File
@@ -0,0 +1,448 @@
CHAT_OUTPUT_BEGIN
COMMAND_ID=FORUM-260923-RCA-DEEP-AUDIT8D42R1
STATUS=FAIL
RC=3
HOST=pve01
MODE=read-only
COMPONENT=forum-deep-audit
REFERENCE_REGISTER_CHECK=OK
REFERENCE_SHA256=5a3d8e5154c41cb582a4e0aca68090be1f0138918bf82131a948df326f9d8d66
ERROR_REGISTER_CHECK=OK
ERROR_REGISTER_SHA256=3b09a553ec0f527ed3afeed4753f52a74ee3036045bbfb3c685e9f8af4ba7ba0
COMMAND_SHA256=6d423c85144f338787f0c2f2a7daa180ba7aead6fa3594088284e715f427cf54
DUPLICATE_FAILED_COMMAND_BLOCKED=false
EXECUTION_STARTED=true
CHANGE_DECLARED=false
RESULT_CONTRACT_VALID=true
RESULT_CONTRACT_STATUS=NOT_APPLICABLE
RESULT_CONTRACT_ERROR=NONE
COMMAND_RC=3
CHANGES_MADE=false
ROLLBACK_STARTED=false
ROLLBACK_RESTORED=null
MUTATION_OUTCOME=NO_MUTATION
SANITIZED=yes
SECRETS_INCLUDED=no
PRIVATE_ADDRESSES_INCLUDED=no
RAW_EVIDENCE_SHA256=216ae7cd459e9bcb0a4ee0015534e8ec40637a2b6ad2108b9f5387cd2e39fee5
SANITIZED_OUTPUT_SHA256=be0ad3901439bb83341b6a18560d8e149eae8c2710481e0b1779cde5c7d7b309
OUTPUT_BEGIN
CID=FORUM-260923-RCA-DEEP-AUDIT8D42R1
MODE=read-only
COMPONENT=forum-deep-audit
MUTATION_BOUNDARY=NONE;NO_MAIL_SEND;NO_DB_WRITE;NO_SERVICE_RESTART;NO_VM_CONFIG;NO_CLOUD
EXECUTION_STARTED=true
CHANGES_MADE=false
ROLLBACK_STARTED=false
ROLLBACK_RESTORED=null
=== AUTHORITY GATES ===
AUTHORITY_SHA256=5a3d8e5154c41cb582a4e0aca68090be1f0138918bf82131a948df326f9d8d66 PATH=/etc/pve/31_HOMELAB_REFERENCE.md
AUTHORITY_SHA256=3b09a553ec0f527ed3afeed4753f52a74ee3036045bbfb3c685e9f8af4ba7ba0 PATH=/etc/pve/HOMELAB_ASSISTANT_ERROR_REGISTER.md
AUTHORITY_SHA256=b248a4c32c9cc64e5747e7dce6c7fc0a23f5124a77c71ce72e27a81aceae9d2d PATH=/usr/local/sbin/homelab-chat-run
=== FRESH HANDOFF ===
HANDOFF_PRESENT=true
HANDOFF_SHA256=82a6060ccc216849c55790a3489ef157d5551a0c7cef0c5ec9abd35f569c7940
11135:- Remote source used: pve02-mail-01-crypt app/forums/forum-prod/fresh5 latest timestamp.
11140:- Proof: /root/evidence/612_FORUM_FRESH5_CLOUD_RESTORE_DRILL_CORRECTED_PROOF.txt.
11142:## FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_OK_20260701
11143:- Five public XenForo forums are healthy over HTTPS.
11144:- Mailcow admin and NetBird UI are reachable after persistent forum SMTP enablement.
11145:- Persistent forum SMTP via msmtp is enabled and tested.
11146:- Local backup exists on VM160 under /var/backups/forums.
11147:- Mail.ru Cloud fresh5 backup is enabled on pve02 via forum-prod-fresh5-cloud-backup.timer.
11149:- Cloud restore drill passed: split parts verified, archive reconstructed, SHA256 matched, tar contained DB/file archives for all five forums.
11150:- Final rollup proof: /root/evidence/614_FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_PROOF.txt.
11152:## FORUM_XENFORO_JOB_RUNNER_CONFIGURED_20260701
11153:- XenForo CLI job runner configured inside VM160 forum-prod.
11154:- Script: /root/scripts/forum-xenforo-run-jobs.sh.
11155:- Cron: /etc/cron.d/forum-xenforo-run-jobs every 5 minutes.
11156:- Purpose: process XenForo job queues and cron tasks independent of forum traffic.
11157:- First run proof: /root/evidence/617_XENFORO_RUN_JOBS_ONCE_PROOF.txt.
11158:- Cron verification proof: /root/evidence/618_XENFORO_JOB_RUNNER_CRON_CONFIG_PROOF.txt.
11160:## FORUM_XENFORO_CRON_DAEMON_AND_JOBRUNNER_OK_20260701
11161:- cron daemon inside VM160 forum-prod is active/enabled and was proven by temporary /etc/cron.d execution proof.
11162:- XenForo job runner script executed successfully after cron daemon verification.
11163:- Due XenForo cron count returned to zero after run.
11164:- Proof: /root/evidence/621_FORUM_CRON_DAEMON_JOBRUNNER_VERIFY_PROOF.txt.
11166:## FORUM_DBTECH_TOR_TIMEOUT_CLASSIFIED_20260701
11167:- Each forum had one historical XenForo error_log row for cURL timeout to check.torproject.org/torbulkexitlist.
11169:- dbtech_security_tornodes option is 0 on all five forums.
11170:- Classified as non-blocking historical external network timeout/noise; not a forum runtime failure.
11174:## FORUM_XENFORO_BUILTIN_MAIL_DELIVERED_BUT_SPAM_20260701
11175:- Built-in XenForo outgoing email test from site "Моды для Hollow Knight" was delivered to <EMAIL>.
11178:- This proves forum SMTP transport works, but deliverability/reputation needs tuning.
11181:- Proof: /root/evidence/627_XENFORO_BUILTIN_MAIL_DELIVERED_SPAM_CLASSIFICATION_PROOF.txt.
11183:## FORUM_YANDEX_DELIVERABILITY_SPAM_WITH_SPF_DKIM_PASS_20260701
11184:- Built-in XenForo outgoing email test was delivered to <EMAIL> but placed in Spam.
11188:- Classification: SMTP/Mailcow/XenForo transport is OK; remaining issue is deliverability/reputation/content filtering.
11190:- Next actions: mark as Not spam in Yandex, add/check pvepro.ru in Yandex Postmaster, warm up sender reputation, optionally plan per-forum sender domains later.
11192:## FORUM_PROJECT_FINAL_ACCEPTANCE_OK_20260701
11193:- Five XenForo forums are published and healthy over HTTPS: codevipe.ru, gamevipe.ru, hkmods.ru, zakrutim.ru, dsmods.ru.
11194:- VM160 forum-prod is the active production forum VM on pve02 at <IP>.
11195:- Runtime services are configured: nginx, MariaDB, PHP 8.3 FPM, qemu-agent, cron, msmtp/sendmail transport.
11196:- NPMplus on edge terminates TLS and proxies all five forums to VM160.
11198:- Forum SMTP transport works through <EMAIL>; Yandex headers proved SPF pass and DKIM pass. Spam placement is classified as reputation/content deliverability, not server failure.
11199:- XenForo job runner cron is configured and cron daemon execution was proven.
11200:- Local backup is configured on VM160 and Mail.ru Cloud fresh5/all-forums backup is configured on pve02.
11201:- Cloud restore drill passed: split parts verified, archive SHA matched, tar contained DB/file/config archives for all five forums.
11203:- Final proof: /root/evidence/629_FORUM_PROJECT_FINAL_ACCEPTANCE_PROOF.txt.
11205:## FORUM_NEXT_CHAT_HANDOFF_FULL_STATE_20260701
11217:### Production forum VM
11219:- Name: forum-prod.
11223:- Runtime: nginx, MariaDB 10.11, PHP 8.3 FPM, qemu-agent, cron, msmtp/sendmail transport.
11224:- Final accepted snapshot: forum-final-accepted-091934Z.
11225:- Mail final snapshot: forum-mail-ok-091815Z.
11226:- Postlaunch snapshot: forum-postlaunch-ok-085501Z.
11228:### Public forums
11238:### Mail
11239:- Forum sender: <EMAIL>.
11240:- Transport: XenForo sendmail -> msmtp -> mail.pvepro.ru.
11241:- Built-in XenForo mail test reached Yandex.
11250:- Local archive path pattern: /var/backups/forums/forum-backup-*.tar.gz.
11251:- pve02 Mail.ru Cloud fresh5/all-forums backup configured.
11252:- New timer: forum-prod-fresh5-cloud-backup.timer active/enabled.
11255: - pve02-mail-01-crypt
11256: - pve02-mail-02-crypt
11257: - pve02-mail-03-crypt
11258: - pve02-mail-04-crypt
11260:- Restore drill passed: split parts verified, archive SHA matched, tar contained DB/file/config archives for all five forums.
11262:### XenForo jobs and cron
11265:- XenForo job runner configured:
11266: - /root/scripts/forum-xenforo-run-jobs.sh
11267: - /etc/cron.d/forum-xenforo-run-jobs
11268:- Purpose: process XenForo jobs and cron tasks independent of visitor traffic.
11271:- Each forum had one historical xf_error_log row for cURL timeout to check.torproject.org.
11273:- dbtech_security_tornodes=0 on all five forums.
11278:- /root/evidence/629_FORUM_PROJECT_FINAL_ACCEPTANCE_PROOF.txt
11279:- /root/evidence/628_FORUM_MAIL_DELIVERABILITY_FINAL_SNAPSHOT_PROOF.txt
11282:- /root/evidence/621_FORUM_CRON_DAEMON_JOBRUNNER_VERIFY_PROOF.txt
11283:- /root/evidence/614_FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_PROOF.txt
11284:- /root/evidence/613_FORUM_FRESH5_CLOUD_RESTORE_DRILL_FINALIZE_PROOF.txt
11285:- /root/evidence/612_FORUM_FRESH5_CLOUD_RESTORE_DRILL_CORRECTED_PROOF.txt
11286:- /root/evidence/609_FORUM_PROD_FRESH5_CLOUD_BACKUP_FINALIZE_PROOF.txt
11287:- /root/evidence/608_FORUM_PROD_FRESH5_CLOUD_BACKUP_FIRST_RUN_PROOF.txt
11288:- /root/evidence/600_FORUM_PERSISTENT_SMTP_FINAL_STABLE_PROOF.txt
11289:- /root/evidence/570_MAIL_DNS_FINAL_CLEAN_AUDIT_PROOF.txt
11290:- /root/evidence/560_FORUM_BACKUP_SCRIPT_SETUP_PROOF.txt
11318:CHECK-9: для forum-prod использовать pve02 и ключ [SENSITIVE_PATH]
11325:VM160 forum-prod is not in Proxmox nightly backup.
11339:Не искать Cloudflare/SMTP сразу по /opt/stacks, MkDocs site, backups и HTML-отчётам.
11352:forum-prod: заходить через pve02, ключ [SENSITIVE_PATH]
11371:20. Ошибка: широкий XenForo SMTP-поиск по src дал шум исходников.
11372:Для XenForo SMTP не grep-ать весь /src; сначала смотреть live src/config.php без значений, затем БД XenForo и xf_option/emailTransport.
11373:Значения DB-паролей и SMTP-паролей не печатать.
11375:21. Ошибка: nested PHP php -r дал Parse error на forum-prod.
11376:Команды 102-104 считать битыми: они не подтвердили и не изменили smtpSsl.
11377:Не продолжать длинные вложенные ssh+php -r с кавычками; для forum-prod лучше войти в VM или использовать короткие mysql/php команды без вложенного кода.
11378:Рабочими считать: DB backup перед SMTP, SMTP password update, JSON postcheck; SMTP auth ещё не закрыт из-за CODE=530 на AUTH LOGIN.
11380:22. Ошибка: самодельный base64 PHP для SMTP auth сломан.
11381:Команда 108 дала PHP Parse error на smtpHost и пустой proof-файл.
11382:Не использовать больше сгенерированные большие base64 PHP для SMTP/XenForo.
11383:Для XenForo SMTP использовать короткие mysql-запросы к xf_option и готовые штатные механизмы XenForo.
11386:Команда 111 не изменила smtpSsl: OLD_SMTPSSL=false, SQL ERROR 1064, NEW_SMTPSSL=false.
11395:25. Ошибка анализа: не надо слепо включать smtpSsl=true при smtpPort=587.
11396:smtpPort=587 обычно означает STARTTLS, а не implicit SMTPS.
11397:CODE=530 на AUTH LOGIN означает, что ручной checker не сделал STARTTLS; это не доказывает неверный SMTP-пароль.
11398:Сначала проверять AUTH через openssl s_client -starttls smtp, без вывода секрета.
11400:26. Ошибка: openssl -crlf вместе с ручным CRLF ломает SMTP AUTH checker.
11402:Это указывает на битую base64-строку checker-а, а не на неверный SMTP-пароль.
11404:Для 587 оставлять smtpSsl=false и проверять STARTTLS корректным checker-ом.
11464:40. Ошибка/вывод: VM150 Mail cloud single-file upload failed with 413.
11465:Факт: Mail/WebDAV rejected one 6.6G vzdump file with 413 Request Entity Too Large.
11469:Факт: first reconciliation wrote STATUS=ERROR because router-running-config-mail-cloud.txt is on pve01, not edge-vm.
11579:## FORUM_PROD_VM160_FIRST_BOOT_SSH255_20260701
11584:## FORUM_PROD_VM160_NESTED_HOSTNAME_PROOF_QUOTING_20260701
11590:## FORUM_PROD_VM160_SWAP_PROOF_QUOTING_20260701
11596:## FORUM_UPLOAD_ARCHIVES_VM_VERIFY_QUOTING_20260701
11600:- Rule: verify VM archive integrity locally inside forum-prod, not through nested quoted loops.
11602:## FORUM_PROD_BULK_IMPORT_PHP85_EMPTY_FRONTEND_20260701
11603:- Context: rebuilt VM160, five XenForo forums imported in bulk on PHP 8.5.7.
11604:- Symptom: all five forums returned HTTP 200 with empty body; XenForo logged Template public:PAGE_CONTAINER/forum_list is unknown.
11606:- Decision: stop bulk repair, treat current VM160 as lab, find a working CodeVipe recipe first, then destroy/rebuild cleanly for all five forums.
11607:- Rule: no more mass rebuild/repair commands across all forums until single-forum CodeVipe recipe is proven.
11609:## XENFORO_REQUIREMENTS_DIRECT_DOWNLOAD_403_20260701
11610:- Context: Tried to download XenForo requirements ZIP directly from xenforo.com inside forum-prod.
11614:- Rule: use operator-uploaded xenforo23-requirements-test.zip from /root/forum-upload instead of relying on direct curl download.
11617:- Context: proof 492 tried to inspect uploaded xenforo23-requirements-test.zip on pve01.
11620:- Rule: verify ZIP with Python zipfile or inside forum-prod rather than installing unzip on Proxmox just for inspection.
11623:- Context: VM160 was destroyed/recreated, then five XenForo forums were deployed from fresh ZIP backups.
11624:- Result: proof 513 confirms all five forums locally healthy.
11635:- Context: proof 525 tried to create forum proxy hosts in NPMplus.
11636:- Issue: script sourced /opt/npmplus/.env, but INITIAL_ADMIN_EMAIL/PASSWORD were not present there; they are in the npmplus container env.
11637:- Impact: no forum proxy hosts were created by proof 525.
11647:- Context: NPMplus API login attempts failed even though the forum VM and edge-to-forum connectivity are healthy.
11650:- Rule: publish forum routes through backed-up manual nginx proxy_host configs, then issue proper certificates after DNS points to edge.
11653:- Context: proof 535 attempted DNS-01 certificate issue for five forum domains.
11654:- Issue: /tmp/forum_cf_token.env was root-owned mode 600, so debian user could not source it directly.
11665:- Context: Cloudflare token discovered on edge verified as a token but could not access forum zones.
11668:- Rule: install a new Cloudflare token with Zone Read and DNS Edit for the five forum zones before retrying certs/DNS.
11670:## FORUM_PUBLICATION_FINAL_SUCCESS_20260701
11671:- Context: five XenForo forums were rebuilt on VM160, routed through edge NPMplus, issued Let’s Encrypt certs using a corrected Cloudflare token, and cut over in DNS.
11676:- Context: proof 556 final health gate passed for all five public forums.
11679:- Impact: forum health was OK, but proof 556 snapshot step was not completed.
11686:- Impact: web routing is OK, but mail-related DNS may still contain stale provider data.
11697:- Impact: domains temporarily had duplicate SPF records, which is invalid for mail validation.
11698:- Fix: delete all SPF TXT records for the five forum zones, then create exactly one v=spf1 -all TXT per zone.
11700:## MSMTP_SECRET_SOURCE_PARSE_ERROR_576_20260701
11701:- Context: proof 576 installed msmtp but sendmail auth test failed.
11702:- Issue: [SENSITIVE_PATH] was sourced as shell, but SMTP password contained shell-special characters; source failed and auth used an invalid/truncated secret path.
11703:- Impact: msmtp package installed, but mail sending was not proven working.
11704:- Rule: never source SMTP secret files containing arbitrary passwords; store password base64 and use msmtp passwordeval helper accessible to www-data.
11706:## FORUM_MSMTP_MAIL_TRANSPORT_STILL_FAILING_20260701
11707:- Context: attempted to fix msmtp config with passwordeval helper.
11708:- Issue: one or more sendmail/PHP mail tests still failed.
11709:- Impact: XenForo mail sending is not yet proven.
11710:- Next step: verify SMTP credential/password and provider policy for <EMAIL>.
11712:## SMTP_PASSWORD_EXPOSED_ROTATE_REQUIRED_20260701
11713:- Context: SMTP password was exposed in terminal output during failed msmtp setup.
11714:- Cause: secret file with raw SMTP_PASS was later parsed/sourced by shell; shell error printed the raw assignment line.
11715:- Impact: treat that SMTP password as compromised.
11716:- Required action: rotate the mailbox/app password in the mail provider panel before any further SMTP testing.
11719:## NETBIRD_MAILCOW_REPORTED_DOWN_AFTER_SMTP_INCIDENT_20260701
11720:- Context: user reported NetBird and Mailcow became unavailable after forum mail-transport work.
11722:- Rule: do not change DNS, firewall, proxy, mail, NetBird, or Mailcow configs before read-only triage proof.
11724:## FORUM_SMTP_FAILED_CONFIG_DISABLED_AFTER_MAILCOW_BAN_20260701
11725:- Context: forum-prod SMTP/msmtp test used invalid or compromised credentials and triggered Mailcow netfilter warnings/ban for <IP>.
11726:- Impact: forum mail transport is not proven and must remain disabled until SMTP password is rotated and tested once carefully.
11727:- Action: remove forum-prod msmtp secret/config files so XenForo cannot keep retrying broken SMTP auth.
11728:- Rule: do not test SMTP auth again until a fresh rotated mailbox/app password is available.
11730:## FORUM_OLD_CODEVIPE_CLOUD_BACKUP_STALE_AFTER_FRESH5_20260701
11731:- Context: pve02 codevipe-cloud-backup.timer existed before fresh five-forum rebuild.
11732:- Issue: old script points to /root/backups/codevipe and old remote path app/codevipe/forum-prod/local.
11733:- Evidence: latest service run failed with missing /root/backups/codevipe; remote inventory contains old codevipe-public/codevipe.sql files, not current five-forum archive.
11734:- Impact: old timer must not be treated as valid current backup for all five forums.
11735:- Fix: create a new forum-prod fresh5 cloud backup using existing pve02 Mail-cloud rclone crypt remotes.
11737:## FORUM_RESTORE_DRILL_610_STATUS_FLAG_BUG_20260701
11738:- Context: restore drill proof 610 downloaded Mail.ru Cloud backup, verified split part SHA256, reconstructed archive, matched full archive SHA256, and confirmed all five forum DB/file archives in tar.
11739:- Issue: final RESTORE_DRILL_OK was set to no because the script tested an unassigned shell variable RCLONE_REMOTE_PVE02_MAIL_01_RC.
11743:## FORUM_XENFORO_MAIL_SMOKE_TEST_FAILED_20260701
11744:- XenForo-level mail smoke test did not return success for all five forums.
11745:- Check proof 623 and msmtp log before retrying.
11747:## XENFORO_MAIL_SMOKE_623_INVALID_ROOT_PATH_20260701
11748:- Context: custom XenForo mail smoke test 623 failed with Could not open input file /root/scripts/623_xenforo_mail_smoke.php.
11750:- Impact: proof 623 is invalid and should not be used to judge mail delivery.
11751:- Superseding evidence: user ran XenForo built-in outgoing email test; message was delivered to Yandex but placed in Spam.
11754:## FORUM_NEXT_CHAT_KNOWN_ERRORS_AND_CLOSED_INCIDENTS_20260701
11757:- SMTP_PASSWORD_EXPOSED_ROTATE_REQUIRED_20260701:
11758: - A previous bad command sourced a raw SMTP secret and printed it.
11760: - Later persistent SMTP was rebuilt using safe files and verified.
11763:- XENFORO_MAIL_SMOKE_623_INVALID_ROOT_PATH_20260701:
11764: - Custom mail proof 623 failed with "Could not open input file".
11766: - Impact: proof 623 is invalid and must not be used to judge mail delivery.
11767: - Superseded by user-observed built-in XenForo test and Yandex header proof.
11769:- FORUM_MSMTP_MAIL_TRANSPORT_STILL_FAILING_20260701 / FORUM_SMTP_FAILED_CONFIG_DISABLED_AFTER_MAILCOW_BAN_20260701:
11770: - Earlier SMTP attempts failed and triggered mailcow netfilter ban for forum public IP.
11772: - Persistent msmtp transport was later enabled and verified.
11773: - Mailcow and NetBird remained reachable after final tests.
11777: - Fixed by deleting duplicates and recreating exactly one SPF per forum domain.
11778: - Final forum domains use v=spf1 -all because sending uses <EMAIL>.
11780:- FORUM_RESTORE_DRILL_610_STATUS_FLAG_BUG_20260701:
11784:- FORUM_DBTECH_TOR_TIMEOUT_CLASSIFIED_20260701:
11785: - One historical timeout row per forum to check.torproject.org.
11790:- Yandex placed built-in XenForo test mail in Spam despite SPF pass and DKIM pass.
11798: - /etc/msmtprc
11799: - /etc/msmtp/*
11822:`service → owner node/VM/container → data path → backup type → local copy → offhost copy → Mail cloud remote → timer → retention → last backup OK → last upload OK → last restore drill OK → alert OK → RPO/RTO → disposable yes/no`.
11830:**Третье: Mail.ru Cloud надо сделать основным offsite-слоем.**
11833:* pve01 cloud: Nextcloud, dns1/unbound1, critical configs, часть P0-копий edge/forum.
11834:* pve02 cloud: forum-prod, dns2/unbound2, копии P0 edge/Nextcloud.
11838:Сейчас Mail-cloud capacity выглядит очень недоиспользованной: largest used remote был около 65 GiB of 1 TiB, а у вас фактически запас гораздо больше. Поэтому можно смело ужесточить retention для критичных вещей.
11870:* VM160 forum-prod: local VM backup + app-level forum backup + cloud restore drill.
11881:* Forum-prod сейчас уже сильнее: fresh5 cloud backup, split parts, SHA256, manifest, restore drill, tar listing всех пяти форумов.
11899:**Forum-prod:**
11903:не делать тупой бесконечный full каждый день, а хранить: DB/config чаще, data/media реже и chunked/manifest-based. VM150 уже имеет Mail-cloud chunked backup with download verification — это правильный паттерн.
11918:2. **pve02 увеличить RAM до 32 GB**, если железо позволяет. Forum-prod + DNS вторички сейчас живут, но N100/15 GiB — слабое место для роста форумов.
11931:* умер pve02 → через N минут поднимаем forum-prod из cloud/local backup;
11939:* pve02-loss drill: восстановить VM160 из cloud backup на временную VM, проверить nginx/php/mysql, один форум, mail disabled/test-safe.
11945:Monitoring уже сильный: Prometheus, Grafana, Alertmanager, Loki, Alloy, Kuma, Gotify, ntfy, Healthchecks, Blackbox exporter и node-exporter описаны. Но pve03 не имел homelab-health-http `:9101`, в отличие от pve01/pve02.
11951:* alert на `backup_age`, `cloud_upload_age`, `restore_drill_age`, `remote_capacity`, `manifest_mismatch`;
11959:Сейчас NPMplus маршруты и сертификаты в хорошем состоянии: активный proxy dir `/opt/npmplus/nginx/proxy_host`, forum routes `200–204`, parked `995–998`, pvepro `994`, taftauto `993`, deploy hooks, forum renewal cron, token files mode 600.
11988:* автоматический контроль прав rclone configs, Cloudflare token files, msmtp secrets;
11989:* отдельный incident runbook: что ротировать при утечке SMTP/Cloudflare/rclone/router.
11998:4. Стандартизировать Mail.ru Cloud layout на всех нодах.
12015:3. Router startup-config сделать не только manual, а регулярным, если доступ позволит; running-config уже имеет recurring Mail-cloud backup.
12025:Главная мысль: **не добавлять сейчас новые красивые штуки, а довести восстановление до состояния “любой узел умер — мы спокойно восстанавливаемся по матрице”**. На вашем кластере это реально сделать без PBS, через Proxmox local vzdump + offhost на соседние ноды + Mail.ru Cloud crypt/remotes + обязательные restore drills.
=== PVE FORUM/ALERT UNIT STATE ===
UNIT=homelab-forum-snuffleupagus-health ACTIVE=inactive ENABLED=static
UNIT=homelab-alerting-health ACTIVE=inactive ENABLED=static
UNIT=postfix ACTIVE=active ENABLED=enabled
TIMER=homelab-forum-snuffleupagus-health ACTIVE=active ENABLED=enabled
TIMER=homelab-alerting-health ACTIVE=active ENABLED=enabled
=== FORUM HEALTH SCRIPT/REPORT ===
META=/usr/local/sbin/homelab-forum-snuffleupagus-health mode=700 owner=root group=root size=3349
SHA256=d7cee34b7d0e93c578f9203ba6e3634d304fee3ea5f3629a576dd683f1b50b29 PATH=/usr/local/sbin/homelab-forum-snuffleupagus-health
4:HEALTH="/var/lib/homelab-health/forum-snuffleupagus.txt"
5:REPORT="/var/lib/homelab-forum-snuffleupagus/latest.md"
6:mkdir -p /var/lib/homelab-health /var/lib/homelab-forum-snuffleupagus
9:OUT="$(ssh -n -o BatchMode=yes -o ConnectTimeout=10 root@pve02 'ssh -n -i [SENSITIVE_PATH] -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new root@<IP> "set +e; echo FORUM_HOST=\$(hostname -s); echo NO_ENV_FILES_READ=YES; php -m 2>/dev/null | grep -i ^snuffleupagus$ >/dev/null && echo MODULE=loaded || echo MODULE=not_loaded; php --ri snuffleupagus 2>/dev/null | grep -Ei \"snuffleupagus support|Version\" | head -n 20 || true; systemctl is-active nginx 2>/dev/null | sed s/^/NGINX=/; systemctl is-active php8.3-fpm 2>/dev/null | sed s/^/PHP83FPM=/; systemctl is-active mariadb 2>/dev/null | sed s/^/MARIADB=/; test -s /etc/php/8.3/snuffleupagus/xenforo-simulation.rules && echo RULES=present || echo RULES=missing; test -s /etc/php/8.3/mods-available/snuffleupagus.ini && echo MOD_INI=present || echo MOD_INI=missing; grep -Ev \"^[[:space:]]*#\" /etc/php/8.3/snuffleupagus/xenforo-simulation.rules 2>/dev/null | grep -q \"sp\\.readonly_exec\" && echo ACTUAL_READONLY_EXEC_RULE=YES || echo ACTUAL_READONLY_EXEC_RULE=NO; grep -Ev \"^[[:space:]]*#\" /etc/php/8.3/snuffleupagus/xenforo-simulation.rules 2>/dev/null | grep -q \"sp\\.ini_protection\" && echo ACTUAL_INI_PROTECTION_RULE=YES || echo ACTUAL_INI_PROTECTION_RULE=NO; cat /var/lib/homelab-forum-hardening/snuffleupagus-health.txt 2>/dev/null || echo FORUM_LOCAL_HEALTH=missing; for d in codevipe.ru gamevipe.ru hkmods.ru zakrutim.ru dsmods.ru; do code=\$(curl -k -sS -o /tmp/forum-snuffle-health-\$d.body -w \"%{http_code}\" --connect-timeout 10 --max-time 20 https://\$d/ 2>/tmp/forum-snuffle-health-\$d.err); bytes=\$(wc -c < /tmp/forum-snuffle-health-\$d.body 2>/dev/null || echo 0); echo PUBLIC_SMOKE site=\$d http_code=\$code bytes=\$bytes; done"' 2>&1)"
10:echo "# Forum Snuffleupagus health" > "$REPORT"
13:echo "target=forum-prod VM160 <IP>" >> "$REPORT"
14:echo "forums=codevipe.ru,gamevipe.ru,hkmods.ru,zakrutim.ru,dsmods.ru" >> "$REPORT"
30:echo "STATUS=$STATUS TS=$TS TYPE=forum-snuffleupagus BAD=$BAD WARN=$WARN MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=$SMOKE_BAD ACTUAL_READONLY_EXEC_RULE=NO REPORT=$REPORT" > "$HEALTH"
META=/usr/local/sbin/homelab-alerting-health mode=700 owner=root group=root size=2318
SHA256=2050bac8c2ca8f1116e6d4fabe476292253bf38c56a49456242606980b4eb797 PATH=/usr/local/sbin/homelab-alerting-health
4:OUT="/var/lib/homelab-health/alerting.txt"
33:echo "STATUS=$STATUS TS=$TS TYPE=alerting BAD=$BAD$DETAILS kuma_state=${KUMA_STATE// /_} gotify_state=${GOTIFY_STATE// /_} monitor=${MON_ROW:-missing}" > "$OUT"
META=/etc/systemd/system/homelab-forum-snuffleupagus-health.service mode=644 owner=root group=root size=167
SHA256=16f2007e56807897865ae510ff84690d41bb7031fe037cc45425585d737f134b PATH=/etc/systemd/system/homelab-forum-snuffleupagus-health.service
META=/etc/systemd/system/homelab-forum-snuffleupagus-health.timer mode=644 owner=root group=root size=160
SHA256=818d7355c53af09a0ef127ab42eceaf1536e93364501160d08995ddf47faf3be PATH=/etc/systemd/system/homelab-forum-snuffleupagus-health.timer
META=/etc/systemd/system/homelab-alerting-health.service mode=644 owner=root group=root size=184
SHA256=2328f1042ac8f6b201603d86941afc87d62191ae12559bee0fd77709c76066f3 PATH=/etc/systemd/system/homelab-alerting-health.service
META=/etc/systemd/system/homelab-alerting-health.timer mode=644 owner=root group=root size=163
SHA256=28ceb1a533a392af460fc8a42886912fcd256ee6fe8da89460e6c8bdb6096ef1 PATH=/etc/systemd/system/homelab-alerting-health.timer
FORUM_REPORT_SHA256=9d41822f17cdaed296b2e80dd5ba331d01870722d788bf6adfb57d82141ec3f3
# Forum Snuffleupagus health
generated_utc=2026-09-23T20:45:17Z
target=forum-prod VM160 <IP>
forums=codevipe.ru,gamevipe.ru,hkmods.ru,zakrutim.ru,dsmods.ru
env_files_read=no
secret_values_printed=no
FORUM_HOST=forum-prod
NO_ENV_FILES_READ=YES
MODULE=loaded
snuffleupagus support => enabled
Version => 0.13.0
NGINX=active
PHP83FPM=active
MARIADB=active
RULES=present
MOD_INI=present
ACTUAL_READONLY_EXEC_RULE=NO
ACTUAL_INI_PROTECTION_RULE=YES
STATUS=OK TS=2026-07-07T03:35:55Z TYPE=snuffleupagus-forums MODE=low-noise-simulation STRICT=NO BAD=0 MODULE=loaded FPM=active FORUMS=5 READONLY_EXEC=disabled_reason_xenforo_code_cache PUBLIC_SMOKE=OK
PUBLIC_SMOKE site=codevipe.ru http_code=200 bytes=44523
PUBLIC_SMOKE site=gamevipe.ru http_code=200 bytes=74958
PUBLIC_SMOKE site=hkmods.ru http_code=200 bytes=111772
PUBLIC_SMOKE site=zakrutim.ru http_code=200 bytes=147223
PUBLIC_SMOKE site=dsmods.ru http_code=200 bytes=173355
=== FORUM HEALTH JOURNAL 24H ===
Sep 23 00:01:09 pve01 homelab-forum-snuffleupagus-health[2154081]: STATUS=OK TS=2026-09-22T21:01:08Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:15:59 pve01 homelab-forum-snuffleupagus-health[2206413]: STATUS=OK TS=2026-09-22T21:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:31:33 pve01 homelab-forum-snuffleupagus-health[2259317]: STATUS=OK TS=2026-09-22T21:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:45:22 pve01 homelab-forum-snuffleupagus-health[2310128]: STATUS=OK TS=2026-09-22T21:45:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:01:02 pve01 homelab-forum-snuffleupagus-health[2351204]: STATUS=OK TS=2026-09-22T22:01:01Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:15:13 pve01 homelab-forum-snuffleupagus-health[2394798]: STATUS=OK TS=2026-09-22T22:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:31:33 pve01 homelab-forum-snuffleupagus-health[2455122]: STATUS=OK TS=2026-09-22T22:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:45:46 pve01 homelab-forum-snuffleupagus-health[2509007]: STATUS=OK TS=2026-09-22T22:45:44Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:00:59 pve01 homelab-forum-snuffleupagus-health[2562448]: STATUS=OK TS=2026-09-22T23:00:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:16:22 pve01 homelab-forum-snuffleupagus-health[2611647]: STATUS=OK TS=2026-09-22T23:16:20Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:31:34 pve01 homelab-forum-snuffleupagus-health[2664129]: STATUS=OK TS=2026-09-22T23:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:45:33 pve01 homelab-forum-snuffleupagus-health[2707466]: STATUS=OK TS=2026-09-22T23:45:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:00:18 pve01 homelab-forum-snuffleupagus-health[2752522]: STATUS=OK TS=2026-09-23T00:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:15:53 pve01 homelab-forum-snuffleupagus-health[2814508]: STATUS=OK TS=2026-09-23T00:15:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:31:40 pve01 homelab-forum-snuffleupagus-health[2907118]: STATUS=OK TS=2026-09-23T00:31:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:46:04 pve01 homelab-forum-snuffleupagus-health[2959189]: STATUS=OK TS=2026-09-23T00:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:00:33 pve01 homelab-forum-snuffleupagus-health[2990112]: STATUS=OK TS=2026-09-23T01:00:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:16:04 pve01 homelab-forum-snuffleupagus-health[3041256]: STATUS=OK TS=2026-09-23T01:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:31:00 pve01 homelab-forum-snuffleupagus-health[3099073]: STATUS=OK TS=2026-09-23T01:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:45:15 pve01 homelab-forum-snuffleupagus-health[3142158]: STATUS=OK TS=2026-09-23T01:45:14Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:01:17 pve01 homelab-forum-snuffleupagus-health[3196314]: STATUS=OK TS=2026-09-23T02:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:15:53 pve01 homelab-forum-snuffleupagus-health[3238867]: STATUS=OK TS=2026-09-23T02:15:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:31:05 pve01 homelab-forum-snuffleupagus-health[3282264]: STATUS=OK TS=2026-09-23T02:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:45:59 pve01 homelab-forum-snuffleupagus-health[3332050]: STATUS=OK TS=2026-09-23T02:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:01:19 pve01 homelab-forum-snuffleupagus-health[3433024]: STATUS=OK TS=2026-09-23T03:01:17Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:15:17 pve01 homelab-forum-snuffleupagus-health[3482327]: STATUS=OK TS=2026-09-23T03:15:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:31:01 pve01 homelab-forum-snuffleupagus-health[3540188]: STATUS=OK TS=2026-09-23T03:30:59Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:45:59 pve01 homelab-forum-snuffleupagus-health[3575511]: STATUS=OK TS=2026-09-23T03:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:01:23 pve01 homelab-forum-snuffleupagus-health[3629329]: STATUS=OK TS=2026-09-23T04:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:15:59 pve01 homelab-forum-snuffleupagus-health[3673169]: STATUS=OK TS=2026-09-23T04:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:31:22 pve01 homelab-forum-snuffleupagus-health[3723216]: STATUS=OK TS=2026-09-23T04:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:45:16 pve01 homelab-forum-snuffleupagus-health[3771308]: STATUS=OK TS=2026-09-23T04:45:15Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:01:39 pve01 homelab-forum-snuffleupagus-health[3831033]: STATUS=OK TS=2026-09-23T05:01:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:15:03 pve01 homelab-forum-snuffleupagus-health[3866230]: STATUS=OK TS=2026-09-23T05:15:02Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:30:53 pve01 homelab-forum-snuffleupagus-health[3917462]: STATUS=OK TS=2026-09-23T05:30:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:45:59 pve01 homelab-forum-snuffleupagus-health[3976070]: STATUS=OK TS=2026-09-23T05:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:00:10 pve01 homelab-forum-snuffleupagus-health[4019194]: STATUS=OK TS=2026-09-23T06:00:09Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:15:59 pve01 homelab-forum-snuffleupagus-health[4062449]: STATUS=OK TS=2026-09-23T06:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:30:18 pve01 homelab-forum-snuffleupagus-health[4108279]: STATUS=OK TS=2026-09-23T06:30:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:45:38 pve01 homelab-forum-snuffleupagus-health[4165927]: STATUS=OK TS=2026-09-23T06:45:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:01:23 pve01 homelab-forum-snuffleupagus-health[21741]: STATUS=OK TS=2026-09-23T07:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:15:17 pve01 homelab-forum-snuffleupagus-health[52177]: STATUS=OK TS=2026-09-23T07:15:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:31:23 pve01 homelab-forum-snuffleupagus-health[110050]: STATUS=OK TS=2026-09-23T07:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:46:17 pve01 homelab-forum-snuffleupagus-health[152931]: STATUS=OK TS=2026-09-23T07:46:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:01:18 pve01 homelab-forum-snuffleupagus-health[202471]: STATUS=OK TS=2026-09-23T08:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:16:23 pve01 homelab-forum-snuffleupagus-health[256676]: STATUS=OK TS=2026-09-23T08:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:31:39 pve01 homelab-forum-snuffleupagus-health[309649]: STATUS=OK TS=2026-09-23T08:31:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:46:05 pve01 homelab-forum-snuffleupagus-health[346987]: STATUS=OK TS=2026-09-23T08:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:00:22 pve01 homelab-forum-snuffleupagus-health[388609]: STATUS=OK TS=2026-09-23T09:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:15:59 pve01 homelab-forum-snuffleupagus-health[440989]: STATUS=OK TS=2026-09-23T09:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:30:49 pve01 homelab-forum-snuffleupagus-health[499444]: STATUS=OK TS=2026-09-23T09:30:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:45:20 pve01 homelab-forum-snuffleupagus-health[532715]: STATUS=OK TS=2026-09-23T09:45:19Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:00:17 pve01 homelab-forum-snuffleupagus-health[583840]: STATUS=OK TS=2026-09-23T10:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:15:45 pve01 homelab-forum-snuffleupagus-health[636067]: STATUS=OK TS=2026-09-23T10:15:44Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:30:17 pve01 homelab-forum-snuffleupagus-health[687754]: STATUS=OK TS=2026-09-23T10:30:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:45:18 pve01 homelab-forum-snuffleupagus-health[740109]: STATUS=OK TS=2026-09-23T10:45:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:01:18 pve01 homelab-forum-snuffleupagus-health[793923]: STATUS=OK TS=2026-09-23T11:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:16:01 pve01 homelab-forum-snuffleupagus-health[828736]: STATUS=OK TS=2026-09-23T11:15:59Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:30:59 pve01 homelab-forum-snuffleupagus-health[878434]: STATUS=OK TS=2026-09-23T11:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:45:18 pve01 homelab-forum-snuffleupagus-health[927971]: STATUS=OK TS=2026-09-23T11:45:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:00:26 pve01 homelab-forum-snuffleupagus-health[977621]: STATUS=OK TS=2026-09-23T12:00:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:15:23 pve01 homelab-forum-snuffleupagus-health[1028167]: STATUS=OK TS=2026-09-23T12:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:30:59 pve01 homelab-forum-snuffleupagus-health[1074965]: STATUS=OK TS=2026-09-23T12:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:45:42 pve01 homelab-forum-snuffleupagus-health[1125071]: STATUS=OK TS=2026-09-23T12:45:40Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:01:34 pve01 homelab-forum-snuffleupagus-health[1175713]: STATUS=OK TS=2026-09-23T13:01:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:16:22 pve01 homelab-forum-snuffleupagus-health[1225835]: STATUS=OK TS=2026-09-23T13:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:30:59 pve01 homelab-forum-snuffleupagus-health[1274841]: STATUS=OK TS=2026-09-23T13:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:45:54 pve01 homelab-forum-snuffleupagus-health[1325565]: STATUS=OK TS=2026-09-23T13:45:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:00:59 pve01 homelab-forum-snuffleupagus-health[1360767]: STATUS=OK TS=2026-09-23T14:00:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:16:23 pve01 homelab-forum-snuffleupagus-health[1414098]: STATUS=OK TS=2026-09-23T14:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:30:52 pve01 homelab-forum-snuffleupagus-health[1466888]: STATUS=OK TS=2026-09-23T14:30:50Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:46:53 pve01 homelab-forum-snuffleupagus-health[1519524]: STATUS=OK TS=2026-09-23T14:46:46Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:01:32 pve01 homelab-forum-snuffleupagus-health[1568613]: STATUS=OK TS=2026-09-23T15:01:30Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:16:05 pve01 homelab-forum-snuffleupagus-health[1618235]: STATUS=OK TS=2026-09-23T15:16:03Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:30:26 pve01 homelab-forum-snuffleupagus-health[1648980]: STATUS=OK TS=2026-09-23T15:30:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:45:59 pve01 homelab-forum-snuffleupagus-health[1702860]: STATUS=OK TS=2026-09-23T15:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:00:16 pve01 homelab-forum-snuffleupagus-health[1748144]: STATUS=OK TS=2026-09-23T16:00:10Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:16:22 pve01 homelab-forum-snuffleupagus-health[1808309]: STATUS=OK TS=2026-09-23T16:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:30:44 pve01 homelab-forum-snuffleupagus-health[1849936]: STATUS=OK TS=2026-09-23T16:30:42Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:45:50 pve01 homelab-forum-snuffleupagus-health[1901311]: STATUS=OK TS=2026-09-23T16:45:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:00:31 pve01 homelab-forum-snuffleupagus-health[1952524]: STATUS=OK TS=2026-09-23T17:00:29Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:15:13 pve01 homelab-forum-snuffleupagus-health[1997547]: STATUS=OK TS=2026-09-23T17:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:31:17 pve01 homelab-forum-snuffleupagus-health[2057552]: STATUS=OK TS=2026-09-23T17:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:45:38 pve01 homelab-forum-snuffleupagus-health[2102176]: STATUS=OK TS=2026-09-23T17:45:37Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:01:11 pve01 homelab-forum-snuffleupagus-health[2150631]: STATUS=OK TS=2026-09-23T18:01:09Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:16:18 pve01 homelab-forum-snuffleupagus-health[2202826]: STATUS=OK TS=2026-09-23T18:16:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:31:38 pve01 homelab-forum-snuffleupagus-health[2255333]: STATUS=OK TS=2026-09-23T18:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:46:08 pve01 homelab-forum-snuffleupagus-health[2306750]: STATUS=OK TS=2026-09-23T18:46:07Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:00:54 pve01 homelab-forum-snuffleupagus-health[2350713]: STATUS=OK TS=2026-09-23T19:00:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:15:43 pve01 homelab-forum-snuffleupagus-health[2399480]: STATUS=OK TS=2026-09-23T19:15:42Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:30:26 pve01 homelab-forum-snuffleupagus-health[2449737]: STATUS=OK TS=2026-09-23T19:30:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:46:18 pve01 homelab-forum-snuffleupagus-health[2502775]: STATUS=OK TS=2026-09-23T19:46:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:01:17 pve01 homelab-forum-snuffleupagus-health[2551419]: STATUS=OK TS=2026-09-23T20:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:15:59 pve01 homelab-forum-snuffleupagus-health[2593718]: STATUS=OK TS=2026-09-23T20:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:31:17 pve01 homelab-forum-snuffleupagus-health[2633602]: STATUS=OK TS=2026-09-23T20:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:45:24 pve01 homelab-forum-snuffleupagus-health[2684007]: STATUS=OK TS=2026-09-23T20:45:17Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
=== PVE POSTFIX/ALIASES ===
POSTFIX_FILE_META=/etc/aliases mode=644 owner=root group=root size=73
POSTFIX_FILE_SHA256=8e81ec4a775c03d4a6d59c6fddb47bb84705a631fd1b142dffd536b824125bde PATH=/etc/aliases
POSTFIX_FILE_MISSING=/etc/aliases.db
POSTFIX_FILE_META=/etc/postfix/main.cf mode=644 owner=root group=root size=572
POSTFIX_FILE_SHA256=d1050a0c110c2d6c59f595abcf791ee8daa5399dee97fc504fafbcf08adc8f7f PATH=/etc/postfix/main.cf
POSTCONF_inet_interfaces=loopback-only
POSTCONF_relayhost=
POSTCONF_mydestination=$myhostname, localhost.$mydomain, localhost
POSTCONF_mynetworks=<IP>/8
POSTCONF_alias_maps=hash:/etc/aliases
POSTCONF_alias_database=hash:/etc/aliases
POSTCONF_transport_maps=
POSTFIX_QUEUE_COUNT=115
POSTFIX_ALIASES_DB_ERRORS_24H=2806
=== ALERTING JOURNAL 24H ===
Sep 23 07:45:12 pve01 systemd[1]: Starting homelab-alerting-health.service - Homelab alerting health check...
Sep 23 07:45:12 pve01 homelab-alerting-health[3763486]: STATUS=OK TS=2026-09-23T04:45:12Z TYPE=alerting BAD=0 endpoint_local=ok endpoint_from_monitoring=ok kuma_container=ok gotify_container=ok kuma_monitor=ok gotify_link=ok kuma_state=running_healthy gotify_state=running monitor=95|1|keyword|http://<IP>:9101/overall.txt|STATUS=OK|Gotify|1
Sep 23 07:45:12 pve01 systemd[1]: homelab-alerting-health.service: Deactivated successfully.
Sep 23 07:45:12 pve01 systemd[1]: Finished homelab-alerting-health.service - Homelab alerting health check.
=== VM INVENTORY HINT ===
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
VM160_STATE=UNKNOWN
FAILURE=VM160_NOT_RUNNING
DECISION=HOLD_DEEP_AUDIT_INCOMPLETE
NEXT_GATE=FIX_DIAGNOSTIC_BLOCKER_ONLY
TASK_RESULT=FAIL_FORUM_DEEP_AUDIT
HOMELAB_RESULT_CONTRACT={"version":1,"command_id":"FORUM-260923-RCA-DEEP-AUDIT8D42R1","status":"FAIL","changes_made":false,"rollback_started":false,"rollback_restored":null}
OUTPUT_END
CHAT_OUTPUT_END
runtime/latest.json
+16 -16
View File
File diff suppressed because one or more lines are too long
runtime/latest.txt
+429 -34
View File
@@ -1,53 +1,448 @@
CHAT_OUTPUT_BEGIN
COMMAND_ID=SUPPORT-260923-SPB-SNIKKET-DNS-CUTOVER-FORCE-9239R1
STATUS=OK
RC=0
COMMAND_ID=FORUM-260923-RCA-DEEP-AUDIT8D42R1
STATUS=FAIL
RC=3
HOST=pve01
MODE=change
COMPONENT=spb-snikket-dns-cutover-force
MODE=read-only
COMPONENT=forum-deep-audit
REFERENCE_REGISTER_CHECK=OK
REFERENCE_SHA256=5a3d8e5154c41cb582a4e0aca68090be1f0138918bf82131a948df326f9d8d66
ERROR_REGISTER_CHECK=OK
ERROR_REGISTER_SHA256=3b09a553ec0f527ed3afeed4753f52a74ee3036045bbfb3c685e9f8af4ba7ba0
COMMAND_SHA256=e9662656e07fa348ef5a7c5227f65b3294f8b7897a4d8575e5c6ef5ed97e8ca0
COMMAND_SHA256=6d423c85144f338787f0c2f2a7daa180ba7aead6fa3594088284e715f427cf54
DUPLICATE_FAILED_COMMAND_BLOCKED=false
EXECUTION_STARTED=true
CHANGE_DECLARED=true
CHANGE_DECLARED=false
RESULT_CONTRACT_VALID=true
RESULT_CONTRACT_STATUS=OK
RESULT_CONTRACT_STATUS=NOT_APPLICABLE
RESULT_CONTRACT_ERROR=NONE
COMMAND_RC=0
CHANGES_MADE=true
COMMAND_RC=3
CHANGES_MADE=false
ROLLBACK_STARTED=false
ROLLBACK_RESTORED=null
MUTATION_OUTCOME=MUTATED
MUTATION_OUTCOME=NO_MUTATION
SANITIZED=yes
SECRETS_INCLUDED=no
PRIVATE_ADDRESSES_INCLUDED=no
RAW_EVIDENCE_SHA256=b700c98b71f0e81d7e8532d01dfad88cbdcfc0eb0c6239aabeeb23cfe6b3a0b9
SANITIZED_OUTPUT_SHA256=b700c98b71f0e81d7e8532d01dfad88cbdcfc0eb0c6239aabeeb23cfe6b3a0b9
RAW_EVIDENCE_SHA256=216ae7cd459e9bcb0a4ee0015534e8ec40637a2b6ad2108b9f5387cd2e39fee5
SANITIZED_OUTPUT_SHA256=be0ad3901439bb83341b6a18560d8e149eae8c2710481e0b1779cde5c7d7b309
OUTPUT_BEGIN
CUT9239_BEGIN=true
MUTATION_BOUNDARY=FORCE_RECREATE_SNIKKET_SERVER_PLUS_REMAINING_CLOUDFLARE_A;SPB_REMAINS_ON
HOME_WAN_SHA16=26ac437d63a8bcf2
SPB_IP_SHA16=6d6149f42864ce03
CF_SPB_A_RECORD_COUNT=29
CF_HOME_A_RECORD_COUNT=5
VM150_COMPOSE_CONFIG=PASS
VM150_EGRESS_HOME=PASS
SNIKKET_FORCE_RECREATE_SPB_STAGE=PASS
HOME_ROUTER_FORWARD_PREFLIGHT=PASS
DNS_OLD_TTL_MAX=296
DNS_WAIT_BUDGET_SECONDS=386
CF_A_RECORDS_CHANGED=29
PUBLIC_CHAT_DNS_HOME=PASS
SNIKKET_FORCE_RECREATE_HOME_STAGE=PASS
EXTERNAL_HOME_POSTCHECK=PASS
SPB_REMAINS_ON=true
TASK_COMPLETE=true
TASK_RESULT=PASS_SPB_SNIKKET_DNS_CUTOVER_STAGE_A
NEXT_ACTION=VERIFY_ALL_PUBLIC_DNS_THEN_SHUTDOWN_SPB
HOMELAB_RESULT_CONTRACT={"version":1,"command_id":"SUPPORT-260923-SPB-SNIKKET-DNS-CUTOVER-FORCE-9239R1","status":"OK","changes_made":true,"rollback_started":false,"rollback_restored":null}
CID=FORUM-260923-RCA-DEEP-AUDIT8D42R1
MODE=read-only
COMPONENT=forum-deep-audit
MUTATION_BOUNDARY=NONE;NO_MAIL_SEND;NO_DB_WRITE;NO_SERVICE_RESTART;NO_VM_CONFIG;NO_CLOUD
EXECUTION_STARTED=true
CHANGES_MADE=false
ROLLBACK_STARTED=false
ROLLBACK_RESTORED=null
=== AUTHORITY GATES ===
AUTHORITY_SHA256=5a3d8e5154c41cb582a4e0aca68090be1f0138918bf82131a948df326f9d8d66 PATH=/etc/pve/31_HOMELAB_REFERENCE.md
AUTHORITY_SHA256=3b09a553ec0f527ed3afeed4753f52a74ee3036045bbfb3c685e9f8af4ba7ba0 PATH=/etc/pve/HOMELAB_ASSISTANT_ERROR_REGISTER.md
AUTHORITY_SHA256=b248a4c32c9cc64e5747e7dce6c7fc0a23f5124a77c71ce72e27a81aceae9d2d PATH=/usr/local/sbin/homelab-chat-run
=== FRESH HANDOFF ===
HANDOFF_PRESENT=true
HANDOFF_SHA256=82a6060ccc216849c55790a3489ef157d5551a0c7cef0c5ec9abd35f569c7940
11135:- Remote source used: pve02-mail-01-crypt app/forums/forum-prod/fresh5 latest timestamp.
11140:- Proof: /root/evidence/612_FORUM_FRESH5_CLOUD_RESTORE_DRILL_CORRECTED_PROOF.txt.
11142:## FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_OK_20260701
11143:- Five public XenForo forums are healthy over HTTPS.
11144:- Mailcow admin and NetBird UI are reachable after persistent forum SMTP enablement.
11145:- Persistent forum SMTP via msmtp is enabled and tested.
11146:- Local backup exists on VM160 under /var/backups/forums.
11147:- Mail.ru Cloud fresh5 backup is enabled on pve02 via forum-prod-fresh5-cloud-backup.timer.
11149:- Cloud restore drill passed: split parts verified, archive reconstructed, SHA256 matched, tar contained DB/file archives for all five forums.
11150:- Final rollup proof: /root/evidence/614_FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_PROOF.txt.
11152:## FORUM_XENFORO_JOB_RUNNER_CONFIGURED_20260701
11153:- XenForo CLI job runner configured inside VM160 forum-prod.
11154:- Script: /root/scripts/forum-xenforo-run-jobs.sh.
11155:- Cron: /etc/cron.d/forum-xenforo-run-jobs every 5 minutes.
11156:- Purpose: process XenForo job queues and cron tasks independent of forum traffic.
11157:- First run proof: /root/evidence/617_XENFORO_RUN_JOBS_ONCE_PROOF.txt.
11158:- Cron verification proof: /root/evidence/618_XENFORO_JOB_RUNNER_CRON_CONFIG_PROOF.txt.
11160:## FORUM_XENFORO_CRON_DAEMON_AND_JOBRUNNER_OK_20260701
11161:- cron daemon inside VM160 forum-prod is active/enabled and was proven by temporary /etc/cron.d execution proof.
11162:- XenForo job runner script executed successfully after cron daemon verification.
11163:- Due XenForo cron count returned to zero after run.
11164:- Proof: /root/evidence/621_FORUM_CRON_DAEMON_JOBRUNNER_VERIFY_PROOF.txt.
11166:## FORUM_DBTECH_TOR_TIMEOUT_CLASSIFIED_20260701
11167:- Each forum had one historical XenForo error_log row for cURL timeout to check.torproject.org/torbulkexitlist.
11169:- dbtech_security_tornodes option is 0 on all five forums.
11170:- Classified as non-blocking historical external network timeout/noise; not a forum runtime failure.
11174:## FORUM_XENFORO_BUILTIN_MAIL_DELIVERED_BUT_SPAM_20260701
11175:- Built-in XenForo outgoing email test from site "Моды для Hollow Knight" was delivered to <EMAIL>.
11178:- This proves forum SMTP transport works, but deliverability/reputation needs tuning.
11181:- Proof: /root/evidence/627_XENFORO_BUILTIN_MAIL_DELIVERED_SPAM_CLASSIFICATION_PROOF.txt.
11183:## FORUM_YANDEX_DELIVERABILITY_SPAM_WITH_SPF_DKIM_PASS_20260701
11184:- Built-in XenForo outgoing email test was delivered to <EMAIL> but placed in Spam.
11188:- Classification: SMTP/Mailcow/XenForo transport is OK; remaining issue is deliverability/reputation/content filtering.
11190:- Next actions: mark as Not spam in Yandex, add/check pvepro.ru in Yandex Postmaster, warm up sender reputation, optionally plan per-forum sender domains later.
11192:## FORUM_PROJECT_FINAL_ACCEPTANCE_OK_20260701
11193:- Five XenForo forums are published and healthy over HTTPS: codevipe.ru, gamevipe.ru, hkmods.ru, zakrutim.ru, dsmods.ru.
11194:- VM160 forum-prod is the active production forum VM on pve02 at <IP>.
11195:- Runtime services are configured: nginx, MariaDB, PHP 8.3 FPM, qemu-agent, cron, msmtp/sendmail transport.
11196:- NPMplus on edge terminates TLS and proxies all five forums to VM160.
11198:- Forum SMTP transport works through <EMAIL>; Yandex headers proved SPF pass and DKIM pass. Spam placement is classified as reputation/content deliverability, not server failure.
11199:- XenForo job runner cron is configured and cron daemon execution was proven.
11200:- Local backup is configured on VM160 and Mail.ru Cloud fresh5/all-forums backup is configured on pve02.
11201:- Cloud restore drill passed: split parts verified, archive SHA matched, tar contained DB/file/config archives for all five forums.
11203:- Final proof: /root/evidence/629_FORUM_PROJECT_FINAL_ACCEPTANCE_PROOF.txt.
11205:## FORUM_NEXT_CHAT_HANDOFF_FULL_STATE_20260701
11217:### Production forum VM
11219:- Name: forum-prod.
11223:- Runtime: nginx, MariaDB 10.11, PHP 8.3 FPM, qemu-agent, cron, msmtp/sendmail transport.
11224:- Final accepted snapshot: forum-final-accepted-091934Z.
11225:- Mail final snapshot: forum-mail-ok-091815Z.
11226:- Postlaunch snapshot: forum-postlaunch-ok-085501Z.
11228:### Public forums
11238:### Mail
11239:- Forum sender: <EMAIL>.
11240:- Transport: XenForo sendmail -> msmtp -> mail.pvepro.ru.
11241:- Built-in XenForo mail test reached Yandex.
11250:- Local archive path pattern: /var/backups/forums/forum-backup-*.tar.gz.
11251:- pve02 Mail.ru Cloud fresh5/all-forums backup configured.
11252:- New timer: forum-prod-fresh5-cloud-backup.timer active/enabled.
11255: - pve02-mail-01-crypt
11256: - pve02-mail-02-crypt
11257: - pve02-mail-03-crypt
11258: - pve02-mail-04-crypt
11260:- Restore drill passed: split parts verified, archive SHA matched, tar contained DB/file/config archives for all five forums.
11262:### XenForo jobs and cron
11265:- XenForo job runner configured:
11266: - /root/scripts/forum-xenforo-run-jobs.sh
11267: - /etc/cron.d/forum-xenforo-run-jobs
11268:- Purpose: process XenForo jobs and cron tasks independent of visitor traffic.
11271:- Each forum had one historical xf_error_log row for cURL timeout to check.torproject.org.
11273:- dbtech_security_tornodes=0 on all five forums.
11278:- /root/evidence/629_FORUM_PROJECT_FINAL_ACCEPTANCE_PROOF.txt
11279:- /root/evidence/628_FORUM_MAIL_DELIVERABILITY_FINAL_SNAPSHOT_PROOF.txt
11282:- /root/evidence/621_FORUM_CRON_DAEMON_JOBRUNNER_VERIFY_PROOF.txt
11283:- /root/evidence/614_FORUM_POSTLAUNCH_BACKUP_MAIL_FINAL_ROLLUP_PROOF.txt
11284:- /root/evidence/613_FORUM_FRESH5_CLOUD_RESTORE_DRILL_FINALIZE_PROOF.txt
11285:- /root/evidence/612_FORUM_FRESH5_CLOUD_RESTORE_DRILL_CORRECTED_PROOF.txt
11286:- /root/evidence/609_FORUM_PROD_FRESH5_CLOUD_BACKUP_FINALIZE_PROOF.txt
11287:- /root/evidence/608_FORUM_PROD_FRESH5_CLOUD_BACKUP_FIRST_RUN_PROOF.txt
11288:- /root/evidence/600_FORUM_PERSISTENT_SMTP_FINAL_STABLE_PROOF.txt
11289:- /root/evidence/570_MAIL_DNS_FINAL_CLEAN_AUDIT_PROOF.txt
11290:- /root/evidence/560_FORUM_BACKUP_SCRIPT_SETUP_PROOF.txt
11318:CHECK-9: для forum-prod использовать pve02 и ключ [SENSITIVE_PATH]
11325:VM160 forum-prod is not in Proxmox nightly backup.
11339:Не искать Cloudflare/SMTP сразу по /opt/stacks, MkDocs site, backups и HTML-отчётам.
11352:forum-prod: заходить через pve02, ключ [SENSITIVE_PATH]
11371:20. Ошибка: широкий XenForo SMTP-поиск по src дал шум исходников.
11372:Для XenForo SMTP не grep-ать весь /src; сначала смотреть live src/config.php без значений, затем БД XenForo и xf_option/emailTransport.
11373:Значения DB-паролей и SMTP-паролей не печатать.
11375:21. Ошибка: nested PHP php -r дал Parse error на forum-prod.
11376:Команды 102-104 считать битыми: они не подтвердили и не изменили smtpSsl.
11377:Не продолжать длинные вложенные ssh+php -r с кавычками; для forum-prod лучше войти в VM или использовать короткие mysql/php команды без вложенного кода.
11378:Рабочими считать: DB backup перед SMTP, SMTP password update, JSON postcheck; SMTP auth ещё не закрыт из-за CODE=530 на AUTH LOGIN.
11380:22. Ошибка: самодельный base64 PHP для SMTP auth сломан.
11381:Команда 108 дала PHP Parse error на smtpHost и пустой proof-файл.
11382:Не использовать больше сгенерированные большие base64 PHP для SMTP/XenForo.
11383:Для XenForo SMTP использовать короткие mysql-запросы к xf_option и готовые штатные механизмы XenForo.
11386:Команда 111 не изменила smtpSsl: OLD_SMTPSSL=false, SQL ERROR 1064, NEW_SMTPSSL=false.
11395:25. Ошибка анализа: не надо слепо включать smtpSsl=true при smtpPort=587.
11396:smtpPort=587 обычно означает STARTTLS, а не implicit SMTPS.
11397:CODE=530 на AUTH LOGIN означает, что ручной checker не сделал STARTTLS; это не доказывает неверный SMTP-пароль.
11398:Сначала проверять AUTH через openssl s_client -starttls smtp, без вывода секрета.
11400:26. Ошибка: openssl -crlf вместе с ручным CRLF ломает SMTP AUTH checker.
11402:Это указывает на битую base64-строку checker-а, а не на неверный SMTP-пароль.
11404:Для 587 оставлять smtpSsl=false и проверять STARTTLS корректным checker-ом.
11464:40. Ошибка/вывод: VM150 Mail cloud single-file upload failed with 413.
11465:Факт: Mail/WebDAV rejected one 6.6G vzdump file with 413 Request Entity Too Large.
11469:Факт: first reconciliation wrote STATUS=ERROR because router-running-config-mail-cloud.txt is on pve01, not edge-vm.
11579:## FORUM_PROD_VM160_FIRST_BOOT_SSH255_20260701
11584:## FORUM_PROD_VM160_NESTED_HOSTNAME_PROOF_QUOTING_20260701
11590:## FORUM_PROD_VM160_SWAP_PROOF_QUOTING_20260701
11596:## FORUM_UPLOAD_ARCHIVES_VM_VERIFY_QUOTING_20260701
11600:- Rule: verify VM archive integrity locally inside forum-prod, not through nested quoted loops.
11602:## FORUM_PROD_BULK_IMPORT_PHP85_EMPTY_FRONTEND_20260701
11603:- Context: rebuilt VM160, five XenForo forums imported in bulk on PHP 8.5.7.
11604:- Symptom: all five forums returned HTTP 200 with empty body; XenForo logged Template public:PAGE_CONTAINER/forum_list is unknown.
11606:- Decision: stop bulk repair, treat current VM160 as lab, find a working CodeVipe recipe first, then destroy/rebuild cleanly for all five forums.
11607:- Rule: no more mass rebuild/repair commands across all forums until single-forum CodeVipe recipe is proven.
11609:## XENFORO_REQUIREMENTS_DIRECT_DOWNLOAD_403_20260701
11610:- Context: Tried to download XenForo requirements ZIP directly from xenforo.com inside forum-prod.
11614:- Rule: use operator-uploaded xenforo23-requirements-test.zip from /root/forum-upload instead of relying on direct curl download.
11617:- Context: proof 492 tried to inspect uploaded xenforo23-requirements-test.zip on pve01.
11620:- Rule: verify ZIP with Python zipfile or inside forum-prod rather than installing unzip on Proxmox just for inspection.
11623:- Context: VM160 was destroyed/recreated, then five XenForo forums were deployed from fresh ZIP backups.
11624:- Result: proof 513 confirms all five forums locally healthy.
11635:- Context: proof 525 tried to create forum proxy hosts in NPMplus.
11636:- Issue: script sourced /opt/npmplus/.env, but INITIAL_ADMIN_EMAIL/PASSWORD were not present there; they are in the npmplus container env.
11637:- Impact: no forum proxy hosts were created by proof 525.
11647:- Context: NPMplus API login attempts failed even though the forum VM and edge-to-forum connectivity are healthy.
11650:- Rule: publish forum routes through backed-up manual nginx proxy_host configs, then issue proper certificates after DNS points to edge.
11653:- Context: proof 535 attempted DNS-01 certificate issue for five forum domains.
11654:- Issue: /tmp/forum_cf_token.env was root-owned mode 600, so debian user could not source it directly.
11665:- Context: Cloudflare token discovered on edge verified as a token but could not access forum zones.
11668:- Rule: install a new Cloudflare token with Zone Read and DNS Edit for the five forum zones before retrying certs/DNS.
11670:## FORUM_PUBLICATION_FINAL_SUCCESS_20260701
11671:- Context: five XenForo forums were rebuilt on VM160, routed through edge NPMplus, issued Let’s Encrypt certs using a corrected Cloudflare token, and cut over in DNS.
11676:- Context: proof 556 final health gate passed for all five public forums.
11679:- Impact: forum health was OK, but proof 556 snapshot step was not completed.
11686:- Impact: web routing is OK, but mail-related DNS may still contain stale provider data.
11697:- Impact: domains temporarily had duplicate SPF records, which is invalid for mail validation.
11698:- Fix: delete all SPF TXT records for the five forum zones, then create exactly one v=spf1 -all TXT per zone.
11700:## MSMTP_SECRET_SOURCE_PARSE_ERROR_576_20260701
11701:- Context: proof 576 installed msmtp but sendmail auth test failed.
11702:- Issue: [SENSITIVE_PATH] was sourced as shell, but SMTP password contained shell-special characters; source failed and auth used an invalid/truncated secret path.
11703:- Impact: msmtp package installed, but mail sending was not proven working.
11704:- Rule: never source SMTP secret files containing arbitrary passwords; store password base64 and use msmtp passwordeval helper accessible to www-data.
11706:## FORUM_MSMTP_MAIL_TRANSPORT_STILL_FAILING_20260701
11707:- Context: attempted to fix msmtp config with passwordeval helper.
11708:- Issue: one or more sendmail/PHP mail tests still failed.
11709:- Impact: XenForo mail sending is not yet proven.
11710:- Next step: verify SMTP credential/password and provider policy for <EMAIL>.
11712:## SMTP_PASSWORD_EXPOSED_ROTATE_REQUIRED_20260701
11713:- Context: SMTP password was exposed in terminal output during failed msmtp setup.
11714:- Cause: secret file with raw SMTP_PASS was later parsed/sourced by shell; shell error printed the raw assignment line.
11715:- Impact: treat that SMTP password as compromised.
11716:- Required action: rotate the mailbox/app password in the mail provider panel before any further SMTP testing.
11719:## NETBIRD_MAILCOW_REPORTED_DOWN_AFTER_SMTP_INCIDENT_20260701
11720:- Context: user reported NetBird and Mailcow became unavailable after forum mail-transport work.
11722:- Rule: do not change DNS, firewall, proxy, mail, NetBird, or Mailcow configs before read-only triage proof.
11724:## FORUM_SMTP_FAILED_CONFIG_DISABLED_AFTER_MAILCOW_BAN_20260701
11725:- Context: forum-prod SMTP/msmtp test used invalid or compromised credentials and triggered Mailcow netfilter warnings/ban for <IP>.
11726:- Impact: forum mail transport is not proven and must remain disabled until SMTP password is rotated and tested once carefully.
11727:- Action: remove forum-prod msmtp secret/config files so XenForo cannot keep retrying broken SMTP auth.
11728:- Rule: do not test SMTP auth again until a fresh rotated mailbox/app password is available.
11730:## FORUM_OLD_CODEVIPE_CLOUD_BACKUP_STALE_AFTER_FRESH5_20260701
11731:- Context: pve02 codevipe-cloud-backup.timer existed before fresh five-forum rebuild.
11732:- Issue: old script points to /root/backups/codevipe and old remote path app/codevipe/forum-prod/local.
11733:- Evidence: latest service run failed with missing /root/backups/codevipe; remote inventory contains old codevipe-public/codevipe.sql files, not current five-forum archive.
11734:- Impact: old timer must not be treated as valid current backup for all five forums.
11735:- Fix: create a new forum-prod fresh5 cloud backup using existing pve02 Mail-cloud rclone crypt remotes.
11737:## FORUM_RESTORE_DRILL_610_STATUS_FLAG_BUG_20260701
11738:- Context: restore drill proof 610 downloaded Mail.ru Cloud backup, verified split part SHA256, reconstructed archive, matched full archive SHA256, and confirmed all five forum DB/file archives in tar.
11739:- Issue: final RESTORE_DRILL_OK was set to no because the script tested an unassigned shell variable RCLONE_REMOTE_PVE02_MAIL_01_RC.
11743:## FORUM_XENFORO_MAIL_SMOKE_TEST_FAILED_20260701
11744:- XenForo-level mail smoke test did not return success for all five forums.
11745:- Check proof 623 and msmtp log before retrying.
11747:## XENFORO_MAIL_SMOKE_623_INVALID_ROOT_PATH_20260701
11748:- Context: custom XenForo mail smoke test 623 failed with Could not open input file /root/scripts/623_xenforo_mail_smoke.php.
11750:- Impact: proof 623 is invalid and should not be used to judge mail delivery.
11751:- Superseding evidence: user ran XenForo built-in outgoing email test; message was delivered to Yandex but placed in Spam.
11754:## FORUM_NEXT_CHAT_KNOWN_ERRORS_AND_CLOSED_INCIDENTS_20260701
11757:- SMTP_PASSWORD_EXPOSED_ROTATE_REQUIRED_20260701:
11758: - A previous bad command sourced a raw SMTP secret and printed it.
11760: - Later persistent SMTP was rebuilt using safe files and verified.
11763:- XENFORO_MAIL_SMOKE_623_INVALID_ROOT_PATH_20260701:
11764: - Custom mail proof 623 failed with "Could not open input file".
11766: - Impact: proof 623 is invalid and must not be used to judge mail delivery.
11767: - Superseded by user-observed built-in XenForo test and Yandex header proof.
11769:- FORUM_MSMTP_MAIL_TRANSPORT_STILL_FAILING_20260701 / FORUM_SMTP_FAILED_CONFIG_DISABLED_AFTER_MAILCOW_BAN_20260701:
11770: - Earlier SMTP attempts failed and triggered mailcow netfilter ban for forum public IP.
11772: - Persistent msmtp transport was later enabled and verified.
11773: - Mailcow and NetBird remained reachable after final tests.
11777: - Fixed by deleting duplicates and recreating exactly one SPF per forum domain.
11778: - Final forum domains use v=spf1 -all because sending uses <EMAIL>.
11780:- FORUM_RESTORE_DRILL_610_STATUS_FLAG_BUG_20260701:
11784:- FORUM_DBTECH_TOR_TIMEOUT_CLASSIFIED_20260701:
11785: - One historical timeout row per forum to check.torproject.org.
11790:- Yandex placed built-in XenForo test mail in Spam despite SPF pass and DKIM pass.
11798: - /etc/msmtprc
11799: - /etc/msmtp/*
11822:`service → owner node/VM/container → data path → backup type → local copy → offhost copy → Mail cloud remote → timer → retention → last backup OK → last upload OK → last restore drill OK → alert OK → RPO/RTO → disposable yes/no`.
11830:**Третье: Mail.ru Cloud надо сделать основным offsite-слоем.**
11833:* pve01 cloud: Nextcloud, dns1/unbound1, critical configs, часть P0-копий edge/forum.
11834:* pve02 cloud: forum-prod, dns2/unbound2, копии P0 edge/Nextcloud.
11838:Сейчас Mail-cloud capacity выглядит очень недоиспользованной: largest used remote был около 65 GiB of 1 TiB, а у вас фактически запас гораздо больше. Поэтому можно смело ужесточить retention для критичных вещей.
11870:* VM160 forum-prod: local VM backup + app-level forum backup + cloud restore drill.
11881:* Forum-prod сейчас уже сильнее: fresh5 cloud backup, split parts, SHA256, manifest, restore drill, tar listing всех пяти форумов.
11899:**Forum-prod:**
11903:не делать тупой бесконечный full каждый день, а хранить: DB/config чаще, data/media реже и chunked/manifest-based. VM150 уже имеет Mail-cloud chunked backup with download verification — это правильный паттерн.
11918:2. **pve02 увеличить RAM до 32 GB**, если железо позволяет. Forum-prod + DNS вторички сейчас живут, но N100/15 GiB — слабое место для роста форумов.
11931:* умер pve02 → через N минут поднимаем forum-prod из cloud/local backup;
11939:* pve02-loss drill: восстановить VM160 из cloud backup на временную VM, проверить nginx/php/mysql, один форум, mail disabled/test-safe.
11945:Monitoring уже сильный: Prometheus, Grafana, Alertmanager, Loki, Alloy, Kuma, Gotify, ntfy, Healthchecks, Blackbox exporter и node-exporter описаны. Но pve03 не имел homelab-health-http `:9101`, в отличие от pve01/pve02.
11951:* alert на `backup_age`, `cloud_upload_age`, `restore_drill_age`, `remote_capacity`, `manifest_mismatch`;
11959:Сейчас NPMplus маршруты и сертификаты в хорошем состоянии: активный proxy dir `/opt/npmplus/nginx/proxy_host`, forum routes `200–204`, parked `995–998`, pvepro `994`, taftauto `993`, deploy hooks, forum renewal cron, token files mode 600.
11988:* автоматический контроль прав rclone configs, Cloudflare token files, msmtp secrets;
11989:* отдельный incident runbook: что ротировать при утечке SMTP/Cloudflare/rclone/router.
11998:4. Стандартизировать Mail.ru Cloud layout на всех нодах.
12015:3. Router startup-config сделать не только manual, а регулярным, если доступ позволит; running-config уже имеет recurring Mail-cloud backup.
12025:Главная мысль: **не добавлять сейчас новые красивые штуки, а довести восстановление до состояния “любой узел умер — мы спокойно восстанавливаемся по матрице”**. На вашем кластере это реально сделать без PBS, через Proxmox local vzdump + offhost на соседние ноды + Mail.ru Cloud crypt/remotes + обязательные restore drills.
=== PVE FORUM/ALERT UNIT STATE ===
UNIT=homelab-forum-snuffleupagus-health ACTIVE=inactive ENABLED=static
UNIT=homelab-alerting-health ACTIVE=inactive ENABLED=static
UNIT=postfix ACTIVE=active ENABLED=enabled
TIMER=homelab-forum-snuffleupagus-health ACTIVE=active ENABLED=enabled
TIMER=homelab-alerting-health ACTIVE=active ENABLED=enabled
=== FORUM HEALTH SCRIPT/REPORT ===
META=/usr/local/sbin/homelab-forum-snuffleupagus-health mode=700 owner=root group=root size=3349
SHA256=d7cee34b7d0e93c578f9203ba6e3634d304fee3ea5f3629a576dd683f1b50b29 PATH=/usr/local/sbin/homelab-forum-snuffleupagus-health
4:HEALTH="/var/lib/homelab-health/forum-snuffleupagus.txt"
5:REPORT="/var/lib/homelab-forum-snuffleupagus/latest.md"
6:mkdir -p /var/lib/homelab-health /var/lib/homelab-forum-snuffleupagus
9:OUT="$(ssh -n -o BatchMode=yes -o ConnectTimeout=10 root@pve02 'ssh -n -i [SENSITIVE_PATH] -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new root@<IP> "set +e; echo FORUM_HOST=\$(hostname -s); echo NO_ENV_FILES_READ=YES; php -m 2>/dev/null | grep -i ^snuffleupagus$ >/dev/null && echo MODULE=loaded || echo MODULE=not_loaded; php --ri snuffleupagus 2>/dev/null | grep -Ei \"snuffleupagus support|Version\" | head -n 20 || true; systemctl is-active nginx 2>/dev/null | sed s/^/NGINX=/; systemctl is-active php8.3-fpm 2>/dev/null | sed s/^/PHP83FPM=/; systemctl is-active mariadb 2>/dev/null | sed s/^/MARIADB=/; test -s /etc/php/8.3/snuffleupagus/xenforo-simulation.rules && echo RULES=present || echo RULES=missing; test -s /etc/php/8.3/mods-available/snuffleupagus.ini && echo MOD_INI=present || echo MOD_INI=missing; grep -Ev \"^[[:space:]]*#\" /etc/php/8.3/snuffleupagus/xenforo-simulation.rules 2>/dev/null | grep -q \"sp\\.readonly_exec\" && echo ACTUAL_READONLY_EXEC_RULE=YES || echo ACTUAL_READONLY_EXEC_RULE=NO; grep -Ev \"^[[:space:]]*#\" /etc/php/8.3/snuffleupagus/xenforo-simulation.rules 2>/dev/null | grep -q \"sp\\.ini_protection\" && echo ACTUAL_INI_PROTECTION_RULE=YES || echo ACTUAL_INI_PROTECTION_RULE=NO; cat /var/lib/homelab-forum-hardening/snuffleupagus-health.txt 2>/dev/null || echo FORUM_LOCAL_HEALTH=missing; for d in codevipe.ru gamevipe.ru hkmods.ru zakrutim.ru dsmods.ru; do code=\$(curl -k -sS -o /tmp/forum-snuffle-health-\$d.body -w \"%{http_code}\" --connect-timeout 10 --max-time 20 https://\$d/ 2>/tmp/forum-snuffle-health-\$d.err); bytes=\$(wc -c < /tmp/forum-snuffle-health-\$d.body 2>/dev/null || echo 0); echo PUBLIC_SMOKE site=\$d http_code=\$code bytes=\$bytes; done"' 2>&1)"
10:echo "# Forum Snuffleupagus health" > "$REPORT"
13:echo "target=forum-prod VM160 <IP>" >> "$REPORT"
14:echo "forums=codevipe.ru,gamevipe.ru,hkmods.ru,zakrutim.ru,dsmods.ru" >> "$REPORT"
30:echo "STATUS=$STATUS TS=$TS TYPE=forum-snuffleupagus BAD=$BAD WARN=$WARN MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=$SMOKE_BAD ACTUAL_READONLY_EXEC_RULE=NO REPORT=$REPORT" > "$HEALTH"
META=/usr/local/sbin/homelab-alerting-health mode=700 owner=root group=root size=2318
SHA256=2050bac8c2ca8f1116e6d4fabe476292253bf38c56a49456242606980b4eb797 PATH=/usr/local/sbin/homelab-alerting-health
4:OUT="/var/lib/homelab-health/alerting.txt"
33:echo "STATUS=$STATUS TS=$TS TYPE=alerting BAD=$BAD$DETAILS kuma_state=${KUMA_STATE// /_} gotify_state=${GOTIFY_STATE// /_} monitor=${MON_ROW:-missing}" > "$OUT"
META=/etc/systemd/system/homelab-forum-snuffleupagus-health.service mode=644 owner=root group=root size=167
SHA256=16f2007e56807897865ae510ff84690d41bb7031fe037cc45425585d737f134b PATH=/etc/systemd/system/homelab-forum-snuffleupagus-health.service
META=/etc/systemd/system/homelab-forum-snuffleupagus-health.timer mode=644 owner=root group=root size=160
SHA256=818d7355c53af09a0ef127ab42eceaf1536e93364501160d08995ddf47faf3be PATH=/etc/systemd/system/homelab-forum-snuffleupagus-health.timer
META=/etc/systemd/system/homelab-alerting-health.service mode=644 owner=root group=root size=184
SHA256=2328f1042ac8f6b201603d86941afc87d62191ae12559bee0fd77709c76066f3 PATH=/etc/systemd/system/homelab-alerting-health.service
META=/etc/systemd/system/homelab-alerting-health.timer mode=644 owner=root group=root size=163
SHA256=28ceb1a533a392af460fc8a42886912fcd256ee6fe8da89460e6c8bdb6096ef1 PATH=/etc/systemd/system/homelab-alerting-health.timer
FORUM_REPORT_SHA256=9d41822f17cdaed296b2e80dd5ba331d01870722d788bf6adfb57d82141ec3f3
# Forum Snuffleupagus health
generated_utc=2026-09-23T20:45:17Z
target=forum-prod VM160 <IP>
forums=codevipe.ru,gamevipe.ru,hkmods.ru,zakrutim.ru,dsmods.ru
env_files_read=no
secret_values_printed=no
FORUM_HOST=forum-prod
NO_ENV_FILES_READ=YES
MODULE=loaded
snuffleupagus support => enabled
Version => 0.13.0
NGINX=active
PHP83FPM=active
MARIADB=active
RULES=present
MOD_INI=present
ACTUAL_READONLY_EXEC_RULE=NO
ACTUAL_INI_PROTECTION_RULE=YES
STATUS=OK TS=2026-07-07T03:35:55Z TYPE=snuffleupagus-forums MODE=low-noise-simulation STRICT=NO BAD=0 MODULE=loaded FPM=active FORUMS=5 READONLY_EXEC=disabled_reason_xenforo_code_cache PUBLIC_SMOKE=OK
PUBLIC_SMOKE site=codevipe.ru http_code=200 bytes=44523
PUBLIC_SMOKE site=gamevipe.ru http_code=200 bytes=74958
PUBLIC_SMOKE site=hkmods.ru http_code=200 bytes=111772
PUBLIC_SMOKE site=zakrutim.ru http_code=200 bytes=147223
PUBLIC_SMOKE site=dsmods.ru http_code=200 bytes=173355
=== FORUM HEALTH JOURNAL 24H ===
Sep 23 00:01:09 pve01 homelab-forum-snuffleupagus-health[2154081]: STATUS=OK TS=2026-09-22T21:01:08Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:15:59 pve01 homelab-forum-snuffleupagus-health[2206413]: STATUS=OK TS=2026-09-22T21:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:31:33 pve01 homelab-forum-snuffleupagus-health[2259317]: STATUS=OK TS=2026-09-22T21:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 00:45:22 pve01 homelab-forum-snuffleupagus-health[2310128]: STATUS=OK TS=2026-09-22T21:45:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:01:02 pve01 homelab-forum-snuffleupagus-health[2351204]: STATUS=OK TS=2026-09-22T22:01:01Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:15:13 pve01 homelab-forum-snuffleupagus-health[2394798]: STATUS=OK TS=2026-09-22T22:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:31:33 pve01 homelab-forum-snuffleupagus-health[2455122]: STATUS=OK TS=2026-09-22T22:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 01:45:46 pve01 homelab-forum-snuffleupagus-health[2509007]: STATUS=OK TS=2026-09-22T22:45:44Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:00:59 pve01 homelab-forum-snuffleupagus-health[2562448]: STATUS=OK TS=2026-09-22T23:00:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:16:22 pve01 homelab-forum-snuffleupagus-health[2611647]: STATUS=OK TS=2026-09-22T23:16:20Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:31:34 pve01 homelab-forum-snuffleupagus-health[2664129]: STATUS=OK TS=2026-09-22T23:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 02:45:33 pve01 homelab-forum-snuffleupagus-health[2707466]: STATUS=OK TS=2026-09-22T23:45:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:00:18 pve01 homelab-forum-snuffleupagus-health[2752522]: STATUS=OK TS=2026-09-23T00:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:15:53 pve01 homelab-forum-snuffleupagus-health[2814508]: STATUS=OK TS=2026-09-23T00:15:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:31:40 pve01 homelab-forum-snuffleupagus-health[2907118]: STATUS=OK TS=2026-09-23T00:31:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 03:46:04 pve01 homelab-forum-snuffleupagus-health[2959189]: STATUS=OK TS=2026-09-23T00:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:00:33 pve01 homelab-forum-snuffleupagus-health[2990112]: STATUS=OK TS=2026-09-23T01:00:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:16:04 pve01 homelab-forum-snuffleupagus-health[3041256]: STATUS=OK TS=2026-09-23T01:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:31:00 pve01 homelab-forum-snuffleupagus-health[3099073]: STATUS=OK TS=2026-09-23T01:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 04:45:15 pve01 homelab-forum-snuffleupagus-health[3142158]: STATUS=OK TS=2026-09-23T01:45:14Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:01:17 pve01 homelab-forum-snuffleupagus-health[3196314]: STATUS=OK TS=2026-09-23T02:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:15:53 pve01 homelab-forum-snuffleupagus-health[3238867]: STATUS=OK TS=2026-09-23T02:15:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:31:05 pve01 homelab-forum-snuffleupagus-health[3282264]: STATUS=OK TS=2026-09-23T02:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 05:45:59 pve01 homelab-forum-snuffleupagus-health[3332050]: STATUS=OK TS=2026-09-23T02:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:01:19 pve01 homelab-forum-snuffleupagus-health[3433024]: STATUS=OK TS=2026-09-23T03:01:17Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:15:17 pve01 homelab-forum-snuffleupagus-health[3482327]: STATUS=OK TS=2026-09-23T03:15:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:31:01 pve01 homelab-forum-snuffleupagus-health[3540188]: STATUS=OK TS=2026-09-23T03:30:59Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 06:45:59 pve01 homelab-forum-snuffleupagus-health[3575511]: STATUS=OK TS=2026-09-23T03:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:01:23 pve01 homelab-forum-snuffleupagus-health[3629329]: STATUS=OK TS=2026-09-23T04:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:15:59 pve01 homelab-forum-snuffleupagus-health[3673169]: STATUS=OK TS=2026-09-23T04:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:31:22 pve01 homelab-forum-snuffleupagus-health[3723216]: STATUS=OK TS=2026-09-23T04:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 07:45:16 pve01 homelab-forum-snuffleupagus-health[3771308]: STATUS=OK TS=2026-09-23T04:45:15Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:01:39 pve01 homelab-forum-snuffleupagus-health[3831033]: STATUS=OK TS=2026-09-23T05:01:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:15:03 pve01 homelab-forum-snuffleupagus-health[3866230]: STATUS=OK TS=2026-09-23T05:15:02Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:30:53 pve01 homelab-forum-snuffleupagus-health[3917462]: STATUS=OK TS=2026-09-23T05:30:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 08:45:59 pve01 homelab-forum-snuffleupagus-health[3976070]: STATUS=OK TS=2026-09-23T05:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:00:10 pve01 homelab-forum-snuffleupagus-health[4019194]: STATUS=OK TS=2026-09-23T06:00:09Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:15:59 pve01 homelab-forum-snuffleupagus-health[4062449]: STATUS=OK TS=2026-09-23T06:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:30:18 pve01 homelab-forum-snuffleupagus-health[4108279]: STATUS=OK TS=2026-09-23T06:30:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 09:45:38 pve01 homelab-forum-snuffleupagus-health[4165927]: STATUS=OK TS=2026-09-23T06:45:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:01:23 pve01 homelab-forum-snuffleupagus-health[21741]: STATUS=OK TS=2026-09-23T07:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:15:17 pve01 homelab-forum-snuffleupagus-health[52177]: STATUS=OK TS=2026-09-23T07:15:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:31:23 pve01 homelab-forum-snuffleupagus-health[110050]: STATUS=OK TS=2026-09-23T07:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 10:46:17 pve01 homelab-forum-snuffleupagus-health[152931]: STATUS=OK TS=2026-09-23T07:46:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:01:18 pve01 homelab-forum-snuffleupagus-health[202471]: STATUS=OK TS=2026-09-23T08:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:16:23 pve01 homelab-forum-snuffleupagus-health[256676]: STATUS=OK TS=2026-09-23T08:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:31:39 pve01 homelab-forum-snuffleupagus-health[309649]: STATUS=OK TS=2026-09-23T08:31:38Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 11:46:05 pve01 homelab-forum-snuffleupagus-health[346987]: STATUS=OK TS=2026-09-23T08:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:00:22 pve01 homelab-forum-snuffleupagus-health[388609]: STATUS=OK TS=2026-09-23T09:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:15:59 pve01 homelab-forum-snuffleupagus-health[440989]: STATUS=OK TS=2026-09-23T09:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:30:49 pve01 homelab-forum-snuffleupagus-health[499444]: STATUS=OK TS=2026-09-23T09:30:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 12:45:20 pve01 homelab-forum-snuffleupagus-health[532715]: STATUS=OK TS=2026-09-23T09:45:19Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:00:17 pve01 homelab-forum-snuffleupagus-health[583840]: STATUS=OK TS=2026-09-23T10:00:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:15:45 pve01 homelab-forum-snuffleupagus-health[636067]: STATUS=OK TS=2026-09-23T10:15:44Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:30:17 pve01 homelab-forum-snuffleupagus-health[687754]: STATUS=OK TS=2026-09-23T10:30:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 13:45:18 pve01 homelab-forum-snuffleupagus-health[740109]: STATUS=OK TS=2026-09-23T10:45:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:01:18 pve01 homelab-forum-snuffleupagus-health[793923]: STATUS=OK TS=2026-09-23T11:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:16:01 pve01 homelab-forum-snuffleupagus-health[828736]: STATUS=OK TS=2026-09-23T11:15:59Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:30:59 pve01 homelab-forum-snuffleupagus-health[878434]: STATUS=OK TS=2026-09-23T11:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 14:45:18 pve01 homelab-forum-snuffleupagus-health[927971]: STATUS=OK TS=2026-09-23T11:45:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:00:26 pve01 homelab-forum-snuffleupagus-health[977621]: STATUS=OK TS=2026-09-23T12:00:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:15:23 pve01 homelab-forum-snuffleupagus-health[1028167]: STATUS=OK TS=2026-09-23T12:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:30:59 pve01 homelab-forum-snuffleupagus-health[1074965]: STATUS=OK TS=2026-09-23T12:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 15:45:42 pve01 homelab-forum-snuffleupagus-health[1125071]: STATUS=OK TS=2026-09-23T12:45:40Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:01:34 pve01 homelab-forum-snuffleupagus-health[1175713]: STATUS=OK TS=2026-09-23T13:01:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:16:22 pve01 homelab-forum-snuffleupagus-health[1225835]: STATUS=OK TS=2026-09-23T13:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:30:59 pve01 homelab-forum-snuffleupagus-health[1274841]: STATUS=OK TS=2026-09-23T13:30:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 16:45:54 pve01 homelab-forum-snuffleupagus-health[1325565]: STATUS=OK TS=2026-09-23T13:45:52Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:00:59 pve01 homelab-forum-snuffleupagus-health[1360767]: STATUS=OK TS=2026-09-23T14:00:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:16:23 pve01 homelab-forum-snuffleupagus-health[1414098]: STATUS=OK TS=2026-09-23T14:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:30:52 pve01 homelab-forum-snuffleupagus-health[1466888]: STATUS=OK TS=2026-09-23T14:30:50Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 17:46:53 pve01 homelab-forum-snuffleupagus-health[1519524]: STATUS=OK TS=2026-09-23T14:46:46Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:01:32 pve01 homelab-forum-snuffleupagus-health[1568613]: STATUS=OK TS=2026-09-23T15:01:30Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:16:05 pve01 homelab-forum-snuffleupagus-health[1618235]: STATUS=OK TS=2026-09-23T15:16:03Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:30:26 pve01 homelab-forum-snuffleupagus-health[1648980]: STATUS=OK TS=2026-09-23T15:30:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 18:45:59 pve01 homelab-forum-snuffleupagus-health[1702860]: STATUS=OK TS=2026-09-23T15:45:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:00:16 pve01 homelab-forum-snuffleupagus-health[1748144]: STATUS=OK TS=2026-09-23T16:00:10Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:16:22 pve01 homelab-forum-snuffleupagus-health[1808309]: STATUS=OK TS=2026-09-23T16:16:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:30:44 pve01 homelab-forum-snuffleupagus-health[1849936]: STATUS=OK TS=2026-09-23T16:30:42Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 19:45:50 pve01 homelab-forum-snuffleupagus-health[1901311]: STATUS=OK TS=2026-09-23T16:45:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:00:31 pve01 homelab-forum-snuffleupagus-health[1952524]: STATUS=OK TS=2026-09-23T17:00:29Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:15:13 pve01 homelab-forum-snuffleupagus-health[1997547]: STATUS=OK TS=2026-09-23T17:15:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:31:17 pve01 homelab-forum-snuffleupagus-health[2057552]: STATUS=OK TS=2026-09-23T17:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 20:45:38 pve01 homelab-forum-snuffleupagus-health[2102176]: STATUS=OK TS=2026-09-23T17:45:37Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:01:11 pve01 homelab-forum-snuffleupagus-health[2150631]: STATUS=OK TS=2026-09-23T18:01:09Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:16:18 pve01 homelab-forum-snuffleupagus-health[2202826]: STATUS=OK TS=2026-09-23T18:16:12Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:31:38 pve01 homelab-forum-snuffleupagus-health[2255333]: STATUS=OK TS=2026-09-23T18:31:32Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 21:46:08 pve01 homelab-forum-snuffleupagus-health[2306750]: STATUS=OK TS=2026-09-23T18:46:07Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:00:54 pve01 homelab-forum-snuffleupagus-health[2350713]: STATUS=OK TS=2026-09-23T19:00:48Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:15:43 pve01 homelab-forum-snuffleupagus-health[2399480]: STATUS=OK TS=2026-09-23T19:15:42Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:30:26 pve01 homelab-forum-snuffleupagus-health[2449737]: STATUS=OK TS=2026-09-23T19:30:24Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 22:46:18 pve01 homelab-forum-snuffleupagus-health[2502775]: STATUS=OK TS=2026-09-23T19:46:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:01:17 pve01 homelab-forum-snuffleupagus-health[2551419]: STATUS=OK TS=2026-09-23T20:01:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:15:59 pve01 homelab-forum-snuffleupagus-health[2593718]: STATUS=OK TS=2026-09-23T20:15:58Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:31:17 pve01 homelab-forum-snuffleupagus-health[2633602]: STATUS=OK TS=2026-09-23T20:31:16Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
Sep 23 23:45:24 pve01 homelab-forum-snuffleupagus-health[2684007]: STATUS=OK TS=2026-09-23T20:45:17Z TYPE=forum-snuffleupagus BAD=0 WARN=0 MODE=low-noise-simulation STRICT=NO MODULE=loaded FORUMS=5 PUBLIC_SMOKE_BAD=0 ACTUAL_READONLY_EXEC_RULE=NO REPORT=/var/lib/homelab-forum-snuffleupagus/latest.md
=== PVE POSTFIX/ALIASES ===
POSTFIX_FILE_META=/etc/aliases mode=644 owner=root group=root size=73
POSTFIX_FILE_SHA256=8e81ec4a775c03d4a6d59c6fddb47bb84705a631fd1b142dffd536b824125bde PATH=/etc/aliases
POSTFIX_FILE_MISSING=/etc/aliases.db
POSTFIX_FILE_META=/etc/postfix/main.cf mode=644 owner=root group=root size=572
POSTFIX_FILE_SHA256=d1050a0c110c2d6c59f595abcf791ee8daa5399dee97fc504fafbcf08adc8f7f PATH=/etc/postfix/main.cf
POSTCONF_inet_interfaces=loopback-only
POSTCONF_relayhost=
POSTCONF_mydestination=$myhostname, localhost.$mydomain, localhost
POSTCONF_mynetworks=<IP>/8
POSTCONF_alias_maps=hash:/etc/aliases
POSTCONF_alias_database=hash:/etc/aliases
POSTCONF_transport_maps=
POSTFIX_QUEUE_COUNT=115
POSTFIX_ALIASES_DB_ERRORS_24H=2806
=== ALERTING JOURNAL 24H ===
Sep 23 07:45:12 pve01 systemd[1]: Starting homelab-alerting-health.service - Homelab alerting health check...
Sep 23 07:45:12 pve01 homelab-alerting-health[3763486]: STATUS=OK TS=2026-09-23T04:45:12Z TYPE=alerting BAD=0 endpoint_local=ok endpoint_from_monitoring=ok kuma_container=ok gotify_container=ok kuma_monitor=ok gotify_link=ok kuma_state=running_healthy gotify_state=running monitor=95|1|keyword|http://<IP>:9101/overall.txt|STATUS=OK|Gotify|1
Sep 23 07:45:12 pve01 systemd[1]: homelab-alerting-health.service: Deactivated successfully.
Sep 23 07:45:12 pve01 systemd[1]: Finished homelab-alerting-health.service - Homelab alerting health check.
=== VM INVENTORY HINT ===
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
VM160_STATE=UNKNOWN
FAILURE=VM160_NOT_RUNNING
DECISION=HOLD_DEEP_AUDIT_INCOMPLETE
NEXT_GATE=FIX_DIAGNOSTIC_BLOCKER_ONLY
TASK_RESULT=FAIL_FORUM_DEEP_AUDIT
HOMELAB_RESULT_CONTRACT={"version":1,"command_id":"FORUM-260923-RCA-DEEP-AUDIT8D42R1","status":"FAIL","changes_made":false,"rollback_started":false,"rollback_restored":null}
OUTPUT_END
CHAT_OUTPUT_END