Normalize key lookups and blank missing template values

2026-02-26 14:08:14 -05:00
1 parent 0f890d4c23
commit 80fd2c9b2f
@@ -11,6 +11,7 @@ This file defines agent behavior, contribution standards, and validation rules f
 - `nessus_remediation_tracker.xlsx` is a generated artifact and must be regenerated when workbook logic changes.
 - Keep workbook logic compatible with both Excel and Google Sheets formulas.
 - Keep dependency documentation in `README.md` accurate when Python module usage changes.
+- Policy: any `Template` cell auto-populated from `Key` lookups must resolve to blank when the source `Remediations` value is missing/empty.

 ## File Standards
 - Use Unix `LF` line endings for all text files.
@@ -1,5 +1,15 @@
 # Changelog

+## 2026-02-26
+- Simplified `Remediations` status options to `exception`, `exception - permanent`, in-progress variants, and `resolved`.
+- Added `Excepted until (date)` to `Remediations` and `Template`, with overdue-date highlighting (red) when the date has passed.
+- Added `First seen` to `Remediations` and `Unresolved for` to `Template` (`days` since first seen when first seen is before today).
+- Fixed `Template` key lookup to match CVEs even when `Remediations` CVE cells include extra leading/trailing spaces.
+- Updated key-driven `Template` formulas to return blank for missing source values (including exception date) instead of displaying Excel zero-date artifacts.
+- Added repository policy in `AGENTS.md` requiring blank output for missing key-driven auto-populated values.
+- Updated `scripts/build_tracker_workbook.py` default seed rows for the new column layout and status set.
+- Updated `README.md` workbook column definitions and behavior notes for exception dates and unresolved metrics.
+
 ## 2026-02-24
 - Switched workbook generation to `openpyxl` in `scripts/build_tracker_workbook.py`.
 - Kept `Remediations` and `Template` structure and matching logic (`CVE` first, then `Plugin ID`).
@@ -23,9 +23,9 @@ python3 -m pip install --user openpyxl xlsxwriter
 ## What it creates

 - `Remediations` tab with columns:
-  - `CVE`, `Plugin ID`, `Severity`, `Name`, `Description`, `Links`, `Status`, `scan 1 notes`, `scan 2 notes`, `scan 3 notes`
+  - `CVE`, `Plugin ID`, `Severity`, `Name`, `Description`, `Links`, `Status`, `Excepted until (date)`, `First seen`, `scan 1 notes`, `scan 2 notes`, `scan 3 notes`
 - `Template` tab with columns:
-  - `Key`, `CVE`, `Plugin ID`, `Severity`, `Name`, `Hosts`, `Owner`, `status`, `scan 1 notes`, `scan 2 notes`, `scan 3 notes`
+  - `Key`, `CVE`, `Plugin ID`, `Severity`, `Name`, `Hosts`, `Owner`, `status`, `Excepted until (date)`, `Unresolved for`, `scan 1 notes`, `scan 2 notes`, `scan 3 notes`

 `Template` behavior:

@@ -39,6 +39,9 @@ python3 -m pip install --user openpyxl xlsxwriter
 - If `Key` is entered and no match is found, fields show `NEW ITEM`.
 - If matched, both `CVE` and `Plugin ID` are auto-populated in `Template`.
 - If matched, `Name` in `Template` is a hyperlink to the matching row in `Remediations`.
+- If matched, `Excepted until (date)` is auto-populated in `Template`.
+- If a matched source value is empty in `Remediations`, the corresponding auto-populated `Template` cell stays blank.
+- `Unresolved for` in `Template` shows elapsed days since `First seen` when `First seen` is before today.
 - `Hosts` and `Owner` are intended for manual entry.
 - `Template` is protected so only `Key`, `Hosts`, and `Owner` are editable (rows 2+).

@@ -46,13 +49,12 @@ python3 -m pip install --user openpyxl xlsxwriter

 - `Severity` has a dropdown: `info`, `low`, `medium`, `high`, `critical`.
 - `Status` has a dropdown:
-  - `exception - 90 days`
-  - `exception - 6 months`
+  - `exception`
  - `exception - permanent`
-  - `exception - nofix available`
  - `In progress - next updates`
  - `In progress - researching`
  - `resolved`
+- `Excepted until (date)` is highlighted red when the date is in the past.
 - Conditional colors are applied for severity/status values.

 ## Generate workbook
@@ -3,6 +3,7 @@

 from __future__ import annotations

+from datetime import date
 from pathlib import Path

 from openpyxl import Workbook
@@ -26,6 +27,8 @@ DEFAULT_REMEDIATIONS_ROWS = [
        "A vulnerability in SonicOS CFS (Content filtering service) returns a large 403 forbidden HTTP response message to the source address when users try to access prohibited resource this allows an attacker to cause HTTP Denial of Service (DoS) attack ",
        None,
        "resolved",
+        None,
+        date(2026, 1, 20),
        "test scan 1 cve-2022-22278",
        None,
        None,
@@ -37,7 +40,9 @@ DEFAULT_REMEDIATIONS_ROWS = [
        "RHEL 8 : grafana (RHSA-2026:3188)",
        "The net/url package does not set a limit on the number of query parameters in a query. While the maximum size of query parameters in URLs is generally limited by the maximum request header size, the net/http.Request.ParseForm method can parse large URL-encoded forms. Parsing a large form containing many unique query parameters can cause excessive memory consumption. ",
        "https://www.tenable.com/cve/CVE-2025-61726",
-        "exception - nofix available",
+        "exception",
+        date(2026, 3, 31),
+        date(2026, 2, 12),
        "test scan 1 cve-2025-61726",
        "test scan 2 cve-612726",
        None,
@@ -50,6 +55,8 @@ DEFAULT_REMEDIATIONS_ROWS = [
        "archive/zip uses a super-linear file name indexing algorithm that is invoked the first time a file in an archive is opened. This can lead to a denial of service when consuming a maliciously constructed ZIP archive. ",
        "https://www.tenable.com/cve/CVE-2025-61728",
        "resolved",
+        None,
+        date(2026, 2, 5),
        "test scan 1 cve-2025-61728",
        "test scan 2 cve-61728",
        "test scan 3 cve-2025-61728",
@@ -62,6 +69,8 @@ DEFAULT_REMEDIATIONS_ROWS = [
        "During session resumption in crypto/tls, if the underlying Config has its ClientCAs or RootCAs fields mutated between the initial handshake and the resumed handshake, the resumed handshake may succeed when it should have failed. This may happen when a user calls Config.Clone and mutates the returned Config, or uses Config.GetConfigForClient. This can cause a client to resume a session with a server that it would not have resumed with during the initial handshake, or cause a server to resume a session with a client that it would not have resumed with during the initial handshake. ",
        "https://www.tenable.com/cve/CVE-2025-68121",
        "exception",
+        date(2026, 2, 10),
+        date(2026, 1, 10),
        "test scan 1 cve-2025-68121",
        None,
        "test scan 3 cve-2025-68121",
@@ -73,7 +82,9 @@ DEFAULT_REMEDIATIONS_ROWS = [
        "Python Unsupported Version Detection",
        "The remote host contains one or more unsupported versions of Python.",
        "https://www.tenable.com/plugins/nessus/148367",
-        "exception",
+        "exception - permanent",
+        None,
+        date(2025, 12, 12),
        "only applies to Python from the Python foundation, not the version maintained on AlmaLinux",
        None,
        None,
@@ -93,6 +104,12 @@ def set_wrap_text(ws, columns: list[str], start_row: int, end_row: int) -> None:
            ws[f"{col}{row}"].alignment = wrap_alignment


+def set_date_format(ws, columns: list[str], start_row: int, end_row: int) -> None:
+    for col in columns:
+        for row in range(start_row, end_row + 1):
+            ws[f"{col}{row}"].number_format = "yyyy-mm-dd"
+
+
 def add_severity_rules(ws, cell_range: str) -> None:
    blue = PatternFill(start_color="FFDBEAFE", end_color="FFDBEAFE", fill_type="solid")
    yellow = PatternFill(start_color="FFFEF3C7", end_color="FFFEF3C7", fill_type="solid")
@@ -136,6 +153,14 @@ def add_status_rules(ws, cell_range: str) -> None:
    )


+def add_expired_date_rule(ws, cell_range: str, date_col: str) -> None:
+    red = PatternFill(start_color="FFFECACA", end_color="FFFECACA", fill_type="solid")
+    ws.conditional_formatting.add(
+        cell_range,
+        FormulaRule(formula=[f'AND(ISNUMBER(${date_col}2),${date_col}2<TODAY())'], fill=red),
+    )
+
+
 def add_remediations_validation_and_formatting(ws) -> None:
    severity_validation = DataValidation(
        type="list",
@@ -147,7 +172,7 @@ def add_remediations_validation_and_formatting(ws) -> None:

    status_validation = DataValidation(
        type="list",
-        formula1='"exception - 90 days,exception - 6 months,exception - permanent,exception - nofix available,In progress - next updates,In progress - researching,resolved"',
+        formula1='"exception,exception - permanent,In progress - next updates,In progress - researching,resolved"',
        allow_blank=True,
    )
    ws.add_data_validation(status_validation)
@@ -155,6 +180,7 @@ def add_remediations_validation_and_formatting(ws) -> None:

    add_severity_rules(ws, f"C2:C{MAX_XLSX_ROWS}")
    add_status_rules(ws, f"G2:G{MAX_XLSX_ROWS}")
+    add_expired_date_rule(ws, f"H2:H{MAX_XLSX_ROWS}", "H")


 def add_template_formatting(ws, max_rows: int) -> None:
@@ -193,6 +219,7 @@ def add_template_formatting(ws, max_rows: int) -> None:
        f"H2:H{max_rows}",
        FormulaRule(formula=['LOWER(H2)="resolved"'], fill=green),
    )
+    add_expired_date_rule(ws, f"I2:I{max_rows}", "I")

 def build_remediations_sheet(wb: Workbook) -> None:
    ws = wb.active
@@ -206,6 +233,8 @@ def build_remediations_sheet(wb: Workbook) -> None:
        "Description",
        "Links",
        "Status",
+        "Excepted until (date)",
+        "First seen",
        "scan 1 notes",
        "scan 2 notes",
        "scan 3 notes",
@@ -219,7 +248,7 @@ def build_remediations_sheet(wb: Workbook) -> None:
        cell.font = Font(bold=True)

    ws.freeze_panes = "A2"
-    ws.auto_filter.ref = "A1:J1"
+    ws.auto_filter.ref = "A1:L1"

    set_column_widths(
        ws,
@@ -231,12 +260,15 @@ def build_remediations_sheet(wb: Workbook) -> None:
            "E": 55,
            "F": 40,
            "G": 16,
-            "H": 35,
-            "I": 35,
+            "H": 20,
+            "I": 14,
            "J": 35,
+            "K": 35,
+            "L": 35,
        },
    )
-    set_wrap_text(ws, ["D", "E", "H", "I", "J"], 2, FORMAT_MAX_ROWS)
+    set_wrap_text(ws, ["D", "E", "J", "K", "L"], 2, FORMAT_MAX_ROWS)
+    set_date_format(ws, ["H", "I"], 2, FORMAT_MAX_ROWS)
    add_remediations_validation_and_formatting(ws)


@@ -252,6 +284,8 @@ def build_template_sheet(wb: Workbook, max_rows: int) -> None:
        "Hosts",
        "Owner",
        "status",
+        "Excepted until (date)",
+        "Unresolved for",
        "scan 1 notes",
        "scan 2 notes",
        "scan 3 notes",
@@ -263,7 +297,7 @@ def build_template_sheet(wb: Workbook, max_rows: int) -> None:
        cell.font = Font(bold=True)

    ws.freeze_panes = "A2"
-    ws.auto_filter.ref = "A1:K1"
+    ws.auto_filter.ref = "A1:M1"

    set_column_widths(
        ws,
@@ -276,16 +310,19 @@ def build_template_sheet(wb: Workbook, max_rows: int) -> None:
            "F": 22,
            "G": 20,
            "H": 16,
-            "I": 28,
-            "J": 28,
+            "I": 20,
+            "J": 16,
            "K": 28,
-            "L": 12,
+            "L": 28,
+            "M": 28,
+            "N": 12,
        },
    )
-    ws.column_dimensions["L"].hidden = True
+    ws.column_dimensions["N"].hidden = True
    ws.protection.sheet = True
    ws.protection.autoFilter = True
-    set_wrap_text(ws, ["E", "I", "J", "K"], 2, max_rows)
+    set_wrap_text(ws, ["E", "K", "L", "M"], 2, max_rows)
+    set_date_format(ws, ["I"], 2, max_rows)

    for row in range(2, max_rows + 1):
        ws[f"A{row}"].protection = Protection(locked=False)
@@ -297,41 +334,55 @@ def build_template_sheet(wb: Workbook, max_rows: int) -> None:
            f'IF(LEFT(UPPER(A{row}),4)="PID:",MID(A{row},5,999),A{row}))'
        )
        lookup_formula = (
-            f'=IF(A{row}="","",IFERROR(MATCH({normalized_key},Remediations!$A:$A,0),'
-            f'IFERROR(MATCH({normalized_key},Remediations!$B:$B,0),"")))'
+            f'=IF(A{row}="","",IFERROR(MATCH(1,INDEX((TRIM(Remediations!$A:$A)=TRIM({normalized_key}))*1,0),0),'
+            f'IFERROR(MATCH(TRIM({normalized_key}),Remediations!$B:$B,0),"")))'
        )

-        ws[f"L{row}"] = lookup_formula
+        ws[f"N{row}"] = lookup_formula
        ws[f"B{row}"] = (
-            f'=IF(OR(A{row}="",$L{row}=""),"",'
-            f'IF(OR(INDEX(Remediations!$A:$A,$L{row})="",INDEX(Remediations!$A:$A,$L{row})=0),'
-            f'"n/a",INDEX(Remediations!$A:$A,$L{row})))'
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(OR(INDEX(Remediations!$A:$A,$N{row})="",INDEX(Remediations!$A:$A,$N{row})=0),'
+            f'"",INDEX(Remediations!$A:$A,$N{row})))'
+        )
+        ws[f"C{row}"] = (
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(OR(INDEX(Remediations!$B:$B,$N{row})="",INDEX(Remediations!$B:$B,$N{row})=0),"",'
+            f'INDEX(Remediations!$B:$B,$N{row})))'
        )
-        ws[f"C{row}"] = f'=IF(OR(A{row}="",$L{row}=""),"",INDEX(Remediations!$B:$B,$L{row}))'
        ws[f"D{row}"] = (
            f'=IF(A{row}="","",'
-            f'IF($L{row}="","NEW ITEM",INDEX(Remediations!$C:$C,$L{row})))'
+            f'IF($N{row}="","NEW ITEM",IF(INDEX(Remediations!$C:$C,$N{row})="","",INDEX(Remediations!$C:$C,$N{row}))))'
        )
        ws[f"E{row}"] = (
            f'=IF(A{row}="","",'
-            f'IF($L{row}="","NEW ITEM",HYPERLINK("#\'Remediations\'!D"&$L{row},'
-            f'INDEX(Remediations!$D:$D,$L{row}))))'
+            f'IF($N{row}="","NEW ITEM",HYPERLINK("#\'Remediations\'!D"&$N{row},'
+            f'IF(INDEX(Remediations!$D:$D,$N{row})="","",INDEX(Remediations!$D:$D,$N{row})))))'
        )
        ws[f"H{row}"] = (
            f'=IF(A{row}="","",'
-            f'IF($L{row}="","NEW ITEM",INDEX(Remediations!$G:$G,$L{row})))'
+            f'IF($N{row}="","NEW ITEM",IF(INDEX(Remediations!$G:$G,$N{row})="","",INDEX(Remediations!$G:$G,$N{row}))))'
        )
        ws[f"I{row}"] = (
-            f'=IF(OR(A{row}="",$L{row}=""),"",'
-            f'IF(INDEX(Remediations!$H:$H,$L{row})="","",INDEX(Remediations!$H:$H,$L{row})))'
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(OR(INDEX(Remediations!$H:$H,$N{row})="",INDEX(Remediations!$H:$H,$N{row})=0),"",'
+            f'INDEX(Remediations!$H:$H,$N{row})))'
        )
        ws[f"J{row}"] = (
-            f'=IF(OR(A{row}="",$L{row}=""),"",'
-            f'IF(INDEX(Remediations!$I:$I,$L{row})="","",INDEX(Remediations!$I:$I,$L{row})))'
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(OR(INDEX(Remediations!$I:$I,$N{row})="",NOT(ISNUMBER(INDEX(Remediations!$I:$I,$N{row})))),"",'
+            f'IF(TODAY()-INDEX(Remediations!$I:$I,$N{row})<=0,"",TODAY()-INDEX(Remediations!$I:$I,$N{row})&" days"))'
        )
        ws[f"K{row}"] = (
-            f'=IF(OR(A{row}="",$L{row}=""),"",'
-            f'IF(INDEX(Remediations!$J:$J,$L{row})="","",INDEX(Remediations!$J:$J,$L{row})))'
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(INDEX(Remediations!$J:$J,$N{row})="","",INDEX(Remediations!$J:$J,$N{row})))'
+        )
+        ws[f"L{row}"] = (
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(INDEX(Remediations!$K:$K,$N{row})="","",INDEX(Remediations!$K:$K,$N{row})))'
+        )
+        ws[f"M{row}"] = (
+            f'=IF(OR(A{row}="",$N{row}=""),"",'
+            f'IF(INDEX(Remediations!$L:$L,$N{row})="","",INDEX(Remediations!$L:$L,$N{row})))'
        )

    add_template_formatting(ws, max_rows)