BXUP Track progress & test yourself — free
IT Support / Service Desk / Sysadmin · Playbook · Core · ~4 min read
IT SUPPORT / SERVICE DESK / SYSADMIN · PLAYBOOK · CORE

Analyze Logs with AI, Verify in the Raw

An LLM reads four hours of event logs in seconds and hands you anomaly candidates and a reconstructed timeline. The word doing the heavy lifting is "candidates": every finding gets confirmed against the raw logs before it enters your diagnosis — and logs with real names in them go to a local model, not the cloud.

Tier: Core · Workflow area: Troubleshooting & diagnosis · ~30 minutes

Log analysis is the AI use case with the best effort-to-value ratio in troubleshooting and the highest hallucination surface: models invent event IDs, shift timestamps, and manufacture tidy narratives from noisy data. The recipe below gets the speed while the verify-in-the-raw step keeps the honesty.

When to use

  • Post-incident "what happened here"a machine misbehaved, rebooted, or slowed at a known time and you need the story from System/Application logs or syslog.
  • Needle-in-haystack correlationthousands of lines across multiple sources where the signal is a sequence (service crash → auth failures → disk warning) rather than a single entry.
  • Unfamiliar log formatsan appliance or app log you rarely read; the model knows the format even when you do not.

When NOT to use

  • Live security investigationssuspected-compromise logs follow your IR process and its approved tooling; pasting them into general AI tools can tip evidence handling and confidentiality rules.
  • When a grep answers it"when did the service last restart" is one Select-String away; save the model for questions about patterns, not lookups.
  • Whole-log dumpspasting 50k unfiltered lines produces vague summaries. Extraction and filtering (step 1) is where your expertise enters; do not skip it.

The recipe

  1. Extract a focused window. Time-bound, level-filtered, from the sources that matter. Windows:

    Get-WinEvent -FilterHashtable @{
      LogName   = 'System','Application'
      Level     = 1,2,3                    # critical, error, warning
      StartTime = (Get-Date).AddHours(-4)
    } | Select-Object TimeCreated, LogName, Id, ProviderName, LevelDisplayName, Message |
      Export-Csv .\events-4h.csv -NoTypeInformation
    

    Linux:

    journalctl --since "-4 hours" -p warning -o short-iso --no-pager > syslog-4h.txt
    

    Aim for hundreds to low thousands of lines. If the incident time is known, cut a ±30-minute window around it and a second, wider warning-level window for context.

  2. Decide the privacy route before anything is pasted. Scan the extract: usernames, hostnames, IPs, email addresses, file paths with names in them? If yes — and for endpoint logs the answer is almost always yes — either sanitize with substitution (keep it consistent: the same user must map to the same placeholder or correlation dies) or run the analysis locally:

    ollama pull llama3.1:8b
    ollama run llama3.1:8b "$(cat analysis-prompt.txt events-4h.csv)"
    

    Local models are noticeably weaker at this than frontier models — compensate with tighter extracts and expect an extra verification pass, not by giving up and pasting raw PII to the cloud.

  3. Prompt for candidates and a timeline — with citation of line evidence.

    You are helping analyze Windows event logs from a machine that froze and
    rebooted around 11:20 local time. The CSV below covers 08:00–12:00.
    
    Produce:
    1. TIMELINE: reconstructed sequence of notable events with timestamps —
       each entry must quote the source line's event ID and provider verbatim.
    2. ANOMALY CANDIDATES: up to 5 patterns that deviate from the log's own
       baseline (new error types, frequency spikes, silent gaps, ordering
       oddities). For each: the evidence lines, and what would confirm or
       refute it as relevant to the freeze.
    3. WHAT'S MISSING: logs or time ranges you would want next.
    
    Rules: cite only lines present in the data — never invent an event ID or
    timestamp. Mark inferences as inferences. A gap in logging is a finding,
    not a place to interpolate.
    
    Log data:
    [paste events-4h.csv]
    
  4. Verify every finding in the raw. For each timeline entry and anomaly candidate, find the actual lines:

    # Confirm the claimed disk warnings actually exist, with the claimed timing
    Get-WinEvent -FilterHashtable @{LogName='System'; ProviderName='disk'; StartTime=(Get-Date).AddHours(-4)} |
      Format-Table TimeCreated, Id, Message -Wrap
    
    grep -n "I/O error" syslog-4h.txt | head -20
    

    You are checking three failure modes: invented entries (an event ID that appears nowhere), shifted times (real events, wrong order — fatal for causal reasoning), and overfitted stories (a clean narrative stitched over unrelated noise). A verified good output looks like:

    TIMELINE (verified):
    10:41  disk / Event 153 — "IO operation retried" (first occurrence in window) ✔ raw line 2211
    10:52–11:14  Event 153 repeats 14x, escalating frequency ✔ count matches: 14
    11:17  Application hangs: Event 1002, OfficeC2RClient ✔
    11:20  Kernel-Power 41 — unclean reboot ✔
    CANDIDATE CONFIRMED: storage degradation preceding freeze.
    Next check: SMART status — not in event logs (correctly flagged as missing).
    
  5. Let the confirmed picture drive the next extraction. Storage story confirmed? Pull SMART data and the storage controller logs. Auth anomalies? Pull the Security log for the same window. Iterating extract → analyze → verify beats one giant paste every time, and each loop's verified findings go into the ticket as you go.

Make it yours

  • SIEM shops (Sentinel, Splunk)do extraction with KQL/SPL and use the assistant on the result set; you keep the SIEM's fidelity and the model's pattern commentary. Copilot-style query generation is pb-13's territory.
  • macOS fleetlog show --last 4h --predicate 'messageType >= 16' --style syslog > out.txt; the unified log is verbose, so halve the window before pasting.
  • Recurring analysesif you run the same extraction weekly, freeze the prompt and extraction script into a repo file so results are comparable across weeks; drift in the prompt silently changes what "anomaly" means.

Quality check

  • No unverified finding leaves the ticketanything you could not locate in the raw logs is deleted or explicitly marked "unconfirmed model suggestion", never silently retained.
  • Timestamps re-checked in orderfor causal chains, manually confirm the ordering of the 3–4 pivotal events; timezone and DST offsets between log sources are the classic silent killer.
  • Gaps treated as dataa 20-minute hole in logging around the incident is often the finding; confirm whether the gap is real (host down) or an artifact of your extraction filter.
  • Sandbox the follow-upsanything the analysis makes you want to change (driver update, service restart on a server) goes through normal change and test discipline; the log story, however convincing, is still one machine's testimony.
Warning

Endpoint and auth logs are personal-data-dense — real names, sign-in times, sometimes URLs that reveal behavior; default to the local-model route for anything user-attributable, and sanitize consistently when you must use a hosted model.

Key Takeaway:

AI turns hours of log reading into minutes of log checking — the speed is real, but only the findings you have re-located in the raw logs are yours to act on; the rest is a well-told story.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in