Defender Spotlight Andy Jackman

Interviewer: Can you tell us about the most interesting incident that you responded to?

Andy: I worked on an incident that targeted a LATAM financial institution. This was a FIN13 threat actor-based attack using the Blue Agave Web Shell. We had the full command line history of every single thing the attacker did, from compromising the RSA Security Console to stealing the token seed file(s) and using them to spoof transactions in attempts to steal money from accounts

Interviewer: What are some recurring challenges you see when working on incidents?

Andy: The most significant issue is visibility, which inevitably impacts the entire incident lifecycle. Right out of the gate, scoping an incident is a routine issue. Regularly, the client doesn’t have the right logging enabled, logging retention for the right logs (you have 365 days of Azure Sign In logs and only 14 days of Azure Audit Logs, for example), or has simply shut down compromised systems, losing valuable memory data for analysis.

Most organizations can identify to some degree whether or not they’ve been compromised. However, when it comes down to it, determining what the attacker could accomplish and kicking the attacker out of the network with any degree of assurance is a very common challenge. We have had incidents where all we can do is, unfortunately, shrug. You’re a BYOD house, only a firewall with no logging enabled, no endpoint controls, no network monitoring, no centralized logging… Do better.

Interviewer: How do you communicate with stakeholders and affected parties throughout the incident response process?

Andy: This depends on the severity of the incident and whether or not it’s post-mortem. During an active major incident, there’s usually an always-on bridge meeting that we remain connected to to have open comms.

Then, we send out daily status reports at the end of each day to keep clients informed of findings and progress. Less severe incidents usually have a daily morning status sync-up followed by a daily status report.

Interviewer: What has not worked well when communicating during an incident?

Andy: When using a bridge for communications during an active incident, it’s critical that it be accompanied by a chat channel of some sort. You need to have a record of key items being discussed in the voice call. Everyone is generally running around with their hair on fire, so people are on-and-off the bridge, and likely won’t catch everything being said.

Also, we’ve had clients just dump us with pages and pages of unorganized notes from their investigation before we were called in, with little guidance on what is actually going on. Precise details of what has been identified and what needs to be done or investigated are key. As well as who is working what.

Interviewer: Are there common issues you see during the remediation phase?

Andy: The most common one is remediating too early before the attacker has been kicked out of the network. It’s a widespread practice that organizations immediately start imaging devices and rolling credentials when they’ve identified they’ve been breached. Still. In 2024. Stop doing this.

You’re tipping off to the attacker that you know they’re there. If you haven’t identified every compromised device before you show your hand, you’re just telling the attacker to embed themselves deeper into your network.

Interviewer: What advice would you offer other incident responders based on your experience?

Andy: There’s a difference between incident response at scale and forensics. You do not have time to run every detail to the ground on every box during an active incident. And you likely never will have this time. Your main goal is to scope the incident, develop IOCs, and further scope it as quickly as possible.

Interviewer: Do you find any specific tools, resources, or training programs particularly valuable?

Andy: I will stand by the Investigation Theory course by Applied Network Defense. Excellent course. As are their others: Practical Threat Hunting, CyberChef, Demystifying Regular Expressions.

For tools, the standard Eric Zimmerman tools, including Kape, are always great. Hayabusa / Chainsaw with Sigma rules are also great for EVTX analysis when you either don’t have a lead on a box you’re looking at or you’re just dead exhausted from working an incident for 16 hours and need a quick win for the time being.

Interviewer: How do you prioritize continuous improvement in incident response capabilities?

Andy: We generally have our post-engagement lessons learned sync-ups. Operations is the biggest issue, and we need to smooth that out and communicate better.

You can have all the technical knowledge in the world, but if you can’t implement that on the fly and communicate effectively, it doesn’t matter. Continued improvement of the operations regarding incident response is key to how we’re tasking and talking about things—the meta of incident response, so to speak

Interviewer: Can you share any insights or observations regarding the motives or tactics of the threat actors behind the ransomware attack, and how did this knowledge inform your response strategy?

Andy: Ransomware typically moves fast. Shorter dwell times on average. This usually results in it being quite a bit noisier than an espionage-based attack, given that correct logging is enabled and centrally ingested. In my experience, generally, it doesn’t take the time to blend in with normal operations, either. This does happen, but on average, the TTPs look more like a smash-and-grab than a stealth operation. After a few major incidents, you start to get a spidey sense of the attacker’s goals.

Interviewer: What’s your favorite story in responding to a ransomware attack? Most challenging/frustrating story?

Andy: Responding to ransomware attacks is honestly usually boring. We usually get called in after the ransomware has detonated and the environment has been encrypted, breaking logging and most things of forensic value.

In rare instances, we have been called in while the attacker is still staging ransomware. We create hosts in vsphere, group servers, and make adjustments for a quicker, more rapid deployment. This is honestly one of the most stressful situations you can be in. Now, you have to make a quick call because, at this stage, you’re down to a few hours before detonation at best.

Interviewer: How often do you see endpoints compromised in ransomware attacks that are protected by EDR, Anti-virus, etc.?

Andy: Like, 100% of the time.

Interviewer: What would you like to see infosec vendors do?

Andy: Focus on UI/UX. If a vendor could knock it out of the park with UI/UX, that’d be amazing. So many times, I find myself wishing I just had raw data and a bash terminal rather than trying to navigate the clunky UI that’s been created to look shiny to CISOs but completely neglects actual useability for responders.

In the ever-changing realm of cybersecurity, professionals like Andy, who specialize in incident response, serve as frontline defenders against threats and work diligently to minimize the fallout from security breaches. Leveraging their extensive experience, specialized skills, and unwavering commitment, they adeptly navigate the intricate landscape of incident response, bolstering resilience and safeguarding organizations’ security amidst evolving threats. We thank Andy for their invaluable contribution to advancing the collective knowledge within the Incident Response community