What to detect during a social engineering test

The second question a social engineering test answers

A social engineering test checks whether people can be talked into something. The question that gets less attention: when they were, did anything on your side notice? This article is about that second question. It covers five channels: email, identity, endpoint, telephone and physical access, and names the record each one should leave.

Scope comes before telemetry. The article on how a social engineering test is scoped explains how pretext, channels and rules of engagement are agreed. Whatever channels that agreement lists, each one needs a named detection owner. A call to the help desk that nobody logs cannot be detected by anyone, however good the SOC is. The social engineering service covers phishing, pretext and physical work, and the detection questions below apply to all three.

Detection is judged against the test's own timeline, not against alert volume. The useful measure is the gap between when the tester acted and when someone on your side recorded it. NIST SP 800-115 lists social engineering among the testing techniques and treats the output as evidence for the organisation to act on. That is the right reading here: the tester supplies timestamps, and your logs either match them or they do not.

Email and click telemetry: the first record on the timeline

The mail gateway sees the message first. Its delivery log should hold the sender, the SPF, DKIM and DMARC results, and a verdict of delivered, quarantined or rejected. A pretext message sent from a lookalike domain should leave at least a delivery record with the authentication result, even when nothing blocks it. If the gateway scores newly registered domains, that score belongs in the same record.

The click is the second record. A URL rewriting or safe-link service logs who followed a link, when, from which client and from which address. Without it, the only evidence of a click is the tester's own report, and you cannot check your side against it. MITRE ATT&CK groups the delivery methods under phishing, technique T1566, which is a useful vocabulary when you map gaps back to detection rules.

The user report is also a detection control, and the only one that depends on a person. A report button that opens a ticket gives you a timestamp for the human response. Measure the time from delivery to first report, and the time from report to analyst action. Set your own target for both before the test, because no external figure describes your staff or your mail flow.

Identity signals: the record that outlasts the email

Email can be purged. The sign-in record stays. When a lure ends in a credential or a token, the identity provider log is where phishing turns into access, and it holds the application, the client, the authentication protocol, the source address and the multi-factor result. A lure that needs no fake login page still leaves a real sign-in. The article on OAuth device code abuse shows how that chain runs, and the sequence below marks where the defender gets a record.

Platform note: the query below uses Microsoft Sentinel table and field names. Splunk, Elastic and Chronicle users will need to map the same ideas onto their own schema, which means protocol, user, application, source address and result. The logic carries over, the field names do not.

Beyond the protocol, watch for a new consent grant to an application nobody approved, a sign-in from an address the user has never used, and token use from a different address than the sign-in. The strongest rule correlates a link click with a sign-in by the same user within a short window. Alone, each event is ordinary. Together they are the shape of the attack.

SigninLogs
| where TimeGenerated > ago(1d)
| where AuthenticationProtocol == "deviceCode"
| project TimeGenerated, UserPrincipalName, AppDisplayName,
          IPAddress, ResultType
| order by TimeGenerated desc

Endpoint, telephone and physical channels: the records people forget

When the pretext carries a payload, the endpoint should speak. The alert classes worth confirming are an Office application starting a script host, a script host launched from a downloaded archive or disk image, an executable run from the Downloads folder with a mark-of-the-web zone identifier, a scheduled task or run key created soon after a download, and a browser starting an unsigned binary. If your EDR raises none of these for a benign test file, the gap is in the product configuration, not in the staff.

Voice phishing leaves nothing on the endpoint. Its records sit in the telephone system and the help desk. Look for call detail records from the PBX or softphone platform, inbound external calls that present an internal extension, and tickets for password or multi-factor resets. Detection here is largely procedural. Did the agent follow the callback procedure, did the ticket record the verification outcome, and did anyone flag the request as unusual?

Physical access has its own log. Badge reader events can show a door held open, a badge denied and then used, or use outside normal hours. The visitor register and time-matched camera footage complete the picture. Tailgating is the awkward case, because a reader records one badge for two people. That is a finding about the control, and it should be written as one. When the test moves on from persuading a person to working from a foothold, the telemetry questions change, and the assumed-breach exercise is built for that stage.

Scoring detection: channel, signal and owner

A scoring table turns the above into something a CISO can sign. For each channel in scope, it names the signal that should fire, the system that records it and the team that owns the answer. A row with no owner is already a finding before the test begins.

The timeline below is an illustrative record, constructed for this article and not drawn from any client. It shows how a test is read afterwards. The user report at 00:41 is the first detection. The device code sign-in nine minutes after the click raised no alert, so the real finding is a missing identity rule rather than the click itself.

Turning a missed row into a rule, and then proving the rule fires, is the work of a purple team session. Re-run the same pretext after the fix and compare the timestamps. A rule nobody has tested against the original action is a hope, not a control.

T+00:00  Pretext email sent      Gateway: delivered, no alert
T+00:07  Link clicked            Click log: user, 203.0.113.20, no alert
T+00:09  Device code sign-in     IdP: protocol deviceCode, new address, no alert
T+00:41  Employee reports mail   Report ticket opened  <- first detection
T+01:55  Analyst revokes tokens  Containment recorded

Detection evidence for social engineering tests in Abu Dhabi and Dubai

In the UAE, four regimes name security testing: CBUAE for banks and other licensed financial institutions, DESC for Dubai government entities and their suppliers, ADHICS for Abu Dhabi healthcare through the Department of Health, and the federal UAE Information Assurance Regulation published by NESA. The article on the UAE rules for social engineering testing sets out what each one says. Read the sources themselves: CBUAE, DESC, the Department of Health for ADHICS, and the UAE Information Assurance Regulation.

The practical point is the evidence a reviewer reads. A Dubai supplier answering to DESC, or an Abu Dhabi hospital working to ADHICS, is better served by a dated record linking each tester action to a log entry than by a click rate. A bank asked by CBUAE what it does about phishing can show the same timeline. Where a channel is in scope, such as the help desk or the front door, its log should be named in the report with an owner, so the finding is a gap in a control and not a vague observation about awareness.

Frequently asked questions