10 Key Questions to Ask Before Your First Phishing Test

Organizations increasingly recognize that cybersecurity awareness training alone is not enough. Employees need opportunities to practice identifying phishing attacks in realistic situations. A phishing test provides a safe and controlled way to measure employee awareness, identify opportunities for improvement, and reinforce secure behaviors.

Phishing tests should be used to measure improvement, provide just-in-time education, and reduce organizational risk over time.

However, launching a phishing test or broader phishing simulation program without proper planning can create confusion, damage trust, and produce misleading results. Before sending the first simulated phishing email to staff, organizations should consider several important questions.

What Is a Phishing Test?

A phishing test is a simulated phishing attack designed to evaluate how employees respond to potentially malicious messages. Organizations use phishing tests to measure awareness, identify risky behaviors, improve reporting, and provide targeted security awareness training.

Unlike a one-time exercise, recurring phishing tests can help organizations measure changes in employee behavior over time.

Here are 10 key questions to ask before implementing your first phishing test:

1. Why are you conducting a phishing test?

It is important to understand what the organization hopes to achieve through testing. Are you trying to:

• Establish a baseline of employee awareness?
• Measure the effectiveness of existing training?
• Meet compliance or cyber insurance requirements?
• Identify high-risk departments or individuals?
• Improve reporting of suspicious emails?
• Reduce successful phishing attacks?

The answer to this question will influence various aspects of the program, from the scenarios selected to the metrics reported. When planning your first phishing test, we recommend you start with a baseline simulation for all employees to understand where users stand today and then track and measure improvement over time.

2. What types of attacks should your phishing test include?

Attackers have evolved beyond just email phishing. When possible, your phishing tests should cover multiple attack vectors to reflect current threats, including:

• Voice phishing (vishing): Vishing involves attackers impersonating trusted entities over the phone and tricking employees into sharing sensitive information.
• SMS phishing (smishing): Sending fraudulent text messages to trick users into clicking malicious links.
• QR code phishing (quishing): Attackers use malicious QR codes to direct victims to malicious websites. Test employees to see if they understand the potential risks of scanning a new QR code without verifying its authenticity.
• MFA phishing: An MFA phishing simulation tests if employees will accept or ignore fake authentication prompts. Due to MFA fatigue, employees will often quickly acknowledge a prompt on their phone without even stopping to think if they initiated the request.
• Spear phishing: Personalized phishing campaigns are often aimed at executives or higher-risk employees with access to sensitive data.
• Business email compromise (BEC): Criminals impersonate an important or trusted person (i.e., the CEO or a common vendor) to reallocate funds to another account or gain access to sensitive data.

3. Are we building a culture of learning or a culture of fear?

One of the biggest mistakes organizations make is treating phishing simulations like a test employees can fail. If employees feel tricked, embarrassed, or punished, they may be defensive and become less engaged with security initiatives over time.

Before launching phishing tests, consider:

• How will the program be communicated?
• Will employees understand the purpose?
• What happens when someone clicks?
• Will immediate coaching be provided?
• Will additional training be required?
• How will managers discuss and utilize results?

A successful phishing program focuses on awareness rather than blame. Phishing simulations should be positioned as educational exercises designed to provide positive reinforcement and opportunities for just-in-time learning.

4. Who should be included?

Not all employee groups face the same risks. Will everyone participate? Will third-party contractors be included? What about student workers, temporary staff, or alumni with active accounts? Should high-risk groups receive different tests?

Role-based testing often produces more meaningful results because simulations can be tailored to the specific threats employees are likely to encounter. For example:

• Finance staff receive invoice or ACH payment fraud scenarios
• Human Resources receive benefits or payroll scams
• IT staff receive credential-related phishing attempts
• Executives receive business email compromise (BEC) attacks

Will executives be tested? Ensuring that leadership also participates sets the tone for the rest of the organization and validates the importance of cybersecurity organization wide.

5. How realistic should scenarios be?

Before building campaigns, consider what scenarios are appropriate and which topics may cause unnecessary anxiety. Confirm there are no legal or HR concerns with the planned tests. Will employees perceive certain messages as deceptive or insensitive?

The goal is to mirror realistic threats while maintaining trust and professionalism. Effective simulations often mimic communications employees commonly see, such as:

• Password expiration notices
• Benefits enrollment updates
• Financial aid requests
• Shared document notifications

Phishing tests that state urgent requests regarding a “compromised account” test employees’ ability to stay calm under pressure and think before they act.

6. How often should phishing tests be sent?

Defining frequency is important. If simulations are conducted too infrequently, users may forget key lessons learned. Running simulations quarterly or even monthly is recommended to ensure awareness is maintained and allows the organization to track improvements as well. Aligning campaigns with your ongoing security awareness training can also be helpful.

7. Do any technical changes need to be made first?

Successful phishing simulations depend on proper technical preparation. Questions include:

• Are simulation domains whitelisted?
• Will emails bypass spam filters?
• Have SPF or email authentication requirements been reviewed?
• Is there a process for reporting suspicious emails?
• Are reporting tools properly configured?
• Are we measuring the right metrics?

Tracking results is critical to understanding how well your organization can respond to phishing threats. Before launching a campaign, determine which metrics matter most and how progress will be measured.

Key metrics include:

• Click rates
• QR code scans
• Credential submissions
• Follow-up training completion rates
• Reporting and response rates: How many employees identified and reported the phishing attempt? How quickly did employees recognize and report the phishing attack?

Your phishing platform should generate detailed executive reports that illustrate key metrics, offer strategic insights, and help your organization understand the overall effectiveness of training and next steps that need to be taken.

8. What happens following a phishing test?

Phishing simulations are only effective if employees receive feedback. Employees who click or fail a phishing test should be provided with immediate feedback that explains clear examples of what they missed.

Providing short, micro-learning training as a follow-up exercise can help reinforce these lessons and improve retention.

You should also decide on organizational actions following a phishing test. Who will review the results? Who will address employee questions, and how will they be handled? Do we have a plan for repeat offenders? If employees repeatedly fail phishing simulations, they may benefit from more focused training or discussions.

9. How is reporting encouraged?

A successful phishing simulation program also encourages reporting. Employees should be rewarded for reporting phishing attempts, whether they identify them during a simulation or in real-life situations. Rewarding individual employees or whole departments with the most employees who reported a test email can be a great way to encourage future reporting.

10. What makes a phishing test successful?

• Establish a baseline

Understand current employee behavior before measuring improvement.

• Use realistic scenarios

Simulations should reflect threats employees actually encounter.

• Provide immediate education

Use failures as opportunities for just-in-time learning.

• Measure behavior over time

Look at trends rather than focusing on one campaign.

• Encourage reporting

A successful program should help employees become more confident in identifying and reporting suspicious messages.

An effective phishing test is about more than finding out who clicks. It is an opportunity to measure awareness, reinforce secure behaviors, improve reporting, and identify areas where additional training is needed.

By asking the right questions before launching a phishing test, organizations can build a program that supports continuous learning, produces meaningful data, and reduces cybersecurity risk over time.

Learn more about CampusGuard’s Phishing Simulator platform or request a demo to see it in action.