Module: AI in Offensive & Defensive Security
Duration: 140-150 minutes
Instructor: Dr. Zhengxiong Li
- Introduction & Overview (15 min)
- AI for Vulnerability Discovery (30 min)
- Automated Exploit Generation (25 min)
- AI-Powered Phishing & Social Engineering (25 min)
- Adversarial ML for Security Applications (20 min)
- AI-Assisted Malware Detection & Defense (25 min)
- Summary & Discussion (10 min)
Duration: 15 minutes
Artificial Intelligence represents both a powerful weapon and a critical defense mechanism in cybersecurity. This fundamental duality—where the same technology can be used for both offense and defense—is what we call the dual-use problem.
Key Concept: Any AI technique that improves security can potentially be inverted or adapted for malicious purposes.
The relationship between AI and security has evolved through several phases:
- Early Days (1990s-2000s): Rule-based expert systems for intrusion detection
- ML Era (2000s-2010s): Statistical learning for anomaly detection
- Deep Learning Revolution (2010s-2020s): Neural networks for advanced threat detection
- LLM Era (2020s-present): Language models for code analysis, automated attacks, and intelligent defense
Understanding AI-powered attacks and defenses is critical because:
- Attackers are already using AI: From automated vulnerability scanning to AI-generated phishing emails
- Defenders need AI to keep pace: The volume and sophistication of attacks require automated analysis
- Ethical implications: We must understand these techniques to build responsible systems
- Career relevance: Security professionals must be fluent in both offensive and defensive AI
By the end of this tutorial, you will be able to:
- Understand the dual-use nature of AI in security contexts
- Analyze various AI-powered attack techniques and their mechanisms
- Evaluate AI-based defense systems and their effectiveness
- Apply principles of responsible AI in security research
Duration: 30 minutes
Vulnerability discovery is the process of identifying security flaws in software, systems, or protocols before they can be exploited by attackers. Traditionally, this required:
- Manual code review by security experts
- Fuzzing with random or semi-random inputs
- Static analysis tools following predefined patterns
AI Changes the Game: Machine learning models can learn patterns of vulnerabilities from historical data and identify new, similar vulnerabilities at scale.
| Aspect | Traditional | AI-Powered |
|---|
| Speed | Hours to weeks | Seconds to minutes |
| Coverage | Limited by human capacity | Scales to millions of lines |
| Pattern Recognition | Explicit rules only | Learns implicit patterns |
| False Positives | Moderate | Can be high (requires tuning) |
| Novel Vulnerabilities | Excellent (human insight) | Limited (depends on training) |
What is Fuzzing? Fuzzing involves feeding random or mutated inputs to a program to trigger crashes or unexpected behavior.
How AI Improves Fuzzing:
- Coverage-guided learning: ML models predict which mutations are most likely to explore new code paths
- Seed selection: Neural networks choose optimal starting inputs
- Mutation strategies: Reinforcement learning adapts fuzzing strategies in real-time
Demo Example: Simple ML-Guided Fuzzer Concept
import numpy as np
from sklearn.ensemble import RandomForestClassifier
class MLGuidedFuzzer:
"""
Simplified ML-guided fuzzer that learns which input mutations
are most likely to trigger interesting program behavior.
"""
def __init__(self):
# Classifier to predict "interesting" inputs
self.model = RandomForestClassifier(n_estimators=100)
self.training_data = []
self.training_labels = []
def extract_features(self, input_bytes):
"""Extract features from input for ML model"""
features = []
# Basic features
features.append(len(input_bytes)) # Length
features.append(input_bytes.count(b'\x00')) # Null bytes
features.append(input_bytes.count(b'\xff')) # Max bytes
# Byte distribution
byte_counts = np.bincount(list(input_bytes), minlength=256)
features.extend(byte_counts[:10]) # First 10 byte frequencies
# Entropy (simplified)
byte_probs = byte_counts / len(input_bytes) if len(input_bytes) > 0 else byte_counts
entropy = -np.sum(byte_probs * np.log2(byte_probs + 1e-10))
features.append(entropy)
return np.array(features)
def mutate(self, input_bytes):
"""Generate mutations of the input"""
mutations = []
# Bit flip mutation
for i in range(min(len(input_bytes), 10)):
mutated = bytearray(input_bytes)
mutated[i] ^= 0xFF
mutations.append(bytes(mutated))
# Byte insertion
for i in range(min(len(input_bytes), 5)):
mutated = bytearray(input_bytes)
mutated.insert(i, 0x41) # Insert 'A'
mutations.append(bytes(mutated))
# Truncation
if len(input_bytes) > 10:
mutations.append(input_bytes[:len(input_bytes)//2])
return mutations
def score_input(self, input_bytes):
"""Predict how 'interesting' an input is"""
if len(self.training_data) < 10:
return np.random.random() # Random until we have training data
features = self.extract_features(input_bytes).reshape(1, -1)
probability = self.model.predict_proba(features)[0][1]
return probability
def train(self, input_bytes, triggered_crash):
"""Train the model on feedback from the target program"""
features = self.extract_features(input_bytes)
self.training_data.append(features)
self.training_labels.append(1 if triggered_crash else 0)
# Retrain periodically
if len(self.training_data) >= 10:
X = np.array(self.training_data)
y = np.array(self.training_labels)
self.model.fit(X, y)
def fuzz(self, seed_input, target_function, iterations=100):
"""Main fuzzing loop"""
queue = [seed_input]
crashes = []
for i in range(iterations):
if not queue:
break
# Select input (prioritize high-scoring inputs)
current = queue.pop(0)
# Test current input
try:
result = target_function(current)
self.train(current, triggered_crash=False)
except Exception as e:
print(f"[!] Crash found with input: {current[:50]}")
crashes.append((current, str(e)))
self.train(current, triggered_crash=True)
continue
# Generate and score mutations
mutations = self.mutate(current)
scored_mutations = [(m, self.score_input(m)) for m in mutations]
# Add best mutations to queue
scored_mutations.sort(key=lambda x: x[1], reverse=True)
queue.extend([m for m, score in scored_mutations[:3]])
return crashes
# Example vulnerable function
def vulnerable_parser(data):
"""Simulated vulnerable parser"""
if len(data) > 100 and data[50:54] == b'EVIL':
raise Exception("Buffer overflow triggered!")
if b'\x00\xff\x00\xff' in data:
raise Exception("Format string vulnerability!")
return "OK"
# Demo the fuzzer
print("=== ML-Guided Fuzzing Demo ===\n")
fuzzer = MLGuidedFuzzer()
seed = b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"
crashes = fuzzer.fuzz(seed, vulnerable_parser, iterations=200)
print(f"\nFound {len(crashes)} crashes:")
for crash_input, error in crashes:
print(f" - {error}")
Expected Output:
=== ML-Guided Fuzzing Demo ===
[!] Crash found with input: b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\nAAAAAAAAAAAA'
[!] Crash found with input: b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n\x00\xff\x00\xff'
Found 2 crashes:
- Buffer overflow triggered!
- Format string vulnerability!
Discussion Points:
- How does the ML model learn which mutations are productive?
- What are the limitations of this approach?
- How might an attacker use this against proprietary software?
Concept: Deep learning models can be trained to understand code semantics and identify vulnerability patterns.
Architecture Example:
Source Code → Tokenization → Code Embeddings → Neural Network → Vulnerability Score
Common Approaches:
- Graph Neural Networks (GNNs): Operate on program dependency graphs
- Transformers: Process source code as sequences (like CodeBERT, GraphCodeBERT)
- Recurrent Networks: Analyze execution traces
Real-World Example: Detecting Buffer Overflows
# Simplified example showing the concept (not a complete implementation)
import re
class VulnerabilityScanner:
"""
Demonstrates pattern-based vulnerability detection
(In practice, this would use trained neural networks)
"""
def __init__(self):
# In a real system, these would be learned patterns
self.dangerous_functions = [
'strcpy', 'strcat', 'gets', 'sprintf', 'scanf'
]
self.vulnerability_patterns = {
'buffer_overflow': [
r'strcpy\s*\([^,]+,\s*[^)]+\)', # strcpy without bounds check
r'gets\s*\([^)]+\)', # gets is always dangerous
],
'format_string': [
r'printf\s*\(\s*[^"\']+\s*\)', # printf with variable format
r'sprintf\s*\([^,]+,\s*[^"\']+\)',
],
'integer_overflow': [
r'malloc\s*\(\s*[^)]*\*[^)]*\)', # malloc with multiplication
]
}
def scan_code(self, source_code):
"""Scan source code for vulnerabilities"""
findings = []
lines = source_code.split('\n')
for line_num, line in enumerate(lines, 1):
# Check for dangerous functions
for func in self.dangerous_functions:
if func in line:
findings.append({
'line': line_num,
'type': 'dangerous_function',
'severity': 'medium',
'function': func,
'code': line.strip()
})
# Check for specific vulnerability patterns
for vuln_type, patterns in self.vulnerability_patterns.items():
for pattern in patterns:
if re.search(pattern, line):
findings.append({
'line': line_num,
'type': vuln_type,
'severity': 'high',
'code': line.strip()
})
return findings
def generate_report(self, findings):
"""Generate vulnerability report"""
if not findings:
print("✓ No vulnerabilities detected")
return
print(f"⚠ Found {len(findings)} potential vulnerabilities:\n")
for i, finding in enumerate(findings, 1):
print(f"{i}. Line {finding['line']}: {finding['type'].upper()}")
print(f" Severity: {finding['severity']}")
print(f" Code: {finding['code']}")
print()
# Demo with vulnerable C code
vulnerable_code = """
#include <stdio.h>
#include <string.h>
void process_input(char *user_input) {
char buffer[64];
strcpy(buffer, user_input); // Vulnerable!
printf(user_input); // Format string vulnerability!
}
int main() {
char input[256];
gets(input); // Very dangerous!
process_input(input);
return 0;
}
"""
print("=== Vulnerability Scanner Demo ===\n")
scanner = VulnerabilityScanner()
findings = scanner.scan_code(vulnerable_code)
scanner.generate_report(findings)
Expected Output:
=== Vulnerability Scanner Demo ===
⚠ Found 5 potential vulnerabilities:
1. Line 6: DANGEROUS_FUNCTION
Severity: medium
Code: strcpy(buffer, user_input); // Vulnerable!
2. Line 6: BUFFER_OVERFLOW
Severity: high
Code: strcpy(buffer, user_input); // Vulnerable!
3. Line 7: DANGEROUS_FUNCTION
Severity: medium
Code: printf(user_input); // Format string vulnerability!
4. Line 7: FORMAT_STRING
Severity: high
Code: printf(user_input); // Format string vulnerability!
5. Line 12: BUFFER_OVERFLOW
Severity: high
Code: gets(input); // Very dangerous!
Real-World Systems:
- DeepCode (Snyk): Uses ML to learn from millions of open-source repositories
- GitHub Copilot Security: Identifies vulnerabilities in AI-generated code
- Google's OSS-Fuzz: ML-guided continuous fuzzing for open-source projects
- Microsoft's SHARD: Deep learning for vulnerability discovery in binaries
Question for Discussion: If you discover a novel vulnerability using AI, what is your ethical responsibility?
Responsible Disclosure:
- Identify and verify the vulnerability
- Notify the vendor/maintainer privately
- Give them time to patch (typically 90 days)
- Publish details only after a fix is available
- Consider the potential for harm
Duration: 25 minutes
Key Distinction:
- Vulnerability: A weakness or flaw in a system
- Exploit: A piece of code or technique that takes advantage of a vulnerability
Why Automation Matters:
- Manual exploit development requires deep expertise
- Time-consuming process (days to weeks)
- AI can generate exploits in minutes to hours
- Enables rapid weaponization of vulnerabilities
Vulnerability → Analysis → Constraint Solving → Payload Generation → Exploit
Symbolic Execution: A technique that analyzes programs by treating inputs as symbolic variables and exploring all possible execution paths.
How AI Enhances It:
- Path prioritization: ML predicts which paths are most likely to be exploitable
- Constraint simplification: Neural networks learn to simplify complex constraints
- Solver guidance: Reinforcement learning guides SMT solvers
Conceptual Example:
class SymbolicExecutionEngine:
"""
Simplified symbolic execution for demonstrating exploit generation
(Real systems use Z3, Angr, or similar frameworks)
"""
def __init__(self):
self.path_constraints = []
self.exploitable_states = []
def analyze_function(self, code, symbolic_input):
"""
Analyze function symbolically to find exploitable conditions
"""
print(f"Analyzing code with symbolic input: {symbolic_input}\n")
# Simulate symbolic execution
# In reality, this would parse AST and track constraints
paths = [
{
'condition': 'length > 100',
'effect': 'buffer_overflow',
'exploitable': True,
'constraint': 'len(input) > 100'
},
{
'condition': 'input contains "admin"',
'effect': 'authentication_bypass',
'exploitable': True,
'constraint': '"admin" in input'
},
{
'condition': 'normal execution',
'effect': 'safe',
'exploitable': False,
'constraint': 'len(input) <= 100 and "admin" not in input'
}
]
print("Discovered execution paths:")
for i, path in enumerate(paths, 1):
status = "⚠ EXPLOITABLE" if path['exploitable'] else "✓ Safe"
print(f"{i}. {status}: {path['condition']}")
print(f" Constraint: {path['constraint']}")
if path['exploitable']:
print(f" Effect: {path['effect']}")
self.exploitable_states.append(path)
print()
return paths
def generate_exploit(self, exploitable_state):
"""Generate exploit payload for an exploitable state"""
exploit_type = exploitable_state['effect']
constraint = exploitable_state['constraint']
print(f"Generating exploit for: {exploit_type}")
print(f"Must satisfy: {constraint}\n")
if exploit_type == 'buffer_overflow':
# Generate payload that overflows buffer
payload = 'A' * 120 + '\x41\x42\x43\x44' # Overflow + return address
print("Generated Payload:")
print(f" Length: {len(payload)} bytes")
print(f" Content: {'A'*120}\\x41\\x42\\x43\\x44")
print(f" Purpose: Overflow buffer and overwrite return address")
elif exploit_type == 'authentication_bypass':
payload = 'admin\x00truncated'
print("Generated Payload:")
print(f" Content: {payload}")
print(f" Purpose: Inject admin string with null byte terminator")
return payload
# Demo
print("=== Automated Exploit Generation Demo ===\n")
vulnerable_function = """
def process_login(username, password):
if len(username) > 100:
# Buffer overflow possible
return "overflow"
if "admin" in username:
# Authentication bypass
return "bypass"
return check_credentials(username, password)
"""
engine = SymbolicExecutionEngine()
paths = engine.analyze_function(vulnerable_function, "symbolic_input")
print("\n--- Generating Exploits ---\n")
for state in engine.exploitable_states:
exploit = engine.generate_exploit(state)
print("\n" + "="*50 + "\n")
Expected Output:
=== Automated Exploit Generation Demo ===
Analyzing code with symbolic input: symbolic_input
Discovered execution paths:
1. ⚠ EXPLOITABLE: length > 100
Constraint: len(input) > 100
Effect: buffer_overflow
2. ⚠ EXPLOITABLE: input contains "admin"
Constraint: "admin" in input
Effect: authentication_bypass
3. ✓ Safe: normal execution
Constraint: len(input) <= 100 and "admin" not in input
--- Generating Exploits ---
Generating exploit for: buffer_overflow
Must satisfy: len(input) > 100
Generated Payload:
Length: 124 bytes
Content: AAAAAAAAAA...(120 A's)...\x41\x42\x43\x44
Purpose: Overflow buffer and overwrite return address
==================================================
Generating exploit for: authentication_bypass
Must satisfy: "admin" in input
Generated Payload:
Content: admin\x00truncated
Purpose: Inject admin string with null byte terminator
Concept: Train an RL agent to interact with a target program and discover exploitation strategies.
Process:
- State: Current program state, memory layout, registers
- Actions: Input modifications, payload injections
- Reward: Getting closer to exploitation (code execution, privilege escalation)
- Policy: Learned strategy for exploiting the vulnerability
Advantages:
- Discovers creative exploitation paths humans might miss
- Adapts to different program variants
- Can bypass certain defenses automatically
Challenges:
- Requires significant computational resources
- Reward engineering is difficult
- May find unintended exploitation methods
Shellcode: Small piece of code used as payload in exploitation, typically to spawn a shell or execute commands.
Traditional Approach: Hand-crafted assembly code
AI Approach: Neural networks trained on existing shellcode can generate new variants
Example Application:
class ShellcodeGenerator:
"""
Demonstrates concept of AI-generated shellcode
(Simplified - real systems use more sophisticated models)
"""
def __init__(self):
# In practice, this would be a trained neural network
self.templates = {
'reverse_shell': [
b'\x31\xc0', # xor eax, eax
b'\x50', # push eax
b'\x68//sh', # push "//sh"
# ... more shellcode bytes
],
'bind_shell': [
b'\x6a\x66', # push 0x66
b'\x58', # pop eax
# ... more shellcode bytes
]
}
def generate(self, shell_type, target_ip, target_port):
"""Generate shellcode for specified type"""
print(f"Generating {shell_type} shellcode")
print(f"Target: {target_ip}:{target_port}\n")
# Simplified generation (real version would use neural network)
template = self.templates.get(shell_type, [])
# Encode parameters
ip_bytes = bytes(map(int, target_ip.split('.')))
port_bytes = target_port.to_bytes(2, byteorder='big')
shellcode = b''.join(template)
print(f"Generated {len(shellcode)} bytes of shellcode")
print(f"Encoded IP: {ip_bytes.hex()}")
print(f"Encoded Port: {port_bytes.hex()}")
return shellcode
# Demo
print("=== AI Shellcode Generator Demo ===\n")
generator = ShellcodeGenerator()
shellcode = generator.generate('reverse_shell', '192.168.1.100', 4444)
Notable Projects:
- Mayhem (ForAllSecure): Competed in DARPA Cyber Grand Challenge
- Automatic Exploit Generation (AEG): Academic research system
- Revery: Automated patch analysis and exploit generation
- Mechanical Phish: DARPA CGC competitor
How Defenders Should Respond:
- Assume exploitation is automated: Patch windows are shrinking
- Zero-day to N-day: Time from discovery to exploitation is decreasing
- Automated patching: Must match speed of automated exploitation
- Virtual patching: WAF rules and IDS signatures deployed immediately
Duration: 25 minutes
Traditional Phishing (1990s-2010s):
- Mass emails with obvious grammar errors
- Generic messages ("Dear User")
- Easy to detect with basic filters
AI-Powered Phishing (2020s):
- Personalized messages at scale
- Perfect grammar and context awareness
- Adaptive to user responses
- Multimodal (text, voice, images)
Spear Phishing: Targeted attacks against specific individuals using personalized information.
How LLMs Enhance Attacks:
- OSINT Automation: Scrape and synthesize public information
- Content Generation: Create convincing, personalized messages
- Context Adaptation: Adjust messaging based on target's industry, role, interests
- A/B Testing: Generate multiple variants and optimize
Demo: AI-Generated Phishing Email
class AIPhishingGenerator:
"""
Demonstrates how LLMs can generate convincing phishing content
(For educational purposes only - do not use maliciously)
"""
def __init__(self):
# Simulated target profile (gathered from OSINT)
self.target_profiles = {
'john_smith': {
'name': 'John Smith',
'role': 'Senior Software Engineer',
'company': 'TechCorp Inc',
'interests': ['cybersecurity', 'Python', 'cloud computing'],
'recent_activity': 'Posted about AWS migration on LinkedIn',
'email_style': 'professional, technical'
}
}
def analyze_target(self, target_id):
"""Analyze target to extract key information"""
profile = self.target_profiles.get(target_id, {})
print("=== Target Analysis ===")
print(f"Name: {profile.get('name')}")
print(f"Role: {profile.get('role')}")
print(f"Company: {profile.get('company')}")
print(f"Recent Activity: {profile.get('recent_activity')}")
print()
return profile
def generate_phishing_email(self, profile, attack_vector):
"""Generate personalized phishing email"""
# Different templates based on attack vector
templates = {
'credential_theft': self._generate_credential_phish,
'malware_delivery': self._generate_malware_phish,
'business_email_compromise': self._generate_bec_phish
}
generator = templates.get(attack_vector, self._generate_credential_phish)
return generator(profile)
def _generate_credential_phish(self, profile):
"""Generate credential harvesting email"""
email = f"""
From: IT Security <security@{profile['company'].lower().replace(' ', '')}.com>
To: {profile['name']} <{profile['name'].lower().replace(' ', '.')}@{profile['company'].lower().replace(' ', '')}.com>
Subject: Urgent: Unusual AWS Activity Detected on Your Account
Hi {profile['name'].split()[0]},
Our security team has detected unusual API calls from your AWS account related to the recent
cloud migration project you've been working on. Given your recent LinkedIn post about the AWS
migration, we wanted to reach out immediately.
We've identified the following suspicious activities:
- Multiple failed authentication attempts from IP: 185.220.101.47 (Russia)
- Unauthorized S3 bucket access attempts
- Anomalous Lambda function deployments
To prevent unauthorized access and secure your account, please verify your credentials
immediately by clicking the link below:
[Verify AWS Account] → https://aws-security-check.com/verify?user={profile['name'].lower().replace(' ', '.')}
This verification must be completed within 2 hours to prevent account suspension.
If you did not initiate these activities, please contact the security team immediately.
Best regards,
AWS Security Team
{profile['company']} IT Department
---
This is an automated security alert. Please do not reply to this email.
"""
return email
def _generate_malware_phish(self, profile):
"""Generate malware delivery email"""
email = f"""
From: GitHub Security <noreply@github.com>
To: {profile['name']}
Subject: Critical Security Vulnerability in Your Python Repository
Hello {profile['name'].split()[0]},
We've identified a critical security vulnerability (CVE-2024-XXXXX) affecting one of your
Python repositories. This vulnerability affects the dependencies you're using and could
expose sensitive data.
Vulnerability Details:
- Severity: CRITICAL (CVSS 9.8)
- Affected Package: requests-auth (used in 3 of your repositories)
- Impact: Remote Code Execution, Data Exfiltration
We've prepared a detailed security report and automated patch tool specifically for your
repositories. Please download and run the security scanner:
[Download Security Scanner] → security-scan-tool-v2.4.exe
The tool will:
1. Scan all your repositories for this vulnerability
2. Generate a detailed report
3. Offer automated patching options
Given your background in cybersecurity, you'll appreciate the urgency of this matter.
Regards,
GitHub Security Team
"""
return email
def _generate_bec_phish(self, profile):
"""Generate Business Email Compromise"""
email = f"""
From: Sarah Chen, CEO <sarah.chen@{profile['company'].lower().replace(' ', '')}.com>
To: {profile['name']}
Subject: Quick Request - Vendor Payment
Hi {profile['name'].split()[0]},
I'm currently in a meeting with investors and need your help with an urgent matter.
We have a critical payment to one of our cloud infrastructure vendors that needs to be
processed today to avoid service interruption. This is related to the AWS migration
you've been working on.
Can you please initiate a wire transfer for $47,500 to our vendor using the details below?
Bank: First National Bank
Account: 4782-9301-5567
Routing: 021000021
Reference: AWS-MIGRATION-Q1
I know this is irregular, but the timing is critical and I'm tied up in meetings all day.
I'll explain everything when I'm back in the office tomorrow.
Thanks for your help on this.
Sarah
--
Sarah Chen
CEO, {profile['company']}
Sent from my iPhone
"""
return email
def detect_red_flags(self, email_content):
"""Identify phishing indicators (defender's perspective)"""
red_flags = []
indicators = {
'Urgency': ['urgent', 'immediately', 'within 2 hours', 'critical'],
'Suspicious Links': ['http://', 'verify', 'security-check'],
'Unusual Requests': ['wire transfer', 'download', 'click the link'],
'Sender Spoofing': ['noreply@', 'security@'],
'Grammar/Formatting': ['Sent from my iPhone', 'automated'],
}
for category, keywords in indicators.items():
for keyword in keywords:
if keyword.lower() in email_content.lower():
red_flags.append(f"{category}: '{keyword}'")
return red_flags
# Demo
print("=== AI-Powered Phishing Generator Demo ===\n")
generator = AIPhishingGenerator()
# Analyze target
profile = generator.analyze_target('john_smith')
# Generate different types of phishing emails
print("\n--- Credential Theft Email ---")
email1 = generator.generate_phishing_email(profile, 'credential_theft')
print(email1)
print("\n--- Detecting Red Flags ---")
flags = generator.detect_red_flags(email1)
print(f"Detected {len(flags)} red flags:")
for flag in flags:
print(f" ⚠ {flag}")
Emerging Threat: AI-generated voice and video for social engineering
Vishing (Voice Phishing) with AI:
- Clone executive's voice from public recordings
- Generate convincing phone calls requesting actions
- Real-time voice synthesis during calls
Example Scenario:
Attacker → Voice Cloning Model → CEO's Voice → Call to CFO → "Urgent wire transfer needed"
Defense Strategies:
- Establish code words for sensitive requests
- Out-of-band verification (call back on known number)
- AI detection tools for synthetic media
- Employee training on deepfake threats
OSINT with LLMs:
class AIReconnaissanceTool:
"""
Demonstrates automated OSINT gathering with AI
(Simplified for educational purposes)
"""
def gather_intel(self, target_name):
"""Simulate gathering public information"""
# In reality, this would scrape multiple sources
intel = {
'professional': {
'linkedin': {
'current_role': 'Senior Engineer at TechCorp',
'skills': ['Python', 'AWS', 'Kubernetes'],
'connections': 847,
'recent_posts': [
'Excited about our new cloud migration!',
'Speaking at CloudConf 2026 next month'
]
},
'github': {
'repositories': ['ml-toolkit', 'security-scanner'],
'languages': ['Python', 'Go'],
'contributions': 'High activity in security projects'
}
},
'personal': {
'twitter': {
'interests': ['cybersecurity', 'AI', 'rock climbing'],
'location': 'San Francisco Bay Area'
},
'instagram': {
'photos': 'Travel, outdoor activities',
'followers': 342
}
},
'digital_footprint': {
'email_pattern': 'firstname.lastname@company.com',
'likely_passwords_patterns': [
'CompanyName + Year',
'Pet names + Numbers (from photos)',
'Hobby + Special dates'
]
}
}
return intel
def synthesize_attack_strategy(self, intel):
"""AI synthesizes intelligence into attack strategy"""
strategy = {
'primary_vector': 'LinkedIn-themed phishing',
'secondary_vector': 'GitHub security notification',
'personalization_points': [
'Reference cloud migration project',
'Mention upcoming CloudConf talk',
'Use technical language matching their expertise'
],
'optimal_timing': 'Monday morning or late Friday afternoon',
'confidence_score': 0.87
}
return strategy
def generate_attack_plan(self, target_name):
"""Complete reconnaissance and planning"""
print(f"=== AI Reconnaissance on {target_name} ===\n")
print("Phase 1: Intelligence Gathering...")
intel = self.gather_intel(target_name)
print(f" ✓ LinkedIn profile analyzed")
print(f" ✓ GitHub activity reviewed")
print(f" ✓ Social media footprint mapped")
print(f" ✓ Email pattern identified\n")
print("Phase 2: Attack Strategy Synthesis...")
strategy = self.synthesize_attack_strategy(intel)
print(f" Primary Vector: {strategy['primary_vector']}")
print(f" Confidence Score: {strategy['confidence_score']:.0%}\n")
print("Phase 3: Personalization Points:")
for point in strategy['personalization_points']:
print(f" • {point}")
return strategy
# Demo
recon = AIReconnaissanceTool()
attack_plan = recon.generate_attack_plan("John Smith")
Multi-Layer Defense Strategy:
- Technical Controls:
- Email authentication (SPF, DKIM, DMARC)
- AI-powered email filtering
- Link analysis and sandboxing
- Domain reputation checking
- User Education:
- Regular security awareness training
- Phishing simulation exercises
- Red flag recognition training
- Incident reporting procedures
- Process Controls:
- Verification procedures for sensitive requests
- Multi-person authorization for financial transactions
- Out-of-band confirmation channels
- Regular security drills
- AI-Powered Defenses:
- Anomaly detection in communication patterns
- Behavioral biometrics
- Real-time deepfake detection
- Natural language analysis for suspicious content
Duration: 20 minutes
We've spent weeks learning how adversarial machine learning can attack AI systems. Now we flip the script: adversarial techniques can actually improve security.
Adversarial Training for Security Models:
import numpy as np
class AdversariallyTrainedDetector:
"""
Demonstrates adversarial training for robust malware detection
"""
def __init__(self):
self.model_weights = np.random.randn(10) # Simplified model
self.threshold = 0.5
def extract_features(self, sample):
"""Extract features from malware sample"""
# Simplified feature extraction
features = np.random.randn(10)
return features
def predict(self, features):
"""Predict if sample is malicious"""
score = np.dot(features, self.model_weights)
return 1 if score > self.threshold else 0
def generate_adversarial_example(self, sample, target_label):
"""
Generate adversarial malware sample
(Simulates evasion attempt)
"""
features = self.extract_features(sample)
# Gradient-based perturbation (simplified)
# In reality: add minimal changes to bypass detector
perturbation = 0.1 * np.sign(self.model_weights)
if target_label == 0: # Want to evade detection
perturbation = -perturbation
adversarial_features = features + perturbation
return adversarial_features
def adversarial_training(self, benign_samples, malware_samples, epochs=10):
"""
Train detector with adversarial examples
"""
print("=== Adversarial Training Process ===\n")
for epoch in range(epochs):
total_loss = 0
correct = 0
total = 0
# Train on normal samples
for sample in benign_samples[:5]:
features = self.extract_features(sample)
pred = self.predict(features)
total += 1
if pred == 0:
correct += 1
for sample in malware_samples[:5]:
features = self.extract_features(sample)
pred = self.predict(features)
total += 1
if pred == 1:
correct += 1
# Generate adversarial example
adv_features = self.generate_adversarial_example(sample, target_label=0)
adv_pred = self.predict(adv_features)
# Update model to correctly classify adversarial example
if adv_pred == 0: # Failed to detect adversarial malware
# Update weights (simplified)
self.model_weights += 0.01 * adv_features
accuracy = correct / total
print(f"Epoch {epoch + 1}/{epochs}: Accuracy = {accuracy:.2%}")
print("\n✓ Training complete! Model is now robust to evasion attacks.")
# Demo
print("=== Adversarial Training for Malware Detection ===\n")
detector = AdversariallyTrainedDetector()
# Simulate training data
benign_samples = [f"benign_{i}" for i in range(10)]
malware_samples = [f"malware_{i}" for i in range(10)]
detector.adversarial_training(benign_samples, malware_samples, epochs=5)
Using Adversarial ML for Penetration Testing:
Concept: Generate adversarial test cases to stress-test security systems
Applications:
- IDS/IPS Testing: Generate traffic that evades detection
- WAF Bypass: Craft requests that slip through web application firewalls
- Antivirus Evasion Testing: Create polymorphic malware variants
- Authentication Testing: Generate adversarial biometric inputs
Example: Testing a Spam Filter
class SpamFilterTester:
"""
Uses adversarial techniques to test spam filter robustness
"""
def __init__(self):
# Simplified spam filter (keyword-based)
self.spam_keywords = ['viagra', 'lottery', 'winner', 'prize', 'click here']
self.spam_patterns = ['!!!', '$$', 'FREE']
def is_spam(self, email_text):
"""Simple spam detection"""
text_lower = email_text.lower()
# Check for spam keywords
keyword_count = sum(1 for kw in self.spam_keywords if kw in text_lower)
# Check for spam patterns
pattern_count = sum(1 for pat in self.spam_patterns if pat in email_text)
return (keyword_count + pattern_count) >= 2
def generate_adversarial_spam(self, base_spam_message):
"""
Generate spam that evades the filter
(Demonstrates adversarial testing)
"""
print("=== Adversarial Spam Generation ===\n")
print(f"Original spam message:\n{base_spam_message}\n")
print(f"Detected as spam: {self.is_spam(base_spam_message)}\n")
# Technique 1: Character substitution
adversarial_v1 = base_spam_message.replace('i', '1').replace('o', '0')
print(f"Version 1 (Character substitution):\n{adversarial_v1}")
print(f"Detected as spam: {self.is_spam(adversarial_v1)}\n")
# Technique 2: Word splitting
adversarial_v2 = base_spam_message.replace('viagra', 'v i a g r a')
adversarial_v2 = adversarial_v2.replace('!!!', '! ! !')
print(f"Version 2 (Word splitting):\n{adversarial_v2}")
print(f"Detected as spam: {self.is_spam(adversarial_v2)}\n")
# Technique 3: Homoglyph attack
adversarial_v3 = base_spam_message.replace('a', 'а') # Cyrillic 'а'
print(f"Version 3 (Homoglyph attack):\n{adversarial_v3}")
print(f"Detected as spam: {self.is_spam(adversarial_v3)}\n")
return [adversarial_v1, adversarial_v2, adversarial_v3]
def evaluate_robustness(self, base_message):
"""Evaluate how many adversarial variants bypass filter"""
variants = self.generate_adversarial_spam(base_message)
bypassed = sum(1 for v in variants if not self.is_spam(v))
total = len(variants)
print(f"=== Robustness Evaluation ===")
print(f"Adversarial variants that bypassed filter: {bypassed}/{total}")
print(f"Filter robustness score: {((total - bypassed) / total):.0%}")
# Demo
tester = SpamFilterTester()
spam_message = "Congratulations!!! You've won the lottery! Click here to claim your prize of $1,000,000. Get your viagra now!!!"
tester.evaluate_robustness(spam_message)
Using Adversarial Examples to Improve Anomaly Detectors:
Key Idea: Train anomaly detection systems on adversarial edge cases
Process:
- Deploy baseline anomaly detector
- Generate adversarial normal/anomalous samples
- Identify where detector fails
- Retrain with these hard examples
- Repeat until robust
Benefits:
- Reduces false positives
- Catches sophisticated attacks
- Improves decision boundaries
Automated Threat Hunting:
class AdversarialThreatHunter:
"""
Uses adversarial thinking to proactively hunt for threats
"""
def __init__(self):
self.known_attack_patterns = [
'PowerShell -encodedcommand',
'eval(base64_decode',
'cmd.exe /c',
]
def hypothesize_attack_variants(self, known_attack):
"""
Generate potential evasion variants of known attacks
"""
print(f"Generating adversarial variants of: {known_attack}\n")
variants = []
# Technique 1: Encoding changes
variants.append({
'variant': known_attack.replace('base64', 'b64'),
'technique': 'Encoding substitution'
})
# Technique 2: Case obfuscation
obfuscated = ''.join(
c.upper() if i % 2 else c.lower()
for i, c in enumerate(known_attack)
)
variants.append({
'variant': obfuscated,
'technique': 'Case obfuscation'
})
# Technique 3: String splitting
split = '+'.join([f'"{c}"' for c in known_attack.split()[0]])
variants.append({
'variant': split + ' ' + ' '.join(known_attack.split()[1:]),
'technique': 'String concatenation'
})
print("Generated variants:")
for i, v in enumerate(variants, 1):
print(f"{i}. {v['technique']}")
print(f" {v['variant']}\n")
return variants
def create_detection_rules(self, variants):
"""Create detection rules for all variants"""
print("=== Generated Detection Rules ===\n")
for i, v in enumerate(variants, 1):
print(f"Rule {i}: Detect {v['technique']}")
print(f" Pattern: {v['variant'][:50]}...")
print(f" Severity: HIGH")
print()
# Demo
print("=== Adversarial Threat Hunting Demo ===\n")
hunter = AdversarialThreatHunter()
known_attack = "PowerShell -encodedcommand base64_payload"
variants = hunter.hypothesize_attack_variants(known_attack)
hunter.create_detection_rules(variants)
Traditional Security: Attack → Detect → Patch → Repeat
Adversarial ML Security:
1. Deploy security system
2. Generate adversarial test cases
3. Identify weaknesses before attackers do
4. Retrain with adversarial examples
5. Achieve robustness
6. Repeat as threat landscape evolves
Duration: 25 minutes
Traditional Approaches:
- Signature-based: Match known malware patterns → Fails against new variants
- Heuristic-based: Rules for suspicious behavior → High false positive rate
- Sandboxing: Execute in isolated environment → Resource intensive, slow
Why AI is Necessary:
- Malware variants: Millions of new samples per day
- Polymorphic/metamorphic malware: Constantly changing code
- Fileless attacks: No traditional signatures to detect
- Zero-day exploits: Never seen before
Static Analysis: Examine malware without executing it
Features Analyzed:
- PE header information
- Import/export tables
- String patterns
- API call sequences
- Code structure
Deep Learning Architecture:
import numpy as np
from collections import Counter
class DeepMalwareDetector:
"""
Simplified deep learning malware detector
Demonstrates concept of static analysis with neural networks
"""
def __init__(self):
# Simplified neural network weights
self.weights = {
'layer1': np.random.randn(50, 100),
'layer2': np.random.randn(100, 50),
'output': np.random.randn(50, 2)
}
def extract_static_features(self, file_path):
"""
Extract static features from executable
(In practice, would use tools like pefile, radare2)
"""
features = {}
# Simulated feature extraction
features['file_size'] = np.random.randint(1000, 1000000)
features['section_count'] = np.random.randint(2, 8)
features['import_count'] = np.random.randint(10, 200)
features['suspicious_strings'] = np.random.randint(0, 50)
# API calls (simplified)
api_calls = [
'CreateFile', 'WriteFile', 'CreateProcess', 'RegSetValue',
'InternetOpen', 'VirtualAlloc', 'LoadLibrary'
]
features['api_calls'] = Counter(
np.random.choice(api_calls, size=20, replace=True)
)
# Entropy (measure of randomness, high in packed malware)
features['entropy'] = np.random.uniform(3.0, 7.5)
# PE characteristics
features['has_digital_signature'] = np.random.choice([True, False])
features['compile_time'] = 'recent' if np.random.random() > 0.7 else 'old'
return features
def features_to_vector(self, features):
"""Convert features to neural network input"""
vector = []
# Normalize file size
vector.append(features['file_size'] / 1000000.0)
# Section and import counts
vector.append(features['section_count'] / 10.0)
vector.append(features['import_count'] / 200.0)
# Suspicious strings count
vector.append(features['suspicious_strings'] / 50.0)
# API call frequencies (most dangerous ones)
dangerous_apis = ['CreateProcess', 'RegSetValue', 'VirtualAlloc']
for api in dangerous_apis:
vector.append(features['api_calls'].get(api, 0) / 20.0)
# Entropy
vector.append(features['entropy'] / 8.0)
# Binary features
vector.append(1.0 if features['has_digital_signature'] else 0.0)
vector.append(1.0 if features['compile_time'] == 'recent' else 0.0)
# Pad to 50 features
while len(vector) < 50:
vector.append(0.0)
return np.array(vector[:50])
def forward_pass(self, feature_vector):
"""Simplified neural network forward pass"""
# Layer 1
hidden1 = np.tanh(np.dot(feature_vector, self.weights['layer1']))
# Layer 2
hidden2 = np.tanh(np.dot(hidden1, self.weights['layer2']))
# Output layer
output = np.dot(hidden2, self.weights['output'])
# Softmax
exp_output = np.exp(output - np.max(output))
probabilities = exp_output / exp_output.sum()
return probabilities
def detect(self, file_path):
"""Detect if file is malware"""
print(f"=== Analyzing: {file_path} ===\n")
# Extract features
print("Step 1: Static Feature Extraction")
features = self.extract_static_features(file_path)
print(f" File size: {features['file_size']:,} bytes")
print(f" Sections: {features['section_count']}")
print(f" Imports: {features['import_count']}")
print(f" Entropy: {features['entropy']:.2f}")
print(f" Digital signature: {features['has_digital_signature']}")
print(f" Suspicious strings: {features['suspicious_strings']}")
# Convert to vector
print("\nStep 2: Feature Vectorization")
feature_vector = self.features_to_vector(features)
print(f" Generated {len(feature_vector)}-dimensional feature vector")
# Run through neural network
print("\nStep 3: Neural Network Classification")
probabilities = self.forward_pass(feature_vector)
is_malware = probabilities[1] > probabilities[0]
confidence = max(probabilities)
print(f" Benign probability: {probabilities[0]:.2%}")
print(f" Malware probability: {probabilities[1]:.2%}")
print(f"\n{'⚠ MALWARE DETECTED' if is_malware else '✓ FILE APPEARS CLEAN'}")
print(f" Confidence: {confidence:.2%}")
# Feature importance
print("\nStep 4: Feature Importance Analysis")
important_features = [
('High entropy', features['entropy'] > 6.5),
('Suspicious APIs', sum(features['api_calls'].values()) > 15),
('No digital signature', not features['has_digital_signature']),
('Recent compile time', features['compile_time'] == 'recent')
]
print(" Contributing factors:")
for feature, is_present in important_features:
if is_present:
print(f" • {feature}")
return is_malware, confidence
# Demo
print("=== Deep Learning Malware Detector ===\n")
detector = DeepMalwareDetector()
# Test on sample files
test_files = ['suspicious_file.exe', 'legitimate_app.exe']
for file in test_files:
is_malware, confidence = detector.detect(file)
print("\n" + "="*60 + "\n")
Dynamic Analysis: Execute malware in controlled environment and observe behavior
Behavioral Features:
- System calls
- Network activity
- File system modifications
- Registry changes
- Process creation
- Memory allocation patterns
class BehavioralMalwareDetector:
"""
Detects malware based on runtime behavior using ML
"""
def __init__(self):
# Behavioral patterns of known malware
self.malware_behaviors = {
'ransomware': {
'file_operations': ['encrypt', 'mass_delete', 'mass_rename'],
'network': ['c2_communication', 'data_exfiltration'],
'persistence': ['registry_run_key', 'scheduled_task'],
'privilege': ['admin_escalation']
},
'trojan': {
'file_operations': ['download_payload', 'create_backdoor'],
'network': ['reverse_shell', 'c2_beacon'],
'persistence': ['service_creation', 'startup_folder'],
'privilege': ['admin_escalation']
},
'spyware': {
'file_operations': ['keylogger_log', 'screenshot_capture'],
'network': ['data_exfiltration', 'periodic_beacon'],
'persistence': ['registry_run_key'],
'privilege': ['normal_user']
}
}
def monitor_behavior(self, process_name):
"""
Monitor process behavior (simulated)
In reality: Use ETW, API hooking, or sandboxing
"""
print(f"=== Monitoring behavior of: {process_name} ===\n")
# Simulated behavioral observations
observed = {
'file_operations': [],
'network': [],
'persistence': [],
'privilege': []
}
# Simulate detection of suspicious activities
activities = [
('file_operations', 'encrypt', 'Encrypted 150 files in Documents folder'),
('file_operations', 'mass_rename', 'Renamed files with .locked extension'),
('network', 'c2_communication', 'Contacted IP 185.220.101.47 on port 8443'),
('persistence', 'registry_run_key', 'Created HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run\\Updater'),
('privilege', 'admin_escalation', 'Attempted UAC bypass using COM elevation')
]
print("Observed behaviors:")
for category, behavior, description in activities:
observed[category].append(behavior)
print(f" ⚠ [{category.upper()}] {description}")
return observed
def classify_malware_family(self, observed):
"""Classify malware family based on behaviors"""
print("\n=== Behavioral Classification ===\n")
scores = {}
for family, patterns in self.malware_behaviors.items():
score = 0
matches = []
for category, behaviors in patterns.items():
for behavior in behaviors:
if behavior in observed[category]:
score += 1
matches.append(f"{category}: {behavior}")
scores[family] = {
'score': score,
'matches': matches
}
# Find best match
best_family = max(scores.keys(), key=lambda f: scores[f]['score'])
best_score = scores[best_family]['score']
print("Similarity to known families:")
for family, data in sorted(scores.items(), key=lambda x: x[1]['score'], reverse=True):
print(f" {family.upper()}: {data['score']} matching behaviors")
print(f"\n⚠ CLASSIFICATION: {best_family.upper()}")
print(f" Confidence: {(best_score / 10) * 100:.0f}%")
print(f"\n Matching behaviors:")
for match in scores[best_family]['matches']:
print(f" • {match}")
return best_family
# Demo
print("=== Behavioral Malware Detection ===\n")
detector = BehavioralMalwareDetector()
observed_behaviors = detector.monitor_behavior("suspicious_process.exe")
malware_family = detector.classify_malware_family(observed_behaviors)
Why Graphs?
- Malware relationships: Files, processes, network connections
- Call graphs: Function invocation patterns
- Infection chains: How malware spreads
GNN Advantages:
- Captures structural patterns
- Invariant to node ordering
- Scales to large graphs
Conceptual Example:
Malware Detection Graph:
- Nodes: Processes, Files, Registry Keys, Network Connections
- Edges: Creates, Modifies, Connects, Reads, Writes
- GNN learns: Patterns of malicious activity graphs
Combining Multiple Detectors:
class EnsembleDefender:
"""
Combines multiple ML models for robust malware detection
"""
def __init__(self):
self.detectors = {
'static': DeepMalwareDetector(),
'behavioral': BehavioralMalwareDetector(),
# In reality: signature-based, heuristic, sandbox, etc.
}
# Weights for each detector (learned from validation data)
self.weights = {
'static': 0.4,
'behavioral': 0.6
}
def detect(self, file_path):
"""Ensemble detection"""
print("=== Ensemble Malware Detection ===\n")
results = {}
# Run all detectors
print("Running individual detectors...\n")
# Static analysis
is_malware_static, conf_static = self.detectors['static'].detect(file_path)
results['static'] = {
'malware': is_malware_static,
'confidence': conf_static
}
print("\n" + "-"*60 + "\n")
# Behavioral analysis (simplified call)
observed = self.detectors['behavioral'].monitor_behavior(file_path)
family = self.detectors['behavioral'].classify_malware_family(observed)
results['behavioral'] = {
'malware': family != 'benign',
'confidence': 0.85 # Simplified
}
# Ensemble decision
print("\n" + "="*60)
print("\n=== Ensemble Decision ===\n")
weighted_score = 0
for detector, result in results.items():
vote = 1 if result['malware'] else 0
weighted_score += vote * result['confidence'] * self.weights[detector]
print(f"{detector.capitalize()} Detector:")
print(f" Vote: {'MALWARE' if result['malware'] else 'BENIGN'}")
print(f" Confidence: {result['confidence']:.2%}")
print(f" Weight: {self.weights[detector]:.2%}\n")
final_decision = weighted_score > 0.5
print(f"Final Weighted Score: {weighted_score:.2%}")
print(f"\n{'⚠ ⚠ ⚠ MALWARE DETECTED ⚠ ⚠ ⚠' if final_decision else '✓ ✓ ✓ FILE APPEARS SAFE ✓ ✓ ✓'}")
return final_decision, weighted_score
AI-Powered Threat Intelligence:
- Automated Threat Feed Processing:
- Ingest millions of threat indicators daily
- ML-based prioritization and correlation
- Anomaly detection in threat patterns
- Predictive Threat Intelligence:
- Predict next attack targets
- Forecast malware campaign evolution
- Early warning systems
- Automated Indicator Extraction:
- Extract IoCs from unstructured threat reports
- NLP for threat actor attribution
- LLMs for threat intelligence synthesis
Adversarial Evasion:
- Attackers can train against defensive models
- Adversarial malware specifically crafted to evade ML detectors
- Arms race between attackers and defenders
False Positives:
- ML models can misclassify legitimate software
- Trade-off between detection rate and false positive rate
- Critical in enterprise environments
Model Drift:
- Malware evolves, models become outdated
- Requires continuous retraining
- Concept drift in threat landscape
Interpretability:
- Deep learning models are "black boxes"
- Difficult to explain why file was flagged
- Compliance and legal requirements
Duration: 10 minutes
1. The Dual-Use Nature of AI in Security
- Every AI technique can be used for both attack and defense
- Understanding offensive techniques is essential for building defenses
- Ethical responsibility comes with technical capability
2. AI-Powered Attacks are Here
- Automated vulnerability discovery
- Intelligent exploit generation
- Sophisticated social engineering at scale
- These are not theoretical threats—they're being used now
3. AI-Powered Defenses are Essential
- Traditional security approaches cannot scale
- ML enables detection of novel attacks
- Adversarial ML makes defenses more robust
- Ensemble approaches provide depth
4. The Arms Race Continues
- Attackers will use AI more effectively
- Defenders must stay ahead through research and innovation
- Continuous learning and adaptation are required
- Collaboration and information sharing are critical
For Security Professionals:
- Learn ML fundamentals—it's no longer optional
- Understand both offensive and defensive AI techniques
- Invest in adversarial testing of your security systems
- Build robust, ensemble-based defenses
- Stay current with latest AI security research
For Organizations:
- Adopt AI-powered security tools
- Train security teams in ML
- Implement adversarial testing programs
- Invest in threat intelligence platforms
- Foster security research culture
For Researchers:
- Focus on robustness and adversarial ML
- Develop interpretable security models
- Address the scalability challenge
- Work on real-time detection systems
- Collaborate across academia and industry
- Ethics: Where should we draw the line in security research? What attack techniques should not be published even for defensive purposes?
- Responsibility: If you develop an AI tool that can be used for both attack and defense, how do you ensure it's used responsibly?
- Regulation: Should AI-powered security tools be regulated? How?
- Future: What do you think will be the next major development in AI-powered attacks or defenses?
- Career: How will AI change the cybersecurity job market in the next 5-10 years?
Next Week (Week 13): Edge AI & IoT Security
- How do resource constraints affect security?
- Unique vulnerabilities in edge AI systems
- Secure deployment strategies
Preparation:
- Review IoT threat landscape
- Read about TinyML and edge computing
- Consider: How does moving AI to the edge change the attack surface?
Research Papers:
- "Intriguing Properties of Adversarial Examples" (Szegedy et al.)
- "Explaining and Harnessing Adversarial Examples" (Goodfellow et al.)
- "Stealing Machine Learning Models via Prediction APIs" (Tramèr et al.)
Tools to Explore:
- Adversarial Robustness Toolbox (ART) by IBM
- CleverHans for adversarial ML
- Foolbox for adversarial attacks
- DeepFool, C&W attack implementations
Online Courses:
- Stanford CS329P: Practical ML Security
- MIT 6.S191: Introduction to Deep Learning
- Adversarial Machine Learning (Coursera)
CTF Challenges:
- AICrowd Adversarial ML Challenge
- NIPS Adversarial Vision Challenge
- DEF CON AI Village CTF
Objective: Build a simple fuzzer that uses ML to guide input mutation
Requirements:
- Choose a target program (simple parser, calculator, etc.)
- Implement basic fuzzing with random mutations
- Add ML component to learn effective mutations
- Compare coverage with and without ML guidance
- Document findings and lessons learned
Deliverable: Code, report, presentation
Objective: Develop an AI-powered phishing email detector
Requirements:
- Collect dataset of phishing and legitimate emails
- Extract features (sender, content, links, etc.)
- Train ML classifier (your choice of algorithm)
- Evaluate against adversarial phishing emails
- Implement defense against evasion
Deliverable: Working system, evaluation report
- Adversarially robust malware detector
- AI-powered penetration testing tool
- Deepfake detection system for security applications
- Automated vulnerability scanner with ML
- Behavioral analysis system for insider threats
All code examples from this tutorial are available at:
https://github.com/ucdenver-csci5773/week12-ai-security
Directory Structure:
week12-ai-security/
├── fuzzing/
│ ├── ml_guided_fuzzer.py
│ └── target_programs/
├── phishing/
│ ├── email_generator.py
│ ├── spam_filter.py
│ └── dataset/
├── malware_detection/
│ ├── static_analyzer.py
│ ├── behavioral_detector.py
│ └── ensemble_defender.py
├── adversarial_ml/
│ ├── adversarial_training.py
│ └── robustness_testing.py
└── README.md
End of Tutorial
Remember: With great power comes great responsibility. Use these techniques ethically and legally.
