Agent Skills in Claude Code and Visual Studio Code: Setup, Benefits, Risks, and a Step-by-Step Example
"Copilot wrote this." That sentence should be a signal to reviewers to look more carefully, not to relax and assume it's safe. The challenge with AI-generated code is that it has a smooth, professional surface that can hide deep problems. The code looks clean, runs well, and passes tests on the happy path. But beneath that polish, systematic patterns of failure and missing context lurk.
AI has a knowledge cutoff based on its training data. It doesn't know about security best practices established after training. It doesn't know about deprecated APIs. It doesn't understand newly-discovered vulnerabilities or modern security standards.
import * as jwt from 'jsonwebtoken';
function createToken(userId: string): string {
return jwt.sign(
{ userId },
process.env.JWT_SECRET!,
{ expiresIn: '7d', algorithm: 'HS256' } // Outdated pattern
);
}This code works technically, but violates modern JWT security standards (RFC 8725). The current best practice: short-lived Access Tokens (15-60 minutes) + separate Refresh Tokens (7 days). A 7-day Access Token means if someone steals the token, the attacker has access for 7 days.
Reviewer check: When seeing security-related code from AI, verify against current standards and OWASP recommendations, not just working code.
AI creates correct code in isolation, but it doesn't know your team's specific conventions, patterns, and accumulated decisions:
@GetMapping("/users")
public ResponseEntity<Page<UserDto>> getUsers(
@RequestParam(defaultValue = "0") int page,
@RequestParam(defaultValue = "20") int size) {
Page<User> users = userRepository.findAll(PageRequest.of(page, size));
Page<UserDto> dtos = users.map(UserDto::from);
return ResponseEntity.ok(dtos);
}This code works and follows REST principles correctly. But your team probably: - Wraps responses in `ApiResponse
AI doesn't see this context. It creates generic, textbook-correct code that doesn't fit your project.
When AI infers intent from names, it creates code that matches the requested name but misses crucial implementation details:
async function deleteUser(userId: string) {
await userRepository.update(userId, { deletedAt: new Date() });
}It's named "delete" and does implement deletion (soft delete). But your service's soft delete policy includes: - Session invalidation - OAuth token revocation - Notification to user - Data retention compliance - Related resource cleanup
This code skips all of those. The name matched the request, but the business logic was incomplete.
AI favors reusability and generic solutions. It might create discount logic handling percentage discounts, fixed amounts, buy-x-get-y, and tiered pricing when you only need simple percentage discounts today.
type DiscountType = 'PERCENTAGE' | 'FIXED_AMOUNT' | 'BUY_X_GET_Y' | 'TIERED';
interface DiscountConfig {
type: DiscountType;
value: number;
minQuantity?: number;
maxDiscount?: number;
tiers?: Array<{ threshold: number; rate: number }>;
}The code works, but you maintain complexity for features you don't need. YAGNI (You Aren't Gonna Need It) principle applies. Implement only what you need now. Expand when you need it.
@GetMapping("/products/{productId}")
public ProductDetail getProduct(@PathVariable String productId) {
return productService.findById(productId);
}AI assumes security has been handled elsewhere. But what if: - This endpoint should only be accessible to logged-in users? - Premium products need access control? - Certain users shouldn't see certain products?
There's no authentication or authorization check.
Require deep review:
Can trust more:
The worst AI-generated code is subtly wrong while looking obviously correct. It looks right, compiles, tests pass, but it's flawed in ways only revealed in production:
public boolean isTokenExpired(String token) {
try {
Claims claims = Jwts.parser()
.setSigningKey(secretKey)
.parseClaimsJws(token)
.getBody();
return claims.getExpiration().before(new Date());
} catch (ExpiredJwtException e) {
return true;
} catch (Exception e) {
return false; // WRONG: treats parsing failure as "not expired"
}
}Look at the final catch block. If the token is malformed, has a bad signature, or is garbage, it returns "not expired." An attacker sends random strings as tokens, and they pass authentication.
The code looks clean and well-structured. It might slip through casual review. But it's a critical vulnerability.
Step 1: Scan before reading. Before diving into the code, ask: What domain is this? Is it security-critical? What's the deployment context?
Step 2: Read the PR description. What does the AI claim it created? Do the claims match what you see in the diff?
Step 3: Check for red flags. Are there new security patterns? Hardcoded values? Complex dependencies?
Step 4: Verify assumptions. Does it assume context that might not exist? Does it handle failures?
Step 5: Check consistency. Does it match team conventions? Does it use the same patterns as existing code?
Step 6: Test assumptions locally. Run the code. Try edge cases. See if it behaves as intended.
AI code saves time when:
AI code wastes time when:
The time saved isn't in coding — it's in careful review of well-written scaffolding.
When AI generates code and you find issues, document your findings. This becomes valuable institutional knowledge:
AI Suggestion: Use ArrayList for general lists
Team decision: We prefer LinkedList for queue operations due to O(1) removal
AI Suggestion: Hardcode constants for simplicity
Team decision: All configuration goes to environment variables
AI Suggestion: Generic error handling
Team decision: We log with userId, requestId, and timestamp always
This documentation helps new team members understand why your codebase looks the way it does and prevents AI from making the same suggestions repeatedly.
Good teams use AI feedback as a teaching tool. When you correct AI-generated code, add a comment explaining why:
// AI generated: const users = userService.findAll();
// Better: const users = userService.findAllActive();
// Why: We should filter at the query level, not application layer.
// This scales better and is more semantically correct.
Over time, you develop better AI prompts and better understanding of what AI does well versus what needs human judgment.
When reviewing AI code, use a dedicated checklist:
This systematic approach catches more issues than casual review and makes your expectations clear to the author.
Over multiple PRs, you learn what AI does well in your codebase and what it consistently misses. Some teams find AI generates excellent utility functions but weak business logic. Others find the opposite. By tracking patterns in your reviews, you can adjust your approach. Invest more review time where AI consistently has issues.
Don't be suspiciously hostile to AI code, and don't unconditionally trust it. Apply different reliability levels to different parts.
The surface-level aspects (grammar, syntax, well-documented algorithms) can be trusted relatively more. But security context, business rules, team conventions, infrastructure assumptions, edge cases, error handling, and domain-specific logic — these absolutely must be verified by a human reviewer with project context and domain expertise.
Treat AI-generated code not as finished work, but as a starting point. Your careful review transforms it from "code that works in theory" to "code that's correct, safe, performant, resilient, and maintainable in your specific organizational context."
As AI tools improve, your role doesn't shrink — it evolves. AI might generate more code faster, but the need for thoughtful review increases. Someone still needs to verify business logic, catch security issues, ensure consistency, and maintain standards. That's you.
The best teams won't be the ones that use AI the most. They'll be the ones that use AI as a tool to speed up routine work while maintaining rigorous review standards for everything that matters.
Comments
Post a Comment