[Back to blog](https://zcp.zsoftly.ca/blog) [Tutorials](https://zcp.zsoftly.ca/blog/category/tutorials) # Zero-Trust Access with AWS SSM: Eliminating SSH Keys and Bastion Hosts Forever Learn how AWS Systems Manager Session Manager enables secure access without SSH keys, open ports, or bastion hosts. Includes practical IAM policy templates, cost savings analysis, and complete migration strategy from traditional SSH to SSM-based access. Staff at ZSoftly January 18, 2025 · Updated June 19, 2026 8 min read Share ![Padlock on keyboard representing zero trust security and access control](https://images.unsplash.com/photo-1614064641938-3bbee52942c7?w=1200&h=600&fit=crop) ## The SSH Key Management Nightmare Every DevOps team has experienced these scenarios: - An engineer leaves the company. How many servers still have their SSH key? - An SSH key gets committed to Git. Now what? - Compliance asks: “How do you audit SSH access?” Silence. - A bastion host goes down. Production access: blocked. Traditional SSH access creates cascading security and operational problems. ### Security Risks of Traditional SSH - Key lifecycle management: SSH keys are difficult to rotate and revoke at scale - Persistent access: Stolen keys provide ongoing access until discovered - No MFA support: Cannot enforce multi-factor authentication on SSH connections - Limited audit trail: Difficult to know who accessed what and when ### Operational Overhead - Bastion maintenance: Patching, monitoring, high availability - Attack surface: Open port 22 invites brute-force attacks - Key distribution: Getting keys to the right people at the right time - Cost: $100-500/month per bastion host --- ## TL;DR SSH keys create cascading security problems—committed to Git, difficult to rotate, no MFA support, limited audit trails. Bastion hosts add operational overhead and cost $100-500/month. AWS Systems Manager Session Manager eliminates both by providing IAM-based, zero-trust access without SSH keys, open ports, or bastion infrastructure. The Transformation: Replace SSH key management and bastion hosts with SSM’s IAM-controlled access. Users authenticate via AWS credentials with MFA, IAM evaluates tag-based policies, sessions logged to CloudTrail and CloudWatch. Key Takeaways: 1. Zero SSH keys, zero open ports - Access controlled through IAM policies. No port 22/3389 security group rules. Instant revocation by removing IAM permissions. 2. IAM-based access with fine-grained control - Tag-based conditions restrict access by environment. Time-based policies for scheduled access. MFA requirements for production. 3. Full audit trail for compliance - CloudTrail logs every session start, command, termination. Session output to CloudWatch/S3. Complete evidence for SOC 2, PCI-DSS, HIPAA. 4. Cost savings: $1,109-$7,289 annually - Eliminate bastion hosts ($15-2,000/month), VPN solutions ($50-500/month), SSH key tools ($20-100/month). SSM is free. Real Results: Mid-Market SaaS with 200 instances saved $18,000/year, achieved 100% MFA enforcement, SOC 2 Type II compliance, recovered 5 hours/week on access management. Core Principle: Least privilege with context—grant minimum necessary access based on role, environment tags, time, and MFA status. Design for auditability from day one. --- ## AWS Systems Manager Session Manager AWS SSM Session Manager provides secure, auditable, zero-trust access to EC2 instances without SSH keys, open ports, or bastion hosts. ### How It Works The access flow is fundamentally different from SSH: 1. User authenticates to AWS via IAM credentials (supports MFA, temporary credentials) 2. IAM evaluates permissions based on policies, conditions, and tags 3. SSM API initiates session through the AWS backbone (not public internet) 4. SSM Agent on instance receives session and provides shell access 5. CloudTrail logs everything for complete audit trail ### The Benefits Zero SSH Keys - No keys to generate, distribute, rotate, or revoke - Access controlled entirely through IAM policies - Instant revocation by removing IAM permissions Zero Open Ports - No security group rules for port 22 or 3389 - Instances have no inbound access from the internet - Eliminates entire class of network-based attacks Full Audit Trail - CloudTrail logs every session start, command, and termination - Session output logged to CloudWatch Logs or S3 - Complete compliance evidence for auditors IAM-Based Access Control - Use existing IAM roles, policies, and groups - Tag-based conditions (e.g., only access instances tagged “dev”) - Require MFA for production access Cost Savings - Eliminate bastion hosts: $500-2000/month - No SSH key management tools needed - SSM Session Manager itself is free ## IAM Policy Design Principles ### The Principle: Least Privilege with Context Design policies that grant minimum necessary access based on role and context. Use conditions to restrict access by environment, time, or source. ### Role-Based Access Patterns Developer Role: Development/Staging Only Developers should access dev and staging environments but not production. Use tag-based conditions to allow `ssm:StartSession` on instances tagged `Environment: dev` or `Environment: staging`, deny access to production instances, and ensure users manage only their own sessions. DevOps Role: Full Access with MFA Operations teams need broader access but with additional controls. Allow access to all instances, require MFA for session start (use `aws:MultiFactorAuthPresent` condition), and allow viewing and managing all active sessions. Security/Audit Role: Read-Only Security teams need visibility without shell access. Allow `DescribeSessions`, `GetConnectionStatus`, `DescribeInstanceInformation`, explicitly deny `StartSession` and `TerminateSession`, and provide audit-only access for compliance. ### Condition Keys for Fine-Grained Control Use IAM condition keys to add context to access decisions: - Tag-based: `ssm:resourceTag/Environment` - Time-based: `aws:CurrentTime` - MFA-based: `aws:MultiFactorAuthPresent` - Source IP: `aws:SourceIp` ## SSM Agent Prerequisites ### The Principle: Agent-Based Security SSM requires an agent running on each instance. This is your security foundation. Pre-installed AMIs - Amazon Linux 2 and 2023 - Windows Server 2016+ - Most recent AWS-published AMIs Manual Installation - Ubuntu/Debian: Install from AWS S3 - CentOS/RHEL: Install from AWS S3 - Verify status with systemctl IAM Instance Profile - Instances need the `AmazonSSMManagedInstanceCore` managed policy - This allows the agent to communicate with SSM endpoints - Without this, instances won’t appear in Session Manager Network Requirements - Instances need outbound HTTPS (443) to SSM endpoints - Options: NAT Gateway, VPC Endpoints, or public IP - VPC Endpoints recommended for private subnets (no internet required) ## Session Logging Configuration ### The Principle: Complete Audit Trail Every session should be logged for security and compliance. Configure logging to CloudWatch Logs or S3. Logging Options: - CloudWatch Logs: Real-time streaming, easy querying, integrates with alarms - S3: Long-term storage, cost-effective, integrates with Athena for analysis What Gets Logged: - Session start/stop timestamps - User identity (IAM principal) - Source IP address - All commands executed (if enabled) - Command output (if enabled) Enable KMS Encryption Encrypt session logs with customer-managed KMS keys for sensitive environments. ## Migration Strategy: SSH to SSM ### Phase 1: Parallel Run (Weeks 1-2) Goal: Run SSM alongside existing SSH access Tasks: 1. Install SSM agent on all instances 2. Grant IAM policies to DevOps team 3. Train team on SSM access methods 4. Monitor usage: SSM vs. SSH Success Metrics: - 100% of instances managed by SSM - 50%+ of access via SSM ### Phase 2: Gradual Cutover (Weeks 3-4) Goal: Shift majority of access to SSM Tasks: 1. Remove SSH keys from non-emergency accounts 2. Keep break-glass SSH access for emergencies only 3. Migrate automation scripts to use SSM 4. Update runbooks and documentation Success Metrics: - 90%+ of access via SSM - Zero SSH-related incidents ### Phase 3: Complete Migration (Weeks 5-6) Goal: Eliminate SSH access entirely Tasks: 1. Remove security group rules for port 22 2. Decommission bastion hosts 3. Revoke all SSH keys 4. Enable comprehensive session logging Success Metrics: - 100% SSM adoption - Bastion costs eliminated - Full audit trail in place ## Cost Savings Analysis ### Before: Traditional SSH + Bastion | Item | Monthly Cost | | --- | --- | | Bastion host (t3.small) | $15 | | ALB for high availability | $16 | | Elastic IP | $3.65 | | VPN solution | $50-500 | | SSH key management tools | $20-100 | | Total | $104.65-634.65 | ### After: SSM Session Manager | Item | Monthly Cost | | --- | --- | | SSM Session Manager | FREE | | VPC Endpoints (optional) | $7.20 | | CloudWatch Logs (optional) | $5-20 | | Total | $12.20-27.20 | Annual Savings: $1,109 - $7,289 ## Real-World Success Story Company: Mid-market SaaS, 200 EC2 instances Challenge: SSH key sprawl, compliance gaps, costly bastion infrastructure Solution: Complete migration to SSM Session Manager Results: - Security: 100% MFA enforcement, zero SSH keys - Compliance: Full audit trail for SOC 2 Type II - Cost: $18,000/year saved (bastion + VPN) - Productivity: 5 hours/week saved on access management ## Advanced Use Cases ### Port Forwarding for Database Access SSM supports port forwarding to access private resources: - Forward RDS port through an EC2 instance - Access databases without exposing them to the internet - Full audit trail of who accessed which database ### Fleet-Wide Command Execution Run commands across multiple instances simultaneously: - Patch all web servers in one command - Collect diagnostics from an entire fleet - Enforce configuration changes immediately ### IDE Integration Configure your IDE to use SSM for remote development: - VS Code Remote SSH with SSM - JetBrains Gateway through SSM tunnels - Same security benefits as interactive sessions ## Security Best Practices 1. Always Enable MFA for production access 2. Use VPC Endpoints for fully private communication 3. Enable Session Logging to S3 and CloudWatch 4. Implement Least Privilege with tag-based conditions 5. Regular Audits of session access patterns 6. Session Timeout for idle sessions 7. Encryption with customer-managed KMS keys ## Troubleshooting Principles ### Instance Not Appearing in Session Manager Check in order: 1. SSM Agent running? (systemctl status) 2. IAM Instance Profile attached with correct policy? 3. Network connectivity to SSM endpoints? 4. Instance in correct region/account? ### Access Denied Errors Check in order: 1. User has `ssm:StartSession` permission? 2. Resource ARN matches policy? 3. Conditions satisfied (tags, MFA)? 4. Session document permissions? ## Next Steps Ready to eliminate SSH keys and bastion hosts from your infrastructure? ZSoftly built and operates ZCP, a sovereign Canadian cloud, and our [Professional Services](https://zcp.zsoftly.ca/professional-services) team helps businesses implement secure, zero-trust access: - Migration planning and execution - IAM policy design and implementation - Session logging and monitoring setup - Team training and documentation Get started today: [Talk to our team](https://zcp.zsoftly.ca/contact) or explore our [services](https://zcp.zsoftly.ca/services). --- Next in series: “Bypassing VPN Failure with AWS SSM Port Forwarding” ## Related articles Tutorials June 19, 2026 ### [Deploy Dokploy on ZCP: Launch a VM and Self-Host Your Apps](https://zcp.zsoftly.ca/blog/deploy-your-first-vm) A step-by-step guide to launching a virtual machine and installing Dokploy on ZSoftly Cloud Platform, entirely from the portal. Self-host your apps and databases with Git deploys and automatic TLS. Tutorials May 16, 2026 ### [Managed DNS on ZCP: how it works and how to set up a domain](https://zcp.zsoftly.ca/blog/dns-on-zcp) A walkthrough of the managed DNS service on ZCP, the two-nameserver architecture behind it, and the portal steps to create and manage a zone. Tutorials May 16, 2026 ### [VMware to Apache CloudStack: a step-by-step migration guide](https://zcp.zsoftly.ca/blog/vmware-to-cloudstack-migration-guide) A practical, step-by-step guide to moving VMware vSphere workloads to Apache CloudStack + KVM. What ZSoftly handles end to end and what is happening underneath.