| name | transfer-family-diagnostics |
| version | 1.0.0 |
| last_updated | 2025-04-12 |
| description | Use this skill to investigate and troubleshoot AWS Transfer Family problems by analyzing server configurations, authentication settings, storage backends, directory mappings, connectivity, and following structured runbooks. Activate when: server creation failures, endpoint configuration issues, protocol misconfiguration, service-managed auth failures, custom IdP errors, SSH key problems, connection failures, firewall/NAT issues, passive FTP problems, S3 backend errors, EFS backend errors, directory mapping issues, home directory configuration, throughput limits, large file transfer failures, host key management, encryption in transit, managed workflow failures, post-upload processing errors, or the user says something is wrong with Transfer Family without naming specific symptoms.
|
| compatibility | Requires AWS CLI or SDK access with Transfer Family, S3, EFS, IAM, Lambda, API Gateway, CloudWatch, CloudTrail, EC2 (security groups, VPC endpoints), and optionally Route 53 and ACM permissions.
|
Transfer Family Diagnostics
When to use
Any Transfer Family investigation where the console alone is insufficient — server configuration analysis, authentication debugging, connectivity troubleshooting, storage backend issues, directory mapping problems, performance optimization, security configuration, workflow failures, or protocol-specific issues.
Investigation workflow
Step 1 — Collect and triage
aws transfer list-servers --query 'Servers[*].{ServerId:ServerId,State:State,EndpointType:EndpointType,Protocols:Protocols}'
aws transfer describe-server --server-id <server-id>
aws transfer list-users --server-id <server-id>
aws transfer describe-user --server-id <server-id> --user-name <username>
aws cloudwatch get-metric-statistics --namespace AWS/Transfer --metric-name FilesIn --dimensions Name=ServerId,Value=<server-id> --start-time <start> --end-time <end> --period 300 --statistics Sum
aws cloudwatch get-metric-statistics --namespace AWS/Transfer --metric-name BytesIn --dimensions Name=ServerId,Value=<server-id> --start-time <start> --end-time <end> --period 300 --statistics Sum
Step 2 — Domain deep dive
aws transfer describe-server --server-id <server-id> --query 'Server.IdentityProviderDetails'
aws transfer describe-server --server-id <server-id> --query 'Server.EndpointDetails'
aws transfer describe-server --server-id <server-id> --query 'Server.ProtocolDetails'
aws transfer describe-server --server-id <server-id> --query 'Server.WorkflowDetails'
aws transfer describe-user --server-id <server-id> --user-name <username> --query 'User.HomeDirectoryMappings'
aws transfer list-executions --workflow-id <workflow-id> --query 'Executions[*].{ExecutionId:ExecutionId,Status:Status}'
Step 3 — Detailed investigation
aws transfer describe-execution --execution-id <execution-id> --workflow-id <workflow-id>
aws transfer describe-server --server-id <server-id> --query 'Server.StructuredLogDestinations'
aws transfer list-host-keys --server-id <server-id>
aws transfer describe-host-key --server-id <server-id> --host-key-id <host-key-id>
aws ec2 describe-security-groups --group-ids <sg-id>
aws ec2 describe-vpc-endpoints --vpc-endpoint-ids <vpce-id>
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventSource,AttributeValue=transfer.amazonaws.com --max-results 10
aws logs filter-log-events --log-group-name /aws/transfer/<server-id> --start-time <epoch-ms> --end-time <epoch-ms>
Read references/transfer-family-guardrails.md before concluding on any Transfer Family issue.
Tool quick reference
| Tool / API | When to use |
|---|
list-servers | List all Transfer Family servers |
describe-server | Full server configuration details |
list-users | List users on a server |
describe-user | User configuration, home directory, role |
list-executions | Check workflow execution status |
describe-execution | Workflow execution details and errors |
list-host-keys | Server host key inventory |
describe-host-key | Host key details and fingerprint |
test-identity-provider | Test custom IdP authentication |
describe-security-policy | Security policy cipher/protocol details |
list-connectors | List SFTP connectors |
describe-connector | Connector configuration details |
Gotchas: Transfer Family
- Endpoint types: PUBLIC endpoints get an AWS-managed IP that can change. VPC endpoints get static private IPs via ENIs. VPC_ENDPOINT (legacy) uses AWS PrivateLink. For static public IPs, use VPC endpoint type with an internet-facing NLB or Elastic IPs on the ENIs.
- Authentication modes: SERVICE_MANAGED stores SSH keys in the Transfer Family service. Custom IdP uses a Lambda function or API Gateway to authenticate. You cannot mix service-managed and custom IdP on the same server. Custom IdP must return a JSON response with Role, HomeDirectory, and optionally Policy and PublicKeys.
- S3 vs EFS backend: S3 supports logical directory mappings, session policies, and object tagging. EFS requires a POSIX profile (Uid/Gid) on each user. EFS does not support logical directory mappings — only physical home directories. S3 does not support POSIX file permissions.
- Directory mappings: Logical directory mappings (HomeDirectoryType=LOGICAL) map virtual paths to S3 or EFS physical paths. The Entry field is the virtual path the user sees; the Target is the actual S3 bucket/prefix or EFS path. Users are chroot-jailed to their mapped directories. Physical home directories (HomeDirectoryType=PATH) set a single home directory without chroot.
- Host key management: SFTP clients cache the server host key fingerprint. If you replace a server or import a new host key, clients will see a "host key mismatch" warning. Export the host key from the old server and import it to the new one to avoid disruption. Each server can have multiple host keys — the most recently added active key is used.
- Passive mode FTP: FTP and FTPS use passive mode by default in Transfer Family. The server sends its endpoint IP in the PASV response. For VPC endpoints behind a NAT or NLB, the PassiveIp must be set in ProtocolDetails to the public-facing IP. Otherwise, clients receive a private IP they cannot reach.
- Protocol differences: SFTP runs over SSH (port 22). FTPS is FTP over TLS (port 21 for control, 8192-8200 for data). FTP is unencrypted (port 21, data 8192-8200). A single server can support multiple protocols, but each protocol has different security implications and port requirements.
- Bandwidth throttling: Transfer Family does not have built-in per-user bandwidth throttling. Throughput depends on the endpoint type, instance size (for VPC endpoints), and the backend storage. S3 supports 5 Gbps per prefix per partition. EFS throughput depends on the file system mode (bursting vs provisioned vs elastic).
- IP whitelisting: Only VPC endpoint type supports security groups for IP whitelisting. PUBLIC endpoint type does not support security groups — use S3 bucket policies or session policies to restrict by source IP. For VPC endpoints, attach security groups to the VPC endpoint ENIs.
- Structured logging: Transfer Family supports structured JSON logging to CloudWatch Logs. Enable via StructuredLogDestinations on the server. Logs include authentication events, file operations (upload, download, delete, rename), and errors. Log group format: /aws/transfer/.
Anti-hallucination rules
- Always cite specific server configurations, user settings, or API responses as evidence.
- Endpoint types have different capabilities — PUBLIC has no security groups, VPC has static IPs. Never conflate them.
- Custom IdP response format is strict — Role and HomeDirectory are required. Never claim optional fields are sufficient.
- EFS does not support logical directory mappings. Never recommend logical mappings for EFS backends.
- PassiveIp must be set for FTP/FTPS behind NAT/NLB. Never assume passive mode works automatically in all network topologies.
- Spend no more than 2 minutes on any single hypothesis. Pivot if inconclusive.
24 runbooks
| Category | IDs | Covers |
|---|
| A — Server | A1-A3 | Server creation failures, endpoint issues, protocol configuration |
| B — Authentication | B1-B3 | Service-managed auth, custom IdP failures, SSH key issues |
| C — Connectivity | C1-C3 | Connection failures, firewall/NAT, passive FTP |
| D — Storage | D1-D2 | S3 backend issues, EFS backend issues |
| E — Directory | E1-E2 | Directory mapping, home directory configuration |
| F — Performance | F1-F2 | Throughput limits, large file transfers |
| G — Security | G1-G2 | Host key management, encryption in transit |
| H — Workflows | H1-H2 | Managed workflow failures, post-upload processing |
| Z — Catch-All | Z1 | General troubleshooting |
