DOCSEARCHbyAlgolia
Submit Docs

Amazon Web Services (AWS) Documentation

website·Indexed 34 days ago

https://docs.outerbounds.com

Comprehensive technical documentation for AWS services, including Amazon RDS, ECR, Athena, SageMaker AI, EventBridge, EKS, Bedrock, and Secrets Manager. Includes guides for the AWS CLI, IAM security best practices, and infrastructure management for AI/ML and HPC workloads.

Tokens
28.9K
Snippets
81
Records
165
Agent score
64%

What's inside AWS

  1. Overview of IAM Roles for Service Accounts (IRSA)

    IAM Roles for Service Accounts (IRSA) allows you to associate an IAM role with a Kubernetes service account. Pods configured to use that service account can then make AWS API requests using the permissions defined in the IAM role. This replaces the need to distribute static AWS credentials to containers or rely on the Amazon EC2 instance profile (node role).

    Key benefits include:

    • Least Privilege: Permissions are scoped to specific service accounts rather than the entire node.
    • Credential Isolation: Pods can only retrieve credentials for their assigned role, preventing access to other Pods' credentials (provided IMDS access is restricted).
    • Auditability: All requests are logged via AWS CloudTrail.

    Constraints: IRSA cannot be used with local clusters for Amazon EKS on AWS Outposts.

  2. Overview of AWS Secrets Manager

    AWS Secrets Manager is a service used to manage, retrieve, and rotate secrets throughout their lifecycle. It is designed to replace hard-coded credentials in application source code with dynamic runtime calls, reducing the risk of credential compromise. Supported secret types include database credentials, application credentials, OAuth tokens, and API keys. Key features include automatic rotation schedules, which allow long-term secrets to be replaced by short-term ones without requiring application redeployment.

  3. Overview of legacy scheduled rules in Amazon EventBridge

    Scheduled rules are a legacy feature of Amazon EventBridge. They allow you to trigger targets at regular intervals or specific times. All scheduled events use the UTC+0 time zone, and the minimum precision is one minute. Note that there may be a delay of several seconds between the trigger time and the target execution due to the distributed nature of the service.

    AWS recommends using EventBridge Scheduler instead of legacy scheduled rules for improved scalability, more flexible time windows, retry limits, and a wider set of target API operations.

  4. Overview of Elastic Fabric Adapter (EFA) for AI/ML and HPC

    The Elastic Fabric Adapter (EFA) is a network device for Amazon EC2 designed to accelerate AI, Machine Learning (ML), and High Performance Computing (HPC) applications. It provides lower and more consistent latency and higher throughput than traditional TCP transport by utilizing the Scalable Reliable Datagram (SRD) protocol and OS-bypass capabilities. EFA is optimized for inter-instance communication and integrates with Libfabric to bypass the operating system kernel, reducing overhead for high-scale workloads.

  5. Overview of exporting Amazon RDS DB snapshots to S3

    Amazon RDS allows exporting DB snapshot data (manual, automated, or AWS Backup snapshots) to an Amazon S3 bucket. The process runs in the background and does not impact active database performance. Data is stored in a compressed, consistent Apache Parquet format.

    Key Constraints:

    • Exported data cannot be restored back to a new DB instance or imported into an existing one.
    • The target S3 bucket must be in the same AWS Region as the snapshot.
    • Exported data is intended for analysis via Amazon Athena, Amazon Redshift Spectrum, or transformation via AWS Glue for subsequent import via AWS DMS or custom scripts.

  6. Understand Amazon Athena data source architecture

    Amazon Athena queries data stored in various locations and formats (CSV, JSON, Avro, Parquet, etc.) by using a combination of a dataset and a data catalog.

    Key concepts include:

    • Dataset: The actual raw data stored in a specific format.
    • Metadata/Schema: Data describing the dataset, such as table names, column names, and data types.
    • Data Catalog (Metastore): The system used to organize metadata.
    • Data Source: The combination of a dataset and the data catalog that describes it.

    Athena natively supports the AWS Glue Data Catalog, which can describe datasets in Amazon S3, Amazon Redshift, and Amazon DynamoDB. For other sources, Athena uses connectors or supports external Hive metastores.

  7. Understand AWS Batch multi-node parallel (MNP) jobs

    Multi-node parallel (MNP) jobs, or gang scheduling, allow a single AWS Batch job to span multiple Amazon EC2 instances. This is used for high-performance computing (HPC) and distributed GPU training using frameworks that support IP-based inter-node communication (e.g., Apache MXNet, TensorFlow, Caffe2, or MPI).

    Key characteristics include:

    • Single-Tenant: Only one job container runs per EC2 instance.
    • Node Structure: A job consists of one 'main node' (launched first) and multiple 'child nodes'.
    • Lifecycle: The job is considered finished only when the main node exits. Once the main node exits, all child nodes are stopped.
    • Job Status: The final status (SUCCEEDED or FAILED) is determined by the main node. Retries are triggered by main node failures, not child node failures.

  8. Permissions for describing SageMaker AI resources

    To retrieve details about SageMaker AI resources using Describe* API calls, you must grant the corresponding sagemaker:Describe* permission for the specific resource ARN. Some operations, such as describing subscribed workteams, require additional AWS Marketplace permissions.
    [
  { "action": "sagemaker:DescribeApp", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:app/{{domain-id}}/{{user-profile-name}}/{{app-type}}/{{appName}}" },
  { "action": "sagemaker:DescribeAppImageConfig", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:app-image-config/{{appImageConfigName}}" },
  { "action": "sagemaker:DescribeAutoMLJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:automl-job/{{autoMLJobName}}" },
  { "action": "sagemaker:DescribeDomain", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:domain/{{domainId}}" },
  { "action": "sagemaker:DescribeEndpoint", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:endpoint/{{endpointName}}" },
  { "action": "sagemaker:DescribeEndpointConfig", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:endpoint-config/{{endpointConfigName}}" },
  { "action": "sagemaker:DescribeFlowDefinition", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:flow-definition/{{flowDefinitionName}}" },
  { "action": "sagemaker:DescribeHumanLoop", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:human-loop/{{humanLoopName}}" },
  { "action": "sagemaker:DescribeHumanTaskUi", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:human-task-ui/{{humanTaskUiName}}" },
  { "action": "sagemaker:DescribeHyperParameterTuningJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:hyper-parameter-tuning-job/{{hyperParameterTuningJob}}" },
  { "action": "sagemaker:DescribeImage", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:image/{{imageName}}" },
  { "action": "sagemaker:DescribeImageVersion", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:image-version/{{imageName}}/{{versionNumber}}" },
  { "action": "sagemaker:DescribeLabelingJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:labeling-job/{{labelingJobName}}" },
  { "action": "sagemaker:DescribeModel", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:model/{{modelName}}" },
  { "action": "sagemaker:DescribeModelPackage", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:model-package/{{modelPackageName}}" },
  { "action": "sagemaker:DescribeModelPackageGroup", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:model-package-group/{{modelPackageGroupName}}" },
  { "action": "sagemaker:DescribeNotebookInstance", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:notebook-instance/{{notebookInstanceName}}" },
  { "action": "sagemaker:DescribePipeline", "resource": "arn:{{aws-partition}}:sagemaker:{{region}}:{{account-id}}:pipeline/{{pipeline-name}}" },
  { "action": "sagemaker:DescribePipelineExecution", "resource": "arn:{{aws-partition}}:sagemaker:{{region}}:{{account-id}}:pipeline/{{pipeline-name}}/execution/{{execution-id}}" },
  { "action": "sagemaker:DescribeProcessingJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:processing-job/{{processingjobname}}" },
  { "action": "sagemaker:DescribeSpace", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:space/{{domain-id}}/{{spaceName}}" },
  { "action": "sagemaker:DescribeTrainingJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:training-job/{{trainingjobname}}" },
  { "action": "sagemaker:DescribeTransformJob", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:transform-job/{{transformjobname}}" },
  { "action": "sagemaker:DescribeUserProfile", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:user-profile/domain-id/{{userProfileName}}" },
  { "action": "sagemaker:DescribeWorkforce", "resource": "arn:aws:sagemaker:{{region}}:{{account-id}}:workforce/*" }
]

  9. Enable Elastic Fabric Adapter (EFA) on Amazon EC2

    Elastic Fabric Adapter (EFA) is an optional networking feature available for supported Amazon EC2 instance types. It can be enabled on any supported instance at no additional cost.

  10. Update AWS CLI to the latest version

    If the AWS CLI was installed via the official install script or an official installer (Linux command line, macOS PKG, or Windows MSI), you can update it using the aws update command. If the CLI was installed for all users, this command must be run from an elevated shell (using sudo on Linux/macOS or an Administrator PowerShell prompt on Windows). The update process preserves the existing installation scope (all users vs. current user).
    aws update

  11. Create a PagerDuty Service

    A technical service in PagerDuty represents a discrete piece of functionality owned by a team. To create a service, navigate to Services $\rightarrow$ Service Directory and click New Service. You must provide a Name and Description, and assign either a new or existing escalation policy. After configuration, you can optionally add integrations to receive events from external tools.

  12. Manage and rotate long-term IAM access keys

    While temporary credentials are preferred, some use cases require long-term programmatic access keys. In these cases, access keys should be updated/rotated when necessary (e.g., when an employee leaves the company). Use the 'IAM access last used information' to determine which keys can be safely updated or removed without disrupting services.