Amazonica Clojure Client

Amazonica provides a comprehensive Clojure client for a wide range of Amazon AWS APIs. Supported services include, but are not limited to:

• Amplify, API Gateway, AppConfig, Application Insights, App Mesh, Augmented AI
• Autoscaling, Autoscaling Plans, Backup, Batch, Budgets
• Certificate Manager, CloudDirectory, CloudFormation, CloudFront, CloudSearch, CloudSearchV2, CloudSearchDomain, CloudWatch, CloudWatchEvents
• CodeBuild, CodeCommit, CodeDeploy, CodePipeline, CodeStar, Cognito, CognitoIdentityProviders, Comprehend, Compute Optimizer, Config, Connect, CostAndUsageReport, CostExplorer
• DatabaseMigrationService, DataPipeline, Data Exchange, Data Sync, Dax, Detective, DeviceFarm, DirectConnect, Directory, DLM, DocDB, DynamoDBV2
• EC2, EC2InstanceConnect, ECR, ECS, ElastiCache, ElasticBeanstalk, ElasticFileSystem, ElasticLoadBalancing, ElasticMapReduce, Elasticsearch, ElasticTranscoder, Event Bridge, Forecast, Fraud Detector
• GameLift, Glacier, Global Accelerator, Glue, GreenGrass, Groundstation, GuardDuty
• IdentityManagement, Image Builder, ImportExport, IoT
• Kafka, Kendra, Kinesis, Kinesis Analytics, KinesisFirehose, Kinesis Video Streams with WebRTC (Signaling Channels), KMS
• Lake Formation, Lambda, Lex, Lightsail, Logs
• MachineLearning, Macie, Managed Blockcahin, MechanicalTurk, MediaConvert, MediaLive, MediaPackage, MediaStore, MigrationHub, Mobile, MQ, MSK (Managed Kafka)
• OpsWorks, Personalize, Pinpoint, Pricing, Polly, QLDB, Quicksight
• RDS, Redshift, Rekognition, Route53, Route53Domains, S3
• Sagemaker, Secrets Manager, Security Hub, Security Token, ServerMigration, ServiceCatalog, Service Discovery, Shield, SimpleDB, SimpleEmail, SimpleSystemsManager, SimpleWorkflow, Snowball, SNS, SQS, StepFunctions, StorageGateway, Support
• Textract, Timestream, Transcribe, Transfer, Translate, WAF, Workspaces, XRay

Amazonica uses reflection to wrap the AWS Java SDK. It creates idiomatically named Clojure Vars in service-specific namespaces.

Mapping Rules:

• Naming: camelCase Java methods are converted to lower-case, hyphenated Clojure functions (e.g., createSnapshot() becomes create-snapshot).
• Arguments: If a Java method takes a Request object (a bean), you pass the bean's properties as keyword arguments to the Clojure function.
• Overloads: If a Java method is overloaded (e.g., one version takes no arguments and another takes a Request object), the Clojure function will be variadic, allowing both (function) and (function :key value) calls.

Example: To call createSnapshot(CreateSnapshotRequest request):

(create-snapshot :volume-id "vol-8a4857fa" :description "my_new_snapshot")

(ns com.example
  (:use [amazonica.aws.ec2]))

(describe-instances)

(create-snapshot :volume-id   "vol-8a4857fa"
                 :description "my_new_snapshot")

Amazonica provides support for both legacy CloudSearch and CloudSearchV2.

For CloudSearchV2, use amazonica.aws.cloudsearchv2 to create domains, index documents, and build suggesters.

For CloudSearchDomain (the low-level service), you can use amazonica.aws.cloudsearchv2/describe-domains to find endpoints, then use set-endpoint to switch between the document and search service endpoints for operations like upload-documents, search, and suggest.

;; CloudSearchV2 example
(ns com.example
  (:use [amazonica.aws.cloudsearchv2]))

(create-domain :domain-name "my-index")
(index-documents :domain-name "my-index")
(build-suggesters :domain-name "my-index")
(list-domains)

;; CloudSearchDomain endpoint switching example
(csd/set-endpoint "doc-domain-name-6fihexkq1234567895wm.us-east-1.cloudsearch.amazonaws.com")
(csd/upload-documents :content-type "application/json" :documents (io/input-stream json-documents))

(csd/set-endpoint "search-domain-name-6fihexkq1234567895wm.us-east-1.cloudsearch.amazonaws.com")
(csd/search :query "drumpf")

Use the amazonica.aws.kinesisfirehose namespace to manage delivery streams.

Common Tasks:

• List streams: list-delivery-streams returns a map containing :delivery-stream-names and :has-more-delivery-streams.
• Create stream: create-delivery-stream requires :delivery-stream-name and :s3DestinationConfiguration (including :role-arn and :bucket-arn).
• Update destination: update-destination allows modifying the S3 configuration, including :BufferingHints (:IntervalInSeconds, :SizeInMBs), :CompressionFormat, and :Prefix.

(ns com.example
  (:require [amazonica.aws.kinesisfirehose :as fh])
  (:import [java.nio ByteBuffer]))

;; List
(fh/list-delivery-streams)

;; Create
(fh/create-delivery-stream :delivery-stream-name "my-test-firehose-2"
                           :s3DestinationConfiguration {:role-arn  "arn:aws:iam:xxxx:role/firehose_delivery_role",
                                                        :bucket-arn "arn:aws:s3:::my-test-bucket"})

;; Put batch (converts to ByteBuffer if possible)
(fh/put-record-batch cred stream-name [[1 2 3 4]["test" 2 3 4] "\"test\",2,3,4" (ByteBuffer. (.getBytes "test,2,3,4"))])

The worker! function is the preferred way to consume Kinesis shards. It manages the lifecycle and checkpointing of the stream consumption.

Configuration Options:

• :app: A unique name for the worker application.
• :stream: The name of the Kinesis stream.
• :processor: A function (fn [records] ...) that receives a sequence of records. To manually control checkpointing, set :checkpoint false and return true from the processor function only when you want a checkpoint to be triggered.
• :checkpoint: (Optional) A numeric value for duration in seconds between checkpoints. If not provided, it defaults to every 60 seconds.
• :credentials: (Optional) Custom authentication. Defaults to the standard Amazonica scheme.
• :dynamodb-adaptor-client?: (Optional) Set to true when consuming streams from DynamoDB.

(worker! :app "app-name"
         :stream "my-stream"
         :checkpoint false
         :processor (fn [records]
                      (doseq [row records]
                        (println (:data row) (:sequence-number row) (:partition-key row)))))

Amazonica provides high-level functions for interacting with Kinesis streams.

Writing Records

• Use put-record to send a single record. If the data is not a java.nio.ByteBuffer, Amazonica transparently serializes and compresses it using Nippy.
• Use put-records for bulk uploads by passing a sequence of maps containing :partition-key and :data.

Reading Records

• Use get-records to read from a specific shard. If you use java.nio.ByteBuffer for data, you must provide a :deserializer function.
• If no :deserializer is provided, Amazonica assumes the data was serialized/compressed via Nippy (e.g., Snappy).

;; Single record
(put-record "my-stream" {:name "data" :col #{"a"}} (str (UUID/randomUUID)))

;; Bulk records
(put-records "my-stream" [{:partition-key "pk1" :data ["foo"]}])

;; Reading with a custom deserializer
(defn- get-raw-bytes [byte-buffer]
  (let [b (byte-array (.remaining byte-buffer))]
    (.get byte-buffer b)
    b))

(get-records :deserializer get-raw-bytes
             :shard-iterator (get-shard-iterator "my-stream" shard-id "TRIM_HORIZON"))

Use amazonica.aws.s3 and amazonica.aws.s3transfer for object storage operations.

Key Operations:

• Create Bucket: create-bucket "bucket-name".
• Put Object: put-object accepts :bucket-name, :key, :metadata (e.g., {:server-side-encryption "AES256"}), and :file.
• Copy Object: copy-object source-bucket source-key dest-bucket dest-key.
• Get Object: get-object returns a map containing an :input-stream. Note: You must close the InputStream (e.g., by using slurp) to avoid exhausting the HTTP connection pool.
• Presigned URLs: generate-presigned-url bucket key duration (e.g., (-> 6 hours from-now)).

(ns com.example
  (:use [amazonica.aws.s3]
        [amazonica.aws.s3transfer]))

(create-bucket "two-peas")

(put-object :bucket-name "two-peas"
            :key "foo"
            :metadata {:server-side-encryption "AES256"}
            :file upload-file)

(copy-object bucket1 "key-1" bucket2 "key-2")

(-> (get-object bucket2 "key-2")
    :input-stream
    slurp)

(delete-object :bucket-name "two-peas" :key "foo")

(generate-presigned-url bucket1 "key-1" (-> 6 hours from-now))

Amazon SQS (Simple Queue Service) allows you to create queues, send/receive messages, and manage Dead Letter Queues (DLQ).

Common patterns:

• Queue Creation: Define attributes like :VisibilityTimeout or :ReceiveMessageWaitTimeSeconds during creation.
• Message Lifecycle: Use receive-message with :delete true to automatically remove messages after receipt, or manually delete using the message object.
• Dead Letter Queues: Use assign-dead-letter-queue to link a primary queue to a DLQ with a specific maxReceiveCount.

(ns com.example
  (:use [amazonica.aws.sqs]))

(create-queue :queue-name "my-queue"
              :attributes
                {:VisibilityTimeout 30
                 :MaximumMessageSize 65536
                 :MessageRetentionPeriod 1209600
                 :ReceiveMessageWaitTimeSeconds 10})

(send-message queue "hello world")

(def msgs (receive-message queue))

(delete-message (-> msgs
                    :messages
                    first
                    (assoc :queue-url queue))) 

Amazonica supports several authentication methods. By default, it uses the AWS SDK's chained provider (Environment Variables $\rightarrow$ Java System Properties $\rightarrow$ ~/.aws/credentials $\rightarrow$ EC2 Instance Profile).

1. Passing credentials per call

Every service function accepts an optional credentials map as its first argument:

(def cred {:access-key "aws-access-key" 
           :secret-key "aws-secret-key" 
           :endpoint "us-west-1"})

(describe-instances cred)

Supported Credential Keys:

• :access-key and :secret-key: Creates BasicAWSCredentials.
• :session-token: Creates BasicSessionCredentials.
• :profile: Creates ProfileCredentialsProvider.
• :cred: Uses the provided AWSCredentialsProvider instance.
• :endpoint: Specifies the service endpoint. If the value is a lowercase hyphenated region (e.g., us-east-1), it uses .withRegion; otherwise, it uses .withEndpointConfiguration.

2. Setting global credentials

Use (defcredential) to set credentials for all subsequent calls in the current process:

(defcredential "aws-access-key" "aws-secret-key" "us-west-1")
(describe-instances)

3. Ad-hoc credential switching

Use the (with-credential) macro to execute a specific block of code with different credentials:

(with-credential ["account-2-key" "secret" "us-east-1"]
  (describe-instances))