Cloud Pub/Sub Client Library for Node.js

To get started with Cloud Pub/Sub in Node.js, follow these steps to set up your environment, install the library, and run a basic publish/subscribe flow.

1. Prerequisites

• Select or create a Google Cloud Platform project.
• Enable billing for your project.
• Enable the Google Cloud Pub/Sub API.
• Set up authentication to allow your local workstation to access the API.

2. Installation

Install the client library using npm:

npm install @google-cloud/pubsub

3. Basic Usage Example

The following example demonstrates how to instantiate a client, create a topic, create a subscription, listen for messages, and publish a message.

// Imports the Google Cloud client library
const {PubSub} = require('@google-cloud/pubsub');

async function quickstart(
  projectId = 'your-project-id', // Your Google Cloud Platform project ID
  topicNameOrId = 'my-topic', // Name for the new topic to create
  subscriptionName = 'my-sub', // Name for the new subscription to create
) {
  // Instantiates a client
  const pubsub = new PubSub({projectId});

  // Creates a new topic
  const [topic] = await pubsub.createTopic(topicNameOrId);
  console.log(`Topic ${topic.name} created.`);

  // Creates a subscription on that new topic
  const [subscription] = await topic.createSubscription(subscriptionName);

  // Receive callbacks for new messages on the subscription
  subscription.on('message', message => {
    console.log('Received message:', message.data.toString());
    process.exit(0);
  });

  // Receive callbacks for errors on the subscription
  subscription.on('error', error => {
    console.error('Received error:', error);
    process.exit(1);
  });

  // Send a message to the topic
  await topic.publishMessage({data: Buffer.from('Test message!')});
}

Pub/Sub supports ingesting data from various external cloud and streaming services. Use the following patterns to create topics with ingestion enabled:

• AWS MSK: Ingest from Amazon Managed Streaming for Apache Kafka.
• Azure Event Hubs: Ingest from Azure Event Hubs.
• Confluent Cloud: Ingest from Confluent Cloud.
• Kinesis: Ingest from Amazon Kinesis.
• Cloud Storage: Ingest from Google Cloud Storage buckets.

# AWS MSK
node createTopicWithAwsMskIngestion.js <topic-name> <cluster-arn> <msk-topic> <role-arn> <gcp-service-account>

# Azure Event Hubs
node createTopicWithAzureEventHubsIngestion.js <topic-name> <cluster-arn> <msk-topic> <role-arn> <gcp-service-account>

# Confluent Cloud
node createTopicWithConfluentCloudIngestion.js <topic-name> <bootstrap-server> <cluster-id> <confluent-topic> <identity-pool-id> <gcp-service-account>

# Kinesis
node createTopicWithKinesisIngestion.js <topic-name> <role-arn> <gcp-service-account> <stream-arn> <consumer-arn>

# Cloud Storage
node createTopicWithCloudStorageIngestion.js <topic-name> <bucket> <input-format> <text-delimiter> <match-glob> <minimum-object-creation-time>

Pub/Sub supports several specialized subscription types. Use the following command patterns to implement specific behaviors:

• Dead Letter Policy: For handling failed messages by sending them to a dead-letter topic.
• Exactly-once delivery: To ensure messages are delivered exactly once.
• Filtering: To only receive messages that match a specific filter string.
• Ordering: To enable message ordering.
• Retry Policy: To configure how the service retries failed deliveries.

# Dead Letter Policy
node createSubscriptionWithDeadLetterPolicy.js <topic-name-or-id> <subscription-name-or-id> <dead-letter-topic-name-or-id>

# Exactly-once delivery
node createSubscriptionWithExactlyOnceDelivery.js <topic-name-or-id> <subscription-name-or-id>

# Filtering
node createSubscriptionWithFiltering.js <topic-name-or-id> <subscription-name-or-id> <filter-string>

# Ordering enabled
node createSubscriptionWithOrdering.js <topic-name-or-id> <subscription-name-or-id>

# Retry Policy
node createSubscriptionWithRetryPolicy.js <topic-name-or-id> <subscription-name-or-id>