fake-gcs-server

You can integrate fake-gcs-server into your Node.js testing workflow using testcontainers. This allows you to spin up a containerized GCS emulator that your application code can interact with via the @google-cloud/storage library.

To set this up:

1. Start a GenericContainer using the fsouza/fake-gcs-server image.
2. Configure the entrypoint to use the -scheme http flag.
3. Expose the required port (e.g., 4443).
4. Retrieve the dynamic host and mapped port from the container.
5. Crucial: Update the server's internal configuration via the /_internal/config endpoint to set the externalUrl. This ensures that the emulator generates correct URLs for operations like signed URLs or redirects.
6. Initialize the @google-cloud/storage client using the emulator's apiEndpoint.

Note: The example uses ky for making the configuration HTTP request.

import { Storage } from "@google-cloud/storage";
import ky from "ky";
import { GenericContainer } from "testcontainers";

const PORT = 4443;

const CONTAINER = await new GenericContainer("fsouza/fake-gcs-server:1.49.0")
  .withEntrypoint(["/bin/fake-gcs-server", "-scheme", "http"])
  .withExposedPorts(PORT)
  .start();

const API_ENDPOINT = `http://${CONTAINER.getHost()}:${CONTAINER.getMappedPort(
  PORT
)}`;

await ky.put(`${API_ENDPOINT}/_internal/config`, {
  json: { externalUrl: API_ENDPOINT },
});

const STORAGE = new Storage({ apiEndpoint: API_ENDPOINT });

// ...

When running fake-gcs-server in a container (e.g., via Testcontainers), the IP and port are dynamic and unknown before startup. Because resumable upload operations require the server to respond with the correct Location header (the external-url), you must update the server's configuration via its internal API after the container has started.

To do this:

1. Retrieve the container's host and mapped port.
2. Construct the external URL (e.g., http://<host>:<port>).
3. Send a PUT request to the /_internal/config endpoint with a JSON body containing the externalUrl key.

@Testcontainers
class FakeGcsServerTest {

    @Container
    static final GenericContainer<?> fakeGcs = new GenericContainer<>("fsouza/fake-gcs-server")
      .withExposedPorts(4443)
      .withCreateContainerCmdModifier(cmd -> cmd.withEntrypoint(
          "/bin/fake-gcs-server",
          "-scheme", "http"
      ));

    @BeforeAll
    static void setUpFakeGcs() throws Exception {
      String fakeGcsExternalUrl = "http://" + fakeGcs.getHost() + ":" + fakeGcs.getFirstMappedPort();

      updateExternalUrlWithContainerUrl(fakeGcsExternalUrl);

      storageClient = StorageOptions.newBuilder()
          .setHost(fakeGcsExternalUrl)
          .setProjectId("test-project")
          .setCredentials(NoCredentials.getInstance())
          .build()
          .getService();
    }

    private static void updateExternalUrlWithContainerUrl(String fakeGcsExternalUrl) throws Exception {
      String modifyExternalUrlRequestUri = fakeGcsExternalUrl + "/_internal/config";
      String updateExternalUrlJson = "{" + "\"externalUrl\": \"" + fakeGcsExternalUrl + "\"}";

      HttpRequest req = HttpRequest.newBuilder()
          .uri(URI.create(modifyExternalUrlRequestUri))
          .header("Content-Type", "application/json")
          .PUT(BodyPublishers.ofString(updateExternalUrlJson))
          .build();
      HttpResponse<Void> response = HttpClient.newBuilder().build()
          .send(req, BodyHandlers.discarding());

      if (response.statusCode() != 200) {
          throw new RuntimeException(
              "error updating fake-gcs-server with external url, response status code " + response.statusCode() + " != 200");
      }
    }

    @Test
    void shouldUploadFileByWriterChannel() throws IOException {
      storageClient.create(BucketInfo.newBuilder("sample-bucket2").build());

      WriteChannel channel = storageClient.writer(BlobInfo.newBuilder("sample-bucket2", "some_file2.txt").build());
      channel.write(ByteBuffer.wrap("line1\n".getBytes()));
      channel.write(ByteBuffer.wrap("line2\n".getBytes()));
      channel.close();

      Blob someFile2 = storageClient.get("sample-bucket2", "some_file2.txt");
      String fileContent = new String(someFile2.getContent());
      assertEquals("line1\nline2\n", fileContent);
    }
}

All server flags can be set using environment variables prefixed with FAKE_GCS_. Command line flags take precedence over environment variables.

Example using environment variables:

docker run -d --name fake-gcs-server -e FAKE_GCS_SCHEME=http -p 4443:4443 fsouza/fake-gcs-server

The Options struct allows fine-grained control over the emulator's behavior: