Amazon S3 Driver

The S3 driver provides full integration with Amazon Simple Storage Service using the AWS SDK v3. It serves as the base class for all S3-compatible drivers (R2, MinIO, B2, DigitalOcean Spaces, Wasabi).

Configuration

Field	Type	Required	Default	Description
driver	's3'	Yes	—	Must be 's3'
bucket	string	Yes	—	S3 bucket name
region	string	Yes	—	AWS region (e.g., 'us-east-1')
key	string	No¹	—	AWS access key ID
secret	string	No¹	—	AWS secret access key
endpoint	string	No	—	Custom endpoint URL (for S3-compatible services)
use_path_style_endpoint	boolean	No	false	Use path-style URLs instead of virtual-hosted
url	string	No	—	Custom base URL for public file URLs
visibility	'public' | 'private'	No	'private'	Default ACL for new objects
cdn	object	No	—	CloudFront CDN configuration

¹ When running on EC2, ECS, Lambda, or any AWS environment with IAM roles, key and secret can be omitted. The SDK will use the instance's IAM credentials automatically.

Basic Setup

import { Module } from '@nestjs/common';
import { StorageModule } from '@fozooni/nestjs-storage';

@Module({
  imports: [
    StorageModule.forRoot({
      default: 's3',
      disks: {
        s3: {
          driver: 's3',
          bucket: 'my-app-uploads',
          region: 'us-east-1',
          key: process.env.AWS_ACCESS_KEY_ID,
          secret: process.env.AWS_SECRET_ACCESS_KEY,
          visibility: 'private',
        },
      },
    }),
  ],
})
export class AppModule {}

Async Configuration with ConfigService

import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { StorageModule } from '@fozooni/nestjs-storage';

@Module({
  imports: [
    ConfigModule.forRoot(),
    StorageModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (config: ConfigService) => ({
        default: 's3',
        disks: {
          s3: {
            driver: 's3',
            bucket: config.getOrThrow('AWS_S3_BUCKET'),
            region: config.getOrThrow('AWS_REGION'),
            key: config.get('AWS_ACCESS_KEY_ID'),
            secret: config.get('AWS_SECRET_ACCESS_KEY'),
            visibility: 'private',
          },
        },
      }),
    }),
  ],
})
export class AppModule {}

IAM Roles in EC2 / ECS / Lambda

When deploying to AWS infrastructure, prefer IAM roles over static credentials. Omit key and secret from the config and the AWS SDK will automatically use the attached IAM role. This eliminates credential rotation concerns and is the recommended production approach.

{
  driver: 's3',
  bucket: 'my-app-uploads',
  region: 'us-east-1',
  // No key/secret — IAM role credentials are used automatically
}

S3-Specific Metadata

When you call getMetadata() on an S3 object, you receive extended metadata in S3FileMetadata:

const meta = await disk.getMetadata('uploads/photo.jpg');

// Standard metadata
console.log(meta.size);         // 1048576
console.log(meta.lastModified); // Date object
console.log(meta.mimeType);     // 'image/jpeg'

// S3-specific metadata
console.log(meta.etag);                  // '"a1b2c3d4..."'
console.log(meta.storageClass);          // 'STANDARD'
console.log(meta.versionId);             // 'abc123' (if versioning enabled)
console.log(meta.serverSideEncryption);  // 'AES256'
console.log(meta.s3Metadata);            // { 'x-amz-meta-author': 'alice' }

Presigned URLs

Generate time-limited URLs that grant temporary access to private objects:

@Injectable()
export class SecureDownloadService {
  constructor(private readonly storage: StorageService) {}

  async getDownloadLink(path: string): Promise<string> {
    const disk = this.storage.disk('s3');

    // URL valid for 15 minutes (900 seconds)
    const url = await disk.temporaryUrl(path, 900);

    return url;
  }

  async getUploadLink(path: string): Promise<string> {
    const disk = this.storage.disk('s3');

    // Presigned PUT URL for client-side uploads
    const url = await disk.temporaryUploadUrl(path, 900, {
      contentType: 'image/jpeg',
      maxSize: 10 * 1024 * 1024, // 10 MB
    });

    return url;
  }
}

Presigned POST

Generate presigned POST data for browser-based uploads with policy constraints:

@Injectable()
export class BrowserUploadService {
  constructor(private readonly storage: StorageService) {}

  async getUploadForm(userId: string): Promise<PresignedPostData> {
    const disk = this.storage.disk('s3');

    const postData = await disk.presignedPost({
      key: `uploads/${userId}/\${filename}`,
      expires: 3600, // 1 hour
      conditions: [
        ['content-length-range', 0, 50 * 1024 * 1024], // max 50 MB
        ['starts-with', '$Content-Type', 'image/'],      // images only
      ],
    });

    return postData;
    // {
    //   url: 'https://my-bucket.s3.us-east-1.amazonaws.com',
    //   fields: {
    //     key: 'uploads/123/${filename}',
    //     policy: 'base64...',
    //     'x-amz-signature': '...',
    //     ...
    //   }
    // }
  }
}

HTML Form Example

<form
  action="https://my-bucket.s3.us-east-1.amazonaws.com"
  method="POST"
  enctype="multipart/form-data"
>
  <!-- Include all fields from presignedPost() response -->
  <input type="hidden" name="key" value="uploads/123/${filename}" />
  <input type="hidden" name="policy" value="base64..." />
  <input type="hidden" name="x-amz-credential" value="..." />
  <input type="hidden" name="x-amz-algorithm" value="AWS4-HMAC-SHA256" />
  <input type="hidden" name="x-amz-date" value="..." />
  <input type="hidden" name="x-amz-signature" value="..." />

  <!-- File input MUST be the last field -->
  <input type="file" name="file" accept="image/*" />
  <button type="submit">Upload</button>
</form>

INFO

Presigned POST is different from presigned PUT URLs. POST allows more granular policy constraints (file size limits, content type restrictions, key prefixes) and is better suited for browser form uploads. PUT URLs are simpler but offer fewer constraints.

Multipart Uploads

For large files, use multipart uploads to improve throughput and enable resumability:

Convenience Method

import { createReadStream } from 'fs';

// Single call — handles chunking, part upload, and completion internally
await disk.putFileMultipart('backups/database.sql.gz', createReadStream('/tmp/dump.sql.gz'), {
  partSize: 10 * 1024 * 1024, // 10 MB parts
  concurrency: 4,              // Upload 4 parts in parallel
});

Manual Control

@Injectable()
export class ChunkedUploadService {
  constructor(private readonly storage: StorageService) {}

  async startUpload(path: string) {
    const disk = this.storage.disk('s3');
    const uploadId = await disk.initMultipartUpload(path);
    return { uploadId };
  }

  async uploadChunk(uploadId: string, partNumber: number, data: Buffer) {
    const disk = this.storage.disk('s3');
    const part = await disk.putPart(uploadId, partNumber, data);
    return part; // { ETag, PartNumber }
  }

  async completeUpload(uploadId: string, parts: { ETag: string; PartNumber: number }[]) {
    const disk = this.storage.disk('s3');
    await disk.completeMultipartUpload(uploadId, parts);
  }

  async abortUpload(uploadId: string) {
    const disk = this.storage.disk('s3');
    await disk.abortMultipartUpload(uploadId);
  }
}

WARNING

Always complete or abort multipart uploads. Incomplete uploads consume storage and incur costs. Consider setting an S3 lifecycle rule to automatically abort incomplete multipart uploads after a few days.

Range Requests

Stream partial content for media playback, resumable downloads, and large file previews:

// Read bytes 0 through 1,048,575 (first 1 MB)
const chunk = await disk.getRange('videos/intro.mp4', {
  start: 0,
  end: 1048575,
});

// Use with StorageService.serveRange() for HTTP range responses
@Controller('video')
export class VideoController {
  constructor(private readonly storage: StorageService) {}

  @Get(':id')
  @RangeServe()
  async stream(@Param('id') id: string) {
    return { disk: 's3', path: `videos/${id}.mp4` };
  }
}

Conditional Writes

Prevent overwrite conflicts using ETag-based conditional operations:

// Get the current ETag
const meta = await disk.getMetadata('config/settings.json');
const currentEtag = meta.etag;

// Write only if the ETag still matches (no concurrent modification)
const success = await disk.putIfMatch(
  'config/settings.json',
  updatedContent,
  currentEtag,
);

if (!success) {
  throw new ConflictException('File was modified since last read');
}

// Write only if the file does NOT exist
const created = await disk.putIfNoneMatch(
  'config/settings.json',
  defaultContent,
);

CDN Integration with CloudFront

Enable CloudFront CDN by adding the cdn configuration:

StorageModule.forRoot({
  default: 's3',
  disks: {
    s3: {
      driver: 's3',
      bucket: 'my-app-assets',
      region: 'us-east-1',
      key: process.env.AWS_ACCESS_KEY_ID,
      secret: process.env.AWS_SECRET_ACCESS_KEY,
      cdn: {
        url: 'https://d1234567890.cloudfront.net',
        keyPairId: process.env.CF_KEY_PAIR_ID,       // For signed URLs
        privateKey: process.env.CF_PRIVATE_KEY,       // For signed URLs
      },
    },
  },
})

When CDN is configured:

• url() returns the CloudFront URL instead of the S3 URL
• temporaryUrl() generates a CloudFront signed URL
• The disk is automatically wrapped with CdnDisk

const disk = this.storage.disk('s3');

// Returns: https://d1234567890.cloudfront.net/images/hero.jpg
const url = await disk.url('images/hero.jpg');

// CloudFront signed URL with expiration
const signedUrl = await disk.temporaryUrl('images/hero.jpg', 3600);

TIP

CloudFront signed URLs require the @aws-sdk/cloudfront-signer package:

pnpm add @aws-sdk/cloudfront-signer

Visibility / ACLs

Control per-object access with S3 ACLs:

// Set visibility on upload
await disk.put('public/logo.png', buffer, { visibility: 'public' });

// Change visibility after upload
await disk.setVisibility('public/logo.png', 'private');

// Check current visibility
const vis = await disk.getVisibility('public/logo.png');
// => 'private'

Visibility	S3 ACL	Effect
'public'	public-read	Anyone can read via the object URL
'private'	private	Only authenticated AWS requests can read

Bucket Policy for Public Access

As of April 2023, S3 blocks public access by default. To use public visibility, you must:

1. Disable "Block Public Access" on the bucket
2. Add a bucket policy allowing s3:GetObject for public objects

Without these changes, setting visibility to public will not make objects publicly accessible.

Custom Metadata

Attach arbitrary metadata to objects via S3 user-defined headers:

await disk.put('documents/contract.pdf', pdfBuffer, {
  metadata: {
    'author': 'alice',
    'department': 'legal',
    'version': '2.1',
  },
});

// Read metadata back
const meta = await disk.getMetadata('documents/contract.pdf');
console.log(meta.s3Metadata);
// { 'x-amz-meta-author': 'alice', 'x-amz-meta-department': 'legal', ... }

Checksum Support

Verify data integrity with checksums:

import { createHash } from 'crypto';

const content = Buffer.from('important data');
const md5 = createHash('md5').update(content).digest('base64');

// S3 verifies the MD5 on upload
await disk.put('data/important.bin', content, {
  contentMd5: md5,
});

// SHA256 checksum
const sha256 = createHash('sha256').update(content).digest('base64');
await disk.put('data/important.bin', content, {
  checksumSHA256: sha256,
});

S3ClientWrapper

For advanced operations not covered by FilesystemContract, access the underlying S3 client:

import { S3ClientWrapper } from '@fozooni/nestjs-storage';

@Injectable()
export class AdvancedS3Service {
  constructor(private readonly storage: StorageService) {}

  async advancedOperation() {
    const disk = this.storage.disk('s3');
    const wrapper = disk.getClient() as S3ClientWrapper;
    const s3Client = wrapper.client;

    // Use the raw AWS SDK S3Client for unsupported operations
    // e.g., S3 Select, Inventory, Analytics, etc.
  }
}

Utility Functions

The S3 driver exports several utility functions:

import { parseS3Url, encodeS3Key, buildS3Url } from '@fozooni/nestjs-storage';

// Parse an S3 URL into bucket and key
const { bucket, key } = parseS3Url('s3://my-bucket/path/to/file.txt');
// => { bucket: 'my-bucket', key: 'path/to/file.txt' }

// Parse HTTPS-style S3 URLs
const parsed = parseS3Url('https://my-bucket.s3.us-east-1.amazonaws.com/path/to/file.txt');
// => { bucket: 'my-bucket', key: 'path/to/file.txt' }

// Encode special characters in S3 keys
const safeKey = encodeS3Key('files/my report (final).pdf');
// => 'files/my%20report%20%28final%29.pdf'

// Build an S3 URL from components
const url = buildS3Url('my-bucket', 'path/to/file.txt', 'us-east-1');
// => 'https://my-bucket.s3.us-east-1.amazonaws.com/path/to/file.txt'

Cross-Region Copy

Copy objects between buckets in different regions:

@Injectable()
export class ReplicationService {
  constructor(private readonly storage: StorageService) {}

  async replicate(path: string) {
    const source = this.storage.disk('us-east');
    const target = this.storage.disk('eu-west');

    // Read from source region, write to target region
    const stream = await source.getStream(path);
    await target.putStream(path, stream);
  }
}

For same-region copies between buckets, use the native S3 copy:

await disk.copy('source/file.txt', 'destination/file.txt');

Complete Example: File Upload Service

import { Injectable, BadRequestException } from '@nestjs/common';
import { StorageService } from '@fozooni/nestjs-storage';
import { v4 as uuid } from 'uuid';

@Injectable()
export class FileUploadService {
  constructor(private readonly storage: StorageService) {}

  private get disk() {
    return this.storage.disk('s3');
  }

  async upload(file: Express.Multer.File) {
    const key = `uploads/${uuid()}/${file.originalname}`;

    await this.disk.put(key, file.buffer, {
      visibility: 'private',
      metadata: {
        originalName: file.originalname,
        uploadedAt: new Date().toISOString(),
      },
    });

    return {
      key,
      url: await this.disk.url(key),
      size: file.size,
    };
  }

  async getPresignedDownload(key: string, expiresIn = 900) {
    if (!(await this.disk.exists(key))) {
      throw new BadRequestException('File not found');
    }

    return this.disk.temporaryUrl(key, expiresIn);
  }

  async remove(key: string) {
    await this.disk.delete(key);
  }
}