S3 Driver
Upload files to S3-compatible storage
Uploads files to S3-compatible storage (AWS S3, Cloudflare R2, MinIO) with pure fetch and hand-rolled SigV4 — no AWS SDK required, so it runs on Node.js, Bun, Deno, and edge runtimes.
import { FilewayClient } from "@fileway/core";
import { S3Driver } from "@fileway/driver-s3";
const client = new FilewayClient({
driver: new S3Driver({
bucket: "my-bucket",
region: "us-east-1",
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
},
}),
});
const result = await client.upload(stream, {
filename: "report.pdf",
path: "invoices/2024",
mimeType: "application/pdf",
size: bytes,
});
console.log(result.meta.bucket);
console.log(result.meta.etag);Upload strategy
The driver picks the S3 upload strategy automatically from options.size:
- Known size ≤ 5 GiB — a single
PUTwith a SigV4aws-chunkedbody. Each chunk is signed lazily as the stream is read, so memory stays constant regardless of file size. - Unknown size — a multipart upload (
CreateMultipartUpload→UploadPart→CompleteMultipartUpload). The stream is read into part-sized buffers (default 8 MiB, minimum 5 MiB), so peak memory is bounded and the file is never buffered whole. - Size > 5 GiB — multipart upload, because a single
PUTcannot exceed 5 GiB.
Pass options.size when you know it to take the leanest path:
const result = await client.upload(fileStream, {
filename: "video.mp4",
size: fileSize,
});- If the stream ends before or after the declared
size, the driver aborts the upload and throws a validation error — a truncated or oversized object is never stored. - Use
partSizeto tune the multipart chunk size, andforceMultipartto always go through multipart.
Cancellation
Pass an AbortSignal in UploadOptions to cancel an upload in flight:
const controller = new AbortController();
const upload = client.upload(stream, {
filename: "large.bin",
signal: controller.signal,
});
// later...
controller.abort();On abort the driver cancels the source stream and rejects with a DOMException
whose name is AbortError:
try {
await upload;
} catch (err) {
if (err instanceof DOMException && err.name === "AbortError") {
// cancelled
}
}For multipart uploads (unknown size, > 5 GiB, or forceMultipart) the driver
first sends an AbortMultipartUpload (DELETE ?uploadId=) so no orphaned parts
are left on the bucket, then rejects with AbortError.
Progress
Pass onProgress to receive { bytes, total?, progress? } as bytes are streamed
(out of the source, and into each part for multipart). Without options.size,
only cumulative bytes is reported:
const result = await client.upload(stream, {
filename: "large.bin",
size: totalBytes,
onProgress: ({ bytes, progress }) => {
console.log(`${Math.round((progress ?? 0) * 100)}%`);
},
});Download
Stream an object back with get(path) — an authenticated SigV4 GET whose response body is returned as a WHATWG ReadableStream<Uint8Array>:
const stream = await client.get("uploads/video.mp4");
const bytes = new Uint8Array(await new Response(stream).arrayBuffer());stream is a WHATWG ReadableStream<Uint8Array>, so you can also pipe it straight to its destination (stream.pipeTo(writable)) with no buffering. Use .arrayBuffer()/.blob() for binary files — .text() would decode binary data as UTF-8 and corrupt it.
Throws with the HTTP status on failure (e.g. 404 when the object does not exist).
Presigned URLs
Generate a time-limited, SigV4-signed download link so anyone — even with no credentials — can fetch the object until it expires:
const url = await client.getPresignedUrl("uploads/report.pdf", { expiresIn: 900 });
// https://bucket.s3.us-east-1.amazonaws.com/uploads/report.pdf?X-Amz-Algorithm=...expiresIndefaults to3600(1 hour) and must be an integer in1..604800(AWS's 7-day cap); invalid values throw aValidationError.- Signing runs entirely with Web Crypto and your configured
credentials— no network call and no SDK. Only thehostheader is signed with anUNSIGNED-PAYLOADhash, so the URL works from any plainfetchor browser<a href>. - Works with
endpoint/forcePathStyletoo, e.g. MinIO, which validates SigV4 presigning.
MinIO
const client = new FilewayClient({
driver: new S3Driver({
bucket: "my-bucket",
endpoint: "http://localhost:9000",
region: "us-east-1",
forcePathStyle: true, // required for MinIO
credentials: {
accessKeyId: process.env.MINIO_ACCESS_KEY!,
secretAccessKey: process.env.MINIO_SECRET_KEY!,
},
}),
});Cloudflare R2
const client = new FilewayClient({
driver: new S3Driver({
bucket: "my-bucket",
endpoint: "https://<accountid>.r2.cloudflarestorage.com",
region: "auto",
credentials: {
accessKeyId: process.env.R2_ACCESS_KEY_ID!,
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
},
// Public URL for browser-accessible files (dev URL or custom domain):
baseUrl: "https://pub-<hash>.r2.dev", // development
// baseUrl: "https://cdn.yourdomain.com", // production
}),
});Config
| Option | Type | Default | Description |
|---|---|---|---|
bucket | string | — | S3 bucket name |
region | string | "us-east-1" | AWS region |
credentials | object | — | { accessKeyId, secretAccessKey } |
endpoint | string | AWS default | Custom endpoint (R2, MinIO) |
forcePathStyle | boolean | false | Use path-style URLs (endpoint/bucket) instead of virtual-hosted (bucket.endpoint). Required for MinIO. |
baseUrl | string | auto-derived | Base URL for generated URLs. For R2, set to the public dev URL or custom domain for browser-accessible links |
partSize | number | 8 MiB | Multipart part size in bytes (clamped to a 5 MiB minimum) |
forceMultipart | boolean | false | Always use multipart uploads, regardless of size |
Metadata
S3Driver implements BaseDriver<{ bucket: string; etag?: string }>
| Field | Type | Description |
|---|---|---|
meta.bucket | string | S3 bucket name |
meta.etag | string | undefined | S3 ETag from upload |
Errors
All failures are StorageErrors from @fileway/core (see the error reference for the full contract — codes, statusCode, provider, cause, and isAbortError). For this driver:
| Code | When |
|---|---|
validation | Malformed input: missing/oversized filename, path separators, absolute path, null bytes, malformed mimeType, non-integer size, invalid expiresIn. |
config | Missing credentials in the driver config. |
not-found | The object does not exist (NoSuchKey, or HTTP 404). |
bucket-not-found | The bucket does not exist (NoSuchBucket). |
auth-failed | Invalid/expired credentials or access denied (AccessDenied, InvalidAccessKeyId, SignatureDoesNotMatch, ExpiredToken, InvalidToken, or HTTP 401/403). |
size-exceeded | Payload over the limit (EntityTooLarge, or HTTP 413). |
network | fetch failed before an HTTP response (DNS, socket, timeout); the original error is preserved as cause. Safe to retry. |
provider-error | Any other non-2xx response (e.g. HTTP 500). Safe to retry. |
S3 codes come from parsing the AWS/MinIO XML <Code> element when present, with an HTTP-status fallback.