File Storage

Overview

Raypx provides a unified file storage interface through the @raypx/storage package. It supports multiple storage backends including AWS S3, Cloudflare R2, and local filesystem, making it easy to switch between providers.

Features

Multiple Providers: S3, R2, and local filesystem support
Image Processing: Built-in Sharp integration for image optimization
Type-Safe: Full TypeScript support
Unified API: Same interface regardless of provider
Presigned URLs: Secure temporary upload/download URLs

Configuration

Environment Variables

Add the following to your .env file:

# ===== Storage Configuration =====
# Provider: "s3" | "r2" | "local"
STORAGE_PROVIDER=s3

# AWS S3 Configuration
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1
AWS_S3_BUCKET=your-bucket-name

# Cloudflare R2 Configuration (alternative to S3)
R2_ACCOUNT_ID=your-account-id
R2_ACCESS_KEY_ID=your-access-key
R2_SECRET_ACCESS_KEY=your-secret-key
R2_BUCKET_NAME=your-bucket-name

# Local Storage Configuration (development only)
STORAGE_LOCAL_PATH=./uploads

Provider Selection

Provider	Use Case	Environment
S3	Production, AWS ecosystem	Production
R2	Production, cost-effective, no egress fees	Production
Local	Development, testing	Development

Quick Start

Basic Upload

import { storage } from "@raypx/storage";

// Upload a file
const file = await storage.upload({
  key: "uploads/avatar-123.jpg",
  body: fileBuffer,
  contentType: "image/jpeg",
});

console.log(file.url); // https://bucket.s3.amazonaws.com/uploads/avatar-123.jpg

Upload from Form Data

import { storage } from "@raypx/storage";

async function handleAvatarUpload(file: File) {
  const buffer = await file.arrayBuffer();
  const key = `avatars/${crypto.randomUUID()}-${file.name}`;

  const result = await storage.upload({
    key,
    body: Buffer.from(buffer),
    contentType: file.type,
  });

  return result;
}

Download File

import { storage } from "@raypx/storage";

// Download file
const file = await storage.download({
  key: "uploads/document.pdf",
});

console.log(file.body); // Buffer
console.log(file.contentType); // application/pdf

Delete File

import { storage } from "@raypx/storage";

await storage.delete({
  key: "uploads/old-file.jpg",
});

Check File Exists

import { storage } from "@raypx/storage";

const exists = await storage.exists({
  key: "uploads/avatar.jpg",
});

if (exists) {
  console.log("File found!");
}

Presigned URLs

For secure client-side uploads, use presigned URLs:

Generate Upload URL

import { storage } from "@raypx/storage";

// Generate presigned upload URL
const uploadUrl = await storage.getPresignedUploadUrl({
  key: "uploads/user-upload.jpg",
  expiresIn: 3600, // 1 hour
  contentType: "image/jpeg",
});

// Client can use this URL to upload directly
console.log(uploadUrl);

Generate Download URL

import { storage } from "@raypx/storage";

// Generate presigned download URL
const downloadUrl = await storage.getPresignedDownloadUrl({
  key: "private/document.pdf",
  expiresIn: 300, // 5 minutes
});

Client-Side Upload Example

// In your API route
const uploadData = await storage.getPresignedUploadUrl({
  key: `uploads/${userId}/${filename}`,
  contentType: file.type,
});

return json({ uploadUrl: uploadData.url, fields: uploadData.fields });

// On the client
async function uploadToS3(file: File, uploadUrl: string, fields: Record<string, string>) {
  const formData = new FormData();
  Object.entries(fields).forEach(([key, value]) => {
    formData.append(key, value);
  });
  formData.append("file", file);

  const response = await fetch(uploadUrl, {
    method: "POST",
    body: formData,
  });

  return response.ok;
}

Image Processing

The storage package includes Sharp for image optimization:

Resize Image

import { processImage } from "@raypx/storage/image";

const processedBuffer = await processImage(fileBuffer, {
  width: 800,
  height: 600,
  fit: "cover",
  format: "jpeg",
  quality: 80,
});

await storage.upload({
  key: "uploads/resized.jpg",
  body: processedBuffer,
  contentType: "image/jpeg",
});

Generate Thumbnail

import { processImage } from "@raypx/storage/image";

async function generateThumbnail(fileBuffer: Buffer) {
  return processImage(fileBuffer, {
    width: 200,
    height: 200,
    fit: "cover",
    format: "webp",
    quality: 75,
  });
}

Available Image Options

interface ImageProcessOptions {
  width?: number;
  height?: number;
  fit?: "cover" | "contain" | "fill" | "inside" | "outside";
  format?: "jpeg" | "png" | "webp" | "avif";
  quality?: number; // 1-100
  blur?: number;
  sharpen?: boolean;
  grayscale?: boolean;
}

Provider Setup

AWS S3

Create S3 Bucket
- Go to AWS S3 Console
- Click "Create bucket"
- Choose a globally unique name
- Select region
- Configure public access settings (block public access for private files)

Create IAM User

Go to IAM Console → Users → Create user
Attach policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::your-bucket/*"
    }
  ]
}

Create access keys and add to .env

Configure CORS (for direct uploads)

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
    "AllowedOrigins": ["https://yourdomain.com"],
    "ExposeHeaders": ["ETag"]
  }
]

Cloudflare R2

Create R2 Bucket
- Go to Cloudflare Dashboard
- Navigate to R2 → Create bucket
- Choose a bucket name
Create API Token
- Go to R2 → Manage R2 API Tokens
- Create API token with "Object Read & Write" permissions
- Copy Access Key ID and Secret Access Key

Configure Environment

STORAGE_PROVIDER=r2
R2_ACCOUNT_ID=your-account-id
R2_ACCESS_KEY_ID=your-key
R2_SECRET_ACCESS_KEY=your-secret
R2_BUCKET_NAME=your-bucket

Configure CORS (for direct uploads)
- Go to bucket settings → CORS Policy
- Add your domain as allowed origin

Local Storage

For development, use local filesystem:

STORAGE_PROVIDER=local
STORAGE_LOCAL_PATH=./uploads

The local provider stores files in the specified directory and serves them via the application.

List Files

import { storage } from "@raypx/storage";

// List files with prefix
const files = await storage.list({
  prefix: "uploads/",
  limit: 100,
});

for (const file of files.objects) {
  console.log(file.key, file.size, file.lastModified);
}

console.log("Has more:", files.hasNextPage);

Get File Metadata

import { storage } from "@raypx/storage";

const metadata = await storage.getMetadata({
  key: "uploads/document.pdf",
});

console.log(metadata.contentType);
console.log(metadata.size);
console.log(metadata.lastModified);
console.log(metadata.etag);

Best Practices

File Naming

// Good: Use organized paths with UUIDs
const key = `uploads/${userId}/${crypto.randomUUID()}-${filename}`;

// Bad: Predictable or user-controlled names
const key = `uploads/${filename}`; // Security risk!

File Validation

import { ALLOWED_MIME_TYPES, MAX_FILE_SIZE } from "@raypx/storage";

function validateFile(file: File): string | null {
  if (file.size > MAX_FILE_SIZE) {
    return "File too large (max 10MB)";
  }

  if (!ALLOWED_MIME_TYPES.includes(file.type)) {
    return "Invalid file type";
  }

  return null;
}

Access Control

// Public files (profile avatars, etc.)
const publicFile = await storage.upload({
  key: "public/avatar.jpg",
  body: buffer,
  contentType: "image/jpeg",
  acl: "public-read",
});

// Private files (documents, etc.)
const privateFile = await storage.upload({
  key: "private/document.pdf",
  body: buffer,
  contentType: "application/pdf",
  acl: "private",
});

Cleanup

// Delete files when no longer needed
async function deleteUserFiles(userId: string) {
  const files = await storage.list({ prefix: `users/${userId}/` });

  for (const file of files.objects) {
    await storage.delete({ key: file.key });
  }
}

Troubleshooting

Upload Fails

Check Credentials: Verify AWS/R2 credentials are correct
Check Bucket: Ensure bucket exists and is accessible
Check Region: Verify region matches bucket region
Check Permissions: Ensure IAM user has write permissions
Check CORS: For direct uploads, verify CORS configuration

Presigned URL Issues

Check Expiration: Ensure URL hasn't expired
Check Content-Type: Must match when uploading
Check Method: Use POST for form uploads, PUT for direct uploads

Image Processing Errors

Check Format: Ensure input is a valid image
Check Memory: Large images may require more memory
Check Dependencies: Ensure Sharp is installed correctly

File Storage

On this page