Uploads & Storage — Overview

Audience: backend, devops Scope: packages/storage, apps/api/src/modules/upload, apps/api/src/utils/multer.ts

Architecture

The storage system uses a local-first, remote-preferred pattern. Files always land on local disk first (via Multer), then the StorageManager pushes them to the configured remote driver and cleans up the local copy on success.

Driver System

The active driver is resolved at startup from the STORAGE_DRIVER env var, or detected automatically from which credentials are present.

auto in production — always set STORAGE_DRIVER=bucket explicitly in production. auto is for local development where bucket credentials may not be available.

File Validation

Two layers of validation run before a file is accepted (added in Subphase 9 of the auth-rbac-security-hardening initiative):

1. fileFilter (Multer, apps/api/src/utils/multer.ts) — rejects a file at upload time if its extension or client-claimed MIME type isn't in the allowlist for the resolved UploadType. This is a fast, first-pass check — it runs before any bytes are written to disk.
2. Magic-byte verification (StorageManager.handleUpload(), packages/storage) — after the file lands on disk, its actual byte signature is sniffed (via file-type) and compared against the same allowlist, ignoring the client-claimed MIME type entirely. A file that passes the fileFilter check by lying about its extension/MIME (e.g. an executable renamed to .png) is caught here and rejected with 400 Bad Request; the temporarily-written local file is deleted before the error propagates.

Both layers share one allowlist (packages/storage/src/upload-allowlist.ts) so they can never drift apart.

Virus/malware scanning is explicitly NOT implemented. A validly-typed file (correct magic bytes, correct extension) that still carries a malicious payload will not be caught by either layer above. This is a known, deliberately-deferred gap — see the Subphase 9 handoff for the reasoning (no existing scanning infrastructure in this deployment; adding it is an infra/ops decision, not a code point-fix).

Folder Structure

Files are routed to different sub-folders based on UploadType, which determines both the local path Multer writes to and the S3 object key prefix.

{uploadRoot}/               ← local disk  (UPLOAD_LOCATION env)
bucket/                     ← MinIO bucket (STORAGE_BUCKET_NAME env)
├── public/                 ← publicly readable without authentication
│   ├── avatar/             ← user profile pictures
│   ├── landing/            ← landing page banners / hero images
│   └── thumbnail/          ← video / content thumbnails
└── uploads/                ← private — signed URLs only, ownership-checked
    ├── image/{userId}/     ← general private images, scoped per uploader
    ├── video/{userId}/     ← private videos, scoped per uploader
    └── file/{userId}/      ← documents, scoped per uploader

Ownership scoping applies only to newly-uploaded private files. Files uploaded before Subphase 9 shipped have no {userId} segment in their key and are grandfathered through with no ownership check (a warning is logged on access) — see API Reference for the access-control behavior this implies.

The split is enforced in two places:

1. Multer destination (apps/api/src/utils/multer.ts) — reads req.query.uploadType and writes to the correct local sub-folder at request time.
2. S3StorageDriver (packages/storage/src/drivers/s3-driver.ts) — derives the S3 key from the local path, preserving the public/ or uploads/ prefix.

Bucket Policy

On every app startup, UploadService.onModuleInit() calls StorageManager.applyBucketPolicy(), which applies a MinIO bucket policy that:

• Allows anonymous s3:GetObject on public/* — direct URLs work without any credentials.
• Leaves uploads/** fully private — only accessible via short-lived pre-signed URLs.

The policy only adds Allow s3:GetObject for public/*. There is no explicit Deny for writes — anonymous writes are denied by default in MinIO when no explicit Allow exists for them.

Signed URL Generation

Pre-signed URLs are generated using HMAC-SHA256 locally — no network round-trip to MinIO is required. The SDK signs the URL on the server and returns it to the client.

For public files (public/* prefix), no signing is needed — the API returns the direct CDN/bucket URL immediately.

Optional: Read-only Credential Separation

For production, configure a second MinIO service account with only s3:GetObject rights and set STORAGE_BUCKET_READONLY_* credentials. The upload module will use this read-only account exclusively for signed URL generation. Even if a signed URL leaks, it cannot be used to upload or delete files.

If STORAGE_BUCKET_READONLY_ACCESS_KEY is not set, the primary StorageManager is used for signing instead — functionally identical, just without the credential isolation.