Pratham Shah

Uploads that don’t grow server memory

Streaming multipart uploads straight to S3 with multer-s3, and committing metadata only after S3 acknowledges.

, Cloud, Node.js, Storage

multer.memoryStorage() holds every file in the Node.js heap until the request finishes. With concurrent large uploads that is a straight path to an out-of-memory crash. Streaming storage pipes each chunk from the socket to S3 as it arrives, so memory use tracks the chunk size, not the file size.

Upload
  1. Multipart request
  2. Multer filterimage/*, 25 MB
  3. multer-s3UUID key
  4. S3 ack
  5. Image.insertMany()

Order of writes

The metadata write happens after S3 confirms every file. If the connection drops mid-upload, there is no database row pointing at an object that doesn’t exist. The worst case is an orphaned object in S3, which a lifecycle rule can clean up, rather than a broken record the UI tries to render.

Reading from a private bucket

With Block Public Access on, the browser can’t load objects directly. A /media/:id route looks up the key and streams GetObject back through the server’s IAM role with the stored Content-Type. The UI tries the direct URL first and falls back to the proxy.

Next step: presigned PUT URLs, so the browser uploads straight to S3 and the server only issues permissions and records metadata.