Uploads that don’t grow server memory
Streaming multipart uploads straight to S3 with multer-s3, and committing metadata only after S3 acknowledges.
, Cloud, Node.js, Storage
multer.memoryStorage() holds every file in the Node.js heap until the request finishes. With concurrent large uploads that is a straight path to an out-of-memory crash. Streaming storage pipes each chunk from the socket to S3 as it arrives, so memory use tracks the chunk size, not the file size.
- Multipart request
- Multer filterimage/*, 25 MB
- multer-s3UUID key
- S3 ack
- Image.insertMany()
Order of writes
The metadata write happens after S3 confirms every file. If the connection drops mid-upload, there is no database row pointing at an object that doesn’t exist. The worst case is an orphaned object in S3, which a lifecycle rule can clean up, rather than a broken record the UI tries to render.
Reading from a private bucket
With Block Public Access on, the browser can’t load objects directly. A /media/:id route looks up the key and streams GetObject back through the server’s IAM role with the stored Content-Type. The UI tries the direct URL first and falls back to the proxy.
Next step: presigned PUT URLs, so the browser uploads straight to S3 and the server only issues permissions and records metadata.