Compute and storagemedium3-5 years

Users upload invoices of up to 200 MB. The current endpoint accepts a multipart body in a Spring controller and streams it to S3 with the SDK; under load the service's heap and threads are exhausted. How would you redesign the upload, what does the service still own, and what does 'S3 is not a filesystem' mean for the rest of the design?

Stop proxying the bytes. The service authenticates the user, decides the object key, and signs a presigned URL that permits one PUT to that one key for a short time (fifteen minutes, say); the client uploads directly to S3 and the bytes never pass through the JVM. The service still owns everything that is a decision: who may upload, the key layout, the content type it signed for, and what happens after the upload, which an S3 event notification to SQS turns into an event instead of a poll. 'Not a filesystem' means there is no rename (it is copy then delete), no append, and directories are only a prefix convention, so keys are designed once, as invoices/<customer>/<date>/<uuid>, not moved around later.

The lesson behind it →
More on Compute and storage