Amazon S3 (Simple Storage Service) is an incredibly powerful and scalable object storage service. When building web applications with Next.js, integrating S3 for file uploads, storage, and retrieval is a common requirement. This guide will walk you through setting up an S3 bucket, configuring it for public and private access, creating an IAM user for secure programmatic access, and then integrating it all into a Next.js application.
We'll cover:
Creating an S3 Bucket.
Setting up a Bucket Policy for public access to a specific folder.
Configuring CORS on the bucket.
Creating an IAM User for programmatic access.
Setting up an empty Next.js project.
Implementing the Next.js code for uploading and listing files (both public and private).
Let's get started!
Prerequisites
An AWS Account.
Node.js and npm (or yarn) installed on your machine.
Step 1: Create an S3 Bucket
First, we need a place to store our files.
Navigate to the AWS S3 console.
Click on "Create bucket".
Bucket name: Choose a globally unique name for your bucket (e.g., yourname-nextjs-s3-uploads).
AWS Region: Select the region closest to your users or your application servers.
Object Ownership: Leave "ACLs disabled (recommended)" selected.
Block Public Access settings for this bucket:
For now, keep "Block all public access" checked. We will refine this with a specific bucket policy later to only allow public access to a designated public/ folder.
Leave other settings (like Bucket Versioning, Tags, Default encryption) as default for this tutorial, or configure them as per your needs.
Click "Create bucket".
Step 2: Setup Bucket Policy for Public Access
We want to allow public read access to files uploaded into a specific "folder" (prefix) within our bucket, let's say public/. All other files should remain private.
Go to your newly created bucket in the S3 console.
Click on the "Permissions" tab.
Scroll down to "Bucket policy" and click "Edit".
Paste the following JSON policy. Remember to replace YOUR_BUCKET_NAME with your actual bucket name.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/public/*" } ] }"Principal": "*" means this policy applies to everyone (public).
"Action": "s3:GetObject" allows reading objects.
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/public/*" restricts this permission only to objects inside the public/ prefix of your bucket.
Click "Save changes".
Now, go back to the "Block public access (bucket settings)" section on the same "Permissions" tab and click "Edit".
Uncheck "Block public access to buckets and objects granted through new public bucket or access point policies" and "Block public access to buckets and objects granted through any public bucket or access point policies". This allows our bucket policy to take effect.
Click "Save changes" and confirm.
Step 3: Setup CORS on the Bucket
Cross-Origin Resource Sharing (CORS) is a security mechanism that browsers use. Since our Next.js app (running on localhost or a different domain) will be making requests to S3 (on an amazonaws.com domain), we need to configure CORS.
In your bucket's "Permissions" tab, scroll down to "Cross-origin resource sharing (CORS)" and click "Edit".
Paste the following JSON configuration. This is a lenient policy for development; you might want to restrict AllowedOrigins in production.
[ { "AllowedHeaders": [ "*" ], "AllowedMethods": [ "GET", "PUT", "POST", "DELETE", "HEAD" ], "AllowedOrigins": [ "*" ], "ExposeHeaders": [] } ]AllowedMethods: We need PUT for uploading files using pre-signed URLs and GET for accessing them.
AllowedOrigins: "*" allows any origin. For production, change this to your application's domain (e.g., "https://your-app.com"). You can also add http://localhost:3000 for local development.
AllowedHeaders: "*" allows all headers.
Click "Save changes".
Step 4: Create an IAM User for Programmatic Access
It's a security best practice not to use your root AWS account credentials. Instead, we'll create an IAM (Identity and Access Management) user with specific permissions to interact with S3.
Navigate to the IAM console in AWS.
In the sidebar, click on "Users" and then "Add users" (or "Create user" if it's your first time).
User name: Give your user a descriptive name (e.g., nextjs-s3-app-user).
Select AWS credential type: Check "Access key - Programmatic access". This will generate an Access Key ID and a Secret Access Key for your Next.js application to use.
Click "Next: Permissions".
Select "Attach existing policies directly".
Search for AmazonS3FullAccess. Check the box next to it.
Note: For a production environment, you should create a custom, more restrictive policy granting only the necessary permissions (e.g., s3:PutObject, s3:GetObject, s3:ListBucket on your specific bucket and prefixes). For this tutorial, AmazonS3FullAccess is simpler.
Click "Next: Tags" (optional, you can skip this).
Click "Next: Review".
Click "Create user".
CRITICAL: On the next screen, you'll see the Access key ID and Secret access key. Copy these down immediately and store them securely. You will not be able to see the Secret Access Key again after leaving this page.
Step 5: Setup an Empty Next.js Project
Now, let's set up our Next.js application.
Open your terminal and run:
npx create-next-app@latest s3-nextjs-demo cd s3-nextjs-demoFollow the prompts (e.g., choose TypeScript, ESLint, Tailwind CSS if you like). For this example, we'll assume you are using the App Router.
Install AWS SDK and other dependencies:
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presignerSet up Environment Variables:
Create a file named .env.local in the root of your project and add your AWS credentials and bucket details:# .env.local AWS_REGION_NEW=your-bucket-region # e.g., ap-south-1 AWS_ACCESS_KEY_ID_NEW=your-iam-user-access-key-id AWS_SECRET_ACCESS_KEY_NEW=your-iam-user-secret-access-key S3_BUCKET_NAME_NEW=your-bucket-nameImportant: Add .env.local to your .gitignore file to prevent committing your credentials to version control.
# .gitignore .env.local
Step 6: Coding the Next.js Project
We'll now add the necessary files and code to interact with S3.
1. Server Actions for S3 (src/app/actions/S3Actions.ts)
Create a new file src/app/actions/S3Actions.ts. This file will contain server-side functions to interact with S3, like generating pre-signed URLs and listing files.
// src/app/actions/S3Actions.ts
"use server";
import {
_Object,
GetObjectCommand,
ListObjectsV2Command,
PutObjectCommand,
S3Client,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
const S3Config = {
region: process.env.AWS_REGION_NEW || "",
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID_NEW || "",
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY_NEW || "",
},
};
const client = new S3Client(S3Config);
const BUCKET_NAME = process.env.S3_BUCKET_NAME_NEW;
if (!BUCKET_NAME) {
throw new Error("S3_BUCKET_NAME_NEW environment variable is not set.");
}
if (!S3Config.region || !S3Config.credentials.accessKeyId || !S3Config.credentials.secretAccessKey) {
throw new Error("AWS credentials or region not fully configured in environment variables.");
}
// Generate pre-signed URL for uploading public files
export const getSignedUrlForPublicUpload = async (fileName: string, fileType: string) => {
const params = {
Bucket: BUCKET_NAME,
Key: `public/${fileName}`, // Files will be stored in the 'public/' prefix
ContentType: fileType,
};
const command = new PutObjectCommand(params);
const url = await getSignedUrl(client, command, { expiresIn: 3600 }); // URL expires in 1 hour
return url;
};
// List files in the 'public/' prefix
export const listPublicFiles = async () => {
const params = {
Bucket: BUCKET_NAME,
Prefix: "public/",
};
const data = await client.send(new ListObjectsV2Command(params));
const contents: _Object[] = data.Contents ?? [];
return contents;
};
// Generate pre-signed URL for uploading user-specific private files
export const getSignedUrlForUserFileUpload = async (
fileName: string,
fileType: string,
userId: string
) => {
const params = {
Bucket: BUCKET_NAME,
Key: `${userId}/${fileName}`, // Files stored under a user-specific prefix
ContentType: fileType,
};
const command = new PutObjectCommand(params);
const url = await getSignedUrl(client, command, { expiresIn: 3600 }); // URL expires in 1 hour
return url;
};
// List files for a specific user (private files)
export const listUserFiles = async (userId: string) => {
const params = {
Bucket: BUCKET_NAME,
Prefix: `${userId}/`,
};
const data = await client.send(new ListObjectsV2Command(params));
const contents: _Object[] = data.Contents ?? [];
return contents;
};
// Generate pre-signed URL for downloading a private file
export const getSignedUrlForDownload = async (fileKey: string) => {
if (!fileKey) {
throw new Error("File key is required to generate a download URL.");
}
const params = {
Bucket: BUCKET_NAME,
Key: fileKey, // Full key of the file (e.g., 'userId/filename.jpg')
};
const command = new GetObjectCommand(params);
// The expiresIn option for getSignedUrl for GetObjectCommand is how long the URL will be valid.
// 20 seconds is very short, adjust as needed.
const url = await getSignedUrl(client, command, { expiresIn: 60 }); // URL expires in 60 seconds
return url;
};Explanation:
"use server";: Marks this module as containing Server Actions.
S3Client: Initializes the S3 client with your credentials and region.
getSignedUrlForPublicUpload & getSignedUrlForUserFileUpload: Generate pre-signed PUT URLs. The client-side will use these URLs to directly upload files to S3. This avoids passing large files through your Next.js server.
Public files are prefixed with public/.
User files are prefixed with userId/.
listPublicFiles & listUserFiles: List objects within the specified S3 prefixes.
getSignedUrlForDownload: Generates a pre-signed GET URL for downloading private files securely. This URL is temporary.
3. UI Components
Create src/components/upload-file.tsx. This component will handle the file selection and upload logic.
// src/components/upload-file.tsx
"use client";
import {
getSignedUrlForPublicUpload,
getSignedUrlForUserFileUpload,
} from "@/app/actions/S3Actions";
import React, { useState } from "react";
import { Button } from "@/components/ui/button"; // Assuming you have this from shadcn/ui or similar
import { Input } from "@/components/ui/input"; // Assuming you have this
import { File as FileIcon } from "lucide-react"; // Using lucide-react for icons
const UploadFile = ({ userId }: { userId?: string }) => {
const [file, setFile] = useState<File | undefined>();
const [buttonText, setButtonText] = useState<string>("Upload");
const [statusMessage, setStatusMessage] = useState<string>("");
const onFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
if (e.target.files && e.target.files[0]) {
setFile(e.target.files[0]);
setStatusMessage(""); // Clear previous status
}
};
const onFileUploadClicked = async () => {
if (!file) {
setStatusMessage("Please select a file first.");
return;
}
setButtonText("Uploading...");
setStatusMessage("Uploading file...");
try {
const fileName = encodeURIComponent(file.name); // Sanitize filename
const fileType = file.type;
let signedUrl = "";
if (userId) {
// Private user file
signedUrl = await getSignedUrlForUserFileUpload(fileName, fileType, userId);
} else {
// Public file
signedUrl = await getSignedUrlForPublicUpload(fileName, fileType);
}
const response = await fetch(signedUrl, {
method: "PUT",
headers: {
"Content-Type": fileType, // Important to set Content-Type for S3
},
body: file,
});
if (response.ok) {
setStatusMessage("File uploaded successfully!");
setFile(undefined); // Clear the file input
// Optionally, trigger a refresh of the file list here
// e.g., by calling router.refresh() if using Next.js App Router
// Or, if you pass a callback function as a prop: onUploadSuccess?.();
window.location.reload(); // Simple way to refresh, consider better state management
} else {
console.error("S3 Upload Error:", response);
setStatusMessage(`Upload failed: ${response.statusText}`);
}
} catch (error) {
console.error("Error uploading file:", error);
setStatusMessage("Error uploading file. See console for details.");
} finally {
setButtonText("Upload");
}
};
return (
<div className="p-4 border rounded-lg space-y-4">
<div className="flex gap-4 items-center">
<div className="relative flex-1">
<Input
type="file"
id="file-upload"
className="opacity-0 absolute inset-0 w-full h-full cursor-pointer z-10" // Ensure input is on top
onChange={onFileChange}
/>
<Button
variant="outline"
className="w-full flex items-center gap-2"
onClick={() => document.getElementById('file-upload')?.click()} // Trigger file input click
type="button" // Important for forms, not to submit
>
<FileIcon size={16} />
{file ? `${file.name}` : "Choose File"}
</Button>
</div>
<Button disabled={!file || buttonText === "Uploading..."} onClick={onFileUploadClicked}>
{buttonText}
</Button>
</div>
{statusMessage && <p className="text-sm">{statusMessage}</p>}
</div>
);
};
export default UploadFile;
Explanation:
"use client";: This is a client component.
It takes an optional userId. If userId is present, it uploads to the user's private folder; otherwise, to the public/ folder.
It calls the appropriate Server Action (getSignedUrlForPublicUpload or getSignedUrlForUserFileUpload) to get a pre-signed URL.
Then, it uses fetch with the PUT method to upload the file directly to S3 using the pre-signed URL.
Added Content-Type header in the PUT request, which is crucial for S3 to store the object correctly.
Create src/components/files-table.tsx. This component will display the list of files.
// src/components/files-table.tsx
"use client";
import { _Object } from "@aws-sdk/client-s3";
import React from "react";
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from "@/components/ui/table"; // Assuming you have these from shadcn/ui or similar
import { Button } from "@/components/ui/button";
import { getSignedUrlForDownload } from "@/app/actions/S3Actions";
const FilesTable = ({
files,
isPublic,
bucketName, // Pass bucketName as a prop
bucketRegion, // Pass bucketRegion as a prop
}: {
files: _Object[];
isPublic: boolean;
bucketName: string;
bucketRegion: string;
}) => {
if (!files || files.length === 0) {
return <p>No files found.</p>;
}
const handleDownloadPrivateFile = async (fileKey: string | undefined) => {
if (!fileKey) return;
try {
const url = await getSignedUrlForDownload(fileKey);
window.open(url, "_blank");
} catch (error) {
console.error("Error generating download URL:", error);
alert("Could not generate download link. See console for details.");
}
};
return (
<Table>
<TableHeader>
<TableRow>
<TableHead>Key (File Name)</TableHead>
<TableHead>Last Modified</TableHead>
<TableHead>Size (Bytes)</TableHead>
<TableHead>Action</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{files.map((file) => (
file.Key && ( // Only render if Key is present
<TableRow key={file.Key}>
<TableCell>{file.Key.split('/').pop()}</TableCell> {/* Show only filename */}
<TableCell>{file.LastModified?.toLocaleDateString()}</TableCell>
<TableCell>{file.Size}</TableCell>
<TableCell>
{isPublic ? (
<a
href={`https://${bucketName}.s3.${bucketRegion}.amazonaws.com/${file.Key}`}
target="_blank"
rel="noopener noreferrer"
className="text-blue-500 hover:underline"
>
Open Public File
</a>
) : (
<Button
variant="outline"
onClick={() => handleDownloadPrivateFile(file.Key)}
>
Download Private File
</Button>
)}
</TableCell>
</TableRow>
)
))}
</TableBody>
</Table>
);
};
export default FilesTable;Explanation:
"use client";: This is a client component as it handles user interaction (clicking download).
It takes files (list of S3 objects), isPublic (boolean), bucketName, and bucketRegion as props.
If isPublic is true, it constructs a direct S3 URL for the file. This works because of the bucket policy we set up.
If isPublic is false, it calls getSignedUrlForDownload to get a temporary, secure URL to download the private file.
4. Page Components
Modify src/app/page.tsx to provide navigation.
// src/app/page.tsx
import Link from "next/link";
export default function Home() {
return (
<main className="container mx-auto p-4">
<div className="flex flex-col items-center gap-6">
<h1 className="text-3xl font-bold">Welcome to S3 File Management Demo</h1>
<div className="flex gap-4">
<Link href="/public-files" className="px-4 py-2 bg-blue-500 text-white rounded hover:bg-blue-600">
Manage Public Files
</Link>
<Link href="/user-files/user123" className="px-4 py-2 bg-green-500 text-white rounded hover:bg-green-600">
Manage Files for User 'user123'
</Link>
<Link href="/user-files/anotherUser789" className="px-4 py-2 bg-purple-500 text-white rounded hover:bg-purple-600">
Manage Files for 'anotherUser789'
</Link>
</div>
<p className="mt-4 text-sm text-gray-600">
(Ensure your S3 bucket, IAM user, and .env.local are configured correctly)
</p>
</div>
</main>
);
}Create src/app/public-files/page.tsx for managing public files.
// src/app/public-files/page.tsx
import FilesTable from "@/components/files-table";
import UploadFile from "@/components/upload-file";
import { listPublicFiles } from "../actions/S3Actions";
export default async function PublicFilesPage() {
const files = await listPublicFiles();
const bucketName = process.env.S3_BUCKET_NAME_NEW!;
const bucketRegion = process.env.AWS_REGION_NEW!;
return (
<div className="container mx-auto p-4 space-y-6">
<h1 className="text-2xl font-semibold">Public Files</h1>
<p>Files uploaded here will be stored in the <code>public/</code> folder of your S3 bucket and will be publicly accessible via their S3 URL.</p>
<UploadFile /> {/* No userId prop, so it uploads to public */}
<h2 className="text-xl font-medium mt-6">Uploaded Public Files</h2>
<FilesTable files={files} isPublic={true} bucketName={bucketName} bucketRegion={bucketRegion} />
</div>
);
}Create src/app/user-files/[id]/page.tsx for managing user-specific private files.
// src/app/user-files/[id]/page.tsx
import { listUserFiles } from "@/app/actions/S3Actions";
import UploadFile from "@/components/upload-file";
import FilesTable from "@/components/files-table";
export default async function UserFilesPage({
params,
}: {
params: { id: string }; // The route parameter 'id' will be the userId
}) {
const userId = params.id;
const userFiles = await listUserFiles(userId);
const bucketName = process.env.S3_BUCKET_NAME_NEW!;
const bucketRegion = process.env.AWS_REGION_NEW!;
return (
<div className="container mx-auto p-4 space-y-6">
<h1 className="text-2xl font-semibold">Files for User: <span className="font-mono bg-gray-100 px-2 py-1 rounded">{userId}</span></h1>
<p>Files uploaded here will be stored in the <code>{userId}/</code> folder of your S3 bucket and will be private. Access is granted via temporary signed URLs.</p>
<UploadFile userId={userId} /> {/* Pass userId to upload to user's private folder */}
<h2 className="text-xl font-medium mt-6">Uploaded Private Files for {userId}</h2>
<FilesTable files={userFiles} isPublic={false} bucketName={bucketName} bucketRegion={bucketRegion} />
</div>
);
}Explanation:
These pages are Server Components that fetch initial file lists.
They pass the necessary props (userId, isPublic, bucketName, bucketRegion) to the client components.
The [id] in the route for user files is used as the userId to scope file operations.
Step 7: Run the Application
Make sure you have saved all files and your .env.local is correctly configured.
Start the Next.js development server:
npm run devOpen your browser and navigate to http://localhost:3000.
You should see links to "Public Files" and "User Files".
Public Files: Try uploading a file. It should appear in the list and be accessible via a direct S3 link. Check your S3 bucket in the public/ folder.
User Files: Navigate to a user page (e.g., for user123). Upload a file. It will be stored under user123/ in S3. Downloading will use a temporary pre-signed URL.
Key Concepts Recap
Public Files: Stored under public/ prefix. Accessible directly via S3 URL thanks to the bucket policy (s3:GetObject for public/*).
Private Files: Stored under user-specific prefixes (e.g., user123/). Not publicly accessible. Downloads are facilitated by short-lived pre-signed URLs generated by your Next.js server.
Pre-signed URLs:
For Uploads (PUT): Your Next.js server generates a temporary URL that grants the client permission to PUT an object directly into S3. This is secure and efficient.
For Downloads (GET): For private files, your server generates a temporary URL that grants permission to GET an object.
Server Actions: Used to securely handle S3 operations (like generating pre-signed URLs and listing files) on the server-side, invoked from client components.
Conclusion
You've now successfully integrated AWS S3 with your Next.js application, supporting both public and private file storage patterns! This setup provides a robust foundation for handling file uploads and downloads.
Further considerations:
Error Handling: Add more comprehensive error handling and user feedback.
UI/UX: Improve the user interface, add loading states, progress bars for uploads.
Security: For production, create fine-grained IAM policies instead of AmazonS3FullAccess. Regularly rotate your access keys.
File Validation: Implement client-side and server-side validation for file types, sizes, etc.
Database Integration: You'll likely want to store metadata about the uploaded files (like original filename, uploader ID, S3 key, etc.) in a database.
Deleting Files: Implement functionality to delete files from S3 (you'll need s3:DeleteObject permission).