section 12 intro s3

• Amazon S3


  One of the main building blocks of AWS, infinitely scaling storage
• A lot of the web relies on Amazon S3
• Many websites use Amazon S3 as a backbone
• Many AWS services will also use Amazon S3 for integrations
• Amazon S3


  • Used for backup and storage
  • Disaster recovery purposes
  • Archival purposes
  • Hybrid cloud storage
  • Hosting applications, media, data lakes
  • Delivering software updates
  • Hosting static websites
• Use cases of Amazon S3

  • Nasdaq stores 7 years of data in S3 Glacier
  • Sysco runs analytics on data in Amazon S3
• Buckets


  Top level directories in Amazon S3 where files (objects) are stored
• Buckets


  • Must have a globally unique name
  • Defined at the region level
• Bucket naming conventions


  • No uppercase, no underscore
  • Between 3 and 63 characters long
  • Must not be an IP
  • Must start with lowercase number or letter
  • Can use letters, numbers, hyphens
• Objects


  Files stored in Amazon S3, have a key which is the full path
• Object key


  Composed of a prefix and an object name
• Amazon S3 does not have a concept of directories, everything is an object key
• Object value


  The content of the body
• Objects


  • Max size is 5 TB
  • If over 5 GB, must use multi-part upload
• Object metadata


  Key-value pairs set by the system or user to indicate information about the file
• Object tags


  Unicode key-value pairs up to 10, useful for security and lifecycles
• Object version ID


  If versioning is enabled
• Creating an S3 bucket

  1. Enter a unique bucket name

  2. Choose a region close to you

  3. Disable ACLs

  4. Block all public access

  5. Leave versioning and default encryption disabled
• The S3 console shows all buckets across all regions in one view, but each bucket is only created and lives within one region
• Uploading an object to an S3 bucket

  1. Click upload

  2. Add files

  3. Select file to upload

  4. Confirm destination bucket
• S3 pre-signed URL


  URL that contains the user's credentials, allowing them to access the object
• Public URL for an S3 object


  Does not work, as it does not contain the user's credentials
• Creating a folder in an S3 bucket


  1. Click to create a new folder

  2. Enter folder name
• Deleting an S3 folder

  1. Navigate to the folder

  2. Type "permanently delete" to confirm

  3. Delete objects
• The user experience of S3 is similar to cloud storage services like Google Drive or Dropbox
• Creating an S3 bucket

  1. Enter a unique bucket name

  2. Choose a region close to you

  3. Disable ACLs

  4. Block all public access

  5. Leave versioning and default encryption disabled
• The S3 console shows all buckets across all regions in one view, but each bucket is only created and lives within one region
• Uploading an object to an S3 bucket

  1. Click upload

  2. Add files

  3. Select file to upload

  4. Confirm destination bucket
• S3 pre-signed URL


  URL that contains the user's credentials, allowing them to access the object
• Public URL for an S3 object


  Does not work, as it does not contain the user's credentials
• Creating a folder in an S3 bucket


  1. Click to create a new folder

  2. Enter folder name
• Deleting an S3 folder

  1. Navigate to the folder

  2. Type "permanently delete" to confirm

  3. Delete objects
• The user experience of S3 is similar to cloud storage services like Google Drive or Dropbox
• S3 Bucket policy


  JSON-based policy that controls access to an S3 bucket and the objects within it
• Structure of an S3 Bucket policy


  1. Resource block (specifies which buckets/objects the policy applies to)

  2. Effect (Allow or Deny)

  3. Actions (e.g. GetObject)

  4. Principal (who the policy applies to)
• Example S3 Bucket policy


  • Allows anyone (Principal *) to GetObject from any object in the example bucket
• S3 Bucket policies


  • Can be used to grant public access to a bucket
  • Can be used to force object encryption
  • Can be used to grant access to another AWS account
• User wants to access files in an S3 bucket

  Attach an S3 Bucket policy that allows public access
• IAM user


  AWS user with permissions defined by an IAM policy
• IAM user wants to access S3 buckets

  Assign IAM permissions to that user through a policy