Skip to main content

Access files on AWS S3 bucket using aws-sdk-s3 with Rails 6

This is a quick post to share few easy methods to access files on AWS S3.

The below custom utility methods uses aws-sdk-s3

Firstly lets define a constant to access files on S3 - 

##to manage S3 Resources
S3 = Aws::S3::Resource.new(
region: "YOUR-REGION",
access_key_id: "YOUR-AWS-KEY",
secret_access_key: "YOUR-AWS-SECRET")

##to query S3
S3_client = Aws::S3::Client.new(
access_key_id: configatron.aws.s3.credentials.access_key_id,
secret_access_key: configatron.aws.s3.credentials.secret_access_key,
region: configatron.aws.s3.region
)

Now, lets use the S3 OBJECT to access files in different ways - 


1. Get all files in a S3 FOLDER

    ##this gives a array of file paths present in the folder specified by prefix attribute    
    s3objects = S3.bucket("my-output-bucket").objects({prefix: "/inputs/any-random-user-folder"}).collect(&:key)
##you can also delete the file if found
##eg: s3objects.batch_delete!


2. Check if file is present/check file content_lenght


          ##using head_object to just fetch required minimal details 
    s3object = S3_client.head_object(bucket: "MY-OUTPUT-BUCKET", key: "my-file-path")
    ##return boolean, if present else false
    !!s3object.content_length

3. Download file from S3

     
data = open(any_valid_s3_url)
send_data data.read, filename: "test-download", stream: false, type: 'any-valid-content-type', disposition: 'attachment'



The above methods are live and tested on production data on RAILS 6 application.
These methods are very simple yet configurable in the way you want by
adding more dynamicity, if needed as per the use case.
We had many use cases regarding -

  • Check if uploaded large file present on S3, after upload.
  • If file is uploaded, especially a video file, check if there are any thumbnails present.
  • If thumbnails present, show the last one else show default image
  • Download a file stored on S3.
There are more efficient ways to do it by setting up IAM roles/ACL attributes
per S3 objects, but that's beyound the scope of this post.













Labels:

Comments

Post a Comment

Popular posts from this blog

Adding Emoji icons to Rails application

Its very easy to add emoji icons/images to your rails application. It seems complex but its all easy using javascript.The steps can be simplified as shown below:- add the javascript(download jemotion ) include it in your view file(where you want :) to be shown as this -----------------------> thats it....done first you need to add this wonderful js which works really great and nice from home page . We are interested in only the js and emotions folder(containing the smileys/GIF images). once you download just include it in your view file and copy/dump the emotions folder(containing images) to app/assets/images for RAILS 3+. as i did where i want the users to view their comments after they submit their comment. You also need to give a unique id to make  the js understand where it needs to convert your symbols to icons.see below code and hope you will understand how to use it. Keep in mind two thing:- dont forget t...
1 comment
Read more

Configure amazon s3 with your rails app in just 5 mins

if you are looking to make use of your uploads using paperclip on AMAZON S3,then this is something that will help you. i tried the same and wasted 5 days but its very easy by using ONLY aws-sdk gem and you can start uploading to S3.  Before starting i assume few thing from you: You have a running rails app ready. You have an active AMAZON S3 account (get the credentials,you will need it now) You are using paperclip 3.5.1 (i prefer) You uploads are saving to your system locally short preview of the sequence you need to understand: START -> get aws-sdk gem -> create new s3.yml -> configure an initializer-paperclip.rb using AMAZON S3 credentials ----END----------THAT'S IT lets get started: Install 'aws-sdk' gem by including in gemfile and running bundle install. Gemfile: gem 'paperclip' , '3.5.1' gem "aws-sdk"   Create new yml file such as config/s3.yml and store your AMAZON S3 credentials as shown below...
Post a Comment
Read more

Using Geocoder to get street name,state,city,country etc in Your Rails app

Recently i implemented Google map for one of the module where User will enter his location/address and other relevant geo-details should get autopopulated.Well,This was posible because of   Geocoder  and by using it you can get every small details such as postalcode,country code,zipcode,street name,state name,state code,latitude,longitude(and more) ready in your hands easily.Its really awesome. I have used to it get all relevant information about the location that the user enter using Geocomplete ,another awesome autocomplete library to get locations. Suppose you have a users table and also few columns such as longitude,latitude,state,postal code,street name,address,country  and want to autopopulate other fields just by using address entered by the user then you are reading the right blog,So its easy,let me show you how. User will enter address using Geocomplete and store that address in address column in users/locations table Then,use Geocoder to fetch other geo...
Post a Comment
Read more