Skip to main content

Access files on AWS S3 bucket using aws-sdk-s3 with Rails 6

This is a quick post to share few easy methods to access files on AWS S3.

The below custom utility methods uses aws-sdk-s3

Firstly lets define a constant to access files on S3 - 

##to manage S3 Resources
S3 = Aws::S3::Resource.new(
region: "YOUR-REGION",
access_key_id: "YOUR-AWS-KEY",
secret_access_key: "YOUR-AWS-SECRET")

##to query S3
S3_client = Aws::S3::Client.new(
access_key_id: configatron.aws.s3.credentials.access_key_id,
secret_access_key: configatron.aws.s3.credentials.secret_access_key,
region: configatron.aws.s3.region
)

Now, lets use the S3 OBJECT to access files in different ways - 


1. Get all files in a S3 FOLDER

    ##this gives a array of file paths present in the folder specified by prefix attribute    
    s3objects = S3.bucket("my-output-bucket").objects({prefix: "/inputs/any-random-user-folder"}).collect(&:key)
##you can also delete the file if found
##eg: s3objects.batch_delete!


2. Check if file is present/check file content_lenght


          ##using head_object to just fetch required minimal details 
    s3object = S3_client.head_object(bucket: "MY-OUTPUT-BUCKET", key: "my-file-path")
    ##return boolean, if present else false
    !!s3object.content_length

3. Download file from S3

     
data = open(any_valid_s3_url)
send_data data.read, filename: "test-download", stream: false, type: 'any-valid-content-type', disposition: 'attachment'



The above methods are live and tested on production data on RAILS 6 application.
These methods are very simple yet configurable in the way you want by
adding more dynamicity, if needed as per the use case.
We had many use cases regarding -

  • Check if uploaded large file present on S3, after upload.
  • If file is uploaded, especially a video file, check if there are any thumbnails present.
  • If thumbnails present, show the last one else show default image
  • Download a file stored on S3.
There are more efficient ways to do it by setting up IAM roles/ACL attributes
per S3 objects, but that's beyound the scope of this post.













Labels:

Comments

Post a Comment

Popular posts from this blog

Convert videos using ffmpeg by ffmpeg_paperclip-Rails

Recently i implemented video uploads at S3 using ffmpeg-paperclip and ffmpeg where my uploaded videos get converted into mp4,webm and ogg(most required formats) along with 2 screenshots(with diff sizes) and stored at S3. its so wonderful in many ways:- your uploaded videos get converted into required formats(compatible on all browsers) before saving using ffmpeg through ffmpeg-paperclip moreover,you will also get screenshot(2 different images)of every video uploaded + AMAZON S3( you can even store locally,refer Paperclip ) if you want the background job to process video conversion then,use delayed_paperclip  i will show you how you can convert your uploaded videos into required formats and why we need to convert ....well refer  this .Before starting make sure that you have ffpmeg installed with all libraries needed,so click the above link instead of searching for how to install ffmpeg and install it( will take around 30mins.but its worthy ). Here i am using medi...
2 comments
Read more

Using Geocoder to get street name,state,city,country etc in Your Rails app

Recently i implemented Google map for one of the module where User will enter his location/address and other relevant geo-details should get autopopulated.Well,This was posible because of   Geocoder  and by using it you can get every small details such as postalcode,country code,zipcode,street name,state name,state code,latitude,longitude(and more) ready in your hands easily.Its really awesome. I have used to it get all relevant information about the location that the user enter using Geocomplete ,another awesome autocomplete library to get locations. Suppose you have a users table and also few columns such as longitude,latitude,state,postal code,street name,address,country  and want to autopopulate other fields just by using address entered by the user then you are reading the right blog,So its easy,let me show you how. User will enter address using Geocomplete and store that address in address column in users/locations table Then,use Geocoder to fetch other geo...
Post a Comment
Read more

Adding Emoji icons to Rails application

Its very easy to add emoji icons/images to your rails application. It seems complex but its all easy using javascript.The steps can be simplified as shown below:- add the javascript(download jemotion ) include it in your view file(where you want :) to be shown as this -----------------------> thats it....done first you need to add this wonderful js which works really great and nice from home page . We are interested in only the js and emotions folder(containing the smileys/GIF images). once you download just include it in your view file and copy/dump the emotions folder(containing images) to app/assets/images for RAILS 3+. as i did where i want the users to view their comments after they submit their comment. You also need to give a unique id to make  the js understand where it needs to convert your symbols to icons.see below code and hope you will understand how to use it. Keep in mind two thing:- dont forget t...
1 comment
Read more