将数据从 EC2 迁移到 S3？

Question

我正在使用 Paperclip 运行一个 Rails 应用程序来处理文件附件和图像大小调整等。该应用程序当前托管在 EngineYard 云上，所有附件都存储在其 EBS 中。考虑使用 S3 处理所有回形针附件。

有谁知道这种迁移的良好且安全的方法？非常感谢！

Answer 1

您可以执行一个 rake 任务来迭代您的附件并将每个附件推送到 S3。我不久前用过这个与attachment_fu——不会有太大不同。这使用了 aws-s3 gem。

基本上这个过程是：
1.从数据库中选择需要移动的文件
2.将它们推送到S3
3. 更新数据库以反映该文件不再存储在本地（这样您可以批量执行这些操作，并且无需担心将同一文件推送两次）。

@attachments = Attachment.stored_locally
@attachments.each do |attachment|

  base_path = RAILS_ROOT + '/public/assets/'
  attachment_folder = ((attachment.respond_to?(:parent_id) && attachment.parent_id) || attachment.id).to_s
  full_filename = File.join(base_path, ("%08d" % attachment_folder).scan(/..../), attachment.filename)
  require 'aws/s3'

  AWS::S3::Base.establish_connection!(
    :access_key_id        => S3_CONFIG[:access_key_id],
    :secret_access_key    => S3_CONFIG[:secret_access_key]
  )

  AWS::S3::S3Object.store(
    'assets/' + attachment_folder + '/' + attachment.filename,
    File.open(full_filename),
    S3_CONFIG[:bucket_name],
    :content_type => attachment.content_type,
    :access => :private
  )

  if AWS::S3::Service.response.success?
    # Update the database
    attachment.update_attribute(:stored_on_s3, true)

    # Remove the file on the local filesystem
    FileUtils.rm full_filename

    # Remove directory also if it is now empty
    Dir.rmdir(File.dirname(full_filename)) if (Dir.entries(File.dirname(full_filename))-['.','..']).empty?
  else
    puts "There was a problem uploading " + full_filename
  end
end

Answer 2

我发现自己处于同样的情况，并采用了 bensie 的代码并使其适合自己 - 这就是我想到的：

require 'aws/s3'

# Ensure you do the following:
#   export AMAZON_ACCESS_KEY_ID='your-access-key'
#   export AMAZON_SECRET_ACCESS_KEY='your-secret-word-thingy'
AWS::S3::Base.establish_connection!


@failed = []
@attachments = Asset.all # Asset paperclip attachment is: has_attached_file :attachment....
@attachments.each do |asset|
  begin
    puts "Processing #{asset.id}"
    base_path = RAILS_ROOT + '/public/'
    attachment_folder = ((asset.respond_to?(:parent_id) && asset.parent_id) || asset.id).to_s
    styles = asset.attachment.styles.keys
    styles << :original
    styles.each do |style|
      full_filename = File.join(base_path, asset.attachment.url(style, false))


      AWS::S3::S3Object.store(
        'attachments/' + attachment_folder + '/' + style.to_s + "/" + asset.attachment_file_name,
        File.open(full_filename),
        "swellnet-assets",
        :content_type => asset.attachment_content_type,
        :access => (style == :original ? :private : :public_read)
      )

      if AWS::S3::Service.response.success?        
        puts "Stored #{asset.id}[#{style.to_s}] on S3..."
      else
        puts "There was a problem uploading " + full_filename
      end
    end
  rescue
    puts "Error with #{asset.id}"
    @failed << asset.id
  end
end

puts "Failed uploads: #{@failed.join(", ")}" unless @failed.empty?

当然，如果您有多个模型，您将需要根据需要进行调整......