Archive from February, 2010
Feb 25, 2010 - Agile & Scrum, Coderkitty    No Comments

Acknowledging the unknown as part of the process

I think this is by far the best, though vague and ambiguous poetry I’ve ever seen and I loved it!

It puts the technical context of programming in a development cycle into lay context. :) I stumbled upon this lovely piece of work, through CodingHorror’s article entitled: Version 1 Sucks, But Ship It Anyway

As we know,
There are known knowns.
There are things we know we know.
We also know
There are known unknowns.
That is to say
We know there are some things
We do not know.
But there are also unknown unknowns,
The ones we don’t know
We don’t know.

Makes me want to keep in mind that knowledge in programming is quite volatile as it comes to real world problems to be solved, and that no accurate solution is definite across a period of time. The unknown is really a part of the process.

The power of toggle

I don’t know if we’ve been missing out on deliberate API lovin’ or that we’re becoming dumber every minute because of too much work, or its just the love month? :) …but we’ve been missing the power of “toggle“.

Toggle is defined as:

any instruction that works first one
way and then the other; it turns
something on the first time it is used
and then turns it off the next time

Needless to say, toggle is very efficient for use with boolean values (defined by either true or false). Let’s take a specific scenario:

1
2
3
4
5
6
7
8
9
10
11
12
  create_table "profiles", :force => true do |t|
    t.string   "salutation",            :limit => 8, :default => ""
    t.string   "name",                  :limit => 200, :default => ""
    t.datetime "created_at"
    t.datetime "updated_at"
  end
 
  create_table "themes", :force => true do |t|
    t.integer "profile_id",
    t.string   "name",            :limit => 100, :default => ""
    t.boolean "primary",        :default => false
  end

So, there are multiple themes per profile. You should get at least that part. Your boolean column should always be set to false by default. Either you replace the name of your column to suit that requirement, or write dirtier code. (I.e., instead of have a “public” column all set to true, use “private” column all set to false by default). Now, of course, your associations should be setup properly similar to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class Profile < ActiveRecord::Base
 
  # filter_chain methods here...
  # validations here...
 
  # associations start...
  has_many :themes do 
    def primary
      find(:first, :conditions => ["primary = ?", true])
    end
  end
  # associations end...
 
end
 
class Theme < ActiveRecord::Base
 
  # filter_chain methods here...
  # validations here...
 
  # associations start...
  belongs_to :profile
  # associations end...
 
  named_scope :current_theme_of, lambda { |profile|
    { :conditions => [ "profile_id = ? and primary = ?", profile.id, true ] }
  }
end

Of course, there would be several records for the Theme table that will belong to a profile. You will offer a checkbox on every create and update of the themes of a particular profile, but how do you ensure that there will only be one primary theme set for each profile record?

This is dirty: (KIDS, DO NOT TRY THIS AT HOME)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class Theme < ActiveRecord::Base
 
  # filter_chain methods here...
  after_save :verify_unique_primary_theme
  # validations here...
 
  # associations start...
  belongs_to :profile
  # associations end...
 
  named_scope :current_theme_of, lambda { |profile|
    { :conditions => [ "profile_id = ? and primary = ?", profile.id, true ] }
  }
 
  def verify_unique_primary_theme
    # get all themes by this profile 
    # set all their primary value to false
    # set this new one's primary value to true
  end
end

Man, I wouldn’t bet on a quick update on a profile whose set of themes would be more than hundreds, or even thousands! This, of course, is always worth a try:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
class Theme < ActiveRecord::Base
 
  # filter_chain methods here...
  # validations here...
 
  # associations start...
  belongs_to :profile
  # associations end...
 
  named_scope :current_theme_of, lambda { |profile|
    { :conditions => [ "profile_id = ? and primary = ?", profile.id, true ] }
  }
 
  def primary?
    primary
  end
 
  def validate
    #ensure that primary true is for one record only
    if primary? and !profile.themes.primary.nil?
      #toggle the old primary theme of this said profile
      profile.themes.primary.toggle!(:primary)
    end
  end
 
end

Simple, yet it works! Each time another theme gets toggled to be the primary theme, you only need to make two writes to your database:

  1. Save your new item
  2. Flip the old primary to false

PS: I did write messy code on purpose to illustrate what I meant for this post.

I hope this toggle magic works for you too the next time you think about boolean switches!

:)

DIY Ruby SFTP to S3

I was faced by a challenge of transferring some files from an FTP server to S3. At first, the nature of the problem seemed simple. I just had to get the files to my local machine and then upload them to S3. Piece of cake! But lo and behold, when I saw that there were over 32K files and each step took me almost 2 minutes each! Oooh lala!

I knew this was a job for fanfare sound here the almighty ruby! :)

And so, I set my priorities straight.. I had to work elegantly and save time altogether. I’m feeling quite exhausted and would really love to get some sleep soon, so here I am, depending on my great yet simple ruby skills and the power of ruby itself to help me achieve this task!

As I write this article, my script is silently running in the background as a very obedient and competent slave. :P But to help you out, here is a brief guide on how to sit back, relax and let the task be accomplished in no time! You should really be sending me gifts.. or better yet, donate to me via Paypal! I’ll love it, I promise.. I might even give you a page of exposure! :)


So, here’s the deal. You have a set of files to download via FTP and a particular folder structure where you want it to go.

STEP 1:

Gather your FTP credentials as follows:

1
2
3
4
host: my.server.com
username: my_username
password: my_password
port: 12345

STEP 2:

We will need some gems here. Now, if you don’t have them yet, now is a good time to install them:

1
2
sudo gem install net-sftp
sudo gem install fastercsv

Net-Sftp is needed for the communication to your FTP server. You can use this even if your access is just set to anonymous. You might want to use another CSV parser if you want, or if you already have something else in mind. You can just tweak the code for that section.

STEP 3:

Now, pull out your beloved spreadsheet editor (mine is OpenOffice) and aggregate your data.

Full remote path to the files Full local path to save into
/your/ultra/deep-stack/of/remote/folders/path/to/filename.doc /Users/coderkitty/attachments/not-so-deep-stack/filename.doc
/your/ultra/deep-stack/of/remote/folders/path/to/filename2.doc /Users/coderkitty/attachments/not-so-deep-stack/filename2.doc
/your/ultra/deep-stack/of/remote/folders/path/to/A super special filename here.doc /Users/coderkitty/attachments/not-so-deep-stack/itsnolongerspecial.doc

STEP 4:

With this, you should export your file to CSV format and remove the Text Delimiter setting.

Export

Export

STEP 5:

Open your CSV file using your favorite text editor (mine is TextWrangler, you might want to try it. Its opensource and free). Make sure that all spaces in your filenames are escaped properly, so that you have something like this:

1
/your/deep-stack/of/remote/folders/A super special filename here.doc

changed into:

1
/your/deep-stack/of/remote/folders/A\ super\ special\ filename\ here.doc

then don’t forget to save your csv file.

STEP 6:

Now, this is a fork. You have two (2) choices you can take. Either one works fine. Just pick your desired approach.

  • Save the remote path and the local path in two separate csv files for loading (version 1), or
  • Use one big csv file of remote, local path (version 2)

STEP 7:

The rest is quite straightforward from this point onwards. Just fill in your constants in the ruby script and then make sure that your files are in the right directories. All set? Execute away!

1
ruby sftp_to_local.rb

You might also find that the generated errors.txt file could help you reiterate on some parts of downloading/uploading your file. Sometimes your path could be wrong, or that you forgot to specify the correct local directory.

STEP 8:

Now, your files should be created in your machine. Now what? For this, you will need the handy-dandy S3Fox. Get it here: http://www.s3fox.net/ Launch the tool from your browser. Now, you can simply sync your folders from your machine to your S3 buckets. Its as easy as that!

STEP ???:

Grab your favorite coffee and wait for the script to complete. Try to sleep for awhile, watch a movie, maybe give me a tip?


PROFIT!!!!

Check your buckets after some time and enjoy the presence of your easily uploaded files. :)

Forcing urls to use https

I found this extremely useful:

1
2
3
def redirect_to_ssl
    redirect_to url_for params.merge({:protocol => 'https://'}) unless request.ssl?
end

Put this in your application controller and call

1
before_filter :redirect_to_ssl

on every controller that would require ssl. You can pick which actions would only require this by extending your before_filter declaration as such:

1
before_filter :redirect_to_ssl, :only => [:new, :purchase]

or even by exclusion, whichever is more convenient for your case:

1
before_filter :redirect_to_ssl, :except => [:show]

PS: There are other resources too, such as this one (that could be of help): http://fuadcse.blogspot.com/2009/01/redirecting-http-request-to-https-in.html

  • Facebook
  • LinkedIn
  • FriendFeed
  • Twitter
  • Tumblr
  • Flickr
  • YouTube
  • Vimeo