Tuesday, December 15, 2009

Extending Ruby on Rails Contacts Gem to invite Facebook users


Contacts Gem is an interface to import email contacts from various providers like Yahoo, Gmail, Hotmail, and Plaxo including AOL to avail invite functionality.


sudo gem install contacts
git clone git://github.com/cardmagic/contacts.git
Do not forget to install Json Gem as well.
gem isntall json

Facebook Inviter

First thing first. You need to have Mechanize Gem including its dependency(Nokogiri). Very simple...
sudo gem install mechanize
this will install Nokogiri as well.

Using Contacts Gem as Plugin

One more thing, I am using Contacts Gem as plugin to get things easier. Making this Gem as plugin is not a big deal! Move the Gem, either getting it from Local Gem repository C:\ruby\lib\ruby\gems\1.8\gems\ or whatever you have downloaded, to '#{RAILS_ROOT}/vendor/plugins'.
Then create initializer init.rb into #{RAILS_ROOT}/vendor/plugins/contacts/.

require 'contacts'
Get to the console(restart to load the plugin) to check it out.
C:\Me\Workspace\contacts>ruby script/console
Loading development environment (Rails 2.3.4)
>> Contacts
=> Contacts
So far so good.

Now back to facebook fetcher Fix a base class(say page_scraper.rb) for all the fetchers(facebook, orkut etc...) that uses mechanize into #{RAILS_ROOT}/vendor/plugins/contacts/lib/contacts extending Contacts's base class as follow.

require 'mechanize'
class Contacts
 class PageScraper < Base
   attr_accessor :agent
   # creates the Mechanize agent used to do the scraping 
   def create_agent
     self.agent = WWW::Mechanize.new
     agent.keep_alive = false
   # Logging in
   def prepare; end # stub

   def validate(page)
      return false if page.nil?
      return false unless page.respond_to?('forms')
      return true
   def strip_html( html )
     html.gsub(/<\/?[^>]*>/, '')
Again in the same location(#{RAILS_ROOT}/vendor/plugins/contacts/lib/contacts) put facebook.rb containing...
require 'page_scraper'
class Contacts
  class Facebook < PageScraper
    URL                 = "http://www.facebook.com/"
    LOGIN_URL      = "http://m.facebook.com/"
    PROTOCOL_ERROR      = "Facebook has changed its protocols, please upgrade this library."
    def real_connect
    def prepare
     page = agent.get(LOGIN_URL)
     raise ConnectionError, PROTOCOL_ERROR unless validate(page)
      login_form = page.forms.first
     login_form.email = @login
     login_form.pass = @password
     login_form.charset_test = ''
     page = login_form.submit
     raise AuthenticationError, "Email and Password do not match!" if page.body =~ /Incorrect email\/password combination/
     #Friends Page
     friends_links = page.links_with(:href => /\/friends.php/)
     friends_link = friends_links.collect{|flink| flink.href if flink.text == 'Friends'}.compact.first
     raise ConnectionError, PROTOCOL_ERROR unless friends_link
     friends_url = 'http://m.facebook.com' + friends_link
     page = agent.get(friends_url)
     #My Friends Page
     my_friends_link = page.links_with(:href => /\/friends.php/)
     @everyone = my_friends_link.collect{|l| l.href if l.text == 'Everyone'}.compact.first
     @everyone = my_friends_link.collect{|l| l.href if l.text == 'Friends'}.compact.first unless @everyone
     @logout_url = page.link_with(:href => /logout/)
     return true
    def contacts
     contacts = []
     next_page = true
     current_page = 2
     everyone = 'http://m.facebook.com' + @everyone
     page = agent.get(everyone)
     raise AuthenticationError, "You must login first!" if page.body =~ /Enter your login and password/
      next_page = false
      data = page.search('//tr[@valign="top"]')
      if data
       data.children.each do |node|
        pf_nodes = node.children.children
        if pf_nodes
         fr_name = msg_link = ''
         pf_nodes.each do |nd|
          fr_name = nd.text if nd.name == 'a' && nd.has_attribute?('href')
          if nd.name == 'span' || nd.text.include?('Message')
           nd.children.each do |n|
            msg_link = n.attributes['href'] if n.name == 'a' && n.text == 'Message' && n.has_attribute?('href') && n.attributes['href'].to_s.include?('compose')
         contacts << [fr_name, msg_link.to_s] unless msg_link.blank?
      end # End Outer if
      data = page.search('//div[@class="pad"]')
    data.each do |node|
     childs = node.children
     next if childs.empty?
     childs.each do |c|
      if c.name == 'a' && c.text.to_i == current_page
       next_page = 'http://m.facebook.com' + c.attributes['href'].to_s
      current_page += 1
    page = agent.get(next_page) unless next_page == false
     end #End While
     return contacts
    def send_message(contact, message, subj=nil)
     success = false
     if contact
      url_inbox = "http://m.facebook.com" + contact.first
      page = agent.get(url_inbox)
      raise AuthenticationError, "You must login first!" if page.body =~ /Enter your login and password/
      return false unless validate(page)
      page.forms.each do |f|
       if f.has_field?('body')
        f.subject = subj if subj && !subj.empty?
        f.body = message
        f.charset_test = ''
        page = agent.submit(f, f.buttons.first)
        return true if page.code == '200'
     return success
     def logout
      agent.click @logout_url if @logout_url
      return true

  TYPES[:facebook] = Facebook
Ultimately, Open lib/contacts.rb and add this at the end of the line as
require 'facebook'
You are done


C:\Me\Workspace\contacts>ruby script/console
Loading development environment (Rails 2.3.4)
>> fb=Contacts::Facebook.new('facebookid','password')
>> contacts = fb.contacts
=> [["User1", "/inbox/?r3a2c6b2e&r39ade298&compose&ids=100000485673107&refid=5"], ["User2", "/inbox/?r46861c8d&rd5ef5fb6&compose&ids=691998819&refid=5"]]
>> contacts.each{|c|fb.send_message(c, 'Your Message', 'Subject')}
Done! but scopes are always there for improvement...:)

Mechanize TypeError: can't convert nil into String


>> page = agent.submit(login_form)
TypeError: can't convert nil into String
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/util.rb:40:in `iconv'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/util.rb:40:in `from_native_charset'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:152:in `from_native_charset'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:144:in `proc_query'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:143:in `map'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:143:in `proc_query'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:166:in `build_query'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:165:in `each'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:165:in `build_query'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/form.rb:214:in `request_data'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize.rb:401:in `post_form'
        from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize.rb:344:in `submit'

Patch(Line 40)

root@khojguru:~# vi /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize/util.rb
def from_native_charset(s, code)
  if Mechanize.html_parser == Nokogiri::HTML
    return unless s
    - Iconv.iconv(code, "UTF-8", s).join("")
    + Iconv.iconv(code.to_s, "UTF-8", s.to_s).join("")
    return s

LoadError: no such file to load -- mechanize


root@khojguru:~# irb
irb(main):001:0> require 'mechanize'
LoadError: no such file to load -- mechanize
        from (irb):1:in `require'
        from (irb):1


Because the of versioned nature of the gems repository, RubyGems doesn’t store the library files directly in standard library search path. It adds the necessary gem packages to the library search path as needed at run time.
This means that RubyGems must be loaded before any gem libraries are accessable.


Add this
export RUBYOPT=rubygems


To reload the changes you need to fire...
root@khojguru:~#source ~/.bashrc

Check it out

irb(main):003:0> require 'mechanize'
=> true

Sunday, August 30, 2009

Overriding Rails Auto Complete Text Field

auto_complete is excellent ajaxified plugin for auto completion, but round-trip server request make it a little bit slow. I could not help myself to make it faster enough for real time data but I did override this to dig through data from distinct field instead of pre-built auto-completion method that does a direct SQL query on the field and view whatever  I wanted by overriding controller and view.
First you'll need to install the auto_complete plugin:
ruby script/plugin install auto_complete
or download it right from here and move to vender/plugin.
This will give you the "text_field_with_auto_complete" view helper method amongst others and a controller hook "auto_complete_for" to implement the auto-complete action/method in your controller as per the auto_complete README, it looks like:
class BlogController < ApplicationController
   auto_compelere_for :post, :title
It would implement a auto_complete_for_post_title method in your controller class. The parameters are the object and field/method of that object. The method(auto_complete_for_post_title) it implements will dig through all the Post records in your database and do a LIKE comparison on the title column, comparing the title to the contents of the post[title] form field. With the results, it will generate the HTML for an unordered list (ul), and return that to the view. Now you can have this line in your view to be done for single field
<%= text_field_with_auto_complete :post, :title %>
Now Its time to play as we want and its as simple as that. Simply implement(override) the method and view according to you. Example: Controller
def auto_complete_for_post_title
   t = params[:post][:title]
   @results = Post.find_by_sql(:all, 
    :select => 'p.id, m.col1, p.col2, p.col3...', :from => 'posts p', 
     :joins => 'LEFT JOIN other o ON p.something = o.something',
    :conditions => ["p.something in (?) AND p.something = 1 AND LOWER(title) LIKE ?", some_array, '%' + v + '%'],
    :order => 'col1,col2,...')
   render :partial => '/common/autocomplete_post_title'
Since I needed same text field on different page, hence I just thrust it into application.rb to avail it anywhere, otherwise you need to put it in appropriate controller. Example: View - common/autocomplete_post_title.rthml
<ul id="something">
   <% for result @results do -%>
      <li id="<%= result.something%>">
       <%= result.something -%>
          <%= result.something %>
    <% end -%>
Tweak it in whatever way... To override in-built plugin's CSS, add {:skip_style => true} as below
<%= text_field_with_auto_complete :post, :title, { :class=>"input", :name => 'something', :value=>"something", :title=>"something", :onclick =>"DoSomething(this);", :onblur =>"DoSomething(this);" }, { :skip_style => true } %>