My Default CentOS Setup

This script will probably only be valid for a week, but I thought I would share my ideas on a good CentOS 5 (64bit) install with Ruby on Rails, MySQL and Apache that will work with a capistrano deployment from subversion:

yum -y update
yum -y install ruby ruby-libs ruby-mode ruby-rdoc ruby-irb ruby-ri ruby-docs ruby-devel mysql mysql-devel mysql-server mysql-admin subversion httpd svn

togglesebool httpd_can_network_connect

cd /tmp
wget http://rubyforge.org/frs/download.php/29548/rubygems-1.0.1.tgz
tar -xvf rubygems-1.0.1.tgz
cd rubygems-1.0.1
ruby setup.rb

gem install rails json rfacebook acts_as_ferret capistrano mongrel mongrel_cluster pdf-toolkit actionmailer actionpack actionwebservice activerecord activeresource activesupport acts_as_taggable acts_as_versioned ferret google4r calendar mysql sources vpim mime-type --include-dependencies

#Fedora 8 also wants
yum -y install rubygem-mongrel gcc ruby-mysql
yum install gd gd-devel zlib-devel openssl-devel
gem install gem_plugin daemons capistrano --include-dependencies
gem install mongrel mongrel_cluster railsmachine --include-dependencies
gem install --version=2.7 mysql -- --with-mysql-config=/usr/bin/mysql_config

or

yum -y install ruby ruby-libs ruby-mode ruby-rdoc ruby-irb ruby-ri ruby-docs ruby-devel mysql mysql-devel mysql-server mysql-admin subversion httpd svn rubygem-mongrel gcc ruby-mysql gd gd-devel zlib-devel openssl-devel partimage
gem install -y rails -v=2.0.2
gem install json rfacebook acts_as_ferret capistrano mongrel mongrel_cluster pdf-toolkit actionmailer actionpack actionwebservice activerecord activeresource activesupport acts_as_taggable acts_as_versioned ferret google4r calendar mysql sources vpim mime-type ferret mime-types solr-ruby ruby-openid mechanize pdf-toolkit mongrel mongrel_cluster railsmachine bio --include-dependencies


I tried to make it as silent and complete as possible so that I could let the computer do all the work.

Mysql Master to Master Replication

This will setup two Amazon EC2 CentOS 5 machines to cluster replicate all their MySQL databases.

Server 1: /etc/my.sql


#replication
server-id=1
log-bin=mysql-bin
binlog-ignore-db=mysql
binlog-ignore-db=test

#information for becoming slave.
master-host =
master-user = replication
master-password = slave
master-port = 3306



Server 2: /etc/my.sql

#replication
server-id=2
log-bin=mysql-bin
binlog-ignore-db=mysql
binlog-ignore-db=test

#information for becoming slave.
master-host =
master-user = replication
master-password = slave
master-port = 3306


Restart /etc/init.d/mysql restart on both servers.

Then run the following SQL on...
server 1:

STOP SLAVE;
CHANGE MASTER TO MASTER_HOST='10.253.15.15',MASTER_USER='replication',MASTER_PASSWORD='slave';
START SLAVE;
GRANT ALL PRIVILEGES ON *.* TO 'replication'@'' identified by 'slave';

server 2:

MASTER_HOST='10.253.65.221',MASTER_USER='replication',MASTER_PASSWORD='slave';
STOP SLAVE;
show slave status\G;
GRANT ALL PRIVILEGES ON *.* TO 'replication'@'' identified by 'slave';


You should then be able to create and remove databases on either servers and it will replicate to the other.

I tried to use "load data from master" but it only worked on very simple data. On large sets, it was better to use mysqlhotcopy (perl script that came with mysql on centos)

On CentOS, you first need to export the mysql directory in the /etc/exports file like this:
var/lib/mysql 10.253.15.15(rw,no_root_squash)
and then run:
exportfs -r

then on the machine with the tables, you run:
mkdir /tmp/mounted_mysql_directory
mount -o vers-3 10.253.65.221:/var/lib/mysql /tmp/mounted_mysql_directory

Now that you have the directory exported and mounted, you can hot copy the files from server 1 to server 2:

mysqlhotcopy --addtodest --resetmaster --resetslave -u root -p yourdatabasename_development /tmp/mounted_mysql_directory

The resetmaster and resetslave are important here. MySQL will keep an index in the replication log where it thinks its master is and where it is as a slave. After I copy the database over, I want them to start fresh. So, from here you can look at the master status on server 1 and master status on server 2 to get these indexes. then you can run CHANGE MASTER TO with an extra parameter:
MASTER_LOG_POS=98;
if the index was 98. This will then get them both in sync and ready to accept further transactions to apply.

You can also skip a given number of replication transactions using:
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = N;
just be careful what you skip.

Coming from a Notes/Domino background, this idea of replicating transactions was a little new. Notes, will keep a sequence number for each record and keep the data in sync. It does not matter how the data got into its current state, it will just do its best at keeping the data replicated (using sequence numbers and time stamps as needed).

MySQL is more transactional based. The documents talk about the my.cnf parameters like "binlog-ignore-db" will act as a filter for how transactions get written to the replication log. To a Notes person, this seems a little inefficient. It means to keep the data in sync, you have to run the exact same transactions on each server (even if the operations are repetitive and act on the same records). In Notes, it will be able to skip ahead to the final state rather than going through each transaction to get to the final state. MySQL has no such ability.

As a result, MySQL cannot just sync itself if you run into problems. This is also true when you add a member to the cluster or replace a failed one. To get things in sync again, you really have to think it through and make sure you lock tables as needed, get the data the same with copy tools like mysqlhotcopy and let the transactions start again on the cluster when you know it is ready.

I would also expect this would have a different scalability path than Notes - if your bottleneck is the amount of transactions you are getting, then adding additional replicas may not help in MySQL where it would in Notes.

Upgrading to rails 2.0.2

There were just a few too many cool features in 2.0 that I wanted so I have taken the plunge in a project. I did wait for 2.0.2, so it is not really bleeding edge - but I feel nice and up to date.
The process was fairly straight forward even though I knew I used a lot of plugins and deprecated methods. I had to perform the following to get it working:

• gem install rails
• script/plugin install svn://errtheblog.com/svn/plugins/classic_pagination
  
• script/plugin install http://svn.rubyonrails.org/rails/plugins/in_place_editing/

I also referenced:

• http://mislav.caboo.se/rails/rails-2-0-taking-the-plunge/
• http://localtone.blogspot.com/2008/01/upgrade-to-rails-20.html

After restarting the server, users will get a warning about cookies. However, this is a small price to pay for cookie based sessions.

bellasante of boston

I tried to make a booking for my wife and a friend's wife yesterday for a spa treatment. So, I called the concierge to book it in for friday afternoon. The problem was they seemed very busy and I didn't know what to order or which location would be best. I didn't mind either Lexington or Wellesley but the concierge seemed in a rush so made me pick one right away. Then I was told after picking Wellesley there was nothing on the day I wanted.
I really felt like they were too busy to deal with me and decided the best way was to get off the phone as quickly as possible and try again with another concierge or spa. After replying to a few "contact us" forms for a site I am working on, I thought I should at least let them know the feedback to see if indeed they are too busy or if it was just a bad time and I got the wrong concierge. In any case, I filled out their web form only to find it was a "post" action to "#" and didn't actually go anywhere.
They seem to have a lot of "best of boston" awards so the spa part might be nice - it just seems that half the challenge is to get to that part.

Jack Cards on the WSJ.

One of the startup web sites we have been working on and hosting at ProjectLounge, was reviewed by the Wall Street Journal. I am looking forward to taking the site to the next level - but a review like this is certainly a good start for the site. It is full of good suggestions and a positive conclusion.
"I think its system would be welcomed by many people looking for good-quality, well-designed cards that catch the eye. It saves users the hassle of last-minute trips to the card store, and still lets greeting cards keep their personal touch, which is what makes them such a permanent fixture in our lives."

Fire Truck Photos

This morning we saw a Fire Truck visit our street - no fire or anything exciting. However, I did capture some photos.

Commando Crawl

This our friends commando Teddy.

Everyone loves cats

Mount ssh/sftp drives into OSX

Dealing with a server that is hosted requires a stronger level of security. This typically means no ftp or file sharing is allowed. However, ssh is secure and you can even prevent passwords and use rsa keys.
However, all this security does not mean you cannot mount a drive using OSX. Using the MacFuse at google, you can mount a remote ssh/sftp drive with a hostname, username, password (or rsa key) and path.
Steps:
1. Download and install MacFuse from the link
2. Restart your OSX (it does some low level magic that makes writing mounted disks easier)
3. Download and copy the sshfs into your Applications from the link.
Now that you have it download, you can run the sshfs application and connect to your disk.
If you do use an RSA key or the like, you will need to first connect via the command line so that the host gets to be trusted and then it will work fine from there on.

DIIOP NotesException: Server access denied

The R5 settings in the server document still play a part in how DIIOP handles permissions in 7.0.2. The getFirstDatabase function in DbDirectory, will return "Server access denied" if the "Allow HTTP clients to browse databases" is not set to "Yes" on the R5 Basics tab under HTTP Internet Protocols in your Domino server document.

Customized iPhone ringtone

One the the missing features of the iPhone is not being able to select our own ringtone. Well, thanks to this web site, I was able to upload my own mp3 file as a ringtone and it worked.
It is not as easy as other phones but still possible with a little instructions.

Outlook Permissions for ICS files

A user recently got the error: "Can't create file: calendarevent.ics. Right-click the folder you want to create the file in, and then click Properties on the shortcut menu to check your permissions for the folder."

After some serching, it is not really a permission problem but rather when you install certain Microsoft Small Business products, the OutlookSecureTempFolder key in the registry is changed from "..\Temp\..." to "...\Temporary Internet Files\...".

To fix this, have your system administrator, go into the registry and change the HKEY_CURRENT_USER\Software\Microsoft\Office\11.0\Outlook\Security registry key OutlookSecureTempFolder value from "...\Temporary Internet Files\..." to "...\Temp\..." and you should be able to open ICS events from QuickPlace or any other service again.

Christmas come early?

Quickr has shipped early and after installing it and the latest fixes from "fix central", I have started the process of picking through the new features.
I have tested our existing themes and they all seem to work just fine so far and with a little more testing, I think we will be safe to upgrade some servers.
I have to say well done to Satwik and his team for getting QuickPlace team buzzing like it was 1999.

It is all happening on the 29th

On the 29th of June, there will be so many things to do, I won't know what to do with myself.
1. Quickr 8 Launches;
2. iPhone is released; and
3. Sicko comes out.
I am sure there will be lots more happening that day, but these are things I have actually been waiting for and anticipating.
This week, I have been very busy sorting out some Windows Vista issues with QuickPlace 7 as well as a ruby on rails development project for jackcards. I am also in the process of installing a NAS for domino which so far indicates 2 to 4 times increase in our current IO performance.
So, if I have been a little quiet lately, I apologize and will be a little more sociable soon (even if it is just to show off an iPhone if I get the courage to buy one).

Rational Climate Change Argument

Rational Climate Change Argument

This is basically the same as Pascal's Wager with some key differences. These are that Pascal was based on religion where this is actually based on science. However, it is a very good use of a truth table

Ruby on Rails Domino Active Record

I have been using this class for a while that allows some ActiveRecord like features for IBM Lotus Domino databases.

An example use would be:


class Contact < DominoRecord
attr_accessor :email_address, :first_name, :last_name

validates_presence_of :first_name, :last_name, :email_address

validates_format_of :email_address, :with =>

/^\s*(?:(?:[^,@\s]+)@(?:(?:[-a-z0-9]+\.)+[a-z]{2,}\s*(,\s*|\z)))+$/i,

:message => "must be a valid email address",

:on => :create

@@database_name = "contacts.nsf" #your database here

@@server_name = 'www/projectlounge' #your server here

end


It currently uses the OLE/COM bridge to get to domino so only will work on windows at this stage. The next step would be to compile it with the NotesAPI to allow it to work on all platforms.

The model source code is here (I do not maintain this anymore but happy for others to take it on under LGPL):


require 'win32ole'

class DominoRecord



attr_accessor :form



def save

@document = @@database.CreateDocument

form = self.class.to_s.downcase

@document.ReplaceItemValue "form", form

self.instance_variables.each do |inst|

field_name = inst.sub(/[@]/, '')

field_value = self.instance_variable_get(inst)

@document.ReplaceItemValue field_name, field_value.to_s

end

@document.Save(true, false)

end



def save!;

save

end

def update_attribute; end

def new_record?; end

include ActiveRecord::Validations



@@session = nil

@@database = nil

@@database_name = nil

@@server_name = nil



def [](key)

instance_variable_get(key)

end



def self.find(*args)

options = extract_options_from_args!(args)

validate_find_options(options)



case args.first

when :first then

@document = find_initial(options)

when :all then find_every(options)

else

@document = find_from_ids(args, options)

end



record = self.new

@document.Items.each do |item|

record.send(item.Name.downcase + "=", item.Text) unless ["document", "errors"].include?(item.Name)

end

record

end



def DominoRecord.human_attribute_name(attribute_key_name)

ActiveRecord::Base.human_attribute_name(attribute_key_name)

end



def initialize(attributes=nil)

unless @@session

@@session = WIN32OLE.new('Lotus.NotesSession')

@@session.Initialize

end

@@database = @@session.GetDatabase(@@server_name,@@database_name) unless @@database



unless form == nil

fields = @@database.GetForm(form).Fields

else

fields = @@database.GetForm(self.class.to_s).Fields

end



fields.each { |field| self.class.__send__(:attr_accessor, field) }



return if attributes.nil?

attributes.each do |k, v|

send(k + "=", v)

end

end



def finalize

@@session.Finalize

end



def session

@@session

end



def database

@@database

end



def document

@document

end



def attributes=(attributes)

return if attributes.nil?

attributes.stringify_keys!

multi_parameter_attributes = []

attributes.each do |k, v|

send(k + "=", v)

end

end



def self.method_missing(method_id, *arguments)



unless @@session

@@session = WIN32OLE.new('Lotus.NotesSession')

@@session.Initialize

end

@@database = @@session.GetDatabase(@@server_name,@@database_name) unless @@database



if match = /find_by_([_a-zA-Z]\w*)/.match(method_id.to_s)



@view = @@database.GetView($1)

puts arguments

@document = @view.GetDocumentByKey(arguments[0])



record = self.new

@document.Items.each do |item|

record.send(item.Name.downcase + "=", item.Text) unless ["document", "errors"].include?(item.Name)

end

record

else

super

end

end



private

def self.find_initial(options)

#not working

end



def self.find_every(options)

#not working

end



def self.find_from_ids(ids, options)

#not working

end



def self.find_one(id, options)



if result = @@database.GetDocumentByUNID( id )

result

else

raise RecordNotFound, "Couldn't find #{name} with ID=#{id}#{conditions}"

end

end



def self.find_some(ids, options)



result = find_every(options)



if result.size == ids.size

result

else

raise RecordNotFound, "Couldn't find all #{name.pluralize} with IDs (#{ids_list})#{conditions}"

end

end





def self.extract_options_from_args!(args) #:nodoc:

args.last.is_a?(Hash) ? args.pop : {}

end



VALID_FIND_OPTIONS = [ :conditions, :include, :joins, :limit, :offset,

:order, :select, :readonly, :group, :from ]



def self.validate_find_options(options) #:nodoc:

options.assert_valid_keys(VALID_FIND_OPTIONS)

end

end


Allow selections not in list (for the web)

One of the neat options with drop down selections in Notes, is the ability to let people type an option that is not in the list. This helps build dynamic lists.

Here is some javascript to use to get this working on the web:


function changed(el, cmp, pmt){
if(el.options[el.selectedIndex].value==cmp) {addoption(el, pmt);}
}
function addoption(el, pmt){
var txt=prompt(pmt,'');
if(txt==null) {return;}
var o=new Option( txt, txt, false, true);
el.options[el.options.length]=o;
}


You can either put this in an included javascript file or wrap it directly inline.

Then you can call it using the onchange event for your selection field:

onchange="changed(this, 'New Group', 'Please enter new group:')"

Where "New Group" is the text you want to trigger the prompt for the text and "Please enter new group" is the prompt you want to display.

A week into Linked In

A customer recently sent me an email to join linked in, a social networking "business" site. It seems myspace is more for music fans, facebook for college or recent graduates where linked in is more for professionals.
The differences are certainly there in the UI. For instance, in facebook, you can say if a linked friend "hooked up" with you whereas there is no option for this in linked in. I guess it still goes on, but it is not something you would publish as it has little business benefit and does not make you look very professional.
I was surprised how many IBM'ers were already on the network and thank those who have linked to me. The purpose is to find connections, so I thought I would give it a little test.
The test worked. I have managed to find one lost friend through our xhtml developer. I worked with David Wyss for a while around the QuickPlace redbook and he had seemingly vanished. I had asked a few common contacts (IBM) what happened to him but they didn't know. When I searched through linked in, it found a connection through Australia (where I was looking in Switzerland) and it came through a non-IBM source (where I looked through IBM'ers). So, it seems we were not very far from each other after all - there was always 2 degrees of separation all this time and we just didn't know.

Change custom field names in QuickPlace

If you have a custom form in quickplace and then change a field name, it will not go through and update all the pages that have been created by this form.
So, for instance, if you create a form that has a field called "categories" and want to change it to "category", here is an agent that will loop through all the documents in this room and all its inner rooms using recursion.

'category correction:

Option Public
Option Declare

Dim server As String
Dim placename As String
Dim s As notessession
Sub fixcategory(roompath)
Dim roomdb As NotesDatabase
Dim roomview As notesview

Set roomdb = s.GetDatabase(server, roompath)
If roomdb.IsOpen = False Then
Call roomdb.Open(server, roompath)
End If


Dim roomindex As notesview
Set roomindex = roomdb.GetView("System\Index")
Dim page As notesdocument

Set page = roomindex.GetFirstDocument

While Not(page Is Nothing)
'Work through all documents
page.c_category = page.GetItemValue("c_categories")

Call page.Save(True,False)
Set page = roomindex.GetNextDocument(page)
Wend

Set roomview = roomdb.GetView("System\Subrooms")
Dim doc As NotesDocument
Set doc = roomview.GetFirstDocument
While Not (doc Is Nothing)
Dim iroomtitle As String
Dim iroomfile As String

iroomtitle = doc.GetItemValue("h_Name")(0)
iroomfile = doc.GetItemValue("h_LocDbName")(0)

Print "Working on: " + iroomfile
Call fixcategory("quickplace/"+placename+"/" + iroomfile)

Set doc = roomview.GetNextDocument(doc)
Wend
End Sub
Sub Initialize
Set s = New notessession
placename = "test_place" 'Your place name
server = "www/projectlounge" 'Your server name

Print "Starting in a room"
'you can change this to main.nsf if you need
Call fixcategory("quickplace/"+placename+"/PageLibrary85257279005D300B.nsf")
End Sub




Rediscovering an ibm.com treasure

While digging up an old email from 2004 for someone, I found that my signature once contained a link to the IBM culture clash web site.
It does have some good information and for those that have traveled to some of these countries, it certainly can bring back some memories.