Tech Your Universe

For anyone waiting to see how the early adopters do, I went to the Mac OSX Leopard Release at the Apple Store in San Francisco and bought myself a Family Pack. I’m happy to report that Leopard upgrade was successful on 3 different computers now.

• 1 Macbook Pro
• 1 Ibook G4
• 1 Imac G4

Very stable on all 3, and it seems faster to me. Maybe this is just the "fresh install" effect though.Important software that I can say works without any compatibility issues:

• Quicksilver
• Microsoft Office
• Cisco VPN
• Firefox
• Camino
• Adium
• smc fan control

My key "wow" features:

1. Spaces kicks ass. This was my biggest reason for upgrading.
2. Mail 3.0 support "todo list" that are stored on the IMAP server, so I can access them from anywhere.
3. Safari 3.0 has the coolest "find" feature I’ve ever seen in a browser.

Trac is decent bug tracking software that is getting a lot of attention these days. It’s very simple to use, which I like. It includes subversion integration so that you can walk through your source code tree and view diffs from one version to another. Handy.

However, Trac’s installation process has a lot of room for improvement in the simplicity department. Here are the steps I went through to install trac on Cent OS 5. I captured them so that others may find it easier to get Trac up and running. Enjoy.

If these instructions work for you, please leave a comment! If they don’t, please leave a comment!

1. Install python and its goodies.
   1. Use yum to install the base package: yum install python
   2. Install mod_python. This is so Apache can run the python scripts as a module:
      yum install mod_python
   3. Install MySQL-python so that python can interact with mysql
      1. Download and untar MySQL-python from http://sourceforge.net/projects/mysql-python
      2. cd $mysqlpythonsourcedir
      3. python setup.py build && python setup.py install
2. Install some devel packages that are needed for compiling trac and svn integration:
   yum install neon neon-devel python-devel swig
3. Install Clearsilver, a templating package that trac depends on.
   1. Download and untar clearsilver from http://www.clearsilver.net/downloads/
   2. Compile cd $clearsilversourcedir; ./configure && make && make install
4. Install trac!
   1. Download and untar trac from http://trac.edgewall.org/wiki/TracDownload
   2. cd $tracsourcedir; python ./setup.py install
   3. Initialize your first trac project:
      trac-admin $pathtoyourtracproject initenv
   4. Tell Apache about trac.. Add the following lines to your Apache configuration file:

        <Location />
          SetHandler mod_python
          PythonHandler trac.web.mod python_frontend
          PythonOption TracEnv $pathtoyourtracproject
          PythonOption TracUriRoot /trac/
        </Location>
      

Finally, restart httpd and try it out by going to:

http://$yourhost/trac/

If it doesn’t work, check your webserver error log for errors:
tail -f /var/log/httpd/error_log

Again, please leave a comment so that you may help others with the same problems you have.

Inevitably mysql replication gets borked. Some sql statement will fail, which stalls mysql replication. While this can be mitigated by restricting access on the slave, anyone who has used mysql replication in production knows that it can, and will happen. So plan for it.

If you see something like this when you run SHOW SLAVE STATUS Tip: Use the \G instead of the semicolon at the end of SQL statements, it produces more readable output.

You can try to skip the failed statement with:

mysql>STOP SLAVE();
mysql>SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1;
mysql>START SLAVE();

But that can’t solve all problems, as that sql statement you just skipped may have been needed to keep the data in sync. So what do you do? It’s time to reset the data on the slave. That’s right, wipe it and grab a fresh copy from the master. It’s the only way.

To grab a fresh copy of the data from the master, here you have several options.

1. Run a mysqldump on the master with –master-data and reimport on slave. Note that while the mysqldump is running on the master, tables are locked and the database is unusable
2. Stop mysql on the master and take a snapshot of all the data using tar or a filesystem snapshot
3. Use ibbackup. Note that ibbackup costs money and it will only work for InnoDB files, so you have to do something else for your MyISAM tables.

The problem with all of these solutions? They require downtime on the master database. That may be fine for some situations, but everywhere I’ve worked having downtime on the master data isn’t a good thing. So this is where you wish that you had thought ahead, and you had another slave that you could grab the master snapshot from. Aha!

Always have more than one slave. Put it on a piece of crap box some where. It’s another copy of your data for redundancy, and it gives you another place besides the master to grab the data from in the event that you need a snapshot of the data.

How do you rebuild a slave from another slave you ask? I’ll write up a step by step guide some day, but here’s a rough overview off the top of my head:

1. Stop mysql on the working slave
2. tar up all the files. This is everything in the "data" directory and the ibdata* files if you are using InnoDB.
3. Make a copy of the master.info file
4. Start mysql on the working slave
5. Stop mysql on the broken slave.
6. Untar all the data files.
7. Edit the my.cnf on the broken slave, and add "skip_slave_start=1″ in the [mysqld] section.
8. Start mysql on broken slave. Note that replication won’t be running yet because of the skip_slave_start line you added in the previous step.
9. Get the position from the master.info, and use "CHANGE MASTER TO" to reset the log file position on the broken slave to match the one on the working slave
10. Run START SLAVE on the working slave, check slave status with SHOW SLAVE STATUS to see if it worked.
11. Remove the skip_slave_start from the my.cnf file so the slave will start next time mysql is restarted.