Handy shell command to list out the Apache processes & what they're up to

ps axo 'pid user size %cpu %mem cmd' | grep http | grep -v "\(root\|grep\)"

I do more & more Linux work from the shell, and it's starting to grow on me. ;)

I used to search through files using this command

find . | xargs grep -s 'keyword'

but now & again get errors like xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

So I found this command works instead

find . -printf '"%p"\n' | xargs grep -s 'keyword'

or you can make a handy shell script (e.g. search.sh) like this

#!/bin/bash

find . -printf '"%p"\n' | xargs grep -s "$1"

and then search files like

search.sh 'keyword'

SliceHost and Affordable Linux Hosting

I'm trying out a new provider, SliceHost, for the upcoming Portland Open Coffee Club and Portland Open Beer Club web sites. SliceHost provides fast & affordable Linux VPSes with a variety of distros to choose from.

The one downside (for me at least) is that the distros come with nothing installed. Well, I shouldn't say nothing (since SSH is on there), but no database, no web server, no ftp, no PHP, no mail. I was surprised when I first uncovered this, since I ordered an Ubuntu 8.04 server VPS & figured it would have a normal LAMP stack installed. But no, it's a minimal Linux install. That's so if you have specific needs, you can install exactly what you want & avoid installing anything you don't. But if you're more of a programmer than a sysadmin ... well, you may want to check out RimuHosting for a VPS. Great support, nice VPSes, good upgrade path, & the boxes are ready for your PHP code.

Or if you want a bit more hand-holding, you can go with GoDaddy or Dreamhost. GoDaddy's support is meh, but they're easy to get started with and are fine for a small site. Dreamhost has better support, but recently I had a really bad performance experience with them & canceled my account.

So anyhow, this wasn't supposed to all be about affordable Linux hosting providers -- I was also going to mention securing SSH. 

Securing SSH

I've been following PickledOnion's great Ubuntu setup guide, and it mentioned securing SSH by moving it to a different port (other than port 22). You can basically edit /etc/ssh/sshd_config, change the port number at the top, and then restart ssh with /etc/init.d/ssh restart.

Note: once you change the port, don't log out of SSH right away! Instead, open a new SSH window & try connecting to your box on the new port. If it doesn't work, you can go back to your previous still-open SSH window & troubleshoot.

Another option in /etc/ssh/sshd_config is to disallow root access, which is definitely a good idea. Although, if you end up SUDOing users so they have access to everything, then it doesn't protect as much as you think, since someone who can log in as you could do almost as much damage as root, including editing /etc/ssh/sshd_config to switch the ports to something different and allow root back in.

An additional way to secure SSH is to edit your firewall to block most IPs from accessing SSH, and block any unneeded ports. I'll refer you to the setup guide again ... scroll down & you'll see the section on iptables.

BTW, experienced Linux admins probably already know the above tips (plus more!), but I figured there's a lot of people getting into Linux admin work these days, so some basic admin knowledge doesn't hurt.

TechTarget has an article on how to set access control lists to define default file permissions. This came in handy for us, as we have several developers who work on our sites. The default file permissions would get annoying, as developer A would create some files which only he could edit, so developer B would have to use sudo to change the file permissions to a developers group.

With the TechTarget article, now all newly created files in our code directory are owned & editable by the developers group, making shared development much easier.

If you need to recursively FTP files (e.g. you're pulling down data, or doing offsite backups), take a look at lftp. It's normally available on Linux boxes. One nice thing is that by default it will only transfer changed/new files. You can use it in a script or cron job.

Rimuhosting has a nice FTP overview on how to use it.

ncftp is another, more powerful FTP client that is also worth checking out.

Happy New Year folks. And, here's an interesting link to some Joomla! 1.5 performance testing. I find it interesting because

1. Joomla! 1.5 beta 2 was slower than good ol' Joomla! 1.0.13 RC
2. There were discussions of a number of PHP caching techniques which can be applied to any PHP system

The caching techniques covered were "file caching" (which sounded like filebased output buffering aka Cache_Lite_Output to me), Alternative PHP Cache (APC), eAccelerator and Memcache. xcache was mentioned in the comments but not tested.

So if you're interested in a quick overview & comparison of some PHP cachine techniques, go ahead and read the article ... I found it interesting that despite memcache's reputation, its performance fell behind other caching methods.

I may have mentioned before that I'm doing a LAMP project these days (among other things). As I go along I'm taking notes on various tasks, etc. so I figured I could post some of them to my blog. Currently the site is hosted on a virtual dedicated server running Fedora Core 4. Anyhow...

Logging MySQL Queries

You may need to turn on query logging for mysql from time to time in order to see what SQL is being passed to the database. Here's how to do so in Linux. This makes some assumptions about directories, so you may need to execute the commands in different places depending on your setup. FYI, I did the following on a Fedora Core 4 distro. Also, if you're running a busy server, your query log will probably get really huge, so be forewarned! I wouldn't leave query logging running all the time, or at least not on a production machine.

Anyhow, shell into your server, log in with an administratively enabled account, and follow the below steps.

Create the MySQL logging file

If you haven't created the MySQL logging file before, you'll need to do so.

First create an empty file with

touch /var/log/mysqlq.log

That makes an empty file. Now change the ownership to the mysql account to it via

chown mysql /var/log/mysqlq.log

Now the mysql user account can write to that file.

Enable MySQL logging

Now you need to stop mysql

/usr/bin/mysqladmin -u root -p shutdown

you'll be prompted for the root password, enter it and mysql will shut down. You may need to hit ENTER after you get the shutdown message in order to get your prompt back.

Now start mysql with logging enabled.

/usr/bin/mysqld_safe --log="/var/log/mysqlq.log" &

The trailing ampersand is important, otherwise you won't get your shell prompt back.

You should see a message about logging and mysql starting. Hit ENTER to get your shell prompt.

Using the Log

You can now run some web sites or do some things that query the MySQL database and the results should be logged.

You can look at the log using vim or other editors. You can look at the last 100 lines of the log via

tail -100 /var/log/mysqlq.log

Note that your log will get pretty big, so to turn off logging, stop MySQL using the shutdown command above, and then start it back up with

/usr/bin/mysqld_safe &

Clearing the Log

You may want to clear your log from time to time if it gets too big. To do first, first make sure that MySQL is not currently logging to the file. You can see if any process is accessing the log by running this command & seeing if anything is returned. If no lines return then nothing is using the log.

fuser /var/log/mysqlq.log

Before clearing the log you could make a backup of it using

cp /var/log/mysqlq.log "$(date +%Y%m%d)-mysqlq.log"

That will create a copy of the log file with today's date in the filename, e.g. 20070408-mysqlq.log

You can then clear the log file using

> /var/log/mysqlq.log

Yes, you type the > in that command. :)

Troubleshooting

If you don't see commands being written to your log, look at the /var/log/mysqld.log log file to see what the problem is.

I've been using Webmin to manage my Linux server, but I wanted the backup files to be timestamped so that subsequent backups didn't overwrite them. Since I didn't see an option for that, I created a simple Linux shell script that will prepend a timestamp to a filename. So for example, if the shell script is called stamp_file, then running the command

./stamp_file /var/backup/apache_config_backup.tar.gz

will make a copy of /var/backup/apache_config_backup.tar.gz file, and call it something like like /var/backup/20070323142367-apache_config_backup.tar.gz. Note that the original file is maintained, and that the copy starts with YYYYMMDDhhmmss.

Anyhow, here's the script

#!/bin/sh
# timestamp a file
# args: <fullfilename>

echo -e "\n\n"

# check arguments
if [ $# -ne 1 ]; then
        echo -e "Usage: $0 <fullfilename> \n\n"
        echo -e "e.g. $0 /var/backups/apache_backup.tar.gz \n\n"
        exit 127
fi

echo -e "Timestamping $1 \n\n"

# get file, directory, and new filename
fname=$(basename $1)
newname="$(date +%Y%m%d%H%M%S)-$fname"
dname=$(dirname $1)

# make a copy
cp -vf $dname/$fname $dname/$newname

echo "\n\nDone, created $dname/$newname\n\n"

Now for all my backup tasks I can just run the stamp_file script to make sure that I don't overwrite older backups, which means I can do daily or weekly backups & go back in time when I need to. Ideas for future expansion might be to have the script make folders for each month, copy the backups into those.

This was inspired by a form post over in startupping, where people were discussing doing projects in ASP.NET or PHP (specifically, LAMP: Linux, Apache, MySQL, and PHP).

I do both ASP.NET (.NET 2.0, C#, SQL Server) & PHP development (LAMP). PHP is pretty easy to pick up (especially if you've programmed in classic ASP), and there's a lot of information, libraries, etc. out for it. For an IDE you could use Zend Studio ($99-$300), PHP Designer ($50+), or free ones like SciTE or PSPad (both are good).

If you just want to try out some PHP development, I'd suggest getting a cheap hosting account from GoDaddy, Dreamhost, or somewhere else. Or even e-rice.net ($10/year). While you could install Linux locally & play around with shell access & administering Linux, Apache, MySQL, and PHP, you could end up spending a lot of time being a Linux sysadmin -- time that you could have instead spent strengthening your PHP & MySQL skills. One step up from a cheap hosting account would be to run LAMP as a virtual machine by downloading the free VM player and ready-to-go LAMP packages.

Once you get beyond coding some basic stuff, you may want to look into better PHP frameworks to help accelerate your development. I like Code Igniter, but here's a list of other frameworks you can read about.

FWIW, I develop in both environments b/c I have a range of clients & projects w/ different needs. One of my current projects is a site that includes a lot of features (social networking, wiki, CMS, data management), but the client wanted to leverage existing open source libraries & avoid building everything from scratch. So LAMP was a natural way to go. Other clients are Microsoft shops, so they want .NET apps built that their existing IT staff can take over.

Like any project, I think deciding on a technology involves many factors -- how skilled are you in it? Do you want to learn a new technology? What does the technology cost? Are there things about this technology (language features, environmental features, pre-existing libraries & applications) that will help the project to be more successful?

Lastly, if it's costs that you're concerned about, IMO Microsoft projects aren't as expensive as some may think. You don't need to spend a lot of money on the IDE -- there are cheap or free IDEs you can develop in, or (if you qualify) you can enroll in the MS Empower for ISVs program for $375, which gets you an MSDN Universal license w/ OSes & dev tools. The .NET framework is free, and you can run IIS on XP Pro. SQL Server Express is free & has (almost) all the features that you'd be using in SQL Server Workgroup/Standard/Enterprise. I have a client that didn't have a very large database, so their production site runs on SQL Express.

Also, if you're using a hosting provider for your site (which you usually should), choosing a Windows OS & SQL Server for your hosting account isn't usually significantly more expensive than Linux.

Still, obviously LAMP is cheaper (free, unless you pay $$ for a better IDE), and Microsoft costs can add up if you're looking at big server farms or are buying your OS & SQL Server licenses outright (for some reason). But of course switching frameworks has its own cost in terms of time & mistakes made while learning. If I were building something simple from the ground up, I might just choose the platform I was strongest in.