rsync only certain files

Create a file on on the origin server and use the –files-from option

cd /
rsync -anvi --files-from=/tmp/only_these_files . to-server:/
 
cat /tmp/only_these_files
/srv/www/mysite/uploaded_files/9a3d3d9046aaad4bbe3074d794a6adaf.jpg
/srv/www/mysite/uploaded_files/5ca1c97efd730427f846e457bd0a0667.png
/srv/www/mysite/uploaded_files/5b0923025b4a1fde3a34119bfcd56567.jpg
/srv/www/mysite/uploaded_files/870ed45d9ad704deace8fb541242425a.jpg

Backgrounded PHP jobs in ‘STOPPED’ state

This could be specific to the PHP package from Ubuntu. Please consider the following a disclaimer.

$ php -v
PHP 5.3.5-1ubuntu7.2 with Suhosin-Patch (cli) (built: May  2 2011 23:00:17) 
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
    with Xdebug v2.1.0, Copyright (c) 2002-2010, by Derick Rethans
 
$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=11.04
DISTRIB_CODENAME=natty
DISTRIB_DESCRIPTION="Ubuntu 11.04"

I have a file with the following contents:

$ cat stdin.php
<?php
echo microtime(), "n";

and I was doing something like this (backgrounds the script. I was using a slightly modified version to make concurrent SOAP requests)

$ for i in `seq 1 2`; do echo $i; sleep 2s; php /tmp/stdin.php &gt;&gt; /tmp/k7.out & done
$ jobs
[1]-  Stopped                 php /tmp/stdin.php &gt;&gt; /tmp/k7.out
[2]+  Stopped                 php /tmp/stdin.php &gt;&gt; /tmp/k7.out

There are a few threads on the Internet relating to the issue and there are a few solutions.
First let us kill the ‘STOPPED’ jobs

$ kill %1
$ kill %2
[1]   Terminated              php /tmp/stdin.php &gt;&gt; /tmp/k7.out
$ jobs
[2]+  Terminated              php /tmp/stdin.php &gt;&gt; /tmp/k7.out

One of the solutions that worked for me was by supplying something to STDIN (perhaps STDIN was blocking, but then again, stream_set_blocking (STDIN, 0) wasn’t of much help).

$ for i in `seq 1 2`; do echo $i; sleep 2s; php /tmp/stdin.php &gt;&gt; /tmp/k7.out &lt; /dev/null & done

and you could simulate arguments

$ for i in `seq 1 2`; do echo $i; sleep 2s; echo arg1 arg2 | php /tmp/stdin.php &gt;&gt; /tmp/k7.out & done
$ tail -f k7.out
0.82262900 1316606606
array(2) {
  [0]=&gt;
  string(4) "arg1"
  [1]=&gt;
  string(4) "arg2"
}
0.83546500 1316606608
array(2) {
  [0]=&gt;
  string(4) "arg1"
  [1]=&gt;
  string(4) "arg2"
}

Checking for the existence of domains from commandline in Bash

while read line; do wget --quiet --spider --timeout=10 $line; if [ $? -ne 0 ]; then echo $line; fi; done < "/tmp/domains.txt" 
 
/tmp/domains.txt would hold the list of domain names separated by newline.
Note this is just a quick hack and not entirely reliable.

SSMTP, Crontab gotchas

I had a few cron entries that did not run and for a long a time I had no idea why.

I had the MAILTO variable set at the top of the crontab and I was expecting to be mailed the output of the cron entries. I was surprised when that did not happen. Looking at the mail.err log I came to know that my ssmtp setup was not working as I could see a few entries like this:

Jan 19 09:48:52 localhost sSMTP[22760]: RCPT TO:<testing ssmtp setup@> (553 5.1.2 or other punctuation after the recipient's email address. o1sm8676654wfl.14)

I had gmail setup using ssmtp and the settings seemed to be right. After doing some research I decided to install the ‘mailx’ package and that solved the mail issue and I started receiving the errors from the crontab.

The errors were indicating that the commands in the crontab had a few syntax errors but, they ran fine when I copied and pasted them into my shell and hit enter. After some trial and error I figured out the piece of code that was triggering the error to be something like this:

$(date +'%y%m%d')

I immediately put those into a shell script and used that in the crontab. That did the trick and my cron jobs were running smoothly. I was not too happy with the solution and continued investigating. One of the error messages:

/bin/sh: Syntax error: EOF in backquote substitution

led me to a solution. It turns out that ‘%’ character in my crontab entry was the source of trouble and it had to be escaped as mentioned in the manpage:

Percent-signs (%) in the command, unless
escaped with backslash (), will be changed into newline characters, and
all data after the first % will be sent to the command as standard input.

So, my modified command looked like this:

$(date +'%y%m%d')

And all was good from there.

Also, please remember to save and quit the editor used to edit crontab entries for the new/updated entries to be installed(that’s what happened when my editor was vim).

Using GNU grep to output just the match(es) and not the entire line containing the match(es)

echo "hello there." | grep -o "ello t"
ello t

The flag ‘o’ (not Zero) does the trick.
From the man page:

-o, –only-matching
Show only the part of a matching line that matches PATTERN.

I couldn’t find that flag as an option in the BSD Grep man page. So this might only be available with GNU version of Grep.