Pentaho Spoon error

If you get an error like this:

Error connecting to database: (using class org.gjt.mm.mysql.Driver)
Unknown initial character set index ‘192’ received from server. Initial client character set can be forced via the ‘characterEncoding’ property.

Then in the database connection settings, do as highlighted in the screenshot below.

kettle_db_character_encoding

SSMTP, Crontab gotchas

I had a few cron entries that did not run and for a long a time I had no idea why.

I had the MAILTO variable set at the top of the crontab and I was expecting to be mailed the output of the cron entries. I was surprised when that did not happen. Looking at the mail.err log I came to know that my ssmtp setup was not working as I could see a few entries like this:

Jan 19 09:48:52 localhost sSMTP[22760]: RCPT TO:<testing ssmtp setup@> (553 5.1.2 or other punctuation after the recipient's email address. o1sm8676654wfl.14)

I had gmail setup using ssmtp and the settings seemed to be right. After doing some research I decided to install the ‘mailx’ package and that solved the mail issue and I started receiving the errors from the crontab.

The errors were indicating that the commands in the crontab had a few syntax errors but, they ran fine when I copied and pasted them into my shell and hit enter. After some trial and error I figured out the piece of code that was triggering the error to be something like this:

$(date +'%y%m%d')

I immediately put those into a shell script and used that in the crontab. That did the trick and my cron jobs were running smoothly. I was not too happy with the solution and continued investigating. One of the error messages:

/bin/sh: Syntax error: EOF in backquote substitution

led me to a solution. It turns out that ‘%’ character in my crontab entry was the source of trouble and it had to be escaped as mentioned in the manpage:

Percent-signs (%) in the command, unless
escaped with backslash (), will be changed into newline characters, and
all data after the first % will be sent to the command as standard input.

So, my modified command looked like this:

$(date +'%y%m%d')

And all was good from there.

Also, please remember to save and quit the editor used to edit crontab entries for the new/updated entries to be installed(that’s what happened when my editor was vim).

Using GNU grep to output just the match(es) and not the entire line containing the match(es)

echo "hello there." | grep -o "ello t"
ello t

The flag ‘o’ (not Zero) does the trick.
From the man page:

-o, –only-matching
Show only the part of a matching line that matches PATTERN.

I couldn’t find that flag as an option in the BSD Grep man page. So this might only be available with GNU version of Grep.

PHP, MongoDB, findAndModify

findAndModify is not supported by the current version of the PHP MongoDB driver, so I had to use the following code

$feed = $db->command(array('findAndModify' => 'feedList', 'query' => array('$where' => new MongoCode('
                                 function(){ 
                                      return ((this.inProgress == false)  
                                          || (currentTime >= (this.lastUpdated + this.updateFrequency))));
                                     }', array('currentTime' => $start))), 'update' => array('$set' => array('inProgress' => true))));

Where:

  • $db = Database handle
  • ‘feedList’ = Name of the Collection(Table)
  • inProgress, currentTime,updateFrequency and lastUpdated are some keys
  • $start = a MongoDate Obj

Essentially I am locking a document(set ‘inProgress’ to true) by finding and updating the first document that that is not locked by another process (‘inProgress’ set to false and a few other conditions)

Notice the use of ‘this’ in the MongoCode Obj. I think it refers to the current Document that is being processed.

Also be aware that findAndModify only finds and modifies the first document found.

GNU sed, replace search string with contents of a file

Say, I want to replace the text “replace me” in all the .txt files in the current directory with the contents of modified.js file

find . -name "*.txt" -print0 | xargs -0 sed -i -e 's/replace me/cat modified.js/e'

Notice the ‘/e’ at the end and that seems to do the trick.

jQuery empty(),remove() and Thickbox

I am a great fan of thickbox and I have used it for a long time. Recently I came across a situation where trying to close a modal that had a table with around 7000 rows (each 8 columns) crashed the tab in Chrome and took a very long time in Firefox.

I went through the thickbox forum and discovered that other people have had similar issues and they were related to the tb_remove() function. I did my own profiling and came to the same conclusion.Of particular interest was this statement:

$("#TB_window").fadeOut("fast",function({$('#TB_window,#TB_overlay,#TB_HideSelect').
trigger("unload").unbind().remove();});

I looked at the documentation and then read through the comments where someone mentioned that remove() can be quite expensive due to stuff that it does and he suggested to use empty() before remove(). Voila, it worked! So for anyone else facing the same issue with thickbox, change the statement above to:

$("#TB_window").fadeOut("fast",function({$('#TB_window,#TB_overlay,#TB_HideSelect').
trigger("unload").unbind().empty().remove();});

and see if that works for you.

Munin: Could not open required defaults file: /root/.my.cnf

For some reason I kept getting this error (even when the file was readable by all) in one of the munin (mysql slave) plugins that I had written.

So this was the solution that worked for me;

under the [mysql*] section in /etc/munin/plugin-conf.d/munin-node 
I added this: 
env.mysqlopts -u{user} -p{password}

and in my plugin

I did something like this:

mysql ${mysqlopts} -e'show slave status G'

and that seemed to have picked up the user and password from env.mysqlopts and the plugin started working (ofcourse after I restarted munin).