JungleDisk and Amazon S3 are Full of Win!

I am now using JungleDisk for backups and Amazon S3 for “cloud” storage. It is very affordable, portable, fast and easy to use. JungleDisk manages the backups, deals with communicating with Amazon S3, and can provide a mounted network drive where I can store files like any other mounted drive.

Below is a screenshot tour of everything, obviously on a Mac, yet the application windows look the same on other platforms.

This is the screenshot of Amazon S3 service summary. It is very very cheap and it will only be cheaper each time you update from the initial update as you’ll be moving less data and making less HTTP requests.

This is a Finder window showing the mounted JungleDisk mounted so that I can use the network drive like any other drive.

Just showing you can create a new directory (or folder) on the network drive in Finder without any issues. “backups” is where JungleDisk stores your backups, and “~VersionArchive” holds previous versions.

Showing data being backed up in the main Jungle Disk window.

Configuration, application settings.



Configuration, Jungle Disk account settings.



Configuration, Jungle Disk networking settings.



Configuration, bandwidth limiting settings.



Configuration, default “Bucket”.



Configuration, default Bucket settings.



Configuration, Jungle Disk Plus settings.



Configuration, network drive settings.



Configuration, encryption settings.



Configuration, previous versions settings.



Configuration, automatic backup settings.



Configuration, my backup settings.



Configuration, my backup schedule settings.



Configuration, pick and choose what directories/folders/files to backup.



Configuration, misc backup settings.



Configuration, add another “bucket”.





So, it is easy to use. Not shown is the restore screens, yet it is pretty much what you’d expect. You browse for what to restore and it defaults to asking to restore the directory/folder/file to the original location. It can then replace the file there.

It’s speed is fast, at least much faster than Mozy, because Mozy is very very slow for home users.

Easy to use, easy to setup, and worry free. The data storage is also on Amazon servers, which is very comforting as they are ran by very smart people and have a lot of experience in data storage and retrieval.

While I still do rsync over ssh to my colo machine, I still want this type of service. I like that I can keep previous versions of files (like Time Machine) and built on reliable solid services and companies.

Give it a shot. It’s very cheap!

UPDATE

You can create a bucket that is in compatibility mode so that it can be browsed, etc with other S3 services. It is in the bucket creation options (as noted by Scott of Jungle Disk below in the comments):



More details on bucket types.

Mozy means waiting for days to backup?

I decided to give Mozy a real shot. It has many nice features and what really sold me on it was the ability to restore via the web, even from different machines.

Here is how ridiculous the upload speed is, from my work, which is a datacenter with bandwidth speeds that are hardware limits (with obvious overhead of protocols, etc):

From reading the FAQ at https://mozy.com/support, it states:

I have LOTS of data. How long will it take to back up?

For a typical system on a typical broadband line, Mozy backs up data at about 2-4 GB per day. But if left undisturbed on a fast connection, you can back up over 9 GB in a single day.

You may experience faster or slower speeds depending on your connection. You may want to check your favorite bandwidth meter test to measure your broadband connection. Here is one from Speakeasy that we like to use. You will be uploading to the Mozy servers when you back up, so pay attention to the “Upload” part of the test results.

Ok, so that makes sense… we all know cable companies seriously limit the upload ability of their internet clients to boost the download speed (which is obviously more desirable).

So, I sent a support email:

The connection at my work, a datacenter and ISP, is very fast yet why is the upload speed to your servers so slow? Is this on purpose or a bug?

I know the documentation states the speed of upload, yet thought that was relative to cable internet clients.

Using http://www.speakeasy.net/speedtest/ as stated at https://mozy.com/support:

- San Francisco
Download Speed: 14554 kbps (1819.3 KB/sec transfer rate)
Upload Speed: 11153 kbps (1394.1 KB/sec transfer rate)

- Dallas
Download Speed: 22945 kbps (2868.1 KB/sec transfer rate)
Upload Speed: 13207 kbps (1650.9 KB/sec transfer rate)

- New York
Download Speed: 76350 kbps (9543.8 KB/sec transfer rate)
Upload Speed: 9608 kbps (1201 KB/sec transfer rate)

Why am I only uploading at under 1000 Kbits/sec, usually around 700-500?

Thanks,
Christopher

Here is the reply I got back:

Hello there,

Is your bandwidth being throttled either by the mozy program or by your work? If the internal network busy at your place of business? Where are you located in the world? Many things contribute to the “slower than theoretically possible” bandwidth issue.

Thank you for choosing Mozy Remote Backup Software for your online backup needs.

Best Regards,

Joe

Now there is nothing on my end blocking upload speed, unless it is a silent “feature” of the Mozy client. Also it is much much “slower than theoretically possible”. Here is the reply I sent:

I don’t see a bandwidth throttle option in the Mac home client, if there is how can I change it?
Bandwidth is not throttled by my work.
The internal network is not busy to slow this down.

I am located in Scranton, PA, we have multiple high bandwidth lines of fiber, so hardware limits from the office to the net.

Do your servers rate limit? Would you like a traceroute?

This is a pain as I have files I want to back up and it states that it will take days, this is a laptop and I got on and offline all the time, so would have to keep restarting for a week just to get the initial backup done.

Thanks,
Chris

Hopefully they’ll fix whatever the problem is. This upload speed to their servers is just not fast enough. If it was fast, I’d be nothing by praising the service.

What a pain…

UPDATE

I quit Mozy yesterday, deleted my account and uninstalled the software. It is just too slow. I am now using JungleDisk (which I’ll blog about shortly).

Here is the response I got this morning (the following day) about this issue, basically “just wait and let it finish”… yeah, right.

Hello there,

I see - I don’t know why but it didn’t occur to me that you were on a mac. We cap unlimited and home users at 1Mbps upload speed, and aside from that we have no constraints… I am leaning towards the idea that there is a bottleneck between us somewhere causing you to not reach the full 1mbps, but it’s difficult to test that.

I suggest just letting it upload when you have time, and eventually it will finish. If you are uploading a very large file, try not to interrupt it because it will restart that file when you resume next time.

Thank you for choosing Mozy Remote Backup Software for your online backup needs.
Best Regards,

Joe

Plurk Theme - Mine

Not pretty and probably can be cleaned up some, yet *meh*.
http://www.plurk.com/user/niroze

#timeline_holder {
background: url(’http://niroze.net/images/KungfuPanda-Boo-1.png’) no-repeat fixed top left;
}
body,html {
background: url(’http://niroze.net/images/wp.png’) repeat fixed;
}

/* Hide Plurk logo and creature */
#dynamic_logo { opacity:0;filter:alpha(opacity=0);zoom:1 }

#plurk_form {
margin-left: auto; margin-right: auto;
width: 760px;
}
#about_me {
background-image: url(’http://niroze.net/images/san_damiano_cross-1.png’);
background-repeat: no-repeat;
background-position: top right;
}
#plurk-dashboard {
border: none;
background: transparent;
width: 750px;
margin-left: auto;
margin-right: auto;
}
.dash-segment {
border: none;
background: transparent;
width: 250px;
}
.segment-content {
border: solid 3px #000;
padding: 4px;
margin: 5px;
background: #333;
-moz-border-radius: 2em;
border-radius: 2em;
}
#dash-stats {
/* Number is your user id number, is the avatar pic that shows on mouseover of the default theme*/
background: url(’http://avatars.plurk.com/353852-big.jpg’) no-repeat top center;
padding-top: 200px;
}
#dash-stats h2 {
position: absolute;
top: -2000px;
left: -2000px;
}
#dash-stats h3 {
font-size: 18px;
color: #000;
background: #ccc;
border: solid 1px #777;
-moz-border-radius: 4px;
border-radius: 4px;
margin-left: 0px;
text-align: center;
}
/* Don’t show the javascript profile picture */
#profile_pic {
position: absolute;
top: -2000px;
left: -2000px;
width: 0px;
height:0px;
padding: 0px;
margin: 0px;
}
html, body {
font-family:  “Helvetica Neue”, “Lucida Grande”, Verdana, Arial, sans-serif;
}
#full_name {
padding: none;
margin: none;
}

TrueCrypt Screenshot Tour

You can find the program and everything about it on their website: truecrypt.org. It is powerful encryption that is free and works on Windows, Mac and Linux. Check it out!

Main screen.

Select “Create Volume” and this is the create wizard start screen.

Select standard type of file container. You can read on the website about the hidden volume.

Select the “Select File” button and then a file dialog comes.

It doesn’t matter what you name the file, just needs to be a file. Note the warning on the previous screen, if you pick an existing file it will be deleted and replaced.

Select “Next >”, again be sure to read the text on this screen.

Select the type of encryption algorithm(s) and the hash. You can read more about encryption algorithms and hash algorithms.

Select the size of the file. This will be a static size, if you have all that used up with files or not, so create what you’ll need.

Create a password that will be needed to mount the volume on the filesystem. You’ll want to always remember this password.

If you select the “Display password” checkbox, you’ll be able to see what you typed.

Select what filesystem type you want to format the volume as. One the Mac, it is just FAT (which is great).

This is the part where you format the volume and generate the encryption keys.

Formatting…

Dialog window pops up letting you know formatting is complete.

Click “Exit” unless you want to create another volume.

On the main screen, select the “Select File” button and an open file dialog comes up. Select the volume you just created.

Choose an empty spot at the top of the main screen. Also ensure you have a directory created on the filesystem to mount the encrypted volume.

Select “Mount” and enter your password. Select the “Options” button and enter the mount point at the bottom. Select the “Ok” button to mount the encrypted volume on the filesystem.

With the volume now mounted, select the slot on the main window and select the “Volume Properties” button. This dialog window will show giving you details about it.

There are some interesting features not shown here like hiding keys in keyfiles that can be any file like an mp3 or picture. There is also the feature of having the volume being hidden and unable to be identified (says their documentation). You can even install an operating system on top of the encrypted volume (how cool is that?!).

You can find out more by reading their documentation (that’s well done).

Using Perl’s Test::Class for Organized Unit Testing

I’m not going to go over the merits of unit testing. I’ve heard it all and discussed both sides till blue in the face. If you’re unsure about unit testing, then please google it or ask some programmers you know about it.

What I do want to discuss is how one does unit testing in Perl. It’s pretty simple, fortunately.

In the world of testing classes in CPAN, there are a few that you’ll actually use: Test::Simple, Test::More, Test::Harness, and Test::Class. There are basically two schools of testing that is either Test::Harness based or Test::Class based. Test::Harness school runs a series of scripts which have tests in them top down, usually with a plan at the top of the file (plan is the number of tests you are planning on running). Test::Class manages itself, using Test::Class for tests and to run them (Test::Class allows you to have a plan per Test::Class subclass subroutine… I’ll explain in a bit).

Today, I’m not going to talk about Test::Harness or Test::Simple. I will some other time.

I will talk about Test::More and Test::Class. They’re pretty awesome and really really simple to use. Don’t believe me?! Let’s jump into some code.

In this example set, we’re going to be dealing with a Hotdog Vendor class. It’s purpose is to provide a way to know about a Hotdog Vendor and what’s in his cart (east coast style). The Hotdog Vendor class only does a few things: takes the vendors name and how many hotdogs he plans on selling. We’re assuming in this example a hotdog is a bun with a frank in it with mustard and kraut (yeah, sounds awesome).

Here’s the Hotdog Vendor class (download here):

#
# HotdogVendor.pm: Provide a Hotdog Vendor
#
package HotdogVendor; 

use strict;
use warnings;

sub new {
my ($class, $name, $how_many) = @_;
my $attrs = {
franks => $how_many,
buns => $how_many,
mustard => $how_many,
kraut => $how_many,
name => $name
};
bless ($attrs, $class);
}

sub _use_product {
my ($self, $product, $how_many) = @_;

if (($self->{$product} - $how_many) < 1) {
die "[$self->{name}] use_$product: Unable to processor order, not enough $product (you wanted $how_many, I only got $self->{$product}";
}

$self->{$product} -= $how_many;
}

sub _has_product {
my ($self, $product) = @_;

if (!defined $self->{$product}) {
die "[$self->{name}] has_$product: undefined product (wrong vendor?!)";
}

return ($self->{$product});
}

# Franks
sub use_franks {
my ($self, $how_many) = @_;

$self->_use_product('franks', $how_many);
}
sub has_franks {
my ($self) = shift;

return ($self->_has_product('franks'));
}

# Buns
sub use_buns {
my ($self, $how_many) = @_;

$self->_use_product('buns', $how_many);
}
sub has_buns {
my ($self) = shift;

return ($self->_has_product('buns'));
}

# Mustard
sub use_mustard {
my ($self, $how_many) = @_;

$self->_use_product('mustard', $how_many);
}
sub has_mustard {
my ($self) = shift;

return ($self->_has_product('mustard'));
}

# Kraut
sub use_kraut {
my ($self, $how_many) = @_;

$self->_use_product('kraut', $how_many);
}
sub has_kraut {
my ($self) = shift;

return ($self->_has_product('kraut'));
}

# Name
sub name {
my ($self) = shift;

return ($self->{name});
}

1;

As you can see, it saves the name and amount for each product. It then provides subroutines to get the amount of product left, use up some product, and see what the vendor’s name is. Pretty simple.. excuse my copy and paste.

Now we need to write some tests to make sure the subroutines work. It’s pretty simple, in fact writing tests is brain dead simple it almost seems like it’s too simple to even bother with. Wrong. Things change over time, simple changes can cause big problems if not checked. Anyways, enough preaching, time for tests (which should have came before the code, but you knew that already, eh?).

Here is the Hotdog Vendor Test class (download here):

#
# HotdogVendor_Test.pl: HotdogVendor.pm Test (Test::Class, unit testing)
# 

package HotdogVendorTest;

use base qw(Test::Class);
use Test::More;

use HotdogVendor;

# Test that name is saved on new vendor creation
sub test_name : Test(1) {
my $name = "Thomas";
my $hotdogVendor = HotdogVendor->new($name, 100);
is ($hotdogVendor->name, $name, 'name saved');
}

# Test product franks is saved and used works
sub test_franks : Test(2) {
my $how_many = 100;
my $hotdogVendor = HotdogVendor->new('Chris', $how_many);
is ($hotdogVendor->has_franks, $how_many, 'franks amount saved');
$hotdogVendor->use_franks(60);
is ($hotdogVendor->has_franks, 40, '100 - 60 franks is 40 franks');
}

# Test product buns is saved and used works
sub test_buns : Test(2) {
my $how_many = 100;
my $hotdogVendor = HotdogVendor->new('Chris', $how_many);
is ($hotdogVendor->has_buns, $how_many, 'buns amount saved');
$hotdogVendor->use_buns(60);
is ($hotdogVendor->has_buns, 40, '100 - 60 buns is 40 buns');
}

# Test product mustard is saved and used works
sub test_mustard : Test(2) {
my $how_many = 100;
my $hotdogVendor = HotdogVendor->new('Chris', $how_many);
is ($hotdogVendor->has_mustard, $how_many, 'mustard amount saved');
$hotdogVendor->use_mustard(60);
is ($hotdogVendor->has_mustard, 40, '100 - 60 mustard is 40 mustard');
}

# Test product mustard is saved and used works
sub test_kraut : Test(2) {
my $how_many = 100;
my $hotdogVendor = HotdogVendor->new('Chris', $how_many);
is ($hotdogVendor->has_kraut, $how_many, 'kraut amount saved');
$hotdogVendor->use_kraut(60);
is ($hotdogVendor->has_kraut, 40, '100 - 60 kraut is 40 kraut');
}
1;

As you can see, it’s using the “secret” Perl attribute technique. You can look that up yourself. Just know that it is pretty much what it seems. You can search about the specifics on the Test::Class CPAN page. The one I’m using is ‘Test(<number of tests in this subroutine>)’, which is pretty simple. A much easier way to organize tests, versus Test::Harness.

How do you run the tests? Here is how (download here):

#!/opt/local/bin/perl
#
use strict;
use warnings; 

use Test::Class;

use HotdogVendorTest;

Test::Class->runtests;

The output:

$ /opt/local/bin/perl run_tests.pl
1..9
ok 1 - buns amount saved
ok 2 - 100 - 60 buns is 40 buns
ok 3 - franks amount saved
ok 4 - 100 - 60 franks is 40 franks
ok 5 - kraut amount saved
ok 6 - 100 - 60 kraut is 40 kraut
ok 7 - mustard amount saved
ok 8 - 100 - 60 mustard is 40 mustard
ok 9 - name saved

Pretty simple? Yes. Unit testing is simple and you can be as resource intensive and take as long as you want to run the tests, as that stuff doesn’t matter. The only thing that matters is testing tiny parts of your class and program modules. It’s pretty easy if your code isn’t one big huge function that does 10 more things than it needs to be doing.

Comments? Suggestions? This is my first in a big series of Perl and Mac development articles. I’m starting off simple so I have something to build on for later articles. Hrm, now that I think about it… what do you want to know about??

Let me know and happy testing!

Catalyst and DBIx::Class: Part 1

[For instructions on the installation of Catalyst, please see the Catalyst Manual page on installation.]

This is an attempt at introducing DBIx::Class and using it in Catalyst, so you can be familiar with it enough to start using it.

Understanding ORM

First, you’ll need to understand the purpose of Object-relational mapping. It is what DBIx::Class is. To quote the wikipedia.org page:

Object-Relational mapping (aka O/RM, ORM, and O/R mapping) is a programming technique for converting data between incompatible type systems in databases and Object-oriented programming languages. This creates, in effect, a “virtual object database” which can be used from within the programming language. 

So, what you end up with is classes that have the ability to look and feel like database tables, yet all read/write operations happen behind the scenes. This enables you to work with the database without having to make database specific calls. This also allows you to code where all you need to worry about in the code is just what to call, not what database is under the hood. DBIx::Class makes this easy and is pretty simple to wrap your brain around.

The basic idea is that you have one controller master class, and then one or more different classes that usually represent each table in the database. The controller master class is where you will specify the database connection information (and other optional settings) and load the table classes.

This may seem complex, yet it makes life much easier and allow your code to be much more flexibly to change. It also fulfills the need of having the Controller layer not be aware of the Model layer details in the Model-View-Controller design pattern (which is most common in serious web applications).

Getting to know DBIx::Class

Please take a moment to go over the examples in the DBIx::Class CPAN page.

Then go over the manuals (they’re pretty brief and include code to explain everything): Intro, Example, Joining, Cookbook with interest on prefetch and joins, and Troubleshooting.

These documents do a much better job at explaining how to use DBIx::Class and all it’s specifics than I could do. Be sure to check out the documentation map also.

Using DBIx::Class in Catalyst

Please table a moment to go over the Catalyst manual tutorial, the database access with DBIx::Class section. This will get you familiar with the basics.

If you already have an existing database schema and it’s in a database, you can have Catalyst create them for you. It’s advised that you just do this once, so that you can save some typing initially (if you already have a database with schema in it… like in this example). First make sure you’re in your Catalyst root directory, made with this:

$ catalyst.pl MyTestApp

Then you can use the Catalyst create script to create your DBIx::Class classes for you:

$ cd MyTestApp
$ ./script/mytestapp_create.pl model DB DBIC::Schema DB::Schema create=static dbi:Pg:dbname=ticketingsystem chris
exists "/home/chris/MyTestApp/script/../lib/MyTestApp/Model"
exists "/home/chris/MyTestApp/script/../t"
Dumping manual schema for DB::Schema to directory /home/chris/MyTestApp/script/../lib ...
Schema dump completed.
created "/home/chris/MyTestApp/script/../lib/MyTestApp/Model/DB.pm"
created "/home/chris/MyTestApp/script/../t/model_DB.t"
$

Ok, now let’s take a look at what it created:

$ head -13 lib/MyTestApp/Model/DB.pm
package MyTestApp::Model::DB; 

use strict;
use base 'Catalyst::Model::DBIC::Schema';

__PACKAGE__->config(
schema_class => 'DB::Schema',
connect_info => [
'dbi:Pg:dbname=ticketingsystem',
'chris',

],
);
$

Ok, so from looking at the Model class for this database, it seems pretty easy to figure out. It creates a class that is based off Catalyst::Model::DBIC::Schema and sets up some configuration values (what the master DBIx::Class is and the database connection information used by DBI). Ok, now lets look at the DB::Schema file:

$ cat lib/DB/Schema.pm
package DB::Schema; 

# Created by DBIx::Class::Schema::Loader v0.03009 @ 2007-09-04 19:26:55

use strict;
use warnings;

use base 'DBIx::Class::Schema';

__PACKAGE__->load_classes;

1;

$

Hrm, even simplier right? ‘__PACKAGE__->load_classes’ subroutine loads up all the configured schema classes in the lib/DB/Schema/ directory. Let’s look at one:

$ cat lib/DB/Schema/Status.pm
package DB::Schema::Status; 

# Created by DBIx::Class::Schema::Loader v0.03009 @ 2007-09-04 19:26:55

use strict;
use warnings;

use base 'DBIx::Class';

__PACKAGE__->load_components("PK::Auto", "Core");
__PACKAGE__->table("status");
__PACKAGE__->add_columns(
"id",
{
data_type => "integer",
default_value => "nextval('status_id_seq'::regclass)",
is_nullable => 0,
size => 4,
},
"name",
{
data_type => "character varying",
default_value => undef,
is_nullable => 0,
size => 40,
},
);
__PACKAGE__->set_primary_key("id");
__PACKAGE__->has_many(
"users_ticket_status_logs",
"DB::Schema::UsersTicketStatusLog",
{ "foreign.status_id" => "self.id" },
);
__PACKAGE__->has_many(
"staff_ticket_status_logs",
"DB::Schema::StaffTicketStatusLog",
{ "foreign.status_id" => "self.id" },
);
__PACKAGE__->has_many(
"tickets",
"DB::Schema::Ticket",
{ "foreign.status_id" => "self.id" },
);

1;

This was obviously generated from PostgreSQL, as you can note the default_value for “id”. Anyways, without focusing too much on that, the important parts are all the function calls (’has_many’, ’set_primary_key’, ‘add_columns’, ‘table’, and ‘load_components’). Here is a breakdown on what they all are for:

• load_components: Loads the components you need, which should almost always be the same (PK::Auto and Core) unless you need custom components detailed in the DBIx::Class component manual. [technically I guess you don't need PK::Auto as it's in Core now]

 

• table: obviously sets the name of the table

 

• add_columns: defines the columns of the table

 

• set_primary_key: sets the primary key of the table

 

• has_many: one of a few relationship definition functions for the table, see more about them in detail here.

 

It’s pretty simple once you read the documentation links I gave. The naming of everything is pretty intuitive, so makes wrapping your brain around what’s going on pretty easy.

Summary

So you should now have an idea of what DBIx::Class’ purpose is and some details about it’s use. You should also know where to look for documentation on each part discussed here and have a good general knowledge of DBIx::Class’ implementation of ORM.

You should have a simple understanding of DBIx::Class and Catalyst. I’ll continue in another article on more details and dive more into using DBIx::Class in Catalyst Controllers and using multiple databases.

Hopefully this was useful to you. I’d like to keep going, yet it’s more appropriate to break this up into parts. If I made any mistakes or feel I should have written something in a better way, let me know and I’ll be sure to update!

Enjoy playing with Catalyst and DBIx::Class!

Using Perl’s Net::SSH::Perl

One important point of Net::SSH::Perl => no humans required. Yeah, a BIG win.

It’s purpose is to provide all the SSH client functionality, yet purely in Perl where the ssh binary is not required. The big feature is that you don’t have to type the password manually or setup passwordless ssh trust relationships (usually a bad idea unless you absolutely trust each side).

So, it provides an easy way to ssh into machines and run commands (and get stdout, stderr, and exit code), providing the password and username in the code.

With SSH 1, each command you run opens up a new session (it’s how SSH1 works, yeah it’s poo now-a-days), and with SSH 2 each command runs in the same session.

One tip you’ll need to know about is that you should install the Math::BigInt::GMP module. Why?! Because it is a fast BigInt implementation that will speed up SSH 2 sessions from like 30 seconds to 1 second.

The awesomeness (if not obvious now): logging into machines via ssh and doing stuff in a perl program.

At work, I use it to monitor and help manage machines, where I keep tabs on many things from software versions (and what needs updated), disk usage, sanity checks, ensuring backups… and an added feature of making sure that passwords are still the same and sshd is working. Much easier to manage all the activity from one place instead of having many machines do the same thing via crontab. You could do many of these things via crontab on all servers, people have for years and years. I prefer to do it from one place where changes and new things are much more scalable and reliable.

Here is an example of logging into a remote machine and checking it’s diskspace, reporting back if it was over a certian amount:

#!/opt/local/bin/perl
#
# checkDiskspace.pl: See if any partitions are greater than a percentage and notify
#
use strict;
use warnings; 

use Net::SSH::Perl;
use Math::BigInt::GMP;  # Don't forget this!

$| = 1;

my $alert_percent = 75;
my $server = 'niroze.net';
my $username = 'christopher';
my $password = '*******************************';

# However you wanna notify yourself (like email)
sub alert_notify {
my $message = shift;
print "STUB: $message\n";
}

# Log into server
print "Creating ssh object... ";
my $ssh = Net::SSH::Perl->new($server); # Error check this
print "done\n";
print "Logging into server... ";
$ssh->login($username, $password); # Error check this
print "done\n";

# Check df
my $command = "df";
print "Running command ($command)... ";
my ($stdout, $stderr, $exit) = $ssh->cmd($command); # Check output
print "done\n";

# Find percentage
foreach my $df_line (split(/\n/, $stdout)) {
# If disk space usage percent > $alert_percent, notify
if ($df_line =~ /\s+(\d+)%\s+(\/.*)/ && $1 >= $alert_percent) {
alert_notify ("[$server] device $2 at $1 percent!");
}
}

This is just a simple example, yet you see how easy it is to do. Here is the output of the script:

Creating ssh object... done
Logging into server... done
Running command (df)... done
STUB: [niroze.net] device / at 75 percent!

Hopefully this handy module will help you streamline your tasks that require managing multiple servers that are sshd enabled. It has helped me greatly at my job.

It’s nice to have programs do work and report back to you. Let the program do some of your work, especially the stuff where it is a lot of the same thing!

Note: there is more the module can do, so be sure to check the Net::SSH::Perl CPAN page for more details.

Using Perl’s Mac::Glue

Are you like me? Did you get a Mac and are in awe of the integration of everything? It was one of the reasons I got a Mac, yet I didn’t look into the details of how everything was integrated. How do applications “talk” to one another?

No, it was not Appletalk (yet that seems fitting word-wise). It’s Applescript.

I may be a Mac newbie, yet I don’t like “programming” Applescript at all. It should be natural, yet it isn’t. It’s actually very frustrating, yet that’s just part of learning it.

To quote devintosh.com FAQ (freenode IRC network #macdev channel):

I have a problem with my AppleScript, can you help?
Probably not. AppleScript is a crappy language that makes it hard to do almost anything. If your AppleScript code is more than a few lines then you’re already going beyond what it’s good at. Learn a real language: it’s harder to start with and it takes time to learn, but in the end you’ll be solving more problems and solving them faster 

Ok, I got that. Am past it… I still don’t like it (thanks to mikeash on #macdev for enlightening me… before I went too far down the wrong rabbit hole).

I want a real programming language to be able to do the same things Applescript does. Isn’t there a way?! Yes! There is Mac::Glue, which gives you all the same functionality of Applescript except you can now do it in Perl. Hrm… sounds like a bunch of bologna? Yeah, I agree… till I poked around into learning it. I’d like to share with you what I found.

First, install Mac::Glue. It’s in MacPorts (p5-mac-glue) or you can install it via CPAN. Either way doesn’t really matter, but in these examples I’ll be using the MacPorts version (it’s Perl is newer). Be sure to read the README file in the tarball. You will need to create the “glue” for programs you want to work with, ex: “sudo /opt/local/bin/gluemac /Applications/iTunes.app“. No big deal, just be sure to do it for programs you plan on working with.

Second, note this isn’t a full coverage of all that Mac::Glue does. This just shows how to port over some Applescript and “talk” to other programs via their bindings.

Now, to my favorite part. Some code! [sorry, it's not that impressive -ed.]

In the first example, lets port over something easy like “/Library/Scripts/Basics/AppleScript Help.scpt”. It opens up ‘Help Viewer.app’, activates it, and then searches looking for ‘AppleScript’.

Here is the AppleScript code:

tell application "Help Viewer"
activate
search looking for "AppleScript"
end tell

Here is the Perl code:

#!/opt/local/bin/perl
#
# Glue requirement:
#   * sudo /opt/local/bin/gluemac "/System/Library/CoreServices/Help Viewer.app"
# Port of:
#   * "/Library/Scripts/Basics/AppleScript Help.scpt"
#
use strict;
use warnings; 

use Mac::Glue;

my $glue = new Mac::Glue 'Help_Viewer';
$glue->search(looking_for => 'AppleScript');

You may be wondering how I knew what subroutine to call. It’s in the glue code pod file. Since I’m using MacPorts in these examples, I did this: “perldoc /opt/local/lib/perl5/vendor_perl/5.8.8/Mac/Glue/glues/Help_Viewer.pod“. Be sure to look in that directory for help on how to use the “glue”.

How about a simple adding someone to your address book?

#!/opt/local/bin/perl
#
# Glue requirement:
#   * sudo /opt/local/bin/gluemac "/Applications/Address Book.app"
# Explanation and original code examples from Chris Nandor:
#   * http://www.nntp.perl.org/group/perl.macosx/2004/04/msg7285.html
#
use strict;
use warnings; 

use Mac::Glue ':all'; # all for 'location'

my $glue = new Mac::Glue 'Address_Book';

my $me = $glue->make (new => 'person',
with_properties => {
'first_name' => 'Steve',
'last_name' => 'Jobs'
}
);
$glue->make (new => 'email', at => location(end => $me->prop('emails')),
with_properties => {
'value' => 'fakestevejobs@gmail.com',
'label' => 'home'
}
);

I’m still a Mac::Glue and AppleScript newbie. There is much to learn, yet all the power that is desired is available at your finger tips. I hope this was communicated enough to educate you that it does exist and you can use it in place of AppleScript for use in real programs.

Much thanks to Chris Nandor and everyone else that may have been involved in Mac::Glue.

I still have much to learn, yet at least we both know that we can do what we need in Perl… at least in this respect. In any place you can use AppleScript, try using Mac::Glue instead. I recommend getting the FastScripts utility if you can, it’s like the regular AppleScripts menu, yet much more flexible as it can run any kind of program and have custom keybindings (very handy).

To be fair, before I get complaints in my inbox, the ability to do the same thing exists for Ruby, Objective-C and Python available at http://appscript.sourceforge.net/. I played with it in Python and Ruby and it seems to work very well. So, give it a try if you’re not a Perl coder.

Do you have any cool Mac::Glue examples? If so, post them and happy hacking!

Speed up Java Development with Grails

It’s hard to not come up with puns and other predictable attempts at humor combinations with these words: Grails, Groovy, IDEA, Spring, Hibernate, and Java. They practically beg for it. Yet, I want to focus on what makes them useful. What is it about them all that makes it a killer combination? … And no there is not an extra ‘G’ in front of ‘rails’ because it is a completely separate project.

A disclaimer: I am not an expert in any of these technologies, yet they are very interesting and worth checking out. It may speed up your Java based web development and that’s worth something to everyone.

Groovy, sounds cool… yet what is it?

Simple! Groovy is a “scripting” language that allows for writing code much faster than writing normal Java. That doesn’t sound so special does it? Well here’s the kicker: it compiles into Java bytecode and can be used in regular Java code. You may also use regular Java code in Groovy. It all compiles into the same bytecode. It is similar to .NET programming where it all compiles into the same CLR format.

Writing Java code, you’ll never ever think, “Wow, It’s amazing how verbose this is, I enjoy writing so much code!” This is where Groovy can help speed up development time, while still keeping your same production and platform environment. All that you need to change is writing Java code text to Groovy code text, where you feel appropriate. It’s all the same JVM bytecode. It really can save you time, so check it out when you have a few moments.

Grails, the apparent non-type-o

Like Rails for Ruby, Grails allows for fast web application development while still being MVC clean. The only kicker is that you have all the power of Java, Groovy, Spring and Hibernate included already. You’re ready to rock from the start.

In the same manner as Ruby’s Rails and Perl’s Catalyst, it is a framework to allow for rapid web application development while also providing scripts and scaffolding for ease of use. It provides a clean environment to start writing code and running it from within itself. The site’s documentation is very helpful and provides you with enough information to begin getting your feet wet immediately.

It also comes with IDE plugins, so that you can get running in an environment you’re already familiar with. I have tested out the plugin with IntelliJ IDEA and it works flawlessly (just need environment variables set).