Schwer Log

The very non-descriptive error below indicates that you have reached the maximum value for the MySQL INT Data Type.

Communicator thread died because of error: Failed to execute mysql_query, perhaps connection died? Err=-1: Duplicate entry ‘2147483647′ for key 1

What you need to do if this happens to you on your PowerDNS records table is the following:

mysql> alter table records modify id BIGINT AUTO_INCREMENT;

If it happens to your domains table, then things get a bit more complicated and you should do the following:

mysql> alter table records drop foreign key `records_ibfk_1`;

mysql> alter table records modify domain_id BIGINT ;

mysql> alter table domains modify id BIGINT AUTO_INCREMENT;

mysql> alter table records add constraint `records_ibfk_1` FOREIGN KEY (`domain
_id`) REFERENCES `domains` (`id`) ON DELETE CASCADE;

The above worked for me, your results may vary.

Here’s a more concise script to feed Twitter an RSS feed; in this case the latest post to the Sonic.net MOTD Blog. The following code grabs the latest entry in an RSS feed, pulls out the URL and as much content as it can, shortens the URL and sends it all to Twitter.

#!/usr/bin/perl

use strict;
use warnings;
use Net::Twitter;
use LWP::Simple;
use XML::RSS::Parser::Lite;
use WWW::Shorten::Metamark;

my $feed = get(‘http://corp.sonic.net/motd/feed/rss’);
my $rp = new XML::RSS::Parser::Lite;

$rp->parse($feed);

my $it = $rp->get(0);

my $long_url = $it->get(‘url’);
my $short_url = makeashorterlink($long_url);
my $message = $it->get(‘description’);

$message =~ s/^.+ — //; # remove dash seperator.
$message =~ s/^\s+//; # Nuke any leading space; every char counts.
$message =~ s/\s+$//; # Nuke any trailing space; every char counts.
$message =~ s/[_]+$//; # Nuke any trailing underscores; happens when message is
short.
$message = substr $message , 0 , 117; # Twitter can only handle 140 chars.
$message .= ‘… ‘; # Append ellipse to denote further content available.
$message .= $short_url; # Append the shortened URL.

my $twit = Net::Twitter->new(username=>”sonicnet”, password=>”password” );
$twit->update($message);

Get updates about your favorite ISP from Twitter; just send “follow sonicnet” from your favorite Twitter device. You can check out the sonicnet page to see what you’ll be getting.

The code for this is made possible by the Net::Twitter Perl module.

#!/usr/bin/perl
use strict;
use warnings;
use Net::Twitter;
my $message = "Super Duper";
my $twit = Net::Twitter->new(username=>"sonicnet", password=>"password" );
$twit->update($message);

Pretty easy huh?

Note: This has been in my drafts for a while, but I think it’s ready for publishing..

At first blush you probably would not expect the following to print “-1″; as you probably expect “system” to execute and return the return code from “echo”.

$SIG{CHLD} = 'IGNORE';
print system('echo');

This stumped me too, so I did some research.

From the Perl PIC Documentation:

On most Unix platforms, the CHLD (sometimes also known as CLD) signal has special behavior with respect to a value of ‘IGNORE’. Setting $SIG{CHLD} to ‘IGNORE’ on such a platform has the effect of not creating zombie processes when the parent process fails to wait() on its child processes (i.e. child processes are automatically reaped). Calling wait() with $SIG{CHLD} set to ‘IGNORE’ usually returns -1 on such platforms.

And perldoc -f system:

The return value is the exit status of the program as returned by the wait call. … Return value of -1 indicates a failure to start the program or an error of the wait(2) system call (inspect $! for the reason).

And wait(2):

POSIX.1-2001 specifies that if the disposition of SIGCHLD is set to SIG_IGN or the SA_NOCLDWAIT flag is set for SIGCHLD (see sigaction(2)), then children that terminate do not become zombies and a call to wait() or waitpid() will block until all children have terminated, and then fail with errno set to ECHILD. (The original POSIX standard left the behaviour of setting SIGCHLD to SIG_IGN unspecified.) Linux 2.6 conforms to this specification. However, Linux 2.4 (and earlier) does not: if a wait() or waitpid() call is made while SIGCHLD is being ignored, the call behaves just as though SIGCHLD were not being ignored, that is, the call blocks until the next child terminates and then returns the process ID and status of that child.

So on Linux 2.6 kernels if you auto. reap zombies by setting $SIG{CHLD} to ‘IGNORE’, then you can’t trust the return code from ’system’ as it will always be ‘-1′.

One solution would be to reap your own zombies with one of the popular zombie reaping code snipets.

sub REAPER { 1 until waitpid(-1 , WNOHANG) == -1 };
$SIG{CHLD} = \&REAPER;
print system('echo');

Be careful when using the special Perl variable ‘$?’ because it will have been altered by the call to ‘waitpid’ in your REAPER.

This is because when a child exits a SIG_CHLD is sent to its parent to see this in action try the code below:

sub REAPER
{ print "REAPER\n"; waitpid(-1 , WNOHANG); }
$SIG{CHLD} = \&REAPER;
system('echo');
print "RV: $?\n";

I have released several PowerDNS modules to CPAN:

http://search.cpan.org/~augie/

PowerDNS::Backend::MySQL Provides an interface to manipulate PowerDNS data in the MySQL Backend.

PowerDNS::Control::Client Provides an interface to control the PowerDNS daemon.

PowerDNS::Control::Server Provides an interface to control the PowerDNS daemon.

The MySQL interface is based on code I have in production, but currently is not what I have running, yet.

The Client/Server Control code I do have running in production; although they currently only implement the features I needed to deploy.

Gin & Tonic

Originally uploaded by Augie Schwer.

Sonic.net had it’s Holiday Party this last Friday; the full set can be found here:

http://flickr.com/photos/augie/sets/72157603462438856/

File::Rotate::Backup is a handy Perl module for backing up directories/files and archiving and rotating them. It’s even more handy after I contributed a patch to add the ‘no_archive’ option.

Last night I gave a talk about MySQL Replication to the Sonoma County System Administrators group; in case you missed it, here is the presentation in PDF:

MySQL Replication [pdf].

I could not find a script to do this anywhere, so I wrote one; it was meant to convert Bind style zone files to Rbldnsd style zone files, but since it first does an AXFR, it can be used to generate rbldnsd zones from any source that supports DNS transfers.

Here is a direct link to the file, feel free to download it: http://schwer.us/files/axfr2rbldnsd.pl; and here’s the code:


#!/usr/bin/perl

# Author: Augie Schwer
# $Id: axfr2rbldnsd.pl 1274 2007-09-23 20:30:17Z augie $
# Purpose: Convert a transfered zone files to rbldnsd style and
# prints to STDOUT.

use strict;

unless ( defined $ARGV[0] ) { print “USAGE : $0 zone_to_transfer (host)\n”; exit; }

my $zone = $ARGV[0];
my $host = $ARGV[1] || ‘localhost’;
my $time = time();
my $tmp_file = “/tmp/$zone.$time”;
my $found_ns = 0;
my $found_soa = 0;

`dig \@$host axfr $zone > $tmp_file`;

open FILE , “$tmp_file” or die “Could not open $tmp_file : $!”;

while ()
{
# Ignore comments
next if /^;/;

# remove everything in front of SOA, but keep the TTL.
if (!$found_soa) # we only need one SOA.
{
if ( /(\d+)\s+IN\s+SOA\s+(.+)$/ )
{
print “\$SOA $1 $2″;
$found_soa = 1;
next;
}
}

# remove everything in front of NS, but keep the TTL.
# concat all NS records together.
if (!$found_ns and /(\d+)\s+IN\s+NS\s+(.+)$/)
{
print “\n\$NS $1 $2 “;
$found_ns = 1;
next;
}
if ( /IN\s+NS\s+(.+)$/ ) { print “$1 “; next; }

# anything that’s not a sub-domain gets an ‘@’ in
# front followed by the type and record.
if ( /^$zone\.\s+(\d+)\s+IN\s+(A|MX)\s+(.+)$/ ) { print “\n@ $1 $2 $3\n”; next; }
# print the rest of the A records.
if ( /^(.+)\.$zone\.\s+(\d+)\s+IN\s+A\s+(.+)$/ ) { print “$1 $2 IN A $3\n”; next; }
}

# clean up after myself.
`rm -f $tmp_file`;

When using InnoDB’s Hot Backup code to take a snapshot of your production database and use that snapshot to populate a new slave you may run into an error about not being able to drop the ibbackup_binlog_marker table and all replication bombs out.

The problem is that the innobackup tool deletes the temporary table ibbackup_binlog_marker but the .frm file does not get removed, so your new slave sees the ibbackup_binlog_marker.frm and thinks the table should exist, but the binary log says you are trying to replay says it’s already been deleted; the result is an inconsistent state that causes MySQL replication to fail.

The solution is to remove the ibbackup_binlog_marker.frm file in the mysql data directory; after that is done, then slave replication should continue as expected.

You know that looping your mail is a bad idea; that is forwarding your email to Gmail and then forwarding that back to yourself, ad infinitum, you get the idea. It’s equally a bad idea to forward DSNs or bounce messages to Gmail (or anywhere really); as it causes the same sort of loop and the same kind of aggravation for your System Administrator. The following is a procmail recipe that works for me.

PMDIR=$HOME/.procmail
LOGFILE=$PMDIR/procmail.log
VERBOSE=yes
SENDMAIL=/usr/sbin/sendmail
FORMAIL=/usr/bin/formail
SUBJECT=`$FORMAIL -xSubject:`

:0
* ^X-Loop: Your_Email_Address@sonic.net
/dev/null

:0E
* ^TO_Your_Email_Address\+0×96rds@sonic\.net
$DEFAULT

:0cE
| $FORMAIL -A “X-Loop: Your_Email_Address@sonic.net” | $SENDMAIL -f Your_Email_Address+0×96rds@sonic.net -oi Your_Email_Address@gmail.com

I recently gave the same talk about PowerDNS to both the Sonoma County System Administrators group and the North Bay Linux Users Group; in case you missed it, here is the presentation in PDF format:

The PowerDNS Name Server[pdf].

I have been doing quite a bit of work with mon, an Open Source software project that is used to monitor the health of the services and systems on your network; which is a very handy thing when you are in charge of hundreds of machines that all need to offer different levels of service. I started submitting quite a few updates to the software for our own internal use and ended up getting on the CVS commiter’s list; which makes me an official developer of the mon project; which is pretty cool in a nerdy kind of way.

PowerDNS is Open Source software that I have been helping bug fix and develop, and the other day I received a nice little public thank you on the various PowerDNS public mailing lists. I’ve trimmed the message a bit; the original can be found here:

http://mailman.powerdns.com/pipermail/pdns-dev/2007-April/000590.html

———- Forwarded message ———-
From: bert hubert
Date: Apr 9, 2007 5:39 AM
Subject: [Pdns-dev] Now is the time to test the 2.9.21 snapshots +
release notes.
To: pdns-users@mailman.powerdns.com,
pdns-announce@mailman.powerdns.com, pdns-dev@mailman.powerdns.com

Hi everybody,

Somewhere in the coming 2 weeks, we will release the PowerDNS Authoritative
Server version 2.9.21.
…
This release would not have been possible without large amounts of help
and support from the PowerDNS Community. We specifically want to thank
Massimo Bandinelli of Italy’s Register.it, Dave Aaldering of Aaldering
ICT, True BV, XS4ALL, Daniel Bilik of Neosystem, EasyDNS, Augie Schwer,
Mark Bergsma, Marcus Rueckert of OpenSUSE, Andre Muraro of Locaweb, Antony
Lesuisse, Norbert Sendetzky, Marco Chiavacci, and Ruben Kerkhof.
…

This is so cool:

http://www.linuxguruz.com/iptables/howto/2.4routing-13.html

/proc/sys/net/ipv4/ip_nonlocal_bind

Set this if you want your applications to be able to bind to an address which doesn’t belong to a device on your system. This can be useful when your machine is on a non-permanent (or even dynamic) link, so your services are able to start up and bind to a specific address when your link is down.

So you have MySQL Replication setup and working and now you want to create more slaves to your existing deployment; these are the steps that have worked for me:

Here Slave One is the slave that is currently in production and Slave Two is the new slave you want to populate and put in to production.

• On Slave One issue the command STOP SLAVE;
• In another terminal on Slave One tar up the MySQL data dir.; tar czf mysql.tgz /var/lib/mysql .
• After the tar finishes go ahead and start the slave process back up on Slave One; START SLAVE;
• On Slave Two un-tar the tarball; tar -C / -xzf mysql.tgz .
• Edit Slave Two’s mysql.cnf to be similar to Slave One’s, but make sure to change the server-id variable.
• Start the MySQL daemon on Slave Two, and issue the following commands: STOP SLAVE; RESET SLAVE;
• Now issue a CHANGE MASTER filling in the correct values for your deployment; most of the info. can be found by issuing a SHOW SLAVE STATUS\G; on Slave One.

mysql> CHANGE MASTER TO
-> MASTER_HOST=’sql_host’,
-> MASTER_USER=’replication_user’,
-> MASTER_PASSWORD=’replication_password’,
-> MASTER_LOG_FILE=’update_log.000007′,
-> MASTER_LOG_POS=560;

• Finally issue a START SLAVE; on Slave Two.

That’s it; you can issue a SHOW SLAVE STATUS\G; on your new slave and see how its health is. This has worked many times for me; the only time I had a problem was when the InnoDB files were in a weird state on the slave I was taring, in which case I waited a bit and re-tarred the MySQL data directory.

The following is a handy little script for replaying DNS traffic and thus verifying that a Name Server will answer as you expect it to.

Updated: Tue Apr 15 16:12:59 PDT 2008.

#!/usr/bin/perl
# Augie Schwer
# dnsreplay.pl - replay a BIND query log and print to STDOUT
# when a query fails against the given name server.
# $Id: dnsreplay.pl 1829 2008-04-15 23:07:12Z augie $

To get the Query Log out of BIND put the following in to your named.conf :