Archive

Archive for the ‘Development’ Category

How-To Create a RESt Webservice with Zend Rest

August 13th, 2010 Ivan Villareal No comments

There are many times when we need to get data from a production application, but the application is running on a different server, or has a different db than the one we are currently working.

So a quick way to get the required data, without modifying the app would be to create a webservice to expose the data needed.

There are several ways to do this, RPC, SOAP, RESt, to name a few, we can even use some non standard methods, but the discussion of this is beyond the scope of this post.

Thanks to the Zend Framework, we can easily write an api and have working REST server within minutes, even if you don’t use the Zend Framework in your application you can just grab the needed files, and you’ll have your server up and running.

This is the way I can quickly expose some data of a current system.

First get the Zend Framework from Latest Zend Framework.

If you don’t want to put all the framework in your library, you can just use the following files to create your REST server.

|-- Exception.php
|-- Rest
|   |-- Client
|   |   |-- Exception.php
|   |   |-- Result
|   |   |   `-- Exception.php
|   |   `-- Result.php
|   |-- Client.php
|   |-- Controller.php
|   |-- Exception.php
|   |-- Route.php
|   |-- Server
|   |   `-- Exception.php
|   `-- Server.php
`-- Server
    |-- Abstract.php
    |-- Cache.php
    |-- Definition.php
    |-- Exception.php
    |-- Interface.php
    |-- Method
    |   |-- Callback.php
    |   |-- Definition.php
    |   |-- Parameter.php
    |   `-- Prototype.php
    |-- Reflection
    |   |-- Class.php
    |   |-- Exception.php
    |   |-- Function
    |   |   `-- Abstract.php
    |   |-- Function.php
    |   |-- Method.php
    |   |-- Node.php
    |   |-- Parameter.php
    |   |-- Prototype.php
    |   `-- ReturnValue.php
    `-- Reflection.php
 
8 directories, 29 files

Once you have saved the above files somewhere (I use lib/Zend), start writing your server.

< ?php
set_include_path('./lib' . PATH_SEPARATOR . get_include_path());
 
/**
 * Include the main class that we will use
 * to get the data from.
 */
require_once 'DomainProcessor.php';
require_once 'Zend/Rest/Server.php';
 
 
 
class domainService
{
    private $_domainProcessor;
 
    public function __construct()
    {
         $this->_domainProcessor = new domainProcessor();
    }
 
    /**
     * This method will expose the data
     * so we make the call and return what we want to
     * publish
     */
    public function getDomainData($domain)
    {
        $domain = strtolower($domain);
        $result = $this->_domainProcessor->_getDomainData($domain);
        if ($result['fullDomainName'] != $domain) {
            $result = false;
        }
        return $result;
    }
}
 
/**
 * We instantiate a Zend_Rest_Server
 * and we pass the class name that we 
 * want to expose, Zend_Rest will take
 * care of everything else, all our public
 * methods can be consumed by a RESt client
 */
$server = new Zend_Rest_Server();
$server->setClass('domainService');
$server->handle();

To consume our webservice we call it either using a GET or a POST, we can use Zend_Rest_Client to write a client, but also we can call it manually just to be sure that everything works.

To call the above class would be something like:

http://blog.ivanvillareal.info/api.php?method=getDomainData&domain=ivanvillareal.com

and it should respond something like this:

01
<domainservice generator="zend" version="1.0">
02
 <getdomaindata>
03
 <response>
04
 <name>ivanvillareal.com</name>
05
 <age>1</age>
06
 <owner>me</owner>
07
 </response>
08
 <status>success</status>
09
 </getdomaindata>
10
</domainservice>

get Name and email address from a “from” email header

June 16th, 2010 Ivan Villareal 1 comment

This is a quick post of a little function I did to get the name and the email of a “from” header, Because this is not a standard thing this function only recognizes the following format:

Some Name <someemail @server.com></someemail>

this is the function I came up:

function extractEmailAddress($subject)
    {
        $result = new stdClass();
        if (preg_match('/\\b[A-Z0-9._%-]+@[A-Z0-9.-]+\\.[A-Z]{2,4}\\b/i', $subject, $regs)) {
            $result->email = $regs[0];
        }
        //try to get the name
        preg_match_all('/(.*)< (.*)>/', $subject, $regs, PREG_PATTERN_ORDER);
        if (isset($regs[1][0])) {
            $result->name = ($regs[1][0] != '') ? $regs[1][0] : '';
        } else {
            $result->name = '';
        }
        return $result;
    }

And this are some results I had:

Case 1:

$testName = "Ivan Villareal <ivan @server.com>";  </ivan>

Result:
object(stdClass)#3 (2) {
  ["email"]=>
  string(15) "ivan@server.com"
  ["name"]=>
  string(15) "Ivan Villareal "
}

Case 2:

$testName = "Ivan Villareal ivan@server.com"; 

Result:
object(stdClass)#3 (2) {
  ["email"]=>
  string(15) "ivan@server.com"
  ["name"]=>
  string(0) ""
}

Case 3:

$testName = "ivan@server.com";

Result:
object(stdClass)#3 (2) {
  ["email"]=>
  string(15) "ivan@server.com"
  ["name"]=>
  string(0) ""
}

Categories: Development, PHP Tags: , , , ,

Fetching mail from a POP3 server with Zend_Mail

June 9th, 2010 Ivan Villareal 1 comment

I was in a need for a class to get mails from a pop server to process them, so I’ve decided to leverage the Zend_Mail class, what I did was to create a wrapper around the Zend_Mail, to return all the messages from the server in a manageable way.

The first step was to put the needed Zend files in my library folder:

lib/Zend/Mail.php
Mail
|-- Exception.php
|-- Message
|   |-- File.php
|   `-- Interface.php
|-- Message.php
|-- Part
|   |-- File.php
|   `-- Interface.php
|-- Part.php
|-- Protocol
|   |-- Abstract.php
|   |-- Exception.php
|   |-- Imap.php
|   |-- Pop3.php
|   |-- Smtp
|   |   `-- Auth
|   |       |-- Crammd5.php
|   |       |-- Login.php
|   |       `-- Plain.php
|   `-- Smtp.php
|-- Storage
|   |-- Abstract.php
|   |-- Exception.php
|   |-- Folder
|   |   |-- Interface.php
|   |   |-- Maildir.php
|   |   `-- Mbox.php
|   |-- Folder.php
|   |-- Imap.php
|   |-- Maildir.php
|   |-- Mbox.php
|   |-- Pop3.php
|   `-- Writable
|       |-- Interface.php
|       `-- Maildir.php
|-- Storage.php
`-- Transport
    |-- Abstract.php
    |-- Exception.php
    |-- Sendmail.php
    `-- Smtp.php

Then I started writing my class:

< ?php
 
set_include_path('lib' . PATH_SEPARATOR . get_include_path());
 
include_once 'Zend/Mail.php';
include_once 'Zend/Mail/Storage/Pop3.php';
include_once 'Zend/Mail/Transport/Smtp.php';
include_once 'Zend/Mime/Part.php';
 
/**
 * Class to get mails from a server
 */
class mailFetcher
{
    private $_emailHost = 'pop.server.com';
    private $_emailUser = 'ivan@ivanvillareal.com';
    private $_emailPass = 'password';
	/**
	 * Zend_Mail_Storage
	 */
    private $_mail;
	/**
	 * Inbox wrapper
	 */
	private $_inbox = array();
 
	/**
	 * on class instantiation we fire connect
	 * to the pop server and start fetching the mails
	 */
    public function __construct()
    {
        try {
            $this->_mail = new Zend_Mail_Storage_Pop3(array(
                                             'host' => $this->_emailHost,
                                             'user' => $this->_emailUser,
                                             'password' => $this->_emailPass));
        } catch (Exception $e) {
            echo 'Mail Retrieval Failed...' . PHP_EOL;
            die();
        }      
        $this->processMail();
    }
 
 
	private function _contentDecoder($encoding, $content)
	{
		switch ($encoding) {
			case 'quoted-printable':
				$result = quoted_printable_decode($content);
			break;
			case 'base64':
				$result = base64_decode($content);
			break;
			default:
				$result = $content;
			break;
		}
		return $result;
	}
 
	/**
	 * This is where everything is done
	 */
	public function processMail()
	{
		if ($this->_mail instanceof Zend_Mail_Storage_Pop3 && $this->_mail->countMessages() > 0) {
			$messageNum = 0;
			foreach ($this->_mail as $message) {
				$messageNum++;
				$rawMail      = $this->_mail->getRawHeader($messageNum) . $this->_mail->getRawContent($messageNum);
				$received     = $message->received;
				$date         = date('Y-m-d H:i:s', strtotime($message->date));
				$messageId    = $message->headerExists('message-id') ? $message->getHeader('message-id') : null;
				//remove <> from message id
				$messageId    = (preg_match('|< (.*?)>|', $messageId, $regs)) ? $regs[1] : $messageId;
				$inReplyTo    = $message->headerExists('in-reply-to') ? $message->getHeader('in-reply-to') : null;
				//remove <> from in-reply-to
				$inReplyTo    = (preg_match('|< (.*?)>|', $inReplyTo, $regs)) ? $regs[1] : $inReplyTo;
				$references   = $message->headerExists('References')  ? $message->getHeader('References') : null;
				//remove <> from references
				$references   = (preg_match('|< (.*?)>|', $references, $regs)) ? $regs[1] : $references;
				$from         = $message->from;
				$subject      = $message->subject;
				$to           = $message->to;
				$plainContent = null;
				$htmlContent  = null;
				$attachments = array();
 
				/**
				 * Check if the message has multiple parts
				 */
		        if ($message->isMultipart()) {
					/**
					 * We have a multipart message
					 * lets extract the plain content,
					 * html content, and attachments
					 */
					foreach (new RecursiveIteratorIterator($message) as $part) {
						try {
							switch (strtok($part->contentType, ';')) {
								case 'text/plain':
									$plainContent = $this->_contentDecoder($part->getHeader('content-transfer-encoding'), $part->getContent());
								break;
								case 'text/html':
									$htmlContent = $this->_contentDecoder($part->getHeader('content-transfer-encoding'), $part->getContent());
								break;
								default: //attachment handle
									$type       = strtok($part->contentType, ';');
									$fileName   = $part->getHeader('content-description');
									$attachment = $this->_contentDecoder($part->getHeader('content-transfer-encoding'), $part->getContent());
									$attachments[] = array(
													   'file_name' => $fileName,
													   'type'      => $type,
													   'content'	=> $attachment);
								break;
							}
						} catch (Zend_Mail_Exception $e) {
							echo "$e \n";
						}
					}
				} else {
					$plainContent = $message->getContent();
				}
				$this->_inbox[] = array('messageNumber' => $messageNum,
									    'rawMail'       => $rawMail,
										'received'		=> $received,
										'date'			=> $date,
										'messageId'		=> $messageId,
										'inReplyTo'		=> $inReplyTo,
										'references'	=> $references,
										'from'			=> $from,
										'subject'		=> $subject,
										'to'			=> $to,
										'plainContent'	=> $plainContent,
										'htmlContent'	=> $htmlContent,
										'attachments'	=> $attachments);
			}
		} else {
			$this->_inbox = false;
		}
	}
	public function getInbox()
	{
		return $this->_inbox;
	}
}

My fetch class does everything on instantiation, and it will have all the mails retreived, so for example if I need to get all messages I can do this:

< ?php
include_once 'mailFetcher.php';
 
$inbox = new mailFetcher();
print_r($inbox->getInbox());

And this would be the result:

Array
(
    [0] => Array
        (
            [messageNumber] => 1
            [rawMail] => Received: from gate55.gate.iad.mailserver.com (11.1.111.1) by
 IAD2HUB01.mex02.mailserver.com (111.11.11.65) with Microsoft SMTP Server (TLS) id
 8.2.247.2; Wed, 9 Jun 2010 17:56:16 -0400
Received: from mail-yw0-f186.google.com (mail-yw0-f186.google.com
 [111.11.111.186])	by gate55.gate.iad.mailserver.com (SMTP Server) with ESMTP id
 B1F6E31D0171	for <asdfasdf @ivanvillareal.com>; Wed,  9 Jun 2010
 17:56:15 -0400 (EDT)
Received: by ywh16 with SMTP id 16so5309053ywh.32        for
 </asdfasdf><asdfasdf @ivanvillareal.com>; Wed, 09 Jun 2010 14:56:15 -0700 (PDT)
Received: by 10.111.111.111 with SMTP id jz1mr465966qcb.170.1276120573865;
 	Wed, 09 Jun 2010 14:56:13 -0700 (PDT)
Received: by 11.111.111.11 with HTTP; Wed, 9 Jun 2010 14:56:13 -0700 (PDT)
From: Ivan Villareal <ivaano @gmail.com>
To: "asdfasdf@ivanvillareal.com" <asdfasdf @ivanvillareal.com>
Date: Wed, 9 Jun 2010 17:56:13 -0400
Subject: attachment test
Thread-Topic: attachment test
Thread-Index: AcsIHpXuE+vWJJMfTGyqikgD2kYkng==
Message-ID: <aanlktingkmpbkefog5kxxoq8wrib7tj1pt1rsy -1ug-1@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: IAD2HUB01.mex02.mailserver.com
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-virus-scanned: OK, catchall
x-originating-ip: [111.11.111.186]
dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed;        d=gmail.com;
 s=gamma;
        h=domainkey-signature:mime-version:received:received:date:message-id
         :subject:from:to:content-type;
        bh=IvMOHmKRabcJ2PrUHPHB/NbNpgQubvvgGprC6gtDY9E=;
        b=cA5Pjycedt9Pylic7CTk4HCF6HXV9eEFpr086KwcMepGKJEv4DWU6d4iFt4Gp7R42V
         +4rVmrmwj2j9yj1vbEUqVMI6/m+Kkwt1r+MAtN/0qZsvEaVBBbbjTBxFLO9cQ7xlaSSm
         eGTHv9mR0qCcBZGLOVFeWUNiYDkhY2F0eDUbU=
domainkey-signature: a=rsa-sha1; c=nofws;        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=VKdlRSfBz/Swij0xE0c7Su8TOwcc2YGmw0K8dvVxoS9TJf0p8nIqtR6nQtmKqabSu4
         g2ALFFHXuVOhGpP+JdvI3wQJ1k9N5EZhcWtSRECrikOlLnaDA8/4z3SYckY6hs7Q+KIe
         qG0IbY+QkuZh7yymcyAN5omTaFDFT2GIPs+1c=
x-orig-to: catchall@ivanvillareal.com
Content-Type: multipart/mixed;
	boundary="_004_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_"
MIME-Version: 1.0
 
--_004_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_
Content-Type: multipart/alternative;
	boundary="_000_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_"
 
--_000_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
 
amhnamhnamhnDQo=
 
--_000_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
 
amhnamhnamhnDQo=
 
--_000_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_--
 
--_004_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_
Content-Type: image/jpeg; name="2010-05-28-112655.jpg"
Content-Description: 2010-05-28-112655.jpg
Content-Disposition: attachment; filename="2010-05-28-112655.jpg"; size=15004;
	creation-date="Wed, 09 Jun 2010 17:56:16 GMT";
	modification-date="Wed, 09 Jun 2010 17:56:16 GMT"
Content-Transfer-Encoding: base64
 
/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a
gq4UlFJmmKwtHakzS0DsMNFLikIxQAUmKKM0AFLSUtABRRRQAlFGeaKACiikouAtGaKPrQAmaKDS
4oASgClxRQKwUUUlAxaKKM0AFBNFGM0AGalHzIKiIp6HgigQuKKKKQyZDlaJAcU2I09+VP0qWSRK
R0zTGXBpuec1IDvX3qihqNhhV5WG0Vn9DUocgUMTJpyCpIqsKezZXmoqENDqQ0dqBmmMBS0UmKAF
oFJS0AJ3oIpQKQk0AAp4JxTBTxQwYE0oFNNOU8UhCkDGKaRin0EZoEiLIpcUuzuKMUyrje9FKQet
J2oGf//Z
 
--_004_AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy1ug1mailgmailcom_--
 
            [received] => from gate55.gate.iad.mailserver.com (11.1.111.1) by IAD2HUB01.mex02.mailserver.com (111.11.11.15) with Microsoft SMTP Server (TLS) id 1.1.147.2; Wed, 9 Jun 2010 17:56:16 -0400
from mail-yw0-f186.google.com (mail-yw0-f186.google.com [209.85.211.186])	by gate55.gate.iad.mailserver.com (SMTP Server) with ESMTP id B1F6E31D0171	for <asdfasdf @ivanvillareal.com>; Wed,  9 Jun 2010 17:56:15 -0400 (EDT)
by ywh16 with SMTP id 16so5309053ywh.32        for </asdfasdf><asdfasdf @ivanvillareal.com>; Wed, 09 Jun 2010 14:56:15 -0700 (PDT)
by 10.229.233.193 with SMTP id jz1mr465966qcb.170.1276120573865; Wed, 09 Jun 2010 14:56:13 -0700 (PDT)
by 10.229.192.82 with HTTP; Wed, 9 Jun 2010 14:56:13 -0700 (PDT)
            [date] => 2010-06-09 14:56:13
            [messageId] => AANLkTingkMPbKeFog5kxXoq8WRib7tJ1pT1Rsy-1ug-1@mail.gmail.com
            [inReplyTo] => 
            [references] => 
            [from] => Ivan Villareal <ivaano @gmail.com>
            [subject] => attachment test
            [to] => "asdfasdf@ivanvillareal.com" <asdfasdf @ivanvillareal.com>
            [plainContent] => jhgjhgjhg
 
            [htmlContent] => jhgjhgjhg
 
            [attachments] => Array
                (
                    [0] => Array
                        (
                            [file_name] => 2010-05-28-112655.jpg
                            [type] => image/jpeg
                        )
 
                )
 
        )
 
)</asdfasdf></ivaano></asdfasdf></aanlktingkmpbkefog5kxxoq8wrib7tj1pt1rsy></asdfasdf></ivaano></asdfasdf>

From this point I can iterate the inbox easily saving the email results or applying other logic to it,

Apache virtual host script in perl

May 27th, 2010 Ivan Villareal 2 comments

This is a very short post about a little script I did to create remove virtual hosts easily.

I’ve found a couple of scripts (http://patrickgibson.com/utilities/virtualhost/, http://www.tamdenholm.com/portfolio/apache)  to do this but they didn’t worked right, I’m using Ubuntu Karmic, maybe thats why. However instead of fixing those scripts I went to my vim editor and started typing this, I’ve spent about 2 hours writing this while I was watching a crappy tv program, but in the end it worked, it surely contains some bugs, or it is unsafe to run it, I just wanted to share this.

#!/usr/bin/perl -w
use strict;
use File::Path qw(mkpath rmtree);
use Getopt::Long;
 
 
our $ipAddress        = '127.0.0.1';
our $apacheConfigFile = 'apache2.conf';
our $apacheConfigDir  = '/etc/apache2';
our $sitesAvailable   = 'sites-available';
 
our $docRootPrefix = '/home/ivan/www';
our $docRoot = 'public_html';
our $logsDir  = 'logs';
 
 
my $del = '';
my $add = '';
my $domain = '';
 
unless (GetOptions ('del' => \$del, 'add' => \$add, 'domain=s' => \$domain) or usage()) {
    usage();
}
 
#print $paramResults;
if ($add || $del) {
    if ($domain) {
        if ($add) {
            createVhost($domain);
        } elsif ($del) {
            deleteVhost($domain);
        }
    } else {
        usage();
    }
   #print 'add aqui';
} else {
    usage();
}
 
sub usage {
    print < <USAGE
This program will add or remove
apache virtual hosts on ubuntu karmic 9.10
 
usage: vhost-updater.pl [--add | --del] --domain newhost.tld 
USAGE
}
 
sub returnVhostPaths
{
    my $vhost = shift;
    my @dir = split(/\//, $docRootPrefix);
    my %res;
 
    push(@dir, $vhost);
 
    my $hostDir = join('/', @dir);
    $res{'docRoot'} = $hostDir . '/' . $docRoot;
    $res{'logsDir'} = $hostDir . '/' . $logsDir;
 
    #todo dir validation
    @dir = split(/\//, $apacheConfigDir);
    push(@dir, $sitesAvailable);
    push(@dir, $vhost);
    $res{'apacheConfig'} = join('/', @dir);
 
    return %res;   
}
 
sub createVhost {
    my $vhost = shift;
    #first create the docRoot
    my %vhostInfo = returnVhostPaths($vhost);
 
    informOut("Creating docroot dir: $vhostInfo{'docRoot'}");
    informOut("Creating log dir: $vhostInfo{'logsDir'}");
    mkpath($vhostInfo{'docRoot'}, $vhostInfo{'logsDir'});
 
 
    informOut("Site File: $vhostInfo{'apacheConfig'}");
 
    my $vhostContent = << "EOF";
<VirtualHost *:80>
    ServerName $vhost
    DocumentRoot $vhostInfo{'docRoot'}
    <directory $vhostInfo{'docRoot'}>
        Options Indexes FollowSymLinks
        AllowOverride All
        Order allow,deny
        Allow from all
    </directory>
        ErrorLog $vhostInfo{'logsDir'}/error_log
        CustomLog $vhostInfo{'logsDir'}/access_log "%h %l %u %t \\"%r\\" %>s %b \\"%{Referer}i\\" \\"%{User-agent}i\\""
        LogLevel debug
 
 
EOF
    informOut("Creating Vhost...");
    open FILE, ">", $vhostInfo{'apacheConfig'} or die $!;
    print FILE $vhostContent;
    close FILE;
 
    informOut("Adding host $vhost");
    open FILE, ">>", '/etc/hosts' or die $!;
    print FILE $ipAddress ."\t". $vhost ."\n";
    close FILE;
 
    my $output = `/usr/sbin/a2ensite $vhost`;
    print $output;
 
    restartApache();
    #print $vhostConten t;
}
 
sub restartApache
{
    informOut("Restarting apache");
    my $output = `/etc/init.d/apache2 restart`;
    print $output; 
}
 
sub deleteVhost
{
    my $vhost = shift;
     my %vhostInfo = returnVhostPaths($vhost);
 
    informOut("Removing $vhost from hosts file");
    open IN, '< ', '/etc/hosts' or die $!;
    my @hostsFile = <IN>;
    close IN;
 
    my @contents = grep(!/^127.0.0.1\t$vhost/, @hostsFile);
 
    open FILE, ">", '/etc/hosts' or die $!;
    print FILE @contents;
    close FILE;
 
    my $output = `/usr/sbin/a2dissite $vhost`;
    print $output;
 
    informOut("Removing  $vhostInfo{'apacheConfig'} file");
    unlink($vhostInfo{'apacheConfig'});
 
    restartApache();
 
    print " manually remove $vhostInfo{'docRoot'}... \n";
 
}
 
sub informOut {
    my $message = shift;
    print "$message \n";
}

Here is a screenshot of the program running

vhost creator Screenshot

Categories: Linux, Perl, configuration Tags:

Bulk domain name resolver with perl

May 7th, 2010 Ivan Villareal No comments

I was in the need of getting the ip address from a long list of domain names, the domains were stored in a MySQL db, so I quickly turned into perl for making a script that got the list and then use an external command to resolve the domain.

At first I was going to use the dig command, but that would have taken to much effort, then I thought about using the ping command, but again to much effort, instead I’ve decided to leverage the IO::Socket::INET module, trying to open a socket to a domain name, will resolve the name and I could use that.

My implementation was very simple and it worked well, here is the relevant part:

 

 

 

#include the module
use IO::Socket::INET;
 
#create a socket to uncommon udp port, because tcp is likely to be firewalled
my $sock = IO::Socket::INET->new(
                           PeerAddr=> "$domain",
                           PeerPort =>40125,
                           Proto=>'udp');
 
#if we can create a socket, then we have an ip
if (defined $sock) {
         $result = $sock->peerhost;
         logOut("Success!");
    } else {
        logOut("Failed trying to connect to $domain");
        return;
    }

 

This was Pretty simple no?, here is the complete script, which gathers all the rows with no ip address and then it will try to get the ip, update the row and continue.

 

#!/usr/bin/perl
use strict;
use DBI;
use IO::Socket::INET;
 
####Global Variables###
 
our %db = ('host' => 'localhost','name' => 'domains','user' => 'ivan','pass' => '1234','port' => '3306');
our $dsn;
our $dbh;
 
#log function
sub logOut {
    my ($msg) = @_;
    print $msg . "\n";
}
 
#db connection
sub initialize {
    logOut("Connecting to db server on $db{'host'}");
    $dsn = "DBI:mysql:database=$db{'name'};host=$db{'host'};port=$db{'port'}";
    $dbh = DBI->connect($dsn, $db{'user'}, $db{'pass'}) or die('failed to connect');
}
 
#function to gather the ip fo a domain
sub getIpAddress {
    my ($domain) = @_;
    logOut("Trying to connect to: $domain...");
    my $result;
    my $sock = IO::Socket::INET->new(
                           PeerAddr=> "$domain",
                           PeerPort =>40125,
                           Proto=>'udp');
    if (defined $sock) {
         $result = $sock->peerhost;
         logOut("Success!");
    } else {
        logOut("Failed trying to connect to $domain");
        return;
    }
    return $result;
}
 
##This function
# gets all domains from domains table
sub getDomains {
    logOut("Fetching db for domains...");
    my $result = $dbh->selectall_hashref("SELECT domain_name FROM domains WHERE ip_address IS NULL", 'domain_name');
    logOut(keys( %$result ) . " domains returned.");
    my $updatedDomains = 0;
    my $noIpDomains    = 0;
    foreach my $id (keys %$result) {
        my $ip = getIpAddress($result->{$id}->{domain_name});
        if ($ip) {
            logOut("Ip found $ip for $result->{$id}->{domain_name}");
            my @values = ($ip, $result->{$id}->{domain_name});
            my $re = $dbh->do("UPDATE domains SET ip_address=? WHERE domain_name=?",undef, @values);
                if ($re == 1) {
                    $updatedDomains++;
                }
        } else {
            logOut("no ip for $result->{$id}->{domain_name}");
            $noIpDomains++;
        }
    }
    logOut("Updated Domains: $updatedDomains");
    logOut("Domains with no Ip found: $noIpDomains");
}
 
#script starts here
initialize();
getDomains();

Maybe there are more efficient methods but as I said, this worked pretty well, it didn’t consume a lot of results and it performs fast enough for my needs.