Blogaria

Ok, step 1. We configured Maven to use an external proxy for Internet contact, with the right proxy credentials. Then we retried. Still no luck. The outgoing proxy would block .jar files with a "403 forbidden" error. I've seen this behavior before, outgoing proxies will try to protect unsuspecting users from fetching e.g. .exe files - which will typically be a Trojan trying to infect the corporate network environment. But I'd never seen the blocking of jars yet. We could reproduce this fairly easily with a wget, as follows:

prompt: export http_proxy=http://proxyuser:proxypass@proxyhost:8080
prompt: wget -SO- http://where.ever.org/some/place/somefile.jar
Proxy request sent, awaiting response... 
  HTTP/1.0 403 Forbidden
  Date: Wed, 03 Aug 2010 10:23:02 GMT
  Content-Length: 257
  Content-Type: text/html
  Server: NetCache appliance (NetApp/5.6.2R1)
2010-08-04 12:23:00 ERROR 403: Forbidden.
prompt:

Now in a browser or at the command line, that's pretty trivial to circumvent. The NetCache can be easily fooled by specifying the final dot in the filename as "%2e", which is the hexadecimal notation for the dot. So, the following command would succeed:

prompt: wget 'http://where.ever.org/some/place/somefile%2ejar'

Right, on to step 2. More hacking and drilling holes. Perl's HTTP::Proxy to the rescue! I wrote up a small intermediate Perl proxy to recode dots into %2e, and forward the request onto the outgoing NetCache proxy. Without ado - here's the code. In order to make this work, you'll probably first need to install HTTP::Proxy using Perl's cpan.

#!/usr/bin/env perl

use strict;
use HTTP::Proxy qw(:log);
use HTTP::Proxy::HeaderFilter;

# Configuration section. Edit only this part.
# -------------------------------------------
my $configuration =  
  {
   # The downstream proxy
   proxyserver => 'the-proxy-hostname',
   proxyport   => 8080,
   proxyuser   => 'the-proxy-username',
   proxypass   => 'the-proxy-password',

   # Hosts to exclude from going to the external proxy
   noproxy     => '127.0.0.1, 10.1.1.1, 10.1.1.2', 

   # Log level of this proxy. Set to NONE to get errors only.
   loglevel    => NONE,
   # loglevel  => PROXY | STATUS | HEADERS,

   # Extensions to fix up (e.g. ".exe" to "%2eexe" to fool the
   # downstream proxies)
   extensions  => [ qw(zip exe jar) ],

   # Who are we.
   version     => '1.00',
  };

# Configuration ends. Do not edit below.
# --------------------------------------

# Reset any external proxy settings.
my $downstream_proxy = 'http://';
$downstream_proxy .=
  $configuration->{proxyuser} . ':' . $configuration->{proxypass} . '@'
  if ($configuration->{proxypass} ne '' and 
      $configuration->{proxyuser} ne '');
$downstream_proxy .= $configuration->{proxyserver};
$downstream_proxy .= ':' . $configuration->{proxyport}
  if ($configuration->{proxyport} ne '');
print("Downstream proxy: $downstream_proxy\n");
for my $e qw(http_proxy https_proxy ftp_proxy) {
    $ENV{$e} = $downstream_proxy;
}
$ENV{no_proxy} = $configuration->{noproxy};

# Package the request header filter.
{
    package MyFilter;
    use base qw(HTTP::Proxy::HeaderFilter);

    sub filter {
	my ($self, $headers, $message) = @_;
	$message->headers->header(User_Agent =>
				  "pproxy/$configuration->{version}");
	my $olduri = $message->uri();
	my $uri = $olduri;
	for my $e (@{ $configuration->{extensions} }) {
	    $uri =~ s{\.$e([^/]*)}{%2e$e$1}g;
	}
	# print("Old uri [$olduri] changed to [$uri]\n");
	$message->uri($uri);
    }
    1;
}

# Set up the logger
my $of;
if (-x '/usr/bin/logger') {
    open($of, '|/usr/bin/logger -t pproxy')
      or die("Cannot start logger pipe: $!\n");
    print("Logging to activity to the system log.\n");
} else {
    open($of, '>>/tmp/pproxy.log')
      or die("Cannot write /tmp/pproxy.log: $!\n");
    print("Logging activity to /tmp/proxy.log.\n");
}

# Instantiate and initialize proxy
my $proxy = HTTP::Proxy->new(port => 3128);
$proxy->push_filter(request => MyFilter->new());
$proxy->logfh($of);
$proxy->logmask($configuration->{loglevel});

# Run tha proxy!
$SIG{PIPE} = 'IGNORE';
print("Contact the proxy on http://localhost:3128.\n");
$proxy->start();

Next up, step 3. We reconfigured Maven to talk to http://localhost:3128 as its outgoing proxy, without supplying proxy credentials ('cuz the above pproxy does that). We fired up the build process using Maven and - success. Maven fetched all that it needed (.jar files, .jar.sha1 files and what not) and got busy.

We also put the following in the system-wide profile, so that any wget will work:

export http_proxy=http://localhost:3128

That's all! Happy hacking & if you have suggestions - as ever, drop me a note.

August 23rd, update: The above shown proxy binds only to the IP address of 'localhost' and hence can't be accessed from other machines. If you want to use the proxy from other IP addresses than the local machine, change the HTTP::Proxy constructor to pass a configuration variable host, with value undef. In that case, the proxy will bind to all IP addresses that are available. (The value can also be a specific IP address if you prefer that.) The constructor then becomes: my $proxy = HTTP::Proxy->new(port => 3128, host => undef); In order to allow the proxy to serve more than just a few requests at a time, the max_clients() setting can be increased: my $proxy = HTTP::Proxy->new(port => 3128, host => undef); $proxy->max_clients(1024); Similarly, the downstream timeout (to the NetCache proxy further on) is by default 60 seconds. If you want to shorten it to say 10 seconds, add: $proxy->timeout(10);