WebSockets are the latest way to provide bi-directional data transfer for HTTP applications. They replace outdated workarounds like AJAX, repeated polling, Comet, etc. WebSockets are a special protocol atop HTTP (which in itself runs over TCP/IP), and can be wrapped in SSL for security (as in HTTPS). Being a Web Technology, it seems to have been developed by the JavaScript people exclusively for the JavaScript people – working with it outside a web browser or Node.js server can seem convoluted. But that’s the world they built, and we just we live in it.

Basic Perl support / documentation for WebSockets was difficult for me to find. The Mojolicious framework (specifically the UserAgent module) has native WebSockets support and can act as a client, but I was looking for info on using with WebSockets on a lower level / without Mojo or other frameworks. Hopefully, this post can shed some light on how you can use Perl to connect to a remote server using WebSockets, and get access to those sweet real-time services using our favorite language.

First off, if you can, you should just use AnyEvent::WebSocket::Client or Net::Async::WebSocket::Client (depending on your preference of async framework). This package has already done the hard work of combining the two packages you’d probably use anyway, Protocol::WebSocket::Client (for encoding / decoding WebSocket frames), and AnyEvent (for cross-platform non-blocking I/O and doing all the messy TCP socket stuff for you). Having already established my status as a Luddite a desire to know what’s really going on, let’s try to reinvent the wheel and write our own client.

A Client for the Echo Test Service

The goal of this project is to interoperate with the WebSocket Echo Server at ws://echo.websocket.org. The Echo Server simply listens to any messages sent to it, and returns the same message to the caller. This is enough to build a simple client that we can then customize for other services. There are two things we need to make this work:

• a plain Internet Socket, for managing the TCP connection to the remote server, and
• a Protocol handler, for encoding / decoding data in the WebSocket format.

The second part of this is already done for us by Protocol::WebSocket::Client: given a stream of bytes, it can identify WebSocket frames and parse them into data from the server, and it can take data from our program and encapsulate it for sending. This tripped me up at first, so pay attention: Protocol::WebSocket does NOT actually do anything with the TCP socket itself – meaning it does not send or receive any data on its own! The class is responsible for only these things: packing/unpacking data, generating a properly formatted handshake for initiating a WebSocket connection, and sending a “close” message to the server signalling intent to disconnect.

Given that Protocol::WebSocket::Client doesn’t do any TCP socket stuff itself, we have to manage all that. Fortunately, there’s the core module IO::Socket::INET which we can use. Protocol::WebSocket::Client also provides some hooks for points in the WebSocket flow, so that we can insert our own handlers at those points. Let’s get started with some code.

Example Code

#!/usr/bin/env perl
use v5.014;
use warnings;

# Simple WebSocket test client using blocking I/O
#  Greg Kennedy 2019

# Core IO::Socket::INET for creating sockets to the Internet
use IO::Socket::INET;
# Protocol handler for WebSocket HTTP protocol
use Protocol::WebSocket::Client;

# Uncomment this line if your IO::Socket::INET is below v1.18 -
#  it enables auto-flush on socket operations.
#$|= 1;

#####################

die "Usage: $0 URL" unless scalar @ARGV == 1;

my $url = $ARGV[0];

# Protocol::WebSocket takes a full URL, but IO::Socket::* uses only a host
#  and port.  This regex section retrieves host/port from URL.
my ($proto, $host, $port, $path);
if ($url =~ m/^(?:(?<proto>ws|wss):\/\/)?(?<host>[^\/:]+)(?::(?<port>\d+))?(?<path>\/.*)?$/)
{
  $host = $+{host};
  $path = $+{path};

  if (defined $+{proto} && defined $+{port}) {
    $proto = $+{proto};
    $port = $+{port};
  } elsif (defined $+{port}) {
    $port = $+{port};
    if ($port == 443) { $proto = 'wss' } else { $proto = 'ws' }
  } elsif (defined $+{proto}) {
    $proto = $+{proto};
    if ($proto eq 'wss') { $port = 443 } else { $port = 80 }
  } else {
    $proto = 'ws';
    $port = 80;
  }
} else {
  die "Failed to parse Host/Port from URL.";
}

say "Attempting to open blocking INET socket to $proto://$host:$port...";

# create a basic TCP socket connected to the remote server.
my $tcp_socket = IO::Socket::INET->new(
  PeerAddr => $host,
  PeerPort => "$proto($port)",
  Proto => 'tcp',
  Blocking => 1
) or die "Failed to connect to socket: $@";

# create a websocket protocol handler
#  this doesn't actually "do" anything with the socket:
#  it just encodes / decode WebSocket messages.  We have to send them ourselves.
say "Trying to create Protocol::WebSocket::Client handler for $url...";
my $client = Protocol::WebSocket::Client->new(url => $url);

# This is a helper function to take input from stdin, and
#  * if "exit" is entered, disconnect and quit
#  * otherwise, send the value to the remote server.
sub get_console_input
{
  say "Type 'exit' to quit, anything else to message the server.";

  # get something from the user
  my $input;
  do { $input = <STDIN>;  chomp $input } while ($input eq '');

  if ($input eq 'exit') {
    $client->disconnect;
    exit;
  } else {
    $client->write($input);
  }
}

# Set up the various methods for the WS Protocol handler
#  On Write: take the buffer (WebSocket packet) and send it on the socket.
$client->on(
  write => sub {
    my $client = shift;
    my ($buf) = @_;

    syswrite $tcp_socket, $buf;
  }
);

# On Connect: this is what happens after the handshake succeeds, and we
#  are "connected" to the service.
$client->on(
  connect => sub {
    my $client = shift;

   # You may wish to set a global variable here (our $isConnected), or
   #  just put your logic as I did here.  Or nothing at all :)
   say "Successfully connected to service!";

   get_console_input();
  }
);

# On Error, print to console.  This can happen if the handshake
#  fails for whatever reason.
$client->on(
  error => sub {
    my $client = shift;
    my ($buf) = @_;

    say "ERROR ON WEBSOCKET: $buf";
    $tcp_socket->close;
    exit;
  }
);

# On Read: This method is called whenever a complete WebSocket "frame"
#  is successfully parsed.
# We will simply print the decoded packet to screen.  Depending on the service,
#  you may e.g. call decode_json($buf) or whatever.
$client->on(
  read => sub {
    my $client = shift;
    my ($buf) = @_;

    say "Received from socket: '$buf'";

    # it's our "turn" to send a message.
    get_console_input();
  }
);

# Now that we've set all that up, call connect on $client.
#  This causes the Protocol object to create a handshake and write it
#  (using the on_write method we specified - which includes sysread $tcp_socket)
say "Calling connect on client...";
$client->connect;

# Now, we go into a loop, calling sysread and passing results to client->read.
#  The client Protocol object knows what to do with the data, and will
#  call our hooks (on_connect, on_read, on_read, on_read...) accordingly.
while ($tcp_socket->connected) {
  # await response
  my $recv_data;
  my $bytes_read = sysread $tcp_socket, $recv_data, 16384;

  if (!defined $bytes_read) { die "sysread on tcp_socket failed: $!" }
  elsif ($bytes_read == 0) { die "Connection terminated." }

  # unpack response - this triggers any handler if a complete packet is read.
  $client->read($recv_data);
}

Running the Example

Save this to a file (blocking-client.pl) and execute it, passing a URL on the command line. If all goes well, you should connect to the remote service, and then are prompted to type messages. Sending a message should return the exact same message from the server. Typing “exit” will send a final “close” packet to the server, and then exit your program. An example session looks like this:

$ ./blocking-client.pl ws://echo.websocket.org
Attempting to open blocking INET socket to ws://echo.websocket.org:80...
Trying to create Protocol::WebSocket::Client handler for ws://echo.websocket.org...
Calling connect on client...
Successfully connected to service!
Type 'exit' to quit, anything else to message the server.
HELLO THERE!
Received from socket: 'HELLO THERE!'
Type 'exit' to quit, anything else to message the server.
It seems to be working.
Received from socket: 'It seems to be working.'
Type 'exit' to quit, anything else to message the server.
exit
$

SSL Support

The example above works for non-encrypted WebSockets only. As WS is a layer atop HTTP, it is also possible to run WebSockets over HTTPS using SSL, usually on port 443 instead. Secure WebSocket URLs begin with wss:// instead of ws://.

However, Net::Socket::INET does not have built-in support for SSL. Thus attempting to connect to the “secure” test server at wss://echo.websocket.org will cause the program to hang and never complete the handshake. This is because you are speaking unencrypted HTTP to a server expecting encrypted HTTP.

There are a few ways around this. The first is to replace IO::Socket::INET with something that is SSL-aware, as in IO::Socket::SSL. With this module, most functions are the same, and it should transparently handle the SSL for you. You may also try a module that provides SSL over an existing INET socket, as in Net::SSL, or for the truly hardcore use Net::SSLeay (Perl bindings to OpenSSL).

Non-Blocking I/O

While this example “works”, it has a major drawback: It uses blocking I/O. This means that socket communication happens in the foreground: when calling send() or recv() on the socket object, your program will halt at that point until data is available. While waiting to recv() some data, you can’t do anything else. That works OK for this Echo test, but remember that WebSockets are bi-directional: you should be able to juggle multiple things at once. For example, some services require you to send a periodic “heartbeat” to keep connected – but if already blocking on recv(), you can’t send() the necessary packet! Even the Echo example is limited: ideally, you should be able to send() two messages, then recv() two responses. But because of the design of the example script, it is forced into a “taking turns” pattern instead.

Again, there are ways around this. For a half-solution, you can continue to use blocking I/O, but with a timeout period. This allows you to wait a certain time to recv() / send() data before “giving up”. The function to adjust socket parameters is setsockopt() –

Another way to handle this is to instead use IO::Select. The Select object lets you pool sockets and then test at once if any have data waiting – thus, you can “multiplex” inputs together for asynchronous operation, even if the underlying sockets still block.

You can also create the sockets in non-blocking mode and poll them – reading when no data is available returns immediately, with an error E_WOULDBLOCK. A further abstraction is an I/O handling library such as IO::Poll, AnyEvent, POE etc. Of course, if you’re going to go THAT route, then maybe you should just do what I said at the start and use AnyEvent::WebSocket::Client – it combines Protocol::WebSocket with AnyEvent and gives a clean, non-blocking interface for interacting with remote web services.

For a detailed look at each of these methods, and a module that can handle some of them for you, I recommend reading the post “IO::Socket::Timeout: socket timeout made easy” on Medium.

A Non-Blocking, SSL-Aware Client

I’ll go ahead and modify my client to address the two issues above. This version uses IO::Socket::SSL instead (for wss:// connections), and IO::Select to handle non-blocking on the socket and STDIN, thus enabling a fully asynchronous connection to a WebSocket service. The initial connection uses a blocking connection until the initial handshake is complete; afterwards, select() is used to multiplex reads from the TCP socket and STDIN at the same time.

#!/usr/bin/env perl
use v5.014;
use warnings;

# Perl WebSocket test client
#  Greg Kennedy 2019

# IO::Socket::SSL lets us open encrypted (wss) connections
use IO::Socket::SSL;
# IO::Select to "peek" IO::Sockets for activity
use IO::Select;
# Protocol handler for WebSocket HTTP protocol
use Protocol::WebSocket::Client;

#####################

die "Usage: $0 URL" unless scalar @ARGV == 1;

my $url = $ARGV[0];

# Protocol::WebSocket takes a full URL, but IO::Socket::* uses only a host
#  and port.  This regex section retrieves host/port from URL.
my ($proto, $host, $port, $path);
if ($url =~ m/^(?:(?<proto>ws|wss):\/\/)?(?<host>[^\/:]+)(?::(?<port>\d+))?(?<path>\/.*)?$/)
{
  $host = $+{host};
  $path = $+{path};

  if (defined $+{proto} && defined $+{port}) {
    $proto = $+{proto};
    $port = $+{port};
  } elsif (defined $+{port}) {
    $port = $+{port};
    if ($port == 443) { $proto = 'wss' } else { $proto = 'ws' }
  } elsif (defined $+{proto}) {
    $proto = $+{proto};
    if ($proto eq 'wss') { $port = 443 } else { $port = 80 }
  } else {
    $proto = 'ws';
    $port = 80;
  }
} else {
  die "Failed to parse Host/Port from URL.";
}

say "Attempting to open SSL socket to $proto://$host:$port...";

# create a connecting socket
#  SSL_startHandshake is dependent on the protocol: this lets us use one socket
#  to work with either SSL or non-SSL sockets.
my $tcp_socket = IO::Socket::SSL->new(
  PeerAddr => $host,
  PeerPort => "$proto($port)",
  Proto => 'tcp',
  SSL_startHandshake => ($proto eq 'wss' ? 1 : 0),
  Blocking => 1
) or die "Failed to connect to socket: $@";

# create a websocket protocol handler
#  this doesn't actually "do" anything with the socket:
#  it just encodes / decode WebSocket messages.  We have to send them ourselves.
say "Trying to create Protocol::WebSocket::Client handler for $url...";
my $client = Protocol::WebSocket::Client->new(url => $url);

# Set up the various methods for the WS Protocol handler
#  On Write: take the buffer (WebSocket packet) and send it on the socket.
$client->on(
  write => sub {
    my $client = shift;
    my ($buf) = @_;

    syswrite $tcp_socket, $buf;
  }
);

# On Connect: this is what happens after the handshake succeeds, and we
#  are "connected" to the service.
$client->on(
  connect => sub {
    my $client = shift;

   # You may wish to set a global variable here (our $isConnected), or
   #  just put your logic as I did here.  Or nothing at all :)
   say "Successfully connected to service!";
  }
);

# On Error, print to console.  This can happen if the handshake
#  fails for whatever reason.
$client->on(
  error => sub {
    my $client = shift;
    my ($buf) = @_;

    say "ERROR ON WEBSOCKET: $buf";
    $tcp_socket->close;
    exit;
  }
);

# On Read: This method is called whenever a complete WebSocket "frame"
#  is successfully parsed.
# We will simply print the decoded packet to screen.  Depending on the service,
#  you may e.g. call decode_json($buf) or whatever.
$client->on(
  read => sub {
    my $client = shift;
    my ($buf) = @_;

    say "Received from socket: '$buf'";
  }
);

# Now that we've set all that up, call connect on $client.
#  This causes the Protocol object to create a handshake and write it
#  (using the on_write method we specified - which includes sysread $tcp_socket)
say "Calling connect on client...";
$client->connect;

# read until handshake is complete.
while (! $client->{hs}->is_done)
{
  my $recv_data;

  my $bytes_read = sysread $tcp_socket, $recv_data, 16384;

  if (!defined $bytes_read) { die "sysread on tcp_socket failed: $!" }
  elsif ($bytes_read == 0) { die "Connection terminated." }

  $client->read($recv_data);
}

# Create a Socket Set for Select.
#  We can then test this in a loop to see if we should call read.
my $set = IO::Select->new($tcp_socket, \*STDIN);

while (1) {
  # call select and see who's got data
  my ($ready) = IO::Select->select($set);

  foreach my $ready_socket (@$ready) {
    # read data from ready socket
    my $recv_data;
    my $bytes_read = sysread $ready_socket, $recv_data, 16384;

    # handler by socket type
    if ($ready_socket == \*STDIN) {
      # Input from user (keyboard, cat, etc)
      if (!defined $bytes_read) { die "Error reading from STDIN: $!" }
      elsif ($bytes_read == 0) {
        # STDIN closed (ctrl+D or EOF)
        say "Connection terminated by user, sending disconnect to remote.";
        $client->disconnect;
        $tcp_socket->close;
        exit;
      } else {
        chomp $recv_data;
        $client->write($recv_data);
      }
    } else {
      # Input arrived from remote WebSocket!
      if (!defined $bytes_read) { die "Error reading from tcp_socket: $!" }
      elsif ($bytes_read == 0) {
        # Remote socket closed
        say "Connection terminated by remote.";
        exit;
      } else {
        # unpack response - this triggers any handler if a complete packet is read.
        $client->read($recv_data);
      }
    }
  }
}