Connecting to Bittorrent's Mainline DHT

01 March 2017

In addition to its fast and efficient file transfer protocol, Bittorrent and other peer-to-peer file sharing networks bring another interesting technology to the masses: the Distributed Hash Table (DHT). In the case of Bittorrent, this is used to look up and download a torrent file based on its magnet link. The DHT network for Bittorrent is called Mainline DHT based the Kademlia DHT, although the network itself is separate from other implementations.

For Mainline DHT, there is an interesting open source client and library, called “mldht”. There are two instances on Github, with moreus/mldht as the original and the fork the8472/mldht. Although, both seems to be somewhat active. Both depend on the EdDSA library which the8472 has also forked. To get started, grab the source, and make sure the “mldht” code depend on or have access to the “ed25519” project. To confirm run the 28 unit tests from “mldht”. They should all pass.

https://github.com/the8472/ed25519-java.git

https://github.com/the8472/mldht.git

The “mldht” project includes a stand-alone DHT server node which can be started through the executable Launcher class. Its main configuration is in “config.xml” which gets written to the current directory if it does not already exist. It’s based on “config-defaults.xml”. To be able to connect to the DHT network, I had to change the option “multihoming” to false in the config file.

Furthermore, in order to use the CLI client utility, the CLI server has to be enabled. I ended up with the following configuration file.

Once started, the activity can be observed in log/dht.log. There will be plenty of noise.

<?xml version="1.0" encoding="UTF-8"?>
<mldht:config
  xmlns:mldht="http://mldht/config/"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://mldht/config/config.xsd ">

  <core>
    <logLevel>Info</logLevel>
    <port>49001</port>
    <useBootstrapServers>true</useBootstrapServers>
    <multihoming>false</multihoming>
    <persistID>true</persistID>
  </core>
    
  <components>
    <component>
      <className>the8472.mldht.cli.Server</className>
    </component>
  </components>

</mldht:config>

While the server is running, the CLI Client executable can be used to issue a few commands. Of interest is the “SAMPLE” command which lists random hash keys from random peers. Using that output, the “GETPEERS” can be used to look up specific hashes (make sure to remove the space formatting from the sample output). Finally, a torrent file can be downloaded with the “GETTORRENT” command.

Assuming the Java classpath is set correctly, example use would look like:

java the8472.mldht.cli.Client SAMPLE
java the8472.mldht.cli.Client GETPEERS f61c5a0dfaac58ba943c5d0c115343477196ad91
java the8472.mldht.cli.Client GETTORRENT f61c5a0dfaac58ba943c5d0c115343477196ad91

The hash used above is the Wikileaks “insurance” file posted last December, with the name “2016-12-09_WL-Insurance.aes256”. The “mldht” project does not contain any tools to actually read the torrent, but we can use the Transmission client:

apt-get install transmission-cli

transmission-show f61c5a0dfaac58ba943c5d0c115343477196ad91.torrent

The expected output would look like this:

Name: 2016-12-09_WL-Insurance.aes256
File: F61C5A0DFAAC58BA943C5D0C115343477196AD91.torrent

GENERAL

  Name: 2016-12-09_WL-Insurance.aes256
  Hash: f61c5a0dfaac58ba943c5d0c115343477196ad91
  Created by: 
  Created on: Unknown
  Piece Count: 42979
  Piece Size: 2.00 MiB
  Total Size: 90.13 GB
  Privacy: Public torrent

TRACKERS

FILES

  2016-12-09_WL-Insurance.aes256 (90.13 GB)

Given that this worked fine, I’m thinking it should be trivial to create a custom ML DHT client which performs the steps above. I hope to come back to that in a future article.

Try Catch Exception

28 February 2017

This post looks at the syntax variations of the try/catch/finally blocks. For further details, see the excellent Java tutorial on the topic.

To start off, below is the basic syntax, with code surrounded by a try-block, and a NullPointerException caught by the catch-block. As can bee seen, the code will fail, since the variable “str” is null, leading to a NullPointerException. The Exception variable in the catch-block is by most common conventions simply “e”. It has a few convenience methods, including printStackTrace() which shows the call trace since the Exception was thrown. Although the print-out might look scary, it does provide useful information to the developer. Thus, keeping the full stack trace is helpful, typically in a detailed log-file. That is beyond the scope of this post.

    try {
      String str = null;
      str.toString();
    } catch (NullPointerException e) {
      e.printStackTrace();
    }

    try {

Exceptions are typed classes, and in the following example the catch-block will not be reached since the expected Exception is not the same or a sub-type of the one which is thrown: NullPointerException vs. ArithmeticException. Instead, the ArithmeticException will be thrown out of the method.

    try {
      int nil = 0;
      int a = 1 / nil;
    } catch (NullPointerException e) {
      System.out.println("This will not trigger.");
    }
    System.out.println("Will not reach this point.");

To handle multiple exception types, there are three options: Declare multiple catch-blocks with different types, as seen in the first part below; or declare multiple Exceptions within the same catch statement, as in the second part. The latter syntax has been available since Java 7. Finally, it’s possible to catch multiple Exceptions by specifying a type higher up the class hierarchy, e.g. using Exception or Throwable.

    try {
      Integer a = null;
      int b = 1 / a;
    } catch (NullPointerException e) {
      System.out.println("NullPointerException");
    } catch (ArithmeticException e) {
      System.out.println("ArithmeticException");
    }

    try {
      Integer a = null;
      int b = 1 / a;
    } catch (NullPointerException | ArithmeticException e) {
      e.printStackTrace();
    }

In addition to the try and catch blocks, there is also a finally-block. It is executed at the end, regardless of whether there was an Exceptions thrown or not. This is useful for setting state or cleaning up, and a common example is closing an IO stream. However, as seen below, this can get crufty since we have to consider that the IO object might not have been opened in the first place, and that the close() method throws its own checked Exception.

    OutputStream out = null;
    try {
      out = new FileOutputStream("/dev/null");
      out.write(0);
    } catch (IOException e) {
      e.printStackTrace();
    } finally {
      if (out != null) {
        try {
          out.close();
        } catch (IOException e) {
          e.printStackTrace();
        }
      }
    }

To clean up the code above, the try-with-resources syntax was introduced in Java 7. It allows the try statement to take an extra block which is executed before its content. In addition, variables declared within this block will be closed at the end through the Closeable interface. This significantly reduces the foot-print of the code above.

    try (OutputStream out = new FileOutputStream("/dev/null")) {
      out.write(0);
    } catch (IOException e) {
      e.printStackTrace();
    }

A good example for multiple resources is the Socket example discussed previously. Here the Socket and both IO streams are closable resources handled by the try-block.

    try (Socket s = new Socket("google.com", 80);
         OutputStream out = s.getOutputStream();
         InputStream in = s.getInputStream()) {
      out.write(0);
      in.read();
    } catch (IOException e) {
      e.printStackTrace();
    }

Finally, a word on messaging and wrapping of Exceptions. As mentioned in the tutorial, it’s poor practice to throw RuntimeExceptions or simply wrap checked Exceptions, as seen below. However, regardless of where you stand in that debate, Exceptions can always be made more helpful and useful by clear messaging and relevant context. The wrapped RuntimeException below adds a more specific message and also includes the filename the IO stream operates on, since it might not be included in all types of IOExceptions. Furthermore, in the case of the File object, is is useful to use the getAbsolutePath() method. It forces the full path to the resolved and included. It really helps when debugging issues where the full path can be copy/pasted and confirmed.

    File file = new File("/dev/null");
    try (OutputStream out = new FileOutputStream(file)) {
      out.write(0);
    } catch (IOException e) {
      throw new RuntimeException("Could not open or read file " + file.getAbsolutePath(), e);
    }

Here is the full listing with all examples as tests.

TryCatchTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.basics;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;

import org.junit.Test;

public class TryCatchTest {

  // The second try-block will throw an uncaught ArithmeticException because of
  // the divide by 0.
  //@Test(expected = ArithmeticException.class)
  public void basic() {
    try {
      String str = null;
      str.toString();
    } catch (NullPointerException e) {
      e.printStackTrace();
    }

    try {
      int nil = 0;
      int a = 1 / nil;
    } catch (NullPointerException e) {
      System.out.println("This will not trigger.");
    }
    System.out.println("Will not reach this point.");
  }

  @Test
  public void multi() {
    try {
      Integer a = null;
      int b = 1 / a;
    } catch (NullPointerException e) {
      System.out.println("NullPointerException");
    } catch (ArithmeticException e) {
      System.out.println("ArithmeticException");
    }

    try {
      Integer a = null;
      int b = 1 / a;
    } catch (NullPointerException | ArithmeticException e) {
      e.printStackTrace();
    }
  }

  @Test
  public void testFinally() {
    OutputStream out = null;
    try {
      out = new FileOutputStream("/dev/null");
      out.write(0);
    } catch (IOException e) {
      e.printStackTrace();
    } finally {
      if (out != null) {
        try {
          out.close();
        } catch (IOException e) {
          e.printStackTrace();
        }
      }
    }
  }

  @Test
  public void tryWith() {
    try (OutputStream out = new FileOutputStream("/dev/null")) {
      out.write(0);
    } catch (IOException e) {
      e.printStackTrace();
    }
  }

  @Test
  public void tryWithMulti() {
    try (Socket s = new Socket("google.com", 80);
         OutputStream out = s.getOutputStream();
         InputStream in = s.getInputStream()) {
      out.write(0);
      in.read();
    } catch (IOException e) {
      e.printStackTrace();
    }
  }

  @Test
  public void message() {
    File file = new File("/dev/null");
    try (OutputStream out = new FileOutputStream(file)) {
      out.write(0);
    } catch (IOException e) {
      throw new RuntimeException("Could not open or read file " + file.getAbsolutePath(), e);
    }
  }
}

Unit test Exceptions

24 February 2017

Testing the “happy path” of the code, when everything goes right, is fine, however error handling and Exceptions is just as much a part of the code under test. In fact, it is possibly a more delicate area, since you want an application which degrades gracefully in the event of error. This post goes through different ways of setting expectations on thrown Exceptions.

In the first method below, the old style pre-Junit 4 way of asserting for an Exception is shown. The idea is that the Exception must be thrown, so if the execution reaches the fail() statement, that did not happen. The expected path is instead that the catch-block engages, with the expected Exception type. If a different type of Exception, which is not a sub-class of the caught Exception is thrown, it propagates out of the method and the test fails. Although this style is a bit clunky and verbose, it has a few advantages over the annotation-style: You have control over exactly what point in the code you expect the Exception to be thrown; you can inspect the Exception and assert its message; you can set a custom error message.

  public void testOldStyle() {
    try {
      Integer.parseInt(INVALID_INTEGER);
      fail("Expected an Exception to be thrown");
    } catch (NumberFormatException e) {
      assertEquals("For input string: \"" + INVALID_INTEGER + "\"", e.getMessage());
    }
  }

Since Java 6 and Junit 4, annotations become available, and the @Test annotation is now the way to declare a test method. It comes with an extra parameter expected which takes a Class type indicating which Exception is expected to be thrown. This approach is clean and even minimalist. If all there is to a test is a one-liner like shown below, this is a perfectly fine way to declaring the expectation. However, it loses the ability to inspect the message or cause. Furthermore, if the test method contains more lines, there is no way to control or verify from where the Exception originated.

  @Test(expected = NumberFormatException.class)
  public void annotation() {
    Integer.parseInt(INVALID_INTEGER);
  }

Enter JUnit 4.7, and the @Rule annotation and ExpectedException rule. It solves the problem with the Test annotation above, but retains a clean way of expressing the assertion. In the example below, the Rule annotation makes sure the thrown field is initialised a-new before every test method. It can then be used right above the method which is expected to throw an Exception, and can assert on its type and message. The expectMessage() method asserts that the message contains rather than equals the expected string.

  @Rule
  public final ExpectedException thrown = ExpectedException.none();

  @Test
  public void testThrown() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(INVALID_INTEGER);

    Integer.parseInt(INVALID_INTEGER);
  }

The ExpectedException class comes with some extra assertion methods which takes the popular Hamcrest matchers. In the code below, the endsWith matcher is used.

import static org.hamcrest.CoreMatchers.endsWith;

  @Test
  public void hamcrest() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(endsWith(INVALID_INTEGER + "\""));

    Integer.parseInt(INVALID_INTEGER);
  }

Finally, the expectCause() method takes another Hamcrest matcher, where for example the type of the contained cause can be asserted. Notice that the outer exception type can be asserted as well. The expectCause assertion only goes one level deep, so if further unnesting is required, a custom Matcher could be implemented. In this example, the wrapping RuntimeException does not alter the message, so it can be asserted directly. If the outer Exception has its own message, another custom Matcher would be needed to assert on the inner message.

import static org.hamcrest.CoreMatchers.isA;

  @Test
  public void cause() {
    thrown.expect(RuntimeException.class);
    thrown.expectMessage(INVALID_INTEGER);
    thrown.expectCause(isA(NumberFormatException.class));

    try {
      Integer.parseInt(INVALID_INTEGER);
    } catch (NumberFormatException e) {
      throw new RuntimeException(e);
    }
  }

Here is the full test case listing.

ExceptionsTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.junit;

import static org.hamcrest.CoreMatchers.endsWith;
import static org.hamcrest.CoreMatchers.isA;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class ExceptionsTest {

  private static final String INVALID_INTEGER = "invalid integer";

  @Rule
  public final ExpectedException thrown = ExpectedException.none();

  @Test
  public void testOldStyle() {
    try {
      Integer.parseInt(INVALID_INTEGER);
      fail("Expected an Exception to be thrown");
    } catch (NumberFormatException e) {
      assertEquals("For input string: \"" + INVALID_INTEGER + "\"", e.getMessage());
    }
  }

  @Test(expected = NumberFormatException.class)
  public void annotation() {
    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void testThrown() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(INVALID_INTEGER);

    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void hamcrest() {
    thrown.expect(NumberFormatException.class);
    thrown.expectMessage(endsWith(INVALID_INTEGER + "\""));

    Integer.parseInt(INVALID_INTEGER);
  }

  @Test
  public void cause() {
    thrown.expect(RuntimeException.class);
    thrown.expectMessage(INVALID_INTEGER);
    thrown.expectCause(isA(NumberFormatException.class));

    try {
      Integer.parseInt(INVALID_INTEGER);
    } catch (NumberFormatException e) {
      throw new RuntimeException(e);
    }
  }
}

Socket client / server example

21 February 2017

After posts on several server libraries, including Sun’s HTTP; Simple Framework’s HTTP; and Websocket, it seems appropriate to include a plain TCP socket server / client example. Without any application protocol to adhere to, it’s straight forward to setup and test. There are two essential classes involved: On the server side there’s a ServerSocket, while the client has a Socket. The only difference is that the ServerSocket accepts one or more incoming connections, and provide a Socket handle for each. After that, both client and server are the same, in that the Socket object provides input and output streams once established.

In the code below, a ServerSocket is set to listen to a specific port, and to accept a single incoming connection. It grabs the IO streams, makes these available to the rest of the test, and finally releases a semaphore lock to allow the test to continue. Normally, the server thread would wait for further incoming requests, and would probably spawn or assign pooled threads to handle the request. Here we focus only on the basic IO parts.

...
    ServerSocket server = new ServerSocket(PORT);
    listen(server);
...

  private void listen(ServerSocket server) {
    new Thread(() -> {
      try {
        Socket socket = server.accept();
        System.out.println("Incoming connection: " + socket);

        serverOut = socket.getOutputStream();
        serverIn = socket.getInputStream();

        lock.release();
        System.out.println("Released lock");
      } catch (IOException e) {
        e.printStackTrace();
      }
    }).start();
  }

On the client side, it’s just as simple: Establish a connection to the severer host and port, and get the IO streams.

    Socket client = new Socket("localhost", PORT);
    OutputStream clientOut = client.getOutputStream();
    InputStream clientIn = client.getInputStream();

The rest of the test code below asserts that messages are received correctly both on the client and the server, using the raw streams and wrapped PrinterWriter helpers. Again, he semaphore is used to wait for the server thread to establish its connection before the rest of the test continues. Without it, using the IO streams will results in NullPointerExceptions, since they are not initialized yet.

SocketTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.net;

import static org.junit.Assert.assertEquals;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.Semaphore;

import org.junit.Test;

/**
 * Simple client / server Socket tests, including a Buffered PrintWriter which
 * has to be flushed.
 */
public class SocketTest {

  private static final int PORT = 8887;

  private OutputStream serverOut;
  private InputStream serverIn;

  /**
   * Shared lock between the "client" and "server" code, to make the test case
   * synchronous.
   */
  private Semaphore lock = new Semaphore(0);

  /**
   * Tests server and client side sockets in one flow. A lock object is used for
   * synchronous between the two sides.
   */
  @Test
  public void testClientServer() throws IOException, InterruptedException {
    ServerSocket server = new ServerSocket(PORT);
    listen(server);

    Socket client = new Socket("localhost", PORT);
    OutputStream clientOut = client.getOutputStream();
    InputStream clientIn = client.getInputStream();

    System.out.println("Waiting for lock");
    lock.acquire();
    System.out.println("Acquired lock");

    write(clientOut, "Hi");
    assertRead(serverIn, "Hi");

    write(serverOut, "Hello");
    assertRead(clientIn, "Hello");

    printWrite(clientOut, "Test printWrite");
    assertRead(serverIn, "Test printWrite");

    printWrite(serverOut, "Test printWrite again");
    assertRead(clientIn, "Test printWrite again");

    client.close();
    server.close();
  }

  /**
   * Writes to an OutputStream. Used for both server and client output streams.
   */
  private void write(OutputStream out, String str) throws IOException {
    out.write(str.getBytes());
    out.flush();
  }

  /**
   * Writes to an OutputStream. Used for both server and client output streams.
   */
  private void printWrite(OutputStream out, String str) throws IOException {
    PrintWriter pw = new PrintWriter(out);
    pw.print(str);
    pw.flush();
  }

  /**
   * Reads from an InputStream. Used for both server and client input streams.
   */
  private void assertRead(InputStream in, String expected) throws IOException {
    assertEquals("Too few bytes available for reading: ", expected.length(), in.available());

    byte[] buf = new byte[expected.length()];
    in.read(buf);
    assertEquals(expected, new String(buf));
  }

  /**
   * Listens for and accepts one incoming request server side on a separate
   * thread. When a request is received, grabs its IO streams and "signals" to
   * the client side above through the shared lock object.
   */
  private void listen(ServerSocket server) {
    new Thread(() -> {
      try {
        Socket socket = server.accept();
        System.out.println("Incoming connection: " + socket);

        serverOut = socket.getOutputStream();
        serverIn = socket.getInputStream();

        lock.release();
        System.out.println("Released lock");
      } catch (IOException e) {
        e.printStackTrace();
      }
    }).start();
  }
}

Styles with JTextPane

20 February 2017

The Swing tutorial for the JTextPane and demo code is good and comprehensive, covering multiple ways to interact with the the StyledDocument and add Styles. So, without repeating all of that, here’s a minimal example including only simple style changes on the insert update and selection events.

In the first snippet below, the implementation for the insertUpdate event is shown. For each typed character or pasted text, the entire text of the document will be searched for the word “foo”. For each occurrence, the bold attribute is set for that word. Notice that the update of the attribute happens on a separate AWT thread.

  @Override
  public void insertUpdate(DocumentEvent event) {
    try {
      String text = doc.getText(0, doc.getLength());

      Pattern p = Pattern.compile(FOO);
      Matcher matcher = p.matcher(text);
      while (matcher.find()) {
        updateAttribute(matcher.start(), FOO.length(), Style.BOLD);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  private void updateAttribute(int pos, int len, Style style) {
    SwingUtilities.invokeLater(() -> {
      doc.setCharacterAttributes(pos, len, style.get(), true);
    });
  }

The other functionality of this small application is on select. If the selected word is “bar”, it is set to italic. Notice the dot and mark positions, which might be at the start or the end of the selection. Furthermore, notice that here the predefined ItalicAction from the StyledEditorKit is used, since we’re dealing with a selection. We just have to translate the CaretEvent into an ActionEvent, or rather just make sure to forward the source component of the selection. (The alternative would have been to go with the plain update as in the example above).

  @Override
  public void caretUpdate(CaretEvent event) {
    int dot = event.getDot();
    int mark = event.getMark();
    int start;
    int end;

    if (dot == mark) {
      return;
    } else if (dot < mark) {
      start = dot;
      end = mark;
    } else {
      start = mark;
      end = dot;
    }

    System.out.println(start + ", " + end);
    try {
      if (doc.getText(start, BAR.length()).startsWith(BAR)) {
        ItalicAction action = new StyledEditorKit.ItalicAction();
        action.actionPerformed(new ActionEvent(event.getSource(), 0, ""));

        // Alternative custom update:
        // updateAttribute(start, BAR.length(), Style.ITALIC);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

Here’s the full file listing, as a stand-alone application.

JTextPaneStylesExample.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.event.ActionEvent;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.swing.JFrame;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyledDocument;
import javax.swing.text.StyledEditorKit;
import javax.swing.text.StyledEditorKit.ItalicAction;

/**
 * Minimal example with JTextPane and StyledDocument. When the string "foo" is
 * typed or pasted it is made bold. When the string "bar" is selected it is made
 * italic.
 */
@SuppressWarnings("serial")
public class JTextPaneStylesExample extends JFrame implements DocumentListener, CaretListener {

  /**
   * Predefined styles.
   */
  enum Style {
    BOLD(StyleConstants.Bold),
    ITALIC(StyleConstants.Italic);

    private MutableAttributeSet attrib;

    private Style(Object style) {
      attrib = new SimpleAttributeSet();
      attrib.addAttribute(style, true);
    }

    AttributeSet get() {
      return attrib;
    }
  }

  private static final String FOO = "foo";

  private static final String BAR = "bar";

  private final StyledDocument doc;

  public static void main(String[] args) {
    new JTextPaneStylesExample();
  }

  public JTextPaneStylesExample() {
    JTextPane editor = new JTextPane();
    doc = editor.getStyledDocument();
    getContentPane().add(editor);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);

    editor.getDocument().addDocumentListener(this);
    editor.addCaretListener(this);
  }

  /**
   * Check of a selection of the string "bar".
   */
  @Override
  public void caretUpdate(CaretEvent event) {
    int dot = event.getDot();
    int mark = event.getMark();
    int start;
    int end;

    if (dot == mark) {
      return;
    } else if (dot < mark) {
      start = dot;
      end = mark;
    } else {
      start = mark;
      end = dot;
    }

    System.out.println(start + ", " + end);
    try {
      if (doc.getText(start, BAR.length()).startsWith(BAR)) {
        ItalicAction action = new StyledEditorKit.ItalicAction();
        action.actionPerformed(new ActionEvent(event.getSource(), 0, ""));

        // Alternative custom update:
        // updateAttribute(start, BAR.length(), Style.ITALIC);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  /**
   * Check for the occurrence of the string "foo".
   */
  @Override
  public void insertUpdate(DocumentEvent event) {
    try {
      String text = doc.getText(0, doc.getLength());

      Pattern p = Pattern.compile(FOO);
      Matcher matcher = p.matcher(text);
      while (matcher.find()) {
        updateAttribute(matcher.start(), FOO.length(), Style.BOLD);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  /**
   * Update the string at the given position and length with the given style.
   * The update happens on a separate AWT thread, to avoid mutations of the
   * Document model while the event is processing.
   */
  private void updateAttribute(int pos, int len, Style style) {
    SwingUtilities.invokeLater(() -> {
      doc.setCharacterAttributes(pos, len, style.get(), true);
    });
  }

  @Override
  public void removeUpdate(DocumentEvent e) {}

  @Override
  public void changedUpdate(DocumentEvent e) {}
}

Line numbers for JEditorPane and JTextPane

19 February 2017

There are many ways to add line numbers to the JEditorPane and JTextPane Swing components. This post looks at two solutions: One providing a custom ParagraphView which paints a child element that contains the line number for that paragraph. The other implements a separate component which is passed as the RowHeaderView of the JScrollPane. Both support scrolling and line wrapping, and in this example the latter includes current line highlighting, as seen in the image below.

Line numbers with RowHeaderView

ParagraphView

The ParagraphView solution is inspired by Stanislav Lapitsky’s old post on the topic, although with some improvements. The basic idea is to paint the line number in the child of the ParagraphView, using the paintChild() method. The advantage is that the relationship between the line and the line number is already established, so alignment is easy. We just have to count which element we’re dealing with to know which number to print. The two methods below show this part. Notice also that only the index 0 of the ParagraphView should be used for the line number, so we don’t count a wrapped line twice. Furthermore, a mono-spaced font helps with padding and alignment.

In order to create this custom ParagraphView, it has to be returned through a ViewFactory, which again has to be provided through an EditorKit. The file below shows the full implementation. Notice how the default ViewFactory is used for all other elements; only the Paragraph Element gets a custom view.

    public void paintChild(Graphics g, Rectangle alloc, int index) {
      super.paintChild(g, alloc, index);

      // Allow of wrapped paragraph lines, but don't print redundant line
      // numbers.
      if (index > 0) {
        return;
      }

      // Pad left so the numbers align
      int lineNumber = getLineNumber() + 1;
      String lnStr = String.format("%3d", lineNumber);

      // Make sure we use a monospaced font.
      font = font != null ? font : new Font(Font.MONOSPACED, Font.PLAIN, getFont().getSize());
      g.setFont(font);

      int x = alloc.x - getLeftInset();
      int y = alloc.y + alloc.height - 3;
      g.drawString(lnStr, x, y);
    }

    private int getLineNumber() {
      // According to the Document.getRootElements() doc, there will "typically"
      // only be one root element.
      Element root = getDocument().getDefaultRootElement();
      int len = root.getElementCount();
      for (int i = 0; i < len; i++) {
        if (root.getElement(i) == thisElement) {
          return i;
        }
      }
      return 0;
    }

setRowHeaderView

The second solution renders the line numbers in a completely separate component, which can be a JComponent (or even an old style AWT Component). This component is passed in to the setRowHeaderView() method of JScrollPane. That way, the coupling becomes a bit cleaner than overriding multiple classes and methods as with the the ParagraphView solution. However, the down-side is that we must do the alignment with the Editor component manually. Luckily, the Swing Text Component API provides a solid API for this purpose.

The code below is inspired by Rob Camick’s article, but simplifies several aspects of his stand-alone API style class. At the centre is the translation between the Document model and its onscreen view. The viewToModel() method translates an x,y point on the screen to the nearest character offset, while the modelToView() method goes the other way. That way we can align the line number at the same y point as the line in the editor. In Camick’s code, he loops through the text by character offsets rather than paragraphs; the getRowEnd() method helps by finding the end of a line. Camick also points out the the line number should take the font decent of the editor font into account when positioning the line number. The FontMetrics class helps with measuring sizes of fonts.

Finally worth noting is the synchronisation between the editor and the margin component. Since they are separate, this must be handled through event listeners registered with the editor component. The DocumentListener and ComponentListener notifies of edit, movement and resizing events, while the CaretListener signals movement in the cursor position. All of these events force a repaint, but through a scheduled AWT thread, to make sure the editor view has updated first, as seen in the documentChanged() method at the bottom of the code section below.

    public void paintComponent(Graphics g) {
      super.paintComponent(g);

      Rectangle clip = g.getClipBounds();
      int startOffset = editor.viewToModel(new Point(0, clip.y));
      int endOffset = editor.viewToModel(new Point(0, clip.y + clip.height));

      while (startOffset <= endOffset) {
        try {
          String lineNumber = getLineNumber(startOffset);
          if (lineNumber != null) {
            int x = getInsets().left + 2;
            int y = getOffsetY(startOffset);

            font = font != null ? font : new Font(Font.MONOSPACED, Font.BOLD, editor.getFont().getSize());
            g.setFont(font);

            g.setColor(isCurrentLine(startOffset) ? Color.RED : Color.BLACK);

            g.drawString(lineNumber, x, y);
          }

          startOffset = Utilities.getRowEnd(editor, startOffset) + 1;
        } catch (BadLocationException e) {
          e.printStackTrace();
          // ignore and continue
        }
      }
    }

    private String getLineNumber(int offset) {
      Element root = editor.getDocument().getDefaultRootElement();
      int index = root.getElementIndex(offset);
      Element line = root.getElement(index);

      return line.getStartOffset() == offset ? String.format("%3d", index + 1) : null;
    }

    private int getOffsetY(int offset) throws BadLocationException {
      FontMetrics fontMetrics = editor.getFontMetrics(editor.getFont());
      int descent = fontMetrics.getDescent();

      Rectangle r = editor.modelToView(offset);
      int y = r.y + r.height - descent;

      return y;
    }

    private boolean isCurrentLine(int offset) {
      int caretPosition = editor.getCaretPosition();
      Element root = editor.getDocument().getDefaultRootElement();
      return root.getElementIndex(offset) == root.getElementIndex(caretPosition);
    }

    private void documentChanged() {
      SwingUtilities.invokeLater(() -> {
        repaint();
      });
    }

Here are the full code listings for both examples, as stand-alone applications.

ParagraphViewLineNumbers.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.Font;
import java.awt.Graphics;
import java.awt.Rectangle;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Random;
import java.util.stream.Collectors;

import javax.swing.JEditorPane;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.text.AbstractDocument;
import javax.swing.text.Element;
import javax.swing.text.ParagraphView;
import javax.swing.text.StyledEditorKit;
import javax.swing.text.View;
import javax.swing.text.ViewFactory;

/**
 * Example use of a left margin line number for a JEditorPane. Using a custom
 * EditorKit and ViewFactory to pass in ParagraphView which paints a child
 * element. Pads the line numbers up to 999, and handles wrapped lines in the
 * editor.
 */
public class ParagraphViewLineNumbers extends JFrame {

  private static final long serialVersionUID = 1L;

  public static void main(String[] args) throws IOException {
    new ParagraphViewLineNumbers();
  }

  public ParagraphViewLineNumbers() throws IOException {
    JEditorPane editor = new JEditorPane();
    editor.setEditorKit(new CustomEditorKit());
    editor.setText(getRandomText());

    JScrollPane scroll = new JScrollPane(editor);
    getContentPane().add(scroll);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);
  }

  /**
   * Returns the first 100 words of the default dictionary (on Ubuntu / Debian)
   * and injects random line breaks.
   */
  private String getRandomText() throws IOException {
    Random rnd = new Random();
    return Files.readAllLines(Paths.get("/usr/share/dict/words")).stream().limit(100)
        .map(word -> (rnd.nextInt(15) == 0 ? "\n" : "") + word).collect(Collectors.joining(" "));
  }

  /**
   * Passes in the custom ViewFactory. Inherits from the StyledEditorKit since
   * that already comes with a default ViewFactory (while the DefaultEditorKit
   * does not).
   */
  class CustomEditorKit extends StyledEditorKit {

    private static final long serialVersionUID = 1L;

    @Override
    public ViewFactory getViewFactory() {
      return new CustomViewFactory(super.getViewFactory());
    }
  }

  /**
   * Produces custom ParagraphViews, but uses the default ViewFactory for all
   * other elements.
   */
  class CustomViewFactory implements ViewFactory {

    private ViewFactory defaultViewFactory;

    CustomViewFactory(ViewFactory defaultViewFactory) {
      this.defaultViewFactory = defaultViewFactory;
    }

    @Override
    public View create(Element elem) {
      if (elem != null && elem.getName().equals(AbstractDocument.ParagraphElementName)) {
        return new CustomParagraphView(elem);
      }
      return defaultViewFactory.create(elem);
    }
  }

  /**
   * Paints a left hand child view with the line number for this Paragraph.
   */
  class CustomParagraphView extends ParagraphView {

    public final short MARGIN_WIDTH_PX = 25;

    private Element thisElement;

    private Font font;

    public CustomParagraphView(Element elem) {
      super(elem);
      thisElement = elem;
      this.setInsets((short) 0, (short) 0, (short) 0, (short) 0);
    }

    @Override
    protected void setInsets(short top, short left, short bottom, short right) {
      super.setInsets(top, (short) (left + MARGIN_WIDTH_PX), bottom, right);
    }

    @Override
    public void paintChild(Graphics g, Rectangle alloc, int index) {
      super.paintChild(g, alloc, index);

      // Allow of wrapped paragraph lines, but don't print redundant line
      // numbers.
      if (index > 0) {
        return;
      }

      // Pad left so the numbers align
      int lineNumber = getLineNumber() + 1;
      String lnStr = String.format("%3d", lineNumber);

      // Make sure we use a monospaced font.
      font = font != null ? font : new Font(Font.MONOSPACED, Font.PLAIN, getFont().getSize());
      g.setFont(font);

      int x = alloc.x - getLeftInset();
      int y = alloc.y + alloc.height - 3;
      g.drawString(lnStr, x, y);
    }

    private int getLineNumber() {
      // According to the Document.getRootElements() doc, there will "typically"
      // only be one root element.
      Element root = getDocument().getDefaultRootElement();
      int len = root.getElementCount();
      for (int i = 0; i < len; i++) {
        if (root.getElement(i) == thisElement) {
          return i;
        }
      }
      return 0;
    }
  }
}
RowHeaderViewLineNumbers.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.Color;
import java.awt.Dimension;
import java.awt.Font;
import java.awt.FontMetrics;
import java.awt.Graphics;
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.event.ComponentEvent;
import java.awt.event.ComponentListener;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Random;
import java.util.stream.Collectors;

import javax.swing.JComponent;
import javax.swing.JEditorPane;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.text.BadLocationException;
import javax.swing.text.Element;
import javax.swing.text.JTextComponent;
import javax.swing.text.Utilities;

/**
 * Example use of a left margin line number for a JEditorPane in a JScrollPane.
 * Uses a separate component for the RowHeaderView of the JScrollPane. Pads the
 * line numbers up to 999, highlights the currently active line, handles wrapped
 * lines and resizing of the editor.
 */
public class RowHeaderViewLineNumbers extends JFrame {

  private static final long serialVersionUID = 1L;

  public final int MARGIN_WIDTH_PX = 28;

  public static void main(String[] args) throws IOException {
    new RowHeaderViewLineNumbers();
  }

  public RowHeaderViewLineNumbers() throws IOException {
    JEditorPane editor = new JTextPane();
    LineNumbersView lineNumbers = new LineNumbersView(editor);

    JScrollPane scroll = new JScrollPane(editor);
    scroll.setRowHeaderView(lineNumbers);
    getContentPane().add(scroll);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);

    editor.setText(getRandomText());
  }

  /**
   * Returns the first 100 words of the default dictionary (on Ubuntu / Debian)
   * and injects random line breaks.
   */
  private String getRandomText() throws IOException {
    Random rnd = new Random();
    return Files.readAllLines(Paths.get("/usr/share/dict/words")).stream().limit(100)
        .map(word -> (rnd.nextInt(15) == 0 ? "\n" : "") + word).collect(Collectors.joining(" "));
  }

  /**
   * Left hand side RowHeaderView for a JEditorPane in a JScrollPane. Highlights
   * the currently selected line. Handles line wrapping, frame resizing.
   */
  class LineNumbersView extends JComponent implements DocumentListener, CaretListener, ComponentListener {

    private static final long serialVersionUID = 1L;

    private JTextComponent editor;

    private Font font;

    public LineNumbersView(JTextComponent editor) {
      this.editor = editor;

      editor.getDocument().addDocumentListener(this);
      editor.addComponentListener(this);
      editor.addCaretListener(this);
    }

    @Override
    public void paintComponent(Graphics g) {
      super.paintComponent(g);

      Rectangle clip = g.getClipBounds();
      int startOffset = editor.viewToModel(new Point(0, clip.y));
      int endOffset = editor.viewToModel(new Point(0, clip.y + clip.height));

      while (startOffset <= endOffset) {
        try {
          String lineNumber = getLineNumber(startOffset);
          if (lineNumber != null) {
            int x = getInsets().left + 2;
            int y = getOffsetY(startOffset);

            font = font != null ? font : new Font(Font.MONOSPACED, Font.BOLD, editor.getFont().getSize());
            g.setFont(font);

            g.setColor(isCurrentLine(startOffset) ? Color.RED : Color.BLACK);

            g.drawString(lineNumber, x, y);
          }

          startOffset = Utilities.getRowEnd(editor, startOffset) + 1;
        } catch (BadLocationException e) {
          e.printStackTrace();
          // ignore and continue
        }
      }
    }

    /**
     * Returns the line number of the element based on the given (start) offset
     * in the editor model. Returns null if no line number should or could be
     * provided (e.g. for wrapped lines).
     */
    private String getLineNumber(int offset) {
      Element root = editor.getDocument().getDefaultRootElement();
      int index = root.getElementIndex(offset);
      Element line = root.getElement(index);

      return line.getStartOffset() == offset ? String.format("%3d", index + 1) : null;
    }

    /**
     * Returns the y axis position for the line number belonging to the element
     * at the given (start) offset in the model.
     */
    private int getOffsetY(int offset) throws BadLocationException {
      FontMetrics fontMetrics = editor.getFontMetrics(editor.getFont());
      int descent = fontMetrics.getDescent();

      Rectangle r = editor.modelToView(offset);
      int y = r.y + r.height - descent;

      return y;
    }

    /**
     * Returns true if the given start offset in the model is the selected (by
     * cursor position) element.
     */
    private boolean isCurrentLine(int offset) {
      int caretPosition = editor.getCaretPosition();
      Element root = editor.getDocument().getDefaultRootElement();
      return root.getElementIndex(offset) == root.getElementIndex(caretPosition);
    }

    /**
     * Schedules a refresh of the line number margin on a separate thread.
     */
    private void documentChanged() {
      SwingUtilities.invokeLater(() -> {
        repaint();
      });
    }

    /**
     * Updates the size of the line number margin based on the editor height.
     */
    private void updateSize() {
      Dimension size = new Dimension(MARGIN_WIDTH_PX, editor.getHeight());
      setPreferredSize(size);
      setSize(size);
    }

    @Override
    public void insertUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void removeUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void changedUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void caretUpdate(CaretEvent e) {
      documentChanged();
    }

    @Override
    public void componentResized(ComponentEvent e) {
      updateSize();
      documentChanged();
    }

    @Override
    public void componentMoved(ComponentEvent e) {
    }

    @Override
    public void componentShown(ComponentEvent e) {
      updateSize();
      documentChanged();
    }

    @Override
    public void componentHidden(ComponentEvent e) {
    }
  }
}

Simple Framework Websocket server

18 February 2017

Following the previous post about a HTTP server based on the Simple Framework library, here’s the Websocket use case. The example gets a bit more complicated with both a HTTP handler, Websocket handler and client, but still a lot simpler than the original example code.

Two interfaces have to be implemented to have a meaningful Websocket server: First, the Service interface, which gets invoked when a new connection and Session is established. From here it is possible to register a FrameListener on the session channel, which is the second interface to implement. The FrameListener methods will get invoked when new Frames, whether it is data or control codes is received. This listener also gets notified about errors and when the session or socket is closed. Two minimal examples of these implementations are shown below, followed by the setup() method which ties it all together and starts the server. Notice how RouterContainer takes both the HTTP and Websocket handlers, and routes the incoming requests appropriately.

  class WebsocketService implements Service {
    @Override
    public void connect(Session session) {
      System.out.println("Session: " + session);
      try {
        FrameChannel channel = session.getChannel();
        channel.register(new WebsocketFrameListener());
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  class WebsocketFrameListener implements FrameListener {

    @Override
    public void onFrame(Session session, Frame frame) {
      System.out.println("onFrame " + frame.getType() + ": " + frame);
    }

    @Override
    public void onError(Session session, Exception cause) {
      System.out.println("onError: " + cause);
    }

    @Override
    public void onClose(Session session, Reason reason) {
      System.out.println("onClose: " + reason);
    }
  }

  public void setup() throws IOException, InterruptedException {
    HttpContainer httpContainer = new HttpContainer();
    Router websocketRouter = new DirectRouter(new WebsocketService());
    RouterContainer routerContainer = new RouterContainer(httpContainer, websocketRouter, 1);

    ContainerSocketProcessor server = new ContainerSocketProcessor(routerContainer, 1);
    serverSocket = new SocketConnection(server);

    serverSocket.connect(new InetSocketAddress(PORT));
  }

To test the core functionality of the Websocket server, a bit more is needed on the client side. First, a HTTP like handshake has to be requested and the response read, then the client must respond to ping control frames, and finally, it can send frames of data itself.

Below is the method which sends the handshake to the server as specified in the Websocket RFC 6455. There’s a few things to note: Currently, details like target path, the host, origin are hard-coded. In a normal client, they should be resolved and set in a proper way. The security key is also hard-coded, but should be generated according to the specification. Finally, note that the line ending has to be “\r\n” or 0xd, 0xa in hex. Therefore, when using a PrintWriter to wrap the raw OutputStream, be careful not to use the println() methods as they might append the incorrect line termination.

What’s useful about a minimal test client like this, is that’s it becomes easy to try out these kind of details in isolation, without distractions from other functionality. Although this example does not go as far as asserting the behaviour, it could easily be extended to an in-memory integration test of the server.

  private void sendHandshake(OutputStream rawOut) {
    String handshake[] = {
      "GET /test HTTP/1.1",
      "Host: loclahost",
      "Upgrade: websocket",
      "Connection: Upgrade",
      "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==",
      "Origin: http://localhost:" + PORT,
      "Sec-WebSocket-Protocol: test",
      "Sec-WebSocket-Version: 13",
      ""
    };

    PrintWriter pwOut = new PrintWriter(rawOut, true);
    for (String str : handshake) {
      pwOut.print(str + "\r\n");
    }
    pwOut.flush();
  }

  private void receiveHandshake(InputStream rawIn) throws IOException {
    BufferedReader bufIn = new BufferedReader(new InputStreamReader(rawIn));

    System.out.println("Response: ");
    String line;
    do {
      line = bufIn.readLine();
      System.out.println("R: '" + line + "'");
    } while (line != null && !line.isEmpty());
  }

Finally, this test method ties the client together: it first opens the socket connection to the server; gets the IO streams; sends and receives the handshake; starts the ping response handler on a separate thread; sends one frame of its own with a ping, waits for some seconds to observe the server in operation, and finally sends a close operation code before closing the connection.

  public void testWebsocketRequest() throws IOException, InterruptedException {
    Socket socket = new Socket("localhost", PORT);

    OutputStream rawOut = socket.getOutputStream();
    InputStream rawIn = socket.getInputStream();

    sendHandshake(rawOut);
    receiveHandshake(rawIn);

    pingPongHandler(rawIn, rawOut);

    sendPing(rawOut);

    Thread.sleep(20000);

    clientClosed = true;
    sendClose(rawOut);

    socket.close();
  }

Here’s the full test case listing, which also includes a HTTP request test. A dependency on the Simple Framework library is needed, for example through the Gradle dependency:

repositories {
  mavenCentral()
}

dependencies {
  compile 'org.simpleframework:simple-http:6.0.1'
}
SfWebsocketServerTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.http;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.URL;
import java.nio.ByteBuffer;
import java.util.Arrays;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.simpleframework.http.Request;
import org.simpleframework.http.Response;
import org.simpleframework.http.core.Container;
import org.simpleframework.http.core.ContainerSocketProcessor;
import org.simpleframework.http.socket.Frame;
import org.simpleframework.http.socket.FrameChannel;
import org.simpleframework.http.socket.FrameListener;
import org.simpleframework.http.socket.Reason;
import org.simpleframework.http.socket.Session;
import org.simpleframework.http.socket.service.DirectRouter;
import org.simpleframework.http.socket.service.Router;
import org.simpleframework.http.socket.service.RouterContainer;
import org.simpleframework.http.socket.service.Service;
import org.simpleframework.transport.connect.SocketConnection;

/**
 * Tests a combined HTTP and Websocket server using the SimpleFramework API.
 * 
 * See http://www.simpleframework.org/
 */
public class SfWebsocketServerTest {

  private static final int PORT = 48889;

  /**
   * Handles incoming HTTP requests.
   */
  class HttpContainer implements Container {

    @Override
    public void handle(Request req, Response resp) {
      System.out.println("HTTP Request: \n" + req);

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap("Hello".getBytes()));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  /**
   * Handles incoming Websocket requests and establishes the frame channel
   * listener.
   */
  class WebsocketService implements Service {
    @Override
    public void connect(Session session) {
      System.out.println("Session: " + session);
      try {
        FrameChannel channel = session.getChannel();
        channel.register(new WebsocketFrameListener());
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  /**
   * Handles incoming Frames from an established channel.
   */
  class WebsocketFrameListener implements FrameListener {

    @Override
    public void onFrame(Session session, Frame frame) {
      System.out.println("onFrame " + frame.getType() + ": " + frame);
    }

    @Override
    public void onError(Session session, Exception cause) {
      System.out.println("onError: " + cause);
    }

    @Override
    public void onClose(Session session, Reason reason) {
      System.out.println("onClose: " + reason);
    }
  }

  private SocketConnection serverSocket;

  private boolean clientClosed;

  @Before
  public void setup() throws IOException, InterruptedException {
    HttpContainer httpContainer = new HttpContainer();
    Router websocketRouter = new DirectRouter(new WebsocketService());
    RouterContainer routerContainer = new RouterContainer(httpContainer, websocketRouter, 1);

    ContainerSocketProcessor server = new ContainerSocketProcessor(routerContainer, 1);
    serverSocket = new SocketConnection(server);

    serverSocket.connect(new InetSocketAddress(PORT));
  }

  @After
  public void close() throws IOException {
    serverSocket.close();
  }

  /**
   * Tests the HTTP part only
   */
  @Test
  public void testHttpRequest() throws Exception {
    URL url = new URL("http://localhost:" + PORT);
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    in.lines().forEach(System.out::println);
    in.close();
  }

  /**
   * Sends a HTTP Websocket handshake to the server; prints its response;
   * listens for pings and sends pongs; sends a ping; sends a close and closes
   * the client socket after 20 seconds.
   */
  @Test
  public void testWebsocketRequest() throws IOException, InterruptedException {
    Socket socket = new Socket("localhost", PORT);

    OutputStream rawOut = socket.getOutputStream();
    InputStream rawIn = socket.getInputStream();

    sendHandshake(rawOut);
    receiveHandshake(rawIn);

    pingPongHandler(rawIn, rawOut);

    sendPing(rawOut);

    Thread.sleep(20000);

    clientClosed = true;
    sendClose(rawOut);

    socket.close();
  }

  private void sendClose(OutputStream out) throws IOException {
    sendOpcode(out, 0x88);
  }

  private void sendPing(OutputStream out) throws IOException {
    sendOpcode(out, 0x89);
    System.out.println("Sent ping");
  }

  private void sendPong(OutputStream out) throws IOException {
    sendOpcode(out, 0x8a);
    System.out.println("Sent pong");
  }

  private void sendOpcode(OutputStream out, int code) throws IOException {
    out.write(new byte[] { (byte) code, (byte) 0 });
    out.flush();
  }

  /**
   * Client side. Starts a separate thread which reads the InputStream stream
   * and checks for ping frames. If one is detected, sends a pong on the
   * OutputStream.
   */
  private void pingPongHandler(InputStream in, OutputStream out) {
    final int pingOpCode = 0x89;

    new Thread(() -> {
      while (!clientClosed) {
        try {
          byte[] buf = readInput(in);

          if (buf.length >= 2 && (buf[0] & 0xff) == pingOpCode) {
            sendPong(out);
          } else if (buf.length > 0) {
            System.out.println("From server:");
            for (int i = 0; i < buf.length; i++) {
              System.out.print(Integer.toHexString(buf[i]) + " ");
            }
            System.out.println();
          }

          Thread.sleep(50);
        } catch (IOException | InterruptedException e) {
          e.printStackTrace();
        }
      }
    }).start();
  }

  /**
   * Client side. Reads available bytes from the InputStream.
   */
  private byte[] readInput(InputStream in) throws IOException {
    byte[] buf = new byte[2000];
    int i = 0;
    while (in.available() > 0) {
      buf[i++] = (byte) in.read();
    }

    return Arrays.copyOf(buf, i);
  }

  /**
   * Client side. Sends the Websocket handshake request to the server.
   */
  private void sendHandshake(OutputStream rawOut) {
    String handshake[] = {
      "GET /test HTTP/1.1",
      "Host: loclahost",
      "Upgrade: websocket",
      "Connection: Upgrade",
      "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==",
      "Origin: http://localhost:" + PORT,
      "Sec-WebSocket-Protocol: test",
      "Sec-WebSocket-Version: 13",
      ""
    };

    PrintWriter pwOut = new PrintWriter(rawOut, true);
    for (String str : handshake) {
      pwOut.print(str + "\r\n");
    }
    pwOut.flush();
  }

  /**
   * Client side. Reads the Websocket handshake response from the server.
   */
  private void receiveHandshake(InputStream rawIn) throws IOException {
    BufferedReader bufIn = new BufferedReader(new InputStreamReader(rawIn));

    System.out.println("Response: ");
    String line;
    do {
      line = bufIn.readLine();
      System.out.println("R: '" + line + "'");
    } while (line != null && !line.isEmpty());
  }
}

Simple Framework HTTP server

12 February 2017

A while back, I ranted about the Simple Framework example code and its unnecessary complexity. It turns out that their HTTP server library is indeed simple, if you just ignore the Spring setup and their example. In fact, as this static file server example shows, it’s very similar to the “hidden” Sun HTTP handler API.

The only interface which has to be implemented is the Container, which is equivalent to the request HttpHandler in the Sun HTTP library. The handle() method takes a normal HTTP request, and the response code and data can be written via the Response object. The first code block below shows a simple handler.

The next block shows how to tie the required Simple Framework classes together to start the server. The Container just mentioned goes into a ContainerSocketProcessor where concurrent processing can be setup up. The processor goes into a SocketConnection which connects to a socket on a given host and port, and the server runs.

  static class StaticFileContainer implements Container {

    private String baseDir;

    StaticFileContainer(String baseDir) {
      this.baseDir = baseDir;
    }

    @Override
    public void handle(Request req, Response resp) {
      System.out.println(req);
      System.out.println("target: " + req.getTarget());

      Path path = Paths.get(baseDir, req.getTarget());

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap(Files.readAllBytes(path)));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }
  public void setup() throws IOException {
    StaticFileContainer container = new StaticFileContainer("com/rememberjava/http");
    ContainerSocketProcessor server = new ContainerSocketProcessor(container, 1);
    connection = new SocketConnection(server);

    connection.connect(new InetSocketAddress(PORT));
  }

Here is the full server example, started and verified as a unit test.

SfHttpServerTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.http;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.simpleframework.http.Request;
import org.simpleframework.http.Response;
import org.simpleframework.http.core.Container;
import org.simpleframework.http.core.ContainerSocketProcessor;
import org.simpleframework.transport.connect.SocketConnection;

public class SfHttpServerTest {

  private static final int PORT = 8889;

  static class StaticFileContainer implements Container {

    private String baseDir;

    StaticFileContainer(String baseDir) {
      this.baseDir = baseDir;
    }

    @Override
    public void handle(Request req, Response resp) {
      System.out.println(req);
      System.out.println("target: " + req.getTarget());

      Path path = Paths.get(baseDir, req.getTarget());

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap(Files.readAllBytes(path)));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  private SocketConnection connection;

  @Before
  public void setup() throws IOException {
    StaticFileContainer container = new StaticFileContainer("com/rememberjava/http");
    ContainerSocketProcessor server = new ContainerSocketProcessor(container, 1);
    connection = new SocketConnection(server);

    connection.connect(new InetSocketAddress(PORT));
  }

  @After
  public void close() throws IOException {
    connection.close();
  }

  // TODO: Read test.txt from Bazel
  //@Test
  public void disabled_testRequest() throws Exception {
    URL url = new URL("http://localhost:8889/test.txt");
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    in.lines().forEach(System.out::println);
    in.close();
  }

  @Test
  public void testDummy() {
  }
}
Newer posts Older posts