Socket client / server example

21 February 2017

After posts on several server libraries, including Sun’s HTTP; Simple Framework’s HTTP; and Websocket, it seems appropriate to include a plain TCP socket server / client example. Without any application protocol to adhere to, it’s straight forward to setup and test. There are two essential classes involved: On the server side there’s a ServerSocket, while the client has a Socket. The only difference is that the ServerSocket accepts one or more incoming connections, and provide a Socket handle for each. After that, both client and server are the same, in that the Socket object provides input and output streams once established.

In the code below, a ServerSocket is set to listen to a specific port, and to accept a single incoming connection. It grabs the IO streams, makes these available to the rest of the test, and finally releases a semaphore lock to allow the test to continue. Normally, the server thread would wait for further incoming requests, and would probably spawn or assign pooled threads to handle the request. Here we focus only on the basic IO parts.

...
    ServerSocket server = new ServerSocket(PORT);
    listen(server);
...

  private void listen(ServerSocket server) {
    new Thread(() -> {
      try {
        Socket socket = server.accept();
        System.out.println("Incoming connection: " + socket);

        serverOut = socket.getOutputStream();
        serverIn = socket.getInputStream();

        lock.release();
        System.out.println("Released lock");
      } catch (IOException e) {
        e.printStackTrace();
      }
    }).start();
  }

On the client side, it’s just as simple: Establish a connection to the severer host and port, and get the IO streams.

    Socket client = new Socket("localhost", PORT);
    OutputStream clientOut = client.getOutputStream();
    InputStream clientIn = client.getInputStream();

The rest of the test code below asserts that messages are received correctly both on the client and the server, using the raw streams and wrapped PrinterWriter helpers. Again, he semaphore is used to wait for the server thread to establish its connection before the rest of the test continues. Without it, using the IO streams will results in NullPointerExceptions, since they are not initialized yet.

SocketTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.net;

import static org.junit.Assert.assertEquals;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.Semaphore;

import org.junit.Test;

/**
 * Simple client / server Socket tests, including a Buffered PrintWriter which
 * has to be flushed.
 */
public class SocketTest {

  private static final int PORT = 8887;

  private OutputStream serverOut;
  private InputStream serverIn;

  /**
   * Shared lock between the "client" and "server" code, to make the test case
   * synchronous.
   */
  private Semaphore lock = new Semaphore(0);

  /**
   * Tests server and client side sockets in one flow. A lock object is used for
   * synchronous between the two sides.
   */
  @Test
  public void testClientServer() throws IOException, InterruptedException {
    ServerSocket server = new ServerSocket(PORT);
    listen(server);

    Socket client = new Socket("localhost", PORT);
    OutputStream clientOut = client.getOutputStream();
    InputStream clientIn = client.getInputStream();

    System.out.println("Waiting for lock");
    lock.acquire();
    System.out.println("Acquired lock");

    write(clientOut, "Hi");
    assertRead(serverIn, "Hi");

    write(serverOut, "Hello");
    assertRead(clientIn, "Hello");

    printWrite(clientOut, "Test printWrite");
    assertRead(serverIn, "Test printWrite");

    printWrite(serverOut, "Test printWrite again");
    assertRead(clientIn, "Test printWrite again");

    client.close();
    server.close();
  }

  /**
   * Writes to an OutputStream. Used for both server and client output streams.
   */
  private void write(OutputStream out, String str) throws IOException {
    out.write(str.getBytes());
    out.flush();
  }

  /**
   * Writes to an OutputStream. Used for both server and client output streams.
   */
  private void printWrite(OutputStream out, String str) throws IOException {
    PrintWriter pw = new PrintWriter(out);
    pw.print(str);
    pw.flush();
  }

  /**
   * Reads from an InputStream. Used for both server and client input streams.
   */
  private void assertRead(InputStream in, String expected) throws IOException {
    assertEquals("Too few bytes available for reading: ", expected.length(), in.available());

    byte[] buf = new byte[expected.length()];
    in.read(buf);
    assertEquals(expected, new String(buf));
  }

  /**
   * Listens for and accepts one incoming request server side on a separate
   * thread. When a request is received, grabs its IO streams and "signals" to
   * the client side above through the shared lock object.
   */
  private void listen(ServerSocket server) {
    new Thread(() -> {
      try {
        Socket socket = server.accept();
        System.out.println("Incoming connection: " + socket);

        serverOut = socket.getOutputStream();
        serverIn = socket.getInputStream();

        lock.release();
        System.out.println("Released lock");
      } catch (IOException e) {
        e.printStackTrace();
      }
    }).start();
  }
}

Styles with JTextPane

20 February 2017

The Swing tutorial for the JTextPane and demo code is good and comprehensive, covering multiple ways to interact with the the StyledDocument and add Styles. So, without repeating all of that, here’s a minimal example including only simple style changes on the insert update and selection events.

In the first snippet below, the implementation for the insertUpdate event is shown. For each typed character or pasted text, the entire text of the document will be searched for the word “foo”. For each occurrence, the bold attribute is set for that word. Notice that the update of the attribute happens on a separate AWT thread.

  @Override
  public void insertUpdate(DocumentEvent event) {
    try {
      String text = doc.getText(0, doc.getLength());

      Pattern p = Pattern.compile(FOO);
      Matcher matcher = p.matcher(text);
      while (matcher.find()) {
        updateAttribute(matcher.start(), FOO.length(), Style.BOLD);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  private void updateAttribute(int pos, int len, Style style) {
    SwingUtilities.invokeLater(() -> {
      doc.setCharacterAttributes(pos, len, style.get(), true);
    });
  }

The other functionality of this small application is on select. If the selected word is “bar”, it is set to italic. Notice the dot and mark positions, which might be at the start or the end of the selection. Furthermore, notice that here the predefined ItalicAction from the StyledEditorKit is used, since we’re dealing with a selection. We just have to translate the CaretEvent into an ActionEvent, or rather just make sure to forward the source component of the selection. (The alternative would have been to go with the plain update as in the example above).

  @Override
  public void caretUpdate(CaretEvent event) {
    int dot = event.getDot();
    int mark = event.getMark();
    int start;
    int end;

    if (dot == mark) {
      return;
    } else if (dot < mark) {
      start = dot;
      end = mark;
    } else {
      start = mark;
      end = dot;
    }

    System.out.println(start + ", " + end);
    try {
      if (doc.getText(start, BAR.length()).startsWith(BAR)) {
        ItalicAction action = new StyledEditorKit.ItalicAction();
        action.actionPerformed(new ActionEvent(event.getSource(), 0, ""));

        // Alternative custom update:
        // updateAttribute(start, BAR.length(), Style.ITALIC);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

Here’s the full file listing, as a stand-alone application.

JTextPaneStylesExample.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.event.ActionEvent;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.swing.JFrame;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.text.AttributeSet;
import javax.swing.text.BadLocationException;
import javax.swing.text.MutableAttributeSet;
import javax.swing.text.SimpleAttributeSet;
import javax.swing.text.StyleConstants;
import javax.swing.text.StyledDocument;
import javax.swing.text.StyledEditorKit;
import javax.swing.text.StyledEditorKit.ItalicAction;

/**
 * Minimal example with JTextPane and StyledDocument. When the string "foo" is
 * typed or pasted it is made bold. When the string "bar" is selected it is made
 * italic.
 */
@SuppressWarnings("serial")
public class JTextPaneStylesExample extends JFrame implements DocumentListener, CaretListener {

  /**
   * Predefined styles.
   */
  enum Style {
    BOLD(StyleConstants.Bold),
    ITALIC(StyleConstants.Italic);

    private MutableAttributeSet attrib;

    private Style(Object style) {
      attrib = new SimpleAttributeSet();
      attrib.addAttribute(style, true);
    }

    AttributeSet get() {
      return attrib;
    }
  }

  private static final String FOO = "foo";

  private static final String BAR = "bar";

  private final StyledDocument doc;

  public static void main(String[] args) {
    new JTextPaneStylesExample();
  }

  public JTextPaneStylesExample() {
    JTextPane editor = new JTextPane();
    doc = editor.getStyledDocument();
    getContentPane().add(editor);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);

    editor.getDocument().addDocumentListener(this);
    editor.addCaretListener(this);
  }

  /**
   * Check of a selection of the string "bar".
   */
  @Override
  public void caretUpdate(CaretEvent event) {
    int dot = event.getDot();
    int mark = event.getMark();
    int start;
    int end;

    if (dot == mark) {
      return;
    } else if (dot < mark) {
      start = dot;
      end = mark;
    } else {
      start = mark;
      end = dot;
    }

    System.out.println(start + ", " + end);
    try {
      if (doc.getText(start, BAR.length()).startsWith(BAR)) {
        ItalicAction action = new StyledEditorKit.ItalicAction();
        action.actionPerformed(new ActionEvent(event.getSource(), 0, ""));

        // Alternative custom update:
        // updateAttribute(start, BAR.length(), Style.ITALIC);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  /**
   * Check for the occurrence of the string "foo".
   */
  @Override
  public void insertUpdate(DocumentEvent event) {
    try {
      String text = doc.getText(0, doc.getLength());

      Pattern p = Pattern.compile(FOO);
      Matcher matcher = p.matcher(text);
      while (matcher.find()) {
        updateAttribute(matcher.start(), FOO.length(), Style.BOLD);
      }
    } catch (BadLocationException e) {
      e.printStackTrace();
    }
  }

  /**
   * Update the string at the given position and length with the given style.
   * The update happens on a separate AWT thread, to avoid mutations of the
   * Document model while the event is processing.
   */
  private void updateAttribute(int pos, int len, Style style) {
    SwingUtilities.invokeLater(() -> {
      doc.setCharacterAttributes(pos, len, style.get(), true);
    });
  }

  @Override
  public void removeUpdate(DocumentEvent e) {}

  @Override
  public void changedUpdate(DocumentEvent e) {}
}

Line numbers for JEditorPane and JTextPane

19 February 2017

There are many ways to add line numbers to the JEditorPane and JTextPane Swing components. This post looks at two solutions: One providing a custom ParagraphView which paints a child element that contains the line number for that paragraph. The other implements a separate component which is passed as the RowHeaderView of the JScrollPane. Both support scrolling and line wrapping, and in this example the latter includes current line highlighting, as seen in the image below.

Line numbers with RowHeaderView

ParagraphView

The ParagraphView solution is inspired by Stanislav Lapitsky’s old post on the topic, although with some improvements. The basic idea is to paint the line number in the child of the ParagraphView, using the paintChild() method. The advantage is that the relationship between the line and the line number is already established, so alignment is easy. We just have to count which element we’re dealing with to know which number to print. The two methods below show this part. Notice also that only the index 0 of the ParagraphView should be used for the line number, so we don’t count a wrapped line twice. Furthermore, a mono-spaced font helps with padding and alignment.

In order to create this custom ParagraphView, it has to be returned through a ViewFactory, which again has to be provided through an EditorKit. The file below shows the full implementation. Notice how the default ViewFactory is used for all other elements; only the Paragraph Element gets a custom view.

    public void paintChild(Graphics g, Rectangle alloc, int index) {
      super.paintChild(g, alloc, index);

      // Allow of wrapped paragraph lines, but don't print redundant line
      // numbers.
      if (index > 0) {
        return;
      }

      // Pad left so the numbers align
      int lineNumber = getLineNumber() + 1;
      String lnStr = String.format("%3d", lineNumber);

      // Make sure we use a monospaced font.
      font = font != null ? font : new Font(Font.MONOSPACED, Font.PLAIN, getFont().getSize());
      g.setFont(font);

      int x = alloc.x - getLeftInset();
      int y = alloc.y + alloc.height - 3;
      g.drawString(lnStr, x, y);
    }

    private int getLineNumber() {
      // According to the Document.getRootElements() doc, there will "typically"
      // only be one root element.
      Element root = getDocument().getDefaultRootElement();
      int len = root.getElementCount();
      for (int i = 0; i < len; i++) {
        if (root.getElement(i) == thisElement) {
          return i;
        }
      }
      return 0;
    }

setRowHeaderView

The second solution renders the line numbers in a completely separate component, which can be a JComponent (or even an old style AWT Component). This component is passed in to the setRowHeaderView() method of JScrollPane. That way, the coupling becomes a bit cleaner than overriding multiple classes and methods as with the the ParagraphView solution. However, the down-side is that we must do the alignment with the Editor component manually. Luckily, the Swing Text Component API provides a solid API for this purpose.

The code below is inspired by Rob Camick’s article, but simplifies several aspects of his stand-alone API style class. At the centre is the translation between the Document model and its onscreen view. The viewToModel() method translates an x,y point on the screen to the nearest character offset, while the modelToView() method goes the other way. That way we can align the line number at the same y point as the line in the editor. In Camick’s code, he loops through the text by character offsets rather than paragraphs; the getRowEnd() method helps by finding the end of a line. Camick also points out the the line number should take the font decent of the editor font into account when positioning the line number. The FontMetrics class helps with measuring sizes of fonts.

Finally worth noting is the synchronisation between the editor and the margin component. Since they are separate, this must be handled through event listeners registered with the editor component. The DocumentListener and ComponentListener notifies of edit, movement and resizing events, while the CaretListener signals movement in the cursor position. All of these events force a repaint, but through a scheduled AWT thread, to make sure the editor view has updated first, as seen in the documentChanged() method at the bottom of the code section below.

    public void paintComponent(Graphics g) {
      super.paintComponent(g);

      Rectangle clip = g.getClipBounds();
      int startOffset = editor.viewToModel(new Point(0, clip.y));
      int endOffset = editor.viewToModel(new Point(0, clip.y + clip.height));

      while (startOffset <= endOffset) {
        try {
          String lineNumber = getLineNumber(startOffset);
          if (lineNumber != null) {
            int x = getInsets().left + 2;
            int y = getOffsetY(startOffset);

            font = font != null ? font : new Font(Font.MONOSPACED, Font.BOLD, editor.getFont().getSize());
            g.setFont(font);

            g.setColor(isCurrentLine(startOffset) ? Color.RED : Color.BLACK);

            g.drawString(lineNumber, x, y);
          }

          startOffset = Utilities.getRowEnd(editor, startOffset) + 1;
        } catch (BadLocationException e) {
          e.printStackTrace();
          // ignore and continue
        }
      }
    }

    private String getLineNumber(int offset) {
      Element root = editor.getDocument().getDefaultRootElement();
      int index = root.getElementIndex(offset);
      Element line = root.getElement(index);

      return line.getStartOffset() == offset ? String.format("%3d", index + 1) : null;
    }

    private int getOffsetY(int offset) throws BadLocationException {
      FontMetrics fontMetrics = editor.getFontMetrics(editor.getFont());
      int descent = fontMetrics.getDescent();

      Rectangle r = editor.modelToView(offset);
      int y = r.y + r.height - descent;

      return y;
    }

    private boolean isCurrentLine(int offset) {
      int caretPosition = editor.getCaretPosition();
      Element root = editor.getDocument().getDefaultRootElement();
      return root.getElementIndex(offset) == root.getElementIndex(caretPosition);
    }

    private void documentChanged() {
      SwingUtilities.invokeLater(() -> {
        repaint();
      });
    }

Here are the full code listings for both examples, as stand-alone applications.

ParagraphViewLineNumbers.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.Font;
import java.awt.Graphics;
import java.awt.Rectangle;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Random;
import java.util.stream.Collectors;

import javax.swing.JEditorPane;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.text.AbstractDocument;
import javax.swing.text.Element;
import javax.swing.text.ParagraphView;
import javax.swing.text.StyledEditorKit;
import javax.swing.text.View;
import javax.swing.text.ViewFactory;

/**
 * Example use of a left margin line number for a JEditorPane. Using a custom
 * EditorKit and ViewFactory to pass in ParagraphView which paints a child
 * element. Pads the line numbers up to 999, and handles wrapped lines in the
 * editor.
 */
public class ParagraphViewLineNumbers extends JFrame {

  private static final long serialVersionUID = 1L;

  public static void main(String[] args) throws IOException {
    new ParagraphViewLineNumbers();
  }

  public ParagraphViewLineNumbers() throws IOException {
    JEditorPane editor = new JEditorPane();
    editor.setEditorKit(new CustomEditorKit());
    editor.setText(getRandomText());

    JScrollPane scroll = new JScrollPane(editor);
    getContentPane().add(scroll);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);
  }

  /**
   * Returns the first 100 words of the default dictionary (on Ubuntu / Debian)
   * and injects random line breaks.
   */
  private String getRandomText() throws IOException {
    Random rnd = new Random();
    return Files.readAllLines(Paths.get("/usr/share/dict/words")).stream().limit(100)
        .map(word -> (rnd.nextInt(15) == 0 ? "\n" : "") + word).collect(Collectors.joining(" "));
  }

  /**
   * Passes in the custom ViewFactory. Inherits from the StyledEditorKit since
   * that already comes with a default ViewFactory (while the DefaultEditorKit
   * does not).
   */
  class CustomEditorKit extends StyledEditorKit {

    private static final long serialVersionUID = 1L;

    @Override
    public ViewFactory getViewFactory() {
      return new CustomViewFactory(super.getViewFactory());
    }
  }

  /**
   * Produces custom ParagraphViews, but uses the default ViewFactory for all
   * other elements.
   */
  class CustomViewFactory implements ViewFactory {

    private ViewFactory defaultViewFactory;

    CustomViewFactory(ViewFactory defaultViewFactory) {
      this.defaultViewFactory = defaultViewFactory;
    }

    @Override
    public View create(Element elem) {
      if (elem != null && elem.getName().equals(AbstractDocument.ParagraphElementName)) {
        return new CustomParagraphView(elem);
      }
      return defaultViewFactory.create(elem);
    }
  }

  /**
   * Paints a left hand child view with the line number for this Paragraph.
   */
  class CustomParagraphView extends ParagraphView {

    public final short MARGIN_WIDTH_PX = 25;

    private Element thisElement;

    private Font font;

    public CustomParagraphView(Element elem) {
      super(elem);
      thisElement = elem;
      this.setInsets((short) 0, (short) 0, (short) 0, (short) 0);
    }

    @Override
    protected void setInsets(short top, short left, short bottom, short right) {
      super.setInsets(top, (short) (left + MARGIN_WIDTH_PX), bottom, right);
    }

    @Override
    public void paintChild(Graphics g, Rectangle alloc, int index) {
      super.paintChild(g, alloc, index);

      // Allow of wrapped paragraph lines, but don't print redundant line
      // numbers.
      if (index > 0) {
        return;
      }

      // Pad left so the numbers align
      int lineNumber = getLineNumber() + 1;
      String lnStr = String.format("%3d", lineNumber);

      // Make sure we use a monospaced font.
      font = font != null ? font : new Font(Font.MONOSPACED, Font.PLAIN, getFont().getSize());
      g.setFont(font);

      int x = alloc.x - getLeftInset();
      int y = alloc.y + alloc.height - 3;
      g.drawString(lnStr, x, y);
    }

    private int getLineNumber() {
      // According to the Document.getRootElements() doc, there will "typically"
      // only be one root element.
      Element root = getDocument().getDefaultRootElement();
      int len = root.getElementCount();
      for (int i = 0; i < len; i++) {
        if (root.getElement(i) == thisElement) {
          return i;
        }
      }
      return 0;
    }
  }
}
RowHeaderViewLineNumbers.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.ui;

import java.awt.Color;
import java.awt.Dimension;
import java.awt.Font;
import java.awt.FontMetrics;
import java.awt.Graphics;
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.event.ComponentEvent;
import java.awt.event.ComponentListener;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Random;
import java.util.stream.Collectors;

import javax.swing.JComponent;
import javax.swing.JEditorPane;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.text.BadLocationException;
import javax.swing.text.Element;
import javax.swing.text.JTextComponent;
import javax.swing.text.Utilities;

/**
 * Example use of a left margin line number for a JEditorPane in a JScrollPane.
 * Uses a separate component for the RowHeaderView of the JScrollPane. Pads the
 * line numbers up to 999, highlights the currently active line, handles wrapped
 * lines and resizing of the editor.
 */
public class RowHeaderViewLineNumbers extends JFrame {

  private static final long serialVersionUID = 1L;

  public final int MARGIN_WIDTH_PX = 28;

  public static void main(String[] args) throws IOException {
    new RowHeaderViewLineNumbers();
  }

  public RowHeaderViewLineNumbers() throws IOException {
    JEditorPane editor = new JTextPane();
    LineNumbersView lineNumbers = new LineNumbersView(editor);

    JScrollPane scroll = new JScrollPane(editor);
    scroll.setRowHeaderView(lineNumbers);
    getContentPane().add(scroll);

    setSize(500, 500);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setVisible(true);

    editor.setText(getRandomText());
  }

  /**
   * Returns the first 100 words of the default dictionary (on Ubuntu / Debian)
   * and injects random line breaks.
   */
  private String getRandomText() throws IOException {
    Random rnd = new Random();
    return Files.readAllLines(Paths.get("/usr/share/dict/words")).stream().limit(100)
        .map(word -> (rnd.nextInt(15) == 0 ? "\n" : "") + word).collect(Collectors.joining(" "));
  }

  /**
   * Left hand side RowHeaderView for a JEditorPane in a JScrollPane. Highlights
   * the currently selected line. Handles line wrapping, frame resizing.
   */
  class LineNumbersView extends JComponent implements DocumentListener, CaretListener, ComponentListener {

    private static final long serialVersionUID = 1L;

    private JTextComponent editor;

    private Font font;

    public LineNumbersView(JTextComponent editor) {
      this.editor = editor;

      editor.getDocument().addDocumentListener(this);
      editor.addComponentListener(this);
      editor.addCaretListener(this);
    }

    @Override
    public void paintComponent(Graphics g) {
      super.paintComponent(g);

      Rectangle clip = g.getClipBounds();
      int startOffset = editor.viewToModel(new Point(0, clip.y));
      int endOffset = editor.viewToModel(new Point(0, clip.y + clip.height));

      while (startOffset <= endOffset) {
        try {
          String lineNumber = getLineNumber(startOffset);
          if (lineNumber != null) {
            int x = getInsets().left + 2;
            int y = getOffsetY(startOffset);

            font = font != null ? font : new Font(Font.MONOSPACED, Font.BOLD, editor.getFont().getSize());
            g.setFont(font);

            g.setColor(isCurrentLine(startOffset) ? Color.RED : Color.BLACK);

            g.drawString(lineNumber, x, y);
          }

          startOffset = Utilities.getRowEnd(editor, startOffset) + 1;
        } catch (BadLocationException e) {
          e.printStackTrace();
          // ignore and continue
        }
      }
    }

    /**
     * Returns the line number of the element based on the given (start) offset
     * in the editor model. Returns null if no line number should or could be
     * provided (e.g. for wrapped lines).
     */
    private String getLineNumber(int offset) {
      Element root = editor.getDocument().getDefaultRootElement();
      int index = root.getElementIndex(offset);
      Element line = root.getElement(index);

      return line.getStartOffset() == offset ? String.format("%3d", index + 1) : null;
    }

    /**
     * Returns the y axis position for the line number belonging to the element
     * at the given (start) offset in the model.
     */
    private int getOffsetY(int offset) throws BadLocationException {
      FontMetrics fontMetrics = editor.getFontMetrics(editor.getFont());
      int descent = fontMetrics.getDescent();

      Rectangle r = editor.modelToView(offset);
      int y = r.y + r.height - descent;

      return y;
    }

    /**
     * Returns true if the given start offset in the model is the selected (by
     * cursor position) element.
     */
    private boolean isCurrentLine(int offset) {
      int caretPosition = editor.getCaretPosition();
      Element root = editor.getDocument().getDefaultRootElement();
      return root.getElementIndex(offset) == root.getElementIndex(caretPosition);
    }

    /**
     * Schedules a refresh of the line number margin on a separate thread.
     */
    private void documentChanged() {
      SwingUtilities.invokeLater(() -> {
        repaint();
      });
    }

    /**
     * Updates the size of the line number margin based on the editor height.
     */
    private void updateSize() {
      Dimension size = new Dimension(MARGIN_WIDTH_PX, editor.getHeight());
      setPreferredSize(size);
      setSize(size);
    }

    @Override
    public void insertUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void removeUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void changedUpdate(DocumentEvent e) {
      documentChanged();
    }

    @Override
    public void caretUpdate(CaretEvent e) {
      documentChanged();
    }

    @Override
    public void componentResized(ComponentEvent e) {
      updateSize();
      documentChanged();
    }

    @Override
    public void componentMoved(ComponentEvent e) {
    }

    @Override
    public void componentShown(ComponentEvent e) {
      updateSize();
      documentChanged();
    }

    @Override
    public void componentHidden(ComponentEvent e) {
    }
  }
}

Simple Framework Websocket server

18 February 2017

Following the previous post about a HTTP server based on the Simple Framework library, here’s the Websocket use case. The example gets a bit more complicated with both a HTTP handler, Websocket handler and client, but still a lot simpler than the original example code.

Two interfaces have to be implemented to have a meaningful Websocket server: First, the Service interface, which gets invoked when a new connection and Session is established. From here it is possible to register a FrameListener on the session channel, which is the second interface to implement. The FrameListener methods will get invoked when new Frames, whether it is data or control codes is received. This listener also gets notified about errors and when the session or socket is closed. Two minimal examples of these implementations are shown below, followed by the setup() method which ties it all together and starts the server. Notice how RouterContainer takes both the HTTP and Websocket handlers, and routes the incoming requests appropriately.

  class WebsocketService implements Service {
    @Override
    public void connect(Session session) {
      System.out.println("Session: " + session);
      try {
        FrameChannel channel = session.getChannel();
        channel.register(new WebsocketFrameListener());
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  class WebsocketFrameListener implements FrameListener {

    @Override
    public void onFrame(Session session, Frame frame) {
      System.out.println("onFrame " + frame.getType() + ": " + frame);
    }

    @Override
    public void onError(Session session, Exception cause) {
      System.out.println("onError: " + cause);
    }

    @Override
    public void onClose(Session session, Reason reason) {
      System.out.println("onClose: " + reason);
    }
  }

  public void setup() throws IOException, InterruptedException {
    HttpContainer httpContainer = new HttpContainer();
    Router websocketRouter = new DirectRouter(new WebsocketService());
    RouterContainer routerContainer = new RouterContainer(httpContainer, websocketRouter, 1);

    ContainerSocketProcessor server = new ContainerSocketProcessor(routerContainer, 1);
    serverSocket = new SocketConnection(server);

    serverSocket.connect(new InetSocketAddress(PORT));
  }

To test the core functionality of the Websocket server, a bit more is needed on the client side. First, a HTTP like handshake has to be requested and the response read, then the client must respond to ping control frames, and finally, it can send frames of data itself.

Below is the method which sends the handshake to the server as specified in the Websocket RFC 6455. There’s a few things to note: Currently, details like target path, the host, origin are hard-coded. In a normal client, they should be resolved and set in a proper way. The security key is also hard-coded, but should be generated according to the specification. Finally, note that the line ending has to be “\r\n” or 0xd, 0xa in hex. Therefore, when using a PrintWriter to wrap the raw OutputStream, be careful not to use the println() methods as they might append the incorrect line termination.

What’s useful about a minimal test client like this, is that’s it becomes easy to try out these kind of details in isolation, without distractions from other functionality. Although this example does not go as far as asserting the behaviour, it could easily be extended to an in-memory integration test of the server.

  private void sendHandshake(OutputStream rawOut) {
    String handshake[] = {
      "GET /test HTTP/1.1",
      "Host: loclahost",
      "Upgrade: websocket",
      "Connection: Upgrade",
      "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==",
      "Origin: http://localhost:" + PORT,
      "Sec-WebSocket-Protocol: test",
      "Sec-WebSocket-Version: 13",
      ""
    };

    PrintWriter pwOut = new PrintWriter(rawOut, true);
    for (String str : handshake) {
      pwOut.print(str + "\r\n");
    }
    pwOut.flush();
  }

  private void receiveHandshake(InputStream rawIn) throws IOException {
    BufferedReader bufIn = new BufferedReader(new InputStreamReader(rawIn));

    System.out.println("Response: ");
    String line;
    do {
      line = bufIn.readLine();
      System.out.println("R: '" + line + "'");
    } while (line != null && !line.isEmpty());
  }

Finally, this test method ties the client together: it first opens the socket connection to the server; gets the IO streams; sends and receives the handshake; starts the ping response handler on a separate thread; sends one frame of its own with a ping, waits for some seconds to observe the server in operation, and finally sends a close operation code before closing the connection.

  public void testWebsocketRequest() throws IOException, InterruptedException {
    Socket socket = new Socket("localhost", PORT);

    OutputStream rawOut = socket.getOutputStream();
    InputStream rawIn = socket.getInputStream();

    sendHandshake(rawOut);
    receiveHandshake(rawIn);

    pingPongHandler(rawIn, rawOut);

    sendPing(rawOut);

    Thread.sleep(20000);

    clientClosed = true;
    sendClose(rawOut);

    socket.close();
  }

Here’s the full test case listing, which also includes a HTTP request test. A dependency on the Simple Framework library is needed, for example through the Gradle dependency:

repositories {
  mavenCentral()
}

dependencies {
  compile 'org.simpleframework:simple-http:6.0.1'
}
SfWebsocketServerTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.http;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.URL;
import java.nio.ByteBuffer;
import java.util.Arrays;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.simpleframework.http.Request;
import org.simpleframework.http.Response;
import org.simpleframework.http.core.Container;
import org.simpleframework.http.core.ContainerSocketProcessor;
import org.simpleframework.http.socket.Frame;
import org.simpleframework.http.socket.FrameChannel;
import org.simpleframework.http.socket.FrameListener;
import org.simpleframework.http.socket.Reason;
import org.simpleframework.http.socket.Session;
import org.simpleframework.http.socket.service.DirectRouter;
import org.simpleframework.http.socket.service.Router;
import org.simpleframework.http.socket.service.RouterContainer;
import org.simpleframework.http.socket.service.Service;
import org.simpleframework.transport.connect.SocketConnection;

/**
 * Tests a combined HTTP and Websocket server using the SimpleFramework API.
 * 
 * See http://www.simpleframework.org/
 */
public class SfWebsocketServerTest {

  private static final int PORT = 48889;

  /**
   * Handles incoming HTTP requests.
   */
  class HttpContainer implements Container {

    @Override
    public void handle(Request req, Response resp) {
      System.out.println("HTTP Request: \n" + req);

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap("Hello".getBytes()));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  /**
   * Handles incoming Websocket requests and establishes the frame channel
   * listener.
   */
  class WebsocketService implements Service {
    @Override
    public void connect(Session session) {
      System.out.println("Session: " + session);
      try {
        FrameChannel channel = session.getChannel();
        channel.register(new WebsocketFrameListener());
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  /**
   * Handles incoming Frames from an established channel.
   */
  class WebsocketFrameListener implements FrameListener {

    @Override
    public void onFrame(Session session, Frame frame) {
      System.out.println("onFrame " + frame.getType() + ": " + frame);
    }

    @Override
    public void onError(Session session, Exception cause) {
      System.out.println("onError: " + cause);
    }

    @Override
    public void onClose(Session session, Reason reason) {
      System.out.println("onClose: " + reason);
    }
  }

  private SocketConnection serverSocket;

  private boolean clientClosed;

  @Before
  public void setup() throws IOException, InterruptedException {
    HttpContainer httpContainer = new HttpContainer();
    Router websocketRouter = new DirectRouter(new WebsocketService());
    RouterContainer routerContainer = new RouterContainer(httpContainer, websocketRouter, 1);

    ContainerSocketProcessor server = new ContainerSocketProcessor(routerContainer, 1);
    serverSocket = new SocketConnection(server);

    serverSocket.connect(new InetSocketAddress(PORT));
  }

  @After
  public void close() throws IOException {
    serverSocket.close();
  }

  /**
   * Tests the HTTP part only
   */
  @Test
  public void testHttpRequest() throws Exception {
    URL url = new URL("http://localhost:" + PORT);
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    in.lines().forEach(System.out::println);
    in.close();
  }

  /**
   * Sends a HTTP Websocket handshake to the server; prints its response;
   * listens for pings and sends pongs; sends a ping; sends a close and closes
   * the client socket after 20 seconds.
   */
  @Test
  public void testWebsocketRequest() throws IOException, InterruptedException {
    Socket socket = new Socket("localhost", PORT);

    OutputStream rawOut = socket.getOutputStream();
    InputStream rawIn = socket.getInputStream();

    sendHandshake(rawOut);
    receiveHandshake(rawIn);

    pingPongHandler(rawIn, rawOut);

    sendPing(rawOut);

    Thread.sleep(20000);

    clientClosed = true;
    sendClose(rawOut);

    socket.close();
  }

  private void sendClose(OutputStream out) throws IOException {
    sendOpcode(out, 0x88);
  }

  private void sendPing(OutputStream out) throws IOException {
    sendOpcode(out, 0x89);
    System.out.println("Sent ping");
  }

  private void sendPong(OutputStream out) throws IOException {
    sendOpcode(out, 0x8a);
    System.out.println("Sent pong");
  }

  private void sendOpcode(OutputStream out, int code) throws IOException {
    out.write(new byte[] { (byte) code, (byte) 0 });
    out.flush();
  }

  /**
   * Client side. Starts a separate thread which reads the InputStream stream
   * and checks for ping frames. If one is detected, sends a pong on the
   * OutputStream.
   */
  private void pingPongHandler(InputStream in, OutputStream out) {
    final int pingOpCode = 0x89;

    new Thread(() -> {
      while (!clientClosed) {
        try {
          byte[] buf = readInput(in);

          if (buf.length >= 2 && (buf[0] & 0xff) == pingOpCode) {
            sendPong(out);
          } else if (buf.length > 0) {
            System.out.println("From server:");
            for (int i = 0; i < buf.length; i++) {
              System.out.print(Integer.toHexString(buf[i]) + " ");
            }
            System.out.println();
          }

          Thread.sleep(50);
        } catch (IOException | InterruptedException e) {
          e.printStackTrace();
        }
      }
    }).start();
  }

  /**
   * Client side. Reads available bytes from the InputStream.
   */
  private byte[] readInput(InputStream in) throws IOException {
    byte[] buf = new byte[2000];
    int i = 0;
    while (in.available() > 0) {
      buf[i++] = (byte) in.read();
    }

    return Arrays.copyOf(buf, i);
  }

  /**
   * Client side. Sends the Websocket handshake request to the server.
   */
  private void sendHandshake(OutputStream rawOut) {
    String handshake[] = {
      "GET /test HTTP/1.1",
      "Host: loclahost",
      "Upgrade: websocket",
      "Connection: Upgrade",
      "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==",
      "Origin: http://localhost:" + PORT,
      "Sec-WebSocket-Protocol: test",
      "Sec-WebSocket-Version: 13",
      ""
    };

    PrintWriter pwOut = new PrintWriter(rawOut, true);
    for (String str : handshake) {
      pwOut.print(str + "\r\n");
    }
    pwOut.flush();
  }

  /**
   * Client side. Reads the Websocket handshake response from the server.
   */
  private void receiveHandshake(InputStream rawIn) throws IOException {
    BufferedReader bufIn = new BufferedReader(new InputStreamReader(rawIn));

    System.out.println("Response: ");
    String line;
    do {
      line = bufIn.readLine();
      System.out.println("R: '" + line + "'");
    } while (line != null && !line.isEmpty());
  }
}

Simple Framework HTTP server

12 February 2017

A while back, I ranted about the Simple Framework example code and its unnecessary complexity. It turns out that their HTTP server library is indeed simple, if you just ignore the Spring setup and their example. In fact, as this static file server example shows, it’s very similar to the “hidden” Sun HTTP handler API.

The only interface which has to be implemented is the Container, which is equivalent to the request HttpHandler in the Sun HTTP library. The handle() method takes a normal HTTP request, and the response code and data can be written via the Response object. The first code block below shows a simple handler.

The next block shows how to tie the required Simple Framework classes together to start the server. The Container just mentioned goes into a ContainerSocketProcessor where concurrent processing can be setup up. The processor goes into a SocketConnection which connects to a socket on a given host and port, and the server runs.

  static class StaticFileContainer implements Container {

    private String baseDir;

    StaticFileContainer(String baseDir) {
      this.baseDir = baseDir;
    }

    @Override
    public void handle(Request req, Response resp) {
      System.out.println(req);
      System.out.println("target: " + req.getTarget());

      Path path = Paths.get(baseDir, req.getTarget());

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap(Files.readAllBytes(path)));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }
  public void setup() throws IOException {
    StaticFileContainer container = new StaticFileContainer("com/rememberjava/http");
    ContainerSocketProcessor server = new ContainerSocketProcessor(container, 1);
    connection = new SocketConnection(server);

    connection.connect(new InetSocketAddress(PORT));
  }

Here is the full server example, started and verified as a unit test.

SfHttpServerTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.http;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.simpleframework.http.Request;
import org.simpleframework.http.Response;
import org.simpleframework.http.core.Container;
import org.simpleframework.http.core.ContainerSocketProcessor;
import org.simpleframework.transport.connect.SocketConnection;

public class SfHttpServerTest {

  private static final int PORT = 8889;

  static class StaticFileContainer implements Container {

    private String baseDir;

    StaticFileContainer(String baseDir) {
      this.baseDir = baseDir;
    }

    @Override
    public void handle(Request req, Response resp) {
      System.out.println(req);
      System.out.println("target: " + req.getTarget());

      Path path = Paths.get(baseDir, req.getTarget());

      resp.setCode(200);
      try {
        resp.getByteChannel().write(ByteBuffer.wrap(Files.readAllBytes(path)));
        resp.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }

  private SocketConnection connection;

  @Before
  public void setup() throws IOException {
    StaticFileContainer container = new StaticFileContainer("com/rememberjava/http");
    ContainerSocketProcessor server = new ContainerSocketProcessor(container, 1);
    connection = new SocketConnection(server);

    connection.connect(new InetSocketAddress(PORT));
  }

  @After
  public void close() throws IOException {
    connection.close();
  }

  @Test
  public void testRequest() throws Exception {
    URL url = new URL("http://localhost:8889/test.txt");
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    in.lines().forEach(System.out::println);
    in.close();
  }
}

Monitoring with Metrics

10 February 2017

Metrics is a minimal library for various server monitoring metrics, and easy reporting. Supported metrics include rate meters; instant gauges; counters; histograms; timers. Furthermore, various reporting formatting is included. See their Getting started page for more details.

It’s available on Maven Central, so the following Gradle config will include the latest 3.1.0 version.

repositories {
  mavenCentral()
}

dependencies {
  compile 'io.dropwizard.metrics:metrics-core:3.1.0'
}

The simplified tests in the code below will output this report.

2/9/17 8:10:50 PM =============================================================

-- Histograms ------------------------------------------------------------------
histogram
             count = 4
               min = 1
               max = 3
              mean = 2.50
            stddev = 0.87
            median = 3.00
              75% <= 3.00
              95% <= 3.00
              98% <= 3.00
              99% <= 3.00
            99.9% <= 3.00


2/9/17 8:10:50 PM =============================================================

-- Gauges ----------------------------------------------------------------------
gauge
             value = 123


2/9/17 8:10:50 PM =============================================================

-- Meters ----------------------------------------------------------------------
meter
             count = 2
         mean rate = 19.68 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second


2/9/17 8:10:50 PM =============================================================

-- Timers ----------------------------------------------------------------------
timer
             count = 1
         mean rate = 4.95 calls/second
     1-minute rate = 0.00 calls/second
     5-minute rate = 0.00 calls/second
    15-minute rate = 0.00 calls/second
               min = 200.13 milliseconds
               max = 200.13 milliseconds
              mean = 200.13 milliseconds
            stddev = 0.00 milliseconds
            median = 200.13 milliseconds
              75% <= 200.13 milliseconds
              95% <= 200.13 milliseconds
              98% <= 200.13 milliseconds
              99% <= 200.13 milliseconds
            99.9% <= 200.13 milliseconds


2/9/17 8:10:50 PM =============================================================

-- Counters --------------------------------------------------------------------
counter
             count = 6

Some metrics examples:

MetricsApiTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.performance;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import com.codahale.metrics.ConsoleReporter;
import com.codahale.metrics.Counter;
import com.codahale.metrics.Gauge;
import com.codahale.metrics.Histogram;
import com.codahale.metrics.Meter;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Timer;
import com.codahale.metrics.Timer.Context;

public class MetricsApiTest {

  private MetricRegistry metrics;

  @Before
  public void setup() {
    metrics = new MetricRegistry();
  }

  @After
  public void report() {
    ConsoleReporter reporter = ConsoleReporter.forRegistry(metrics).build();
    reporter.report();
  }

  @Test
  public void testMeter() throws InterruptedException {
    Meter meter = metrics.meter("meter");
    meter.mark();
    Thread.sleep(100);
    meter.mark();
  }

  @Test
  public void testGauge() {
    Gauge<Integer> g = (() -> 123);
    metrics.register("gauge", g);
  }

  @Test
  public void testCounter() {
    Counter counter = metrics.counter("counter");
    counter.inc();
    counter.inc(5);
  }

  @Test
  public void testHistogram() {
    Histogram histogram = metrics.histogram("histogram");
    histogram.update(1);
    histogram.update(3);
    histogram.update(3);
    histogram.update(3);
  }

  @Test
  public void testTimer() throws InterruptedException {
    Timer timer = metrics.timer("timer");
    Context time = timer.time();
    Thread.sleep(200);
    time.stop();
  }
}

Identity Map Collector Helpers

09 February 2017

When generating a Map, the basis is very often a list of unique elements (i.e. a Set), and often the original elements take part in the new Map, either as keys or values. When the original elements are keys, we might be talking about a cache, where the key is the pointer and the value is the result of some expensive operation. E.g. the key is a filename and the value is the bytes of the file loaded into memory for fast serving. Conversely, when the original elements are values of the new map, we’re often dealing with a look-up table, e.g. from an ID to its User object, or as in the previous article from a short-form string to a specific class reference.

Streams iterating over collections, and lambda functions take some of the dull boilerplate out of creating these kind of maps. However, some of the helper classes, like Collectors, come with rather long method names. To take away even these parts, here are a few helper methods for the identity maps. They come in two forms: One which returns a Collector which can be used in the collect() method of the Stream. Typically, this is useful if other operations, like filtering, is also used. The other helper methods operate directly on the Collection, and take only the mapping function for keys or values. See the full file for further examples and documentation below.

  public static <T, U> Collector<T, ?, Map<T, U>> identityToValue(
      Function<? super T, ? extends U> valueMapper) {
    return Collectors.toMap(Function.identity(), valueMapper);
  }

  public static <T, U> Map<T, U> identityToValue(
      Collection<T> c, Function<? super T, ? extends U> valueMapper) {
      return c.stream().collect(identityToValue(valueMapper));
  }
  public static <T, K> Collector<T, ?, Map<K, T>> keytoIdentity(
      Function<? super T, ? extends K> keyMapper) {
    return Collectors.toMap(keyMapper, Function.identity());
  }

  public static <T, K> Map<K, T> keytoIdentity(
      Collection<T> c, Function<? super T, ? extends K> keyMapper) {
    return c.stream().collect(keytoIdentity(keyMapper));
  }

The full code listing, with examples and documentation. This code is under GPL3.

IdentityCollectorsTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.lambda;

import static org.junit.Assert.assertEquals;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collector;
import java.util.stream.Collectors;

import org.junit.Test;

public class IdentityCollectorsTest {

  /**
   * Returns a specialised {@link Collector} which results in a {@link Map}
   * where the keys are the elements of the stream, and the values is provided
   * by the given mapping function.
   * 
   * @param <T> the type of the input elements, and keys of the Map
   * @param <U> the output type of the value mapping function
   * 
   * @param valueMapper a mapping function to produce values
   * @return a {@code Collector} which collects elements into a {@code Map}
   *         whose keys are as given by the stream, and values are the result of
   *         applying the given mapping function to the input elements.
   */
  public static <T, U> Collector<T, ?, Map<T, U>> identityToValue(
      Function<? super T, ? extends U> valueMapper) {
    return Collectors.toMap(Function.identity(), valueMapper);
  }
  
  /**
   * Returns a {@link Map} based on the given {@link Collection}. The keys are
   * the elements of the Collection, and the values is provided by the given
   * mapping function.
   * 
   * @param <T> the type of the input elements, and keys of the Map
   * @param <U> the output type of the value mapping function
   * 
   * @param c Collection of elements to map
   * @param valueMapper a mapping function to produce values
   * @return a {@code Map} whose keys are as given by the Collection, and values
   *         are the result of applying the given mapping function to the input
   *         elements.
   */
  public static <T, U> Map<T, U> identityToValue(
      Collection<T> c, Function<? super T, ? extends U> valueMapper) {
      return c.stream().collect(identityToValue(valueMapper));
  }

  /**
   * Returns a specialised {@link Collector} which results in a {@link Map}
   * where the keys are provided by the given mapping function, and the values
   * are the elements of the stream. 
   * 
   * @param <T> the type of the input elements, and values of the Map
   * @param <K> the output type of the key mapping function
   * 
   * @param keyMapper a mapping function to produce keys
   * @return a {@code Collector} which collects elements into a {@code Map}
   *         whose keys are the result of applying the given mapping function
   *         to the input elements, and values are as given by the stream.
   */
  public static <T, K> Collector<T, ?, Map<K, T>> keytoIdentity(
      Function<? super T, ? extends K> keyMapper) {
    return Collectors.toMap(keyMapper, Function.identity());
  }

  /**
   * Returns a {@link Map} based on the given {@link Collection}. The keys are
   * provided by the given mapping function, and the values are the elements of
   * the Collection. 
   * 
   * @param <T> the type of the input elements, and values of the Map
   * @param <K> the output type of the key mapping function
   * 
   * @param c Collection of elements to map
   * @param valueMapper a mapping function to produce values
   * @return a {@code Map} whose keys are the result of applying the given
   *         mapping function to the input elements, and values are as given
   *         by the stream.
   */
  public static <T, K> Map<K, T> keytoIdentity(
      Collection<T> c, Function<? super T, ? extends K> keyMapper) {
    return c.stream().collect(keytoIdentity(keyMapper));
  }

  @Test
  public void keytoIdentityTest() {
    List<Class<?>> classes = Arrays.asList(String.class, ArrayList.class);
    
    Map<String, Class<?>> nameMap = classes.stream()
        .filter(c -> c.getName().contains("java.lang"))
        .collect(keytoIdentity(c -> c.getSimpleName()));
    assertEquals(String.class, nameMap.get("String"));

    Map<String, Class<?>> nameMap2 = keytoIdentity(classes, c -> c.getSimpleName());
    assertEquals(ArrayList.class, nameMap2.get("ArrayList"));
  }

  @Test
  public void identityToValueTest() {
    List<String> files = Arrays.asList("index.html", "about.html");

    Map<String, byte[]> fileCache = files.stream()
        .filter(f -> f.contains("index"))
        .collect(identityToValue(f -> loadFile(f)));
    assertEquals("index.html", new String(fileCache.get("index.html")));

    Map<String, byte[]> fileCache2 = identityToValue(files, this::loadFile);
    assertEquals("about.html", new String(fileCache2.get("about.html")));
  }

  public byte[] loadFile(String name) {
    // Returns a dummy value for the sake of the test. 
    return name.getBytes();
  }
}

Object construction through reflection

27 January 2017

In the last post I ranted about dependency injection framework magic, but even custom magic can cause problems. When objects are constructed and invoked through reflection, the execution flow is no longer apparent in the code. Yet, it still has its place in certain applications, typically mapping external input to object instances, be it language parsing or lexing, or DB output to object mapping. But how to best balance between the abstract world of reflected classes and concrete type-safe code?

The key lies in simplicity, and imposing certain constrains. Dynamically resolving classes and recursive reflective construction of a full object a hierarchy is probably not a good idea. Furthermore, as discussed previously, Java 8 method references now makes it cleaner to pass in factory methods, and avoid the Factory Factory Factory trap.

The following snippet shows the reflective construction of an object, given a few constraints: First off, the name of the input token can be mapped to typed classes. In this example that is done with a generated String to Class map, based on the concrete class references we’re dealing with. They have to be available at compile time, and at the time of writing the code. The advantage is that we avoid the Class.forName() look-up, and the package hierarchy confusion and possible mismatch of tokens and classes. If this is too strict, we can modify the map, e.g. by allowing for case-insensitive matching (by converting both map key and look-up to the same case). Or if there is not a literal mapping, an enum could define the relationship. Either way, the idea is that the code make it clear which classes we plan to deal with, in strongly typed manner.

The next assumption in this example is that the class to be created has only one constructor method, and that it is public. Or if that is not feasible, the constructor to be used could be marked with an Annotation. Just don’t go full-on Guice, and you’ll be fine.

Finally, we assume that the relevant constructor takes zero or more parameters, and if it does have parameters that they can themselves we represented and created from a single String object. These parameter objects are initialized in a helper method, discussed below.

  List<Class<?>> validClasses = Arrays.asList(Foo.class, Account.class);

  Map<String, Class<?>> classNameMap = validClasses.stream()
      .collect(Collectors.toMap(c -> c.getSimpleName(), Function.identity()));

  Object construct(String type, String... args) throws Exception {
    if (!classNameMap.containsKey(type)) {
      throw new IllegalArgumentException("Invalid class name: " + type);
    }

    Class<?> klass = classNameMap.get(type);
    Constructor<?>[] constructors = klass.getConstructors();
    Constructor<?> constructor = constructors[0];

    Class<?>[] parameterTypes = constructor.getParameterTypes();
    Object[] parameters = createParameters(parameterTypes, args);
    return constructor.newInstance(parameters);
  }

The construction of the parameter classes also contains several assumptions and restrictions. As with the first order classes, we limit ourselves to a pre-defined set of classes. This make it possible to define which constructor or factory methods should be used. In this example, the constructor which takes a single String is used, except for the Password class, where a factory method is used, again taking a single String.

  Map<Class<?>, Function<String, ?>> classConstructorMap = new HashMap<Class<?>, Function<String, ?>>() { {
    put(String.class, String::new);
    put(Integer.TYPE, Integer::new);
    put(Double.class, Double::new);
    put(Email.class, Email::new);
    put(Password.class, Password::hash);
  } };

  Object[] createParameters(Class<?>[] types, String[] args) {
    if (types.length != args.length) {
      throw new IllegalArgumentException("Expects: " + Arrays.asList(types));
    }

    Object[] result = new Object[types.length];
    for (int i = 0; i < types.length; i++) {
      Class<?> klass = types[i];
      Function<String, ?> constuctor = classConstructorMap.get(klass);
      if (constuctor == null) {
        throw new IllegalArgumentException("Constructor for " + klass + " not declared.");
      }
      result[i] = constuctor.apply(args[i]);
    }
    return result;
  }

That is all it takes to construct objects through reflection, including its parameter types. The examples above maintain some type-safety, and also restricts the supported types. Finally, some error handling and messaging is in place, and more could be added to make it very clear which classes and input is allowed.

The full code listing below shows the example classes to be constructed and test code to verify them.

ObjectConstructorTest.java
GitHub Raw
/* Copyright rememberjava.com. Licensed under GPL 3. See http://rememberjava.com/license */
package com.rememberjava.reflect;

import static org.junit.Assert.assertEquals;

import java.lang.reflect.Constructor;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;

import org.junit.Test;

public class ObjectConstructorTest {

  static class Foo {
    final String str;
    final int i;
    final double d;

    public Foo(String str, int i, Double d) {
      this.str = str;
      this.i = i;
      this.d = d;
    }
  }

  static class Account {
    final Email email;
    final Password password;

    public Account(Email email, Password password) {
      this.email = email;
      this.password = password;
    }
  }

  static class Email {
    final String email;

    Email(String email) {
      this.email = email;
    }
  }

  static class Password {
    final String hash;

    Password(String hash) {
      this.hash = hash;
    }

    static Password hash(String pw) {
      try {
        MessageDigest digest = MessageDigest.getInstance("SHA");
        digest.update(pw.getBytes());
        digest.update("some salt".getBytes());

        String base64 = Base64.getEncoder().encodeToString(digest.digest());

        return new Password(base64);
      } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
      }
      return null;
    }
  }

  List<Class<?>> validClasses = Arrays.asList(Foo.class, Account.class);

  Map<String, Class<?>> classNameMap = validClasses.stream()
      .collect(Collectors.toMap(c -> c.getSimpleName(), Function.identity()));

  Object construct(String type, String... args) throws Exception {
    if (!classNameMap.containsKey(type)) {
      throw new IllegalArgumentException("Invalid class name: " + type);
    }

    Class<?> klass = classNameMap.get(type);
    Constructor<?>[] constructors = klass.getConstructors();
    Constructor<?> constructor = constructors[0];

    Class<?>[] parameterTypes = constructor.getParameterTypes();
    Object[] parameters = createParameters(parameterTypes, args);
    return constructor.newInstance(parameters);
  }

  @SuppressWarnings("serial")
  Map<Class<?>, Function<String, ?>> classConstructorMap = new HashMap<Class<?>, Function<String, ?>>() { {
    put(String.class, String::new);
    put(Integer.TYPE, Integer::new);
    put(Double.class, Double::new);
    put(Email.class, Email::new);
    put(Password.class, Password::hash);
  } };

  Object[] createParameters(Class<?>[] types, String[] args) {
    if (types.length != args.length) {
      throw new IllegalArgumentException("Expects: " + Arrays.asList(types));
    }

    Object[] result = new Object[types.length];
    for (int i = 0; i < types.length; i++) {
      Class<?> klass = types[i];
      Function<String, ?> constuctor = classConstructorMap.get(klass);
      if (constuctor == null) {
        throw new IllegalArgumentException("Constructor for " + klass + " not declared.");
      }
      result[i] = constuctor.apply(args[i]);
    }
    return result;
  }

  @Test
  public void testFoo() throws Exception {
    Foo foo = (Foo) construct("Foo", "ABC", "123", "3.14");
    assertEquals("ABC", foo.str);
    assertEquals(123, foo.i);
    assertEquals(3.14, foo.d, 2);
  }

  @Test
  public void testAccount() throws Exception {
    Account account = (Account) construct("Account", "bob@example.com", "some password");
    assertEquals("V0PYfuPn4u9Gize+0DZ0nLgQQPk=", account.password.hash);
  }

  @Test(expected = IllegalArgumentException.class)
  public void testUndefinedClass() throws Exception {
    construct("NotFound");
  }

  @Test(expected = NumberFormatException.class)
  public void testInvalidParameterType() throws Exception {
    construct("Foo", "ABC", "ABC", "3.14");
  }

  @Test(expected = IllegalArgumentException.class)
  public void testInvalidParameterCount() throws Exception {
    construct("Foo", "ABC");
  }
}
Newer posts Older posts