Hello World with JavaFx

2017-01-22 14:07:26 +0000

JavaFx is a comprehensive platform and set of APIs to develop desktop, web and even mobile applications. Although intended to replace the Swing GUI API, it can get significantly more complex, with FXML for XML based object specification; CSS styling; and special IDE tool chains.

Since JDK 8, it is supposedly included in the main install, however, on Debian I had to install it separately:

apt-get install openjfx openjfx-source

The Oracle documentation and tutorial is detailed and well written, so I will not go into further detail here. The following is a simple Hello World app.

GitHub Raw
package com.rememberjava.ui;

import javafx.application.Application;
import javafx.collections.ObservableList;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;

@SuppressWarnings("restriction")
public class JavaFxHelloWorld extends Application {

  public static void main(String[] args) {
    launch(args);
  }

  @Override
  public void start(Stage primaryStage) {
    primaryStage.setTitle("My App");

    Label label = new Label();

    Button button = new Button();
    button.setText("Say 'Hello World'");
    button.setOnAction((e) -> {
      label.setText("Hi!");
    });

    VBox root = new VBox();
    ObservableList<Node> children = root.getChildren();
    children.add(button);
    children.add(label);

    primaryStage.setScene(new Scene(root));
    primaryStage.show();
  }
}

Terminal and ncurses Java libraries

2017-01-22 11:40:29 +0000

There’s a few Java libraries for creating terminal and ncurses like applications. They abstract away the ANSI escape codes and low level control like tput. I’ve briefly looked at three of them: Lanterna, Charva and JCurses. They all come with basic terminal control, and some widgets and window managing to create terminal based GUI applications.

JCurses seems to be the oldest, and possibly unmaintained. It requires ncurses to be installed, and also a native library. It is available through the Maven Central repository, however, make sure to set up the path for access to the libjcurses.so (in the tar-ball from SourceForge, or within the .jar from Maven). Documentation is sparse, but it seems it’s possible to interface with the terminal with little code.

Charva is also ncurses based and requires a native lib. One of it’s selling points is that it can act as a drop-in replacement for java.awt UIs. It seems unlikely that that will work well for all but the simplest UIs, however, at least it offers an easy transition path.

Finally, Lanterna seems to be the best maintained, and clearly documented project of the three. It is unique in that is handles all terminal details itself, and does not require external libraries. It is compatible with Linux, Windows, Cygwin, OS X terminals. Furthermore, if launched from an X or similar GUI based environment instead of a terminal, it will create its own GUI window into which it will render the terminal. Neat.

The drawback with Lanterna is its rather verbose and involved API. For “serious” apps, maybe that’s OK, but for simple positioning of the cursor, it can get a bit too much. The follow renders a text label with a border:

  public void testWindow() throws IOException, InterruptedException {
    Terminal terminal = new DefaultTerminalFactory().createTerminal();
    Screen screen = new TerminalScreen(terminal);
    screen.startScreen();

    Label label = new Label("test");

    BasicWindow window = new BasicWindow();
    window.setComponent(label);

    MultiWindowTextGUI gui = new MultiWindowTextGUI(screen, new DefaultWindowManager(),
        new EmptySpace(TextColor.ANSI.BLACK));
    gui.setTheme(LanternaThemes.getRegisteredTheme("businessmachine"));
    gui.addWindowAndWait(window);
  }

Lanterna is available through Maven Central, so the following gradle.build lines will do.

repositories {
  mavenCentral()
}

dependencies {
  compile 'com.googlecode.lanterna:lanterna:3.0.0-beta3'
}

Here’s the full test example, including a main-method to easily run it stand-alone from the terminal.

GitHub Raw
package com.rememberjava.ui;

import java.io.IOException;

import org.junit.Test;

import com.googlecode.lanterna.TextColor;
import com.googlecode.lanterna.bundle.LanternaThemes;
import com.googlecode.lanterna.gui2.BasicWindow;
import com.googlecode.lanterna.gui2.DefaultWindowManager;
import com.googlecode.lanterna.gui2.EmptySpace;
import com.googlecode.lanterna.gui2.Label;
import com.googlecode.lanterna.gui2.MultiWindowTextGUI;
import com.googlecode.lanterna.screen.Screen;
import com.googlecode.lanterna.screen.TerminalScreen;
import com.googlecode.lanterna.terminal.DefaultTerminalFactory;
import com.googlecode.lanterna.terminal.Terminal;

public class LanternaCliTest {

  public static void main(String[] args) throws Exception {
    new LanternaCliTest().testWindow();
  }

  @Test
  public void testWindow() throws IOException, InterruptedException {
    Terminal terminal = new DefaultTerminalFactory().createTerminal();
    Screen screen = new TerminalScreen(terminal);
    screen.startScreen();

    Label label = new Label("test");

    BasicWindow window = new BasicWindow();
    window.setComponent(label);

    MultiWindowTextGUI gui = new MultiWindowTextGUI(screen, new DefaultWindowManager(),
        new EmptySpace(TextColor.ANSI.BLACK));
    gui.setTheme(LanternaThemes.getRegisteredTheme("businessmachine"));
    gui.addWindowAndWait(window);
  }
}

A simple HTTP server

2017-01-20 04:50:47 +0000

The unofficial com.sun packages which is still part of the main JDK, include a few convenience classes for running a HTTP server. However, since these are not part of the official API, there’s typically a warning in the IDEs not to use them. The annotation @SuppressWarnings(“restriction”) can also be used to ignore this warning. Besides that, they will work fine.

Here’s what it takes to start the server on a given port, and serve static files from a specific directory tree:

  void start() throws IOException {
    server = HttpServer.create(new InetSocketAddress(PORT), 0);

    server.createContext("/static", new StaticFileHandler(BASEDIR));

    server.start();
  }

The second argument to the create() method, which is currently 0, is the number of queued incoming queries. Setting a higher number here will allow the server to handle more parallel incoming requests, however potentially at the expense of overall throughput.

The start() is non-blocking, so a typically server application would have to go into an internal loop.

As for the handler, it listens to requests where the path of the URI starts with the specified string, in this case “/static”. It can then use the request URI to map this to hard-coded files, or files with the same name as in this example:

  public void handle(HttpExchange ex) throws IOException {
    URI uri = ex.getRequestURI();
    String name = new File(uri.getPath()).getName();
    File path = new File(baseDir, name);

    Headers h = ex.getResponseHeaders();
    // Could be more clever about the content type based on the filename here.
    h.add("Content-Type", "text/html");

    OutputStream out = ex.getResponseBody();

    if (path.exists()) {
      ex.sendResponseHeaders(200, path.length());
      Files.copy(path, out);
    } else {
      System.err.println("File not found: " + path.getAbsolutePath());

      ex.sendResponseHeaders(404, 0);
      out.write("404 File not found.".getBytes());
    }

    out.close();
  }

Finally, to test the running server, here’s two unit tests. The first downloads a file which exists, and prints its lines. The second requests a file which does not exist. Notice that the 404 return code passed from sendResponseHeaders() in the handler, is enough to make the openStream() call throw a FileNotFoundException on the client side.

  @Test
  public void testDownload() throws IOException {
    server.start();

    URL url = new URL("http://localhost:9999/static/test.txt");
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    in.lines().forEach(System.out::println);
    in.close();
  }



  @Test(expected = FileNotFoundException.class)
  public void testFilenNotFound() throws IOException {
    server.start();

    URL url = new URL("http://localhost:9999/static/not_found");
    url.openStream();
  }

Here’s the main class:

GitHub Raw
package com.rememberjava.http;

import java.io.IOException;
import java.net.InetSocketAddress;

import com.sun.net.httpserver.HttpServer;

@SuppressWarnings("restriction")
public class SimpleHttpServer {

  private static final String BASEDIR = "com/rememberjava/http";

  private static final int PORT = 9999;

  private HttpServer server;

  public static void main(String[] args) throws Exception {
    SimpleHttpServer server = new SimpleHttpServer();
    server.start();
  }

  void start() throws IOException {
    server = HttpServer.create(new InetSocketAddress(PORT), 0);

    server.createContext("/static", new StaticFileHandler(BASEDIR));

    server.start();
  }

  public void stop() {
    server.stop(0);
  }
}

And here’s the Handler for static files:

GitHub Raw
package com.rememberjava.http;

import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.net.URI;

import com.google.common.io.Files;
import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;

@SuppressWarnings("restriction")
public class StaticFileHandler implements HttpHandler {

  private final String baseDir;

  public StaticFileHandler(String baseDir) {
    this.baseDir = baseDir;
  }

  @Override
  public void handle(HttpExchange ex) throws IOException {
    URI uri = ex.getRequestURI();
    String name = new File(uri.getPath()).getName();
    File path = new File(baseDir, name);

    Headers h = ex.getResponseHeaders();
    // Could be more clever about the content type based on the filename here.
    h.add("Content-Type", "text/html");

    OutputStream out = ex.getResponseBody();

    if (path.exists()) {
      ex.sendResponseHeaders(200, path.length());
      Files.copy(path, out);
    } else {
      System.err.println("File not found: " + path.getAbsolutePath());

      ex.sendResponseHeaders(404, 0);
      out.write("404 File not found.".getBytes());
    }

    out.close();
  }

}

Send and Receive money with bitcoinj

2017-01-15 11:37:40 +0000

The bitcoinj library is easy to use for Bitcoin wallet and transaction functions for both native Java and Android applications. Although there are certain features missing, it seems mature enough to be included in a Bitcoin walled app or service.

Sometimes the source code leaves a bit to be desired in structure and readability: anonymous inner classes and other deep nesting blocks sometimes makes it difficult to follow; inheritance is often used where composition would have been be better; the Collections classes could have been used over arrays in many places. All of this might come back to haunt the developers later, for now they seem to be plowing on.

At least the basics are straight forward. The following code will read a test walled from disk, or create a new one if it does not already exist. The TestNet3 block chain and network is used. Since the bitcoinj library relies heavily on the Google Guava (com.google.common) classes, there are frequent artifacts of the threading and callback handling showing up. In this example, we want the code to block and wait, therefore the extra await-functions are required.

  public void testSync() {
    params = TestNet3Params.get();
    kit = new WalletAppKit(params, new File("/dev/shm"), "test");

    kit.startAsync();
    kit.awaitRunning();

    kit.stopAsync();
    kit.awaitTerminated();
  }

The nice thing about the test block chain is that it is a real public live block chain, with miners and a block chain explorer, but with no value in the coins. In fact, you can get free coins to test with from faucet.xeno-genesis.com or tpfaucet.appspot.com. (The latter has been timing out over the last days).

To get some free test coins, run the following code, wait for the prompt which shows the next receiving address, and head over to faucet.xeno-genesis.com to ask them to send some money there. It should show up as received within a few seconds. Your wallet now contains some coins.

  public void testReceive() {
    startSync();

    wallet.addCoinsReceivedEventListener(this::coinsReceived);

    Address toAdr = wallet.currentReceiveKey().toAddress(params);
    System.out.println("Waiting to receive coins on: " + toAdr);

    while (!receivedCoins) {
      sleep(100);
    }
  }



  private void coinsReceived(Wallet wallet, Transaction tx, Coin prevBalance, Coin newBalance) {
    Coin value = tx.getValueSentToMe(wallet);
    System.out.println("Received tx for " + value.toFriendlyString() + ": " + tx);
    receivedCoins = true;
  }

Since the test network is a real network with real miners, it’s good etiquette to return your test coins to the pool for others to use once you’re done with them. The following code takes care of that, returning them to the TP Faucet default return address “n2eMqTT929pb1RDNuqEnxdaLau1rxy3efi”. You can return all your coins, or just a fraction if you want to experiment more. This will also wait a few seconds for the callback confirmation.

  public void testSend() throws InsufficientMoneyException {
    startSync();

    // Adjust how many coins to send. E.g. the minimum; or everything.
    Coin sendValue = Transaction.REFERENCE_DEFAULT_MIN_TX_FEE;
    // Coin sendValue = wallet.getBalance().minus(Transaction.DEFAULT_TX_FEE);
    
    Address sendToAdr = Address.fromBase58(params, TPFAUCET_RETURN_ADR);
    SendRequest request = SendRequest.to(sendToAdr, sendValue);

    SendResult result = wallet.sendCoins(request);

    result.broadcastComplete.addListener(() -> {
      println("Coins were sent. Transaction hash: " + result.tx.getHashAsString());
      sentCoins = true;
    }, MoreExecutors.sameThreadExecutor());

    while (!sentCoins) {
      sleep(100);
    }
  }

Finally, it’s worth noting that bitcoinj is a “live” library, in development and with the latest update available through Gradle. To make this work, there’s a few settings and dependencies to take care of. The logging framework used by bitcoinj is SL4J, and an actual implementation library (e.g. “sl4j-simple”) is also need. It can be downloaded, or included as a Gradle build dependency as seen below.

Then, your source code directory structure might not match the default Gradle “main”, “test” structure. My current structure keeps all source code under the directory “src”, so I have specified that.

Gradle integrates OK with Eclipse, but be careful with the “refresh” option. It tends to insist on changing the classpath setting of the project, so the packages disappear. It’s a good idea to keep the .classpath setting file under version control.

apply plugin: 'java'

sourceSets {
    main {
        java {
            srcDirs = ['src']
        }
    }
}

repositories {
    jcenter()
    mavenCentral()
}

dependencies {
    compile 'org.slf4j:slf4j-api:1.7.21'
    compile 'org.slf4j:slf4j-simple:1.7.21'
    compile 'org.bitcoinj:bitcoinj-core:0.14.3'

    compile 'com.googlecode.lanterna:lanterna:3.0.0-beta3'

    testCompile 'junit:junit:4.12'
}

The following listing shows all the tests. It demonstrates similar functionality as seen in the ForwardingService class in the main bitcoinj getting started guide. Hopefully, the code is a bit easier to read and run this way.

GitHub Raw
package com.rememberjava.bitcoin;

import java.io.File;

import org.bitcoinj.core.Address;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.InsufficientMoneyException;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.kits.WalletAppKit;
import org.bitcoinj.params.TestNet3Params;
import org.bitcoinj.wallet.SendRequest;
import org.bitcoinj.wallet.Wallet;
import org.bitcoinj.wallet.Wallet.SendResult;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.impl.StaticLoggerBinder;

import com.google.common.util.concurrent.MoreExecutors;

/**
 * Multiple large file and network I/O tests for the Testnet3 block chain.
 * Comment in and out @Test annotations as need.
 * 
 * For more info:<br>
 * https://en.bitcoin.it/wiki/Testnet<br>
 * 
 * For test coins: <br>
 * http://faucet.xeno-genesis.com <br>
 * http://tpfaucet.appspot.com <br>
 * 
 * For test block explorer: <br>
 * https://testnet.blockexplorer.com
 *
 */
public class BitcoinjApiTest {

  /**
   * http://tpfaucet.appspot.com
   */
  private static final String TPFAUCET_RETURN_ADR = "n2eMqTT929pb1RDNuqEnxdaLau1rxy3efi";

  private static final File WALLET_DIR = new File("/tmp");

  private static final String WALLET_PREFIX = "rjtest";

  private NetworkParameters params;

  private WalletAppKit kit;

  private Wallet wallet;

  private volatile boolean receivedCoins;

  private volatile boolean sentCoins;

  /**
   * For all tests, connect to the TestNet3 network.
   */
  @Before
  public void setup() {
    params = TestNet3Params.get();
    kit = new WalletAppKit(params, WALLET_DIR, WALLET_PREFIX);
  }

  /**
   * Connect, sync and wait till done.
   */
  private void startSync() {
    kit.startAsync();
    kit.awaitRunning();
    sleep(1000);

    wallet = kit.wallet();
  }

  /**
   * Disconnect and wait till done.
   */
  @After
  public void teardown() {
    kit.stopAsync();
    kit.awaitTerminated();
  }

  /**
   * Even though it is possible to run without the logger implementation classes
   * in place, it is very confusing and misleading.
   * 
   * The bitcoinj logs output connection and transaction information. Without
   * them, it's difficult to know what is happening, and the application might
   * seem "stuck".
   */
  @Test
  public void checkLogger() {
    StaticLoggerBinder.getSingleton();
  }

  /**
   * Connects to the TestNet3 network and disconnects. If there is no walled
   * stored at the specified location, a new one is created and relevant blocks
   * are downloaded. This can take 1 to 2 minutes. If information is stale, it
   * can also take a minute to update.
   * 
   * The walled used in this test is separate from the other tests.
   */
  @Test
  public void testSync() {
    params = TestNet3Params.get();
    kit = new WalletAppKit(params, new File("/dev/shm"), "test");

    kit.startAsync();
    kit.awaitRunning();

    kit.stopAsync();
    kit.awaitTerminated();
  }

  /**
   * Connects and prints address information about the test wallet (from the
   * TestNet3 network).
   */
  @Test
  public void printAddresses() {
    startSync();

    System.out.println("Receive Addresses:");
    wallet.getIssuedReceiveAddresses().stream().forEach(this::println);

    System.out.println("Watched Addresses:");
    wallet.getWatchedAddresses().stream().forEach(this::println);

    System.out.println("Current change address:");
    System.out.println(wallet.currentChangeAddress());
  }

  /**
   * Connects and prints wallet information (from the TestNet3 network).
   */
  @Test
  public void printWalletInfo() {
    startSync();

    Coin balance = wallet.getBalance();
    println("Balance satoshis: " + balance.toString());
    println("Balance friendly: " + balance.toFriendlyString());
    println("Version: " + wallet.getVersion());
  }

  /**
   * Waits for coins to be received on the prompted address. This test blocks
   * until the coinsReceived callback is called.
   * 
   * (This test is commented out to avoid blocking the other tests).
   */
  // @Test
  public void testReceive() {
    startSync();

    wallet.addCoinsReceivedEventListener(this::coinsReceived);

    Address toAdr = wallet.currentReceiveKey().toAddress(params);
    System.out.println("Waiting to receive coins on: " + toAdr);

    while (!receivedCoins) {
      sleep(100);
    }
  }

  /**
   * Callback called when the coins have been received.
   */
  private void coinsReceived(Wallet wallet, Transaction tx, Coin prevBalance, Coin newBalance) {
    Coin value = tx.getValueSentToMe(wallet);
    System.out.println("Received tx for " + value.toFriendlyString() + ": " + tx);
    receivedCoins = true;
  }

  /**
   * Sends test coins back to the TPFAUCET test network address. This test
   * blocks until the broadcastComplete callback is called.
   * 
   * (This test is commented out to avoid blocking the other test and
   * inadvertently sending away our money.
   */
  // @Test
  public void testSend() throws InsufficientMoneyException {
    startSync();

    // Adjust how many coins to send. E.g. the minimum; or everything.
    Coin sendValue = Transaction.REFERENCE_DEFAULT_MIN_TX_FEE;
    // Coin sendValue = wallet.getBalance().minus(Transaction.DEFAULT_TX_FEE);
    
    Address sendToAdr = Address.fromBase58(params, TPFAUCET_RETURN_ADR);
    SendRequest request = SendRequest.to(sendToAdr, sendValue);

    SendResult result = wallet.sendCoins(request);

    result.broadcastComplete.addListener(() -> {
      println("Coins were sent. Transaction hash: " + result.tx.getHashAsString());
      sentCoins = true;
    }, MoreExecutors.sameThreadExecutor());

    while (!sentCoins) {
      sleep(100);
    }
  }

  private void sleep(long millis) {
    try {
      Thread.sleep(millis);
    } catch (InterruptedException e) {
      e.printStackTrace();
    }
  }

  private void println(Object str) {
    System.out.println(str);
  }
}

MIDI basics

2017-01-13 18:58:04 +0000

Working with MIDI in Java is easy. The standard API have classes covering MIDI file I/O, device I/O, sequencing and sound synthesis. This comprhensive tutorial covers most aspects. It also helps to know something about the MIDI format. Juan Bello’s slide deck is an excellent introduction. midi.org is also a good source, including their full message reference table.

To play a single C note through the default included “Gervill” soft synthezier, the following snippet will do. As commented in the code, there’s come discrepancy on pitch or octave labelling, but the 60th note is still on the save octave as A at 440 Hz.

  public void testPlayNote() throws InvalidMidiDataException, MidiUnavailableException {
    // 60 = middle C, C4 (or C3 or C5)
    // https://en.wikipedia.org/wiki/Scientific_pitch_notation#Similar_systems
    int channel = 0;
    int note = 60;
    int velocity = 127; // velocity (i.e. volume); 127 = high
    ShortMessage msg = new ShortMessage();
    msg.setMessage(ShortMessage.NOTE_ON, channel, note, velocity);

    long timeStamp = -1;
    MidiDevice synthesizer = getSynthesizer("Gervill");
    synthesizer.open();
    Receiver receiver = synthesizer.getReceiver();
    receiver.send(msg, timeStamp);
    receiver.close();
  }

This gives a brief overview of the MIDI devices on the system, both software based and hardware devices. Depending on drivers and OS, the various devices might show up under different names and types.

  public void testGetMidiDeviceInfo() throws MidiUnavailableException {
    Info[] infos = MidiSystem.getMidiDeviceInfo();
    for (Info info : infos) {
      MidiDevice device = MidiSystem.getMidiDevice(info);
      boolean isSequencer = device instanceof Sequencer;
      boolean isSynthesizer = device instanceof Synthesizer;

      System.out.println(
          "Info: " + device + ", '" + 
              info.getName() + "',  '" + 
              info.getVendor() + "', '" + 
              info.getVersion() + "', '" + 
              info.getDescription() + "', " + 
              "sequencer=" + isSequencer + 
              ", synthesizer=" + isSynthesizer);
    }
  }

Finally, the following methods demonstream MIDI file I/O.

  public void testWriteFile() throws InvalidMidiDataException, IOException {
    int volume = 0x70;
    int tick = 0;

    Sequence sequence = new Sequence(Sequence.PPQ, 24);
    Track track = sequence.createTrack();

    MetaMessage meta = new MetaMessage();
    String trackName = "First track";
    meta.setMessage(0x03, trackName.getBytes(), trackName.length());
    track.add(new MidiEvent(meta, tick));

    int notes[] = IntStream.range(60, 73).toArray();

    for (int note : notes) {
      ShortMessage on = new ShortMessage(ShortMessage.NOTE_ON, note, volume);
      track.add(new MidiEvent(on, tick));
      tick += 4;

      ShortMessage off = new ShortMessage(ShortMessage.NOTE_OFF, note, 0);
      track.add(new MidiEvent(off, tick));
      tick += 4;
    }

    MidiSystem.write(sequence, 1, MIDI_FILE);
  }




  public void testReadFile() throws InvalidMidiDataException, IOException {
    Sequence sequence = MidiSystem.getSequence(MIDI_FILE);
    
    System.out.println("tracks: " + sequence.getTracks().length);
    System.out.println("patches: " + sequence.getPatchList().length);
    
    for (Track track : sequence.getTracks()) {
      int size = track.size();
      System.out.println("track size: "+size);
      
      IntStream.range(0, size).boxed()
        .map(track::get)
        .map(RawMessage::new)
        .map(AbstractMessage::getNoteOctave)
        .map(str -> str + " ")
        .forEach(System.out::print);
      System.out.println();
    }
  }




  public void testMidiFileFormat() throws InvalidMidiDataException, IOException {
    MidiFileFormat format = MidiSystem.getMidiFileFormat(MIDI_FILE);

    System.out.println("bytes: " + format.getByteLength());
    System.out.println("length ms: " + format.getMicrosecondLength());
    System.out.println("divisionType: " + format.getDivisionType());
    System.out.println("resolution: " + format.getResolution());
    System.out.println("type: " + format.getType());
    System.out.println("properties: " + format.properties().size());

    format.properties().entrySet().stream()
      .map(e -> e.getValue() + "=" + e.getKey())
      .forEach(System.out::println);
  }

The full test case can be seen below. There’s a few more helper classes and details in the same package here. Then there’s some special implementation and details for the Roland TB-03 and TR-8 devices here.

GitHub Raw
package com.rememberjava.midi;

import static com.rememberjava.midi.MidiUtils.filteredDeviceStream;
import static com.rememberjava.midi.MidiUtils.onClassnameEquals;

import java.io.File;
import java.io.IOException;
import java.util.stream.IntStream;

import javax.sound.midi.InvalidMidiDataException;
import javax.sound.midi.MetaMessage;
import javax.sound.midi.MidiDevice;
import javax.sound.midi.MidiDevice.Info;
import javax.sound.midi.MidiEvent;
import javax.sound.midi.MidiFileFormat;
import javax.sound.midi.MidiSystem;
import javax.sound.midi.MidiUnavailableException;
import javax.sound.midi.Receiver;
import javax.sound.midi.Sequence;
import javax.sound.midi.Sequencer;
import javax.sound.midi.ShortMessage;
import javax.sound.midi.Synthesizer;
import javax.sound.midi.Track;
import javax.sound.midi.Transmitter;

import org.junit.Test;


/**
 * See also: https://docs.oracle.com/javase/tutorial/sound/overview-MIDI.html
 */
public class MidiApiTest {

  private static final String SOFT_SYNTHESIZER = "SoftSynthesizer";

  private static final String MIDI_FILE_PATH = "com/rememberjava/midi/test.mid";

  private static final File MIDI_FILE = new File(MIDI_FILE_PATH);

  @Test
  public void testGetMidiDeviceInfo() throws MidiUnavailableException {
    Info[] infos = MidiSystem.getMidiDeviceInfo();
    for (Info info : infos) {
      MidiDevice device = MidiSystem.getMidiDevice(info);
      boolean isSequencer = device instanceof Sequencer;
      boolean isSynthesizer = device instanceof Synthesizer;

      System.out.println(
          "Info: " + device + ", '" + 
              info.getName() + "',  '" + 
              info.getVendor() + "', '" + 
              info.getVersion() + "', '" + 
              info.getDescription() + "', " + 
              "sequencer=" + isSequencer + 
              ", synthesizer=" + isSynthesizer);
    }
  }

  @Test
  public void testGetSequencer() throws MidiUnavailableException {
    Sequencer sequencer = MidiSystem.getSequencer();
    Transmitter transmitter = sequencer.getTransmitter();

    System.out.println("Sequencer: " + sequencer.getDeviceInfo());
    System.out.println("Transmitter: " + transmitter);
  }

  @Test
  public void testGetSynthesizer() throws MidiUnavailableException {
    Synthesizer synthesizer = MidiSystem.getSynthesizer();
    Receiver receiver = synthesizer.getReceiver();

    System.out.println("Synthesizer: " + synthesizer.getDeviceInfo());
    System.out.println("Receiver: " + receiver);
  }

  @Test
  public void testPlayNote() throws InvalidMidiDataException, MidiUnavailableException {
    // 60 = middle C, C4 (or C3 or C5)
    // https://en.wikipedia.org/wiki/Scientific_pitch_notation#Similar_systems
    int channel = 0;
    int note = 60;
    int velocity = 127; // velocity (i.e. volume); 127 = high
    ShortMessage msg = new ShortMessage();
    msg.setMessage(ShortMessage.NOTE_ON, channel, note, velocity);

    long timeStamp = -1;
    MidiDevice synthesizer = getSynthesizer("Gervill");
    synthesizer.open();
    Receiver receiver = synthesizer.getReceiver();
    receiver.send(msg, timeStamp);
    receiver.close();
  }
  
  private MidiDevice getSynthesizer(String deviceName) throws MidiUnavailableException {
     return filteredDeviceStream(deviceName)
         .filter(onClassnameEquals(SOFT_SYNTHESIZER))
         .findFirst().get();
  }

  @Test
  public void testWriteFile() throws InvalidMidiDataException, IOException {
    int volume = 0x70;
    int tick = 0;

    Sequence sequence = new Sequence(Sequence.PPQ, 24);
    Track track = sequence.createTrack();

    MetaMessage meta = new MetaMessage();
    String trackName = "First track";
    meta.setMessage(0x03, trackName.getBytes(), trackName.length());
    track.add(new MidiEvent(meta, tick));

    int notes[] = IntStream.range(60, 73).toArray();

    for (int note : notes) {
      ShortMessage on = new ShortMessage(ShortMessage.NOTE_ON, note, volume);
      track.add(new MidiEvent(on, tick));
      tick += 4;

      ShortMessage off = new ShortMessage(ShortMessage.NOTE_OFF, note, 0);
      track.add(new MidiEvent(off, tick));
      tick += 4;
    }

    MidiSystem.write(sequence, 1, MIDI_FILE);
  }

  @Test
  public void testReadFile() throws InvalidMidiDataException, IOException {
    Sequence sequence = MidiSystem.getSequence(MIDI_FILE);
    
    System.out.println("tracks: " + sequence.getTracks().length);
    System.out.println("patches: " + sequence.getPatchList().length);
    
    for (Track track : sequence.getTracks()) {
      int size = track.size();
      System.out.println("track size: "+size);
      
      IntStream.range(0, size).boxed()
        .map(track::get)
        .map(RawMessage::new)
        .map(AbstractMessage::getNoteOctave)
        .map(str -> str + " ")
        .forEach(System.out::print);
      System.out.println();
    }
  }

  @Test
  public void testMidiFileFormat() throws InvalidMidiDataException, IOException {
    MidiFileFormat format = MidiSystem.getMidiFileFormat(MIDI_FILE);

    System.out.println("bytes: " + format.getByteLength());
    System.out.println("length ms: " + format.getMicrosecondLength());
    System.out.println("divisionType: " + format.getDivisionType());
    System.out.println("resolution: " + format.getResolution());
    System.out.println("type: " + format.getType());
    System.out.println("properties: " + format.properties().size());

    format.properties().entrySet().stream()
      .map(e -> e.getValue() + "=" + e.getKey())
      .forEach(System.out::println);
  }
}

Method references kills the Factory class

2017-01-06 16:24:10 +0000

When writing reusable code, we often want it to be as general and flexible as possible. So layers of abstractions are added; generic types; abstract class hierarchies; and let’s not forget the Factory pattern. Joel Spolsky had a famous rant about the factory factory factory pattern, and it can get ugly in the real world as well.

One of the reasons for the often clunky factory class is that it has not been possible to pass methods and constructors. Method references in Java 8 changes that, even for constructors. They can be passed for any class or array, e.g. String::new, String[]::new. Combined with generics, the type of the newly created object can also be specified.

In the example class below, the constructor happen to take two arguments, first a String and then an int. Therefore, the BiFunction function method fits, however, it would probably be more appropriate to define a more specific functional interface, which would also make the code more readable. The return value is of the type T, which should then be the same type as the generated object. The use is demonstrated in the test method below.

The restriction with this setup is of course that the number of arguments to the constructor is fixed. We could write fixes around that as well, but that would require the general class to know something about the classes it is instantiating, which defetes the purpose. There’s always the old Factory class, though.

  class Generator<T> {

    private final BiFunction<String, Integer, T> constructor;

    Generator(BiFunction<String, Integer, T> constructor) {
      this.constructor = constructor;
    }

    T generate() {
      return constructor.apply(TEST, _123);
    }
  }




  public void testGenerateA() {
    Generator<A> genA = new Generator<>(A::new);
    A a = genA.generate();
    assertEquals(TEST + _123, a.content);
  }

Now, it can be argued that this still constitutes a factory pattern, even if an external factory class is not used. The methods of Collectors highlights this, e.g.:

    public static <T, C extends Collection<T>>
    Collector<T, ?, C> toCollection(Supplier<C> collectionFactory) {
        return new CollectorImpl<>(collectionFactory, Collection<T>::add,
                                   (r1, r2) -> { r1.addAll(r2); return r1; },
                                   CH_ID);
    }

The full code listing of the example:

GitHub Raw
package com.rememberjava.lambda;

import static org.junit.Assert.assertEquals;

import java.util.function.BiFunction;

import org.junit.Test;

public class GenericConstructor {

  private static final String TEST = "test";
  private static final int _123 = 123;

  class Generator<T> {

    private final BiFunction<String, Integer, T> constructor;

    Generator(BiFunction<String, Integer, T> constructor) {
      this.constructor = constructor;
    }

    T generate() {
      return constructor.apply(TEST, _123);
    }
  }

  class A {
    final String content;

    A(String str, int i) {
      content = str + i;
    }
  }

  class B {
    final String str;
    final int i;

    B(String str, int i) {
      this.str = str;
      this.i = i;
    }
  }

  @Test
  public void testGenerateA() {
    Generator<A> genA = new Generator<>(A::new);
    A a = genA.generate();
    assertEquals(TEST + _123, a.content);
  }

  @Test
  public void testGenerateB() {
    Generator<B> genB = new Generator<>(B::new);
    B b = genB.generate();
    assertEquals(TEST, b.str);
    assertEquals(_123, b.i);
  }
}

Limited size queue

2017-01-04 10:51:25 +0000

The Java Collections Queue implementations will either grow without limit, or block if it grows beyond a given size, like the LinkedBlockingDeque. However, what if you need a non-blocking queue which drops its oldest elements? The Apache Commons CircularFifoQueue covers that. The snippet below shows typical use, with a queue size of two, and where the first element of three is dropped.

  public void testDrop() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(2);

    queue.add(1);
    queue.add(2);
    queue.add(3);

    assertTrue(2 == queue.poll());
    assertTrue(3 == queue.poll());
    assertTrue(queue.isEmpty());
  }

To install the Apache Commons 4.0 library on Debian / Ubuntu:

apt-get install libcommons-collections4-java libcommons-collections4-java-doc
apt-get source libcommons-collections4-java

The relevant files will be located at:

/usr/share/java/commons-collections4.jar  
/usr/share/maven-repo/org/apache/commons/commons-collections4/4.0/commons-collections4-4.0-javadoc.jar

Often, a queue is populated on one thread, and consumed by another. In this case, the access methods have to be synchronized, as seen in this example. Both offer() and poll() methods are non-blocking, and null is returned if the queue is empty.

  public void testMultithreaded() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(5);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        synchronized (queue) {
          queue.offer(i++);
        }
      }
    });
    insertThread.start();

    sleep(1);
    for (int i = 0; i < 10; i++) {
      synchronized (queue) {
        System.out.println("" + i + ": " + queue.poll());
      }
    }
  }

Finally, how does this queue work with Streams? In a single-thread context, there shouldn’t be a problem. However, when multithreaded it gets more tricky. The example below fails since the two threads operate on the queue concurrently, and a NoSuchElementException is often thrown. The ConcurrentLinkedQueue is thread-safe, but unbounded. Furthermore, its documentation states that “the size method is NOT a constant-time operation. Because of the asynchronous nature of these queues, determining the current number of elements requires a traversal of the elements, and so may report inaccurate results if this collection is modified during traversal”. Which means we’re back to square one.

  public void streamFail() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(10);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        queue.offer(i++);
      }
    });
    insertThread.start();

    sleep(1000);
    System.out.println("size=" + queue.size());
    // throws NoSuchElementException
    queue.stream().forEach(System.out::println);
  }

There are a few work-arounds, mentioned in this discussion. One trick is to use the Stream.generate() method, which will loop indefinilty, and synchronize on the queue within. The problem is, that this will never stop, which might be okey depending on your application. However, you’d have to run this on a spearate thread. Alternativly, use the limit() method or a stream terminating operation (e.g. findFirst()).

  public void streamGenerate() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(10);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        synchronized (queue) {
          queue.offer(i++);
        }
      }
    });
    insertThread.start();

    Stream.generate(() -> {
      synchronized (queue) {
        return queue.poll();
      }
    })
    // Never stops without the limit
    .limit(20)
    .forEach(System.out::println);
  }

Also worth mentioning, is the Google Guava implementation EvictingQueue. However, it it also not thread-safe.

Here’s the full listing with all test methods.

GitHub Raw
package com.rememberjava.apache;

import static org.junit.Assert.assertTrue;

import java.util.stream.Stream;

import org.apache.commons.collections4.queue.CircularFifoQueue;
import org.junit.Test;

public class CircularFifoQueueTest {

  @Test
  public void testDrop() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(2);

    queue.add(1);
    queue.add(2);
    queue.add(3);

    assertTrue(2 == queue.poll());
    assertTrue(3 == queue.poll());
    assertTrue(queue.isEmpty());
  }

  @Test
  public void testMultithreaded() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(5);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        synchronized (queue) {
          queue.offer(i++);
        }
      }
    });
    insertThread.start();

    sleep(1);
    for (int i = 0; i < 10; i++) {
      synchronized (queue) {
        System.out.println("" + i + ": " + queue.poll());
      }
    }
  }

  @Test
  public void streamFail() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(10);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        queue.offer(i++);
      }
    });
    insertThread.start();

    sleep(1000);
    System.out.println("size=" + queue.size());
    // throws NoSuchElementException
    queue.stream().forEach(System.out::println);
  }

  @Test
  public void streamGenerate() {
    CircularFifoQueue<Integer> queue = new CircularFifoQueue<>(10);

    Thread insertThread = new Thread(() -> {
      int i = 0;
      while (true) {
        synchronized (queue) {
          queue.offer(i++);
        }
      }
    });
    insertThread.start();

    Stream.generate(() -> {
      synchronized (queue) {
        return queue.poll();
      }
    })
    // Never stops without the limit
    .limit(20)
    .forEach(System.out::println);
  }

  private void sleep(long millis) {
    try {
      Thread.sleep(millis);
    } catch (InterruptedException e) {
      e.printStackTrace();
    }
  }
}

Sequential vs. Parallel Streams

2016-12-31 07:13:32 +0000

The Java 8 Streams API offers functional-style operations, and a simple way to execute such operations in parallel. To jump right into an example, the following two test methods show the difference between sequential and parallel execution of the the println() method over each of the elements of the stream.

    range = IntStream.range(1, RANGE);
  }

  @Test
  public void sequential() {
    range.forEach(PRINTLN);
  }

  @Test
  public void parallel() {
    range.parallel().forEach(PRINTLN);
  }

As is expected, the default sequential execution will print the elements of the stream, which in this example are integers from 1 to 14, in natural order. By using the parallel() method to create a parallel stream and call the same print method, the only noticeable difference is that they are printed out of order.

 --- sequential ---      --- parallel ---
1                       9
2                       12
3                       5
4                       4
5                       13
6                       7
7                       6
8                       11
9                       10
10                      1
11                      2
12                      14
13                      3
14                      8

To see how the parallel stream behaves, the last test applies a different method to each of the elements of the stream. In this example, the time spent by the work() method on each element is linearly proportional to its value. That is, for the element of value 1 it spends 100ms, for 2 its 200 ms and so on. The “work” it does, is simply to sleep for intervals of 100 ms, and the rest of the code is dedicated to printing and formatting the table below. However, it serves the purpose of demonstrating how parallel execution behaves, and how it relates to the underlying CPU(s).

    range.parallel().forEach(this::work);

In the result table, each element of the stream is represented by a row. Each row shows the value of the element, the time-slots the work() method was executing, and to the right the time in milliseconds when work started and finished. Each 100 ms slot is represented by a hash. As can be seen, the first row was element 9, thus it marked off nine slots, and the starting time was at 6 ms, and finish at 909 ms.

Furthermore, since this was run on a machine with four CPU cores, the stream will execute four calls in parallel. This can be seen by both the hashes and the start times of the first four rows. Next, when element 2 (fourth row) finishes at 207 ms, a new element is immediately started (element 3, fifth row).

In this example, the total number of 100 ms “units of work” can be found by the formula for the triangular number where n = 14, or 14 * (14 + 1) / 2 = 105. Meaning that, sequential execution would have taken 10.5 seconds, while four parallel CPUs managed in 3 seconds.

In the second table below, the same code is executed on a dual core CPU, and it is clear that now only two methods execute in parallel. That will of course lead to a longer overall runtime, of about 5.4 seconds for this example. This could lead to a discussion on task and scheduling optimisation, however it goes beyond this article, and what is possible with the simple parallel Stream construct.

 --- parallelWork ---
CPU count: 4

    0000000000111111111122222222223333
    0123456789012345678901234567890123
 9:  #########                          [   6 -  909]
13:  #############                      [   6 - 1312]
 5:  #####                              [   6 -  507]
 2:  ##                                 [   6 -  207]
 3:    ###                              [ 207 -  507]
 4:       ####                          [ 507 -  909]
 1:       #                             [ 507 -  608]
 7:        #######                      [ 609 - 1312]
10:           ##########                [ 911 - 1915]
12:           ############              [ 911 - 2116]
14:               ##############        [1313 - 2719]
 6:               ######                [1313 - 1915]
 8:                     ########        [1915 - 2719]
11:                     ###########     [1915 - 3021]
    0123456789012345678901234567890123
    0000000000111111111122222222223333
 --- parallelWork ---
CPU count: 2

    00000000001111111111222222222233333333334444444444555555555
    01234567890123456789012345678901234567890123456789012345678
 8:  ########                                                    [  23 -  825]
 4:  ####                                                        [  23 -  424]
 5:      #####                                                   [ 424 -  926]
 9:          #########                                           [ 826 - 1727]
 6:           ######                                             [ 926 - 1527]
 7:                 #######                                      [1527 - 2228]
10:                   ##########                                 [1727 - 2729]
 1:                        #                                     [2229 - 2329]
 2:                         ##                                   [2329 - 2529]
 3:                           ###                                [2530 - 2830]
13:                             #############                    [2729 - 4031]
11:                              ###########                     [2831 - 3932]
12:                                         ############         [3933 - 5134]
14:                                          ##############      [4031 - 5433]
    01234567890123456789012345678901234567890123456789012345678
    00000000001111111111222222222233333333334444444444555555555

The full code list is here.

GitHub Raw
package com.rememberjava.lambda;

import java.time.Duration;
import java.time.Instant;
import java.time.temporal.TemporalAmount;
import java.util.Arrays;
import java.util.List;
import java.util.Vector;
import java.util.function.IntConsumer;
import java.util.stream.IntStream;

import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;

public class ParallelCount {

  @Rule
  public TestName name = new TestName();

  private static final IntConsumer PRINTLN = System.out::println;

  private static final int RANGE = 15;

  private int axisWidth;

  private static final int WORK_UNIT_MILLIS = 100;

  private List<StringBuilder> results = new Vector<>(RANGE);

  private TemporalAmount start;

  private IntStream range;

  @Before
  public void setup() {
    String methodName = name.getMethodName().split("\\[")[0];
    System.out.println("\n --- " + methodName + " ---");

    int cpus = Runtime.getRuntime().availableProcessors();
    int workUnits = RANGE * (RANGE + 1) / 2;
    axisWidth = (int) ((workUnits / cpus) * 1.3);

    start = Duration.between(Instant.EPOCH, Instant.now());

    range = IntStream.range(1, RANGE);
  }

  @Test
  public void sequential() {
    range.forEach(PRINTLN);
  }

  @Test
  public void parallel() {
    range.parallel().forEach(PRINTLN);
  }

  @Test
  public void parallelWork() {
    System.out.printf("CPU count: %d\n\n",
        Runtime.getRuntime().availableProcessors());
    printAxis(true);

    range.parallel().forEach(this::work);

    results.stream().map(sb -> sb.toString()).forEach(System.out::println);
    printAxis(false);
  }

  void work(int units) {
    StringBuilder result = new StringBuilder();
    results.add(result);

    char[] timestampSlots = new char[axisWidth];
    Arrays.fill(timestampSlots, ' ');

    long startMillis = getNowMillis();

    try {
      for (int i = 0; i < units; i++) {
        Thread.sleep(WORK_UNIT_MILLIS);

        int nowSlot = (int) (getNowMillis() / WORK_UNIT_MILLIS);
        timestampSlots[nowSlot] = '#';
      }
    } catch (InterruptedException e) { // ignore
    }

    long endMillis = getNowMillis();

    result.append(String.format("%2d: %s [%4d - %4d]",
        units, new String(timestampSlots), startMillis, endMillis));
  }

  private void printAxis(boolean header) {
    String row1 = "";
    String row2 = "";
    for (int i = 0; i < axisWidth - 1; i++) {
      row1 += i / 10;
      row2 += i % 10;
    }

    System.out.println("    " + (header ? row1 : row2));
    System.out.println("    " + (header ? row2 : row1));
  }

  private long getNowMillis() {
    return Instant.now().minus(start).toEpochMilli();
  }
}
Older posts