About Apache Compression and Content-Length Header

Just resolved an interesting problem today, one of our code breaks because the response header set by the web server did not include Content-Length.

Spent quite a while investigating and turns out this is due to gzip compression. As seen below Content-Encoding is gzip and I think this causes Content-Length to be omitted.

apache-resp-headers

Gzip compression can be disabled on apache via .htaccess config. In my case I disabled all compression to swf file by adding following configuration

<FilesMatch "\.swf$">
  SetEnv no-gzip 1
</FilesMatch>

Using File Protocol to Deploy Maven Project To Windows Shared Folder

Our development team is fairly small and we’re not at the point where we need Nexus yet, so I decided we can try using a simple Windows server share as our internal Maven repository.

On each of our shared Maven project pom.xml we add following distributionManagement configuration:

<distributionManagement>
  <repository>
    <id>enfinium</id>
    <name>Enfinium Internal Repository</name>
    <url>file://mavenrepo/maven_repository</url>
  </repository>
</distributionManagement>

Where¬†//mavenrepo/maven_repository is the Windows server share we’ve setup. We’ve made sure each team member has correct read/write permission.

However every time we deploy Maven would say everything is successful but the file is nowhere to be found on the remote repository. (We’re using Maven 3.1.0

Turns out with file protocol and Windows server share, this is the syntax that works for us (yes Maven just fail silently and said everything was SUCCESSFUL)

file://\/\/mavenrepo/maven_repository

Generating Visual C++ Crash Dump For Debugging

Detecting cause of problem that occurs only in production environment is hard. For Visual C++ created native windows application, this can be done using DebugDiag tool.

Download and install the tool on the production server, run it, and create a Crash rule type

vcppdebug1

Select a specific process target:

vcppdebug2

And search for your process

vcppdebug3

Tick This process instance only if multiple processes with same name are running but you’re only interested in one.

Click next and accept all defaults. Then when the crash occurs, you will get a .dmp file on C:\Program Files\DebugDiag\Logs\Crash rule for process id NNNN folder.

This .dmp file can be copied into your PC and opened in Visual Studio for stack trace analysis. Don’t forget you need to tell Visual Studio where to find the symbol (.pdb) file.

Select Set symbol paths on Visual Studio Action bar and add the folder containing your .pdb file.

vcppdebug4

Then start the debugging session by selecting Debug with Native Only. Visual Studio will tell you the stack trace when the crash happens.

Thanks to Ganesh R for providing this solution on Stack Overflow.

Also checkout Microsoft’s documentation on DebugDiag.

Debugging Maven Unit Test in Eclipse / STS

Maven unit tests (run by surefire plugin) can be debugged with eclipse. Firstly check you’re using surefire plugin version 2.16 or newer. Observe / add following in your pom.xml

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.16</version>
</plugin>

Ensure you’ve set some breakpoints to debug the faulty codes. Then on your run configuration, add -Dmaven.surefire.debug=true parameter:

maven-debug1

When this run configuration is executed, maven will wait for you to attach the remote debugger instead of launching the tests:

maven-debug2

Now open eclipse debug configuration, create a Remote Java Application config with your source code and set the port to 5005:

maven-debug3

When the debugger attaches the unit test will proceed and when any breakpoints hit you can debug it like a normal java/eclipse application

Debugging Event Driven Multithreaded Java Code

So here’s the scenario: you have few potentially time consuming work need to be done. They need to be executed in the order of submission, but only one work at a time can run. Simple solution is to use single thread executor.

First let’s define the Job class. This is a simple class implementing Runnable, when the job runs / stops it will print to console. A random sleep between 1 to 5 seconds is introduced to simulate some reality.

public class Job implements Runnable {

  private String id;

  public Job(String id) {
    this.id = id;
  }
  
  @Override
  public void run() {
    System.out.println("Job " + id + " started");
    
    long duration = (new Random(System.currentTimeMillis()).nextInt(5) + 1) * 1000;
    try {
      Thread.sleep(duration);
    } catch (InterruptedException e) {
      System.out.println("Job " + id + " interrupted");
      e.printStackTrace();
    }
    
    System.out.println("Job " + id + " completed");
  }

}

To run the job I have a JobRunner class that setups the ExecutorService. Just call submit(job) and the job will be queued and run when the worker thread is free.

public class JobRunner {

  private ExecutorService executor = Executors.newSingleThreadExecutor();

  public void enqueue(Job job) {
    executor.submit(job);
  }

}

But a debugging problem starts to appear. It is now obscure where does the code goes after calling submit()? Which thread runs it? Unless the programmer put a big comment informing it is actually a Runnable this could be very hard to guess.

Further Problems with Observer Pattern

Another common patterns often used is the observer pattern. The idea is you want specific objects to “react” when a particular event occured.

For example, let’s do two observers: OddJobObservers and EvenJobObservers. When an odd/even job completed the corresponding observer will print to console.

public class OddJobObserver implements Observer {

  @Override
  public void update(Observable arg0, Object arg1) {
    int id = (int) arg1;
    if(id % 2 == 1)
      System.out.println("Odd job " + id + " completed");
  }

}
public class EvenJobObserver implements Observer {

  @Override
  public void update(Observable o, Object arg) {
    int id = (int) arg;
    if(id % 2 == 0)
      System.out.println("Even job " + id + " finished");
  }

}

For this purpose we’ll refactor JobRunner to extend Observable so we can add the observers into it. The observer instances are created and registered on the constructor.

Each time a job is created it will also have reference back to JobRunner. We’ll also add a method jobFinished(int id) for the job to call when it’s done running.

public class JobRunner extends Observable {

  private ExecutorService executor = Executors.newSingleThreadExecutor();

  public JobRunner() {
    addObserver(new OddJobObserver());
    addObserver(new EvenJobObserver());
  }

  public void enqueue(Job job) {
    job.setJobRunner(this);
    executor.submit(job);
  }

  /**
   * This will be invoked by the Job's run method in the worker thread
   * to indicate the runner that this particular job is finished
   */
  public void jobFinished(int id) {
    setChanged();
    notifyObservers();
  }
}

And on the Job class once it’s finshed running we’ll notify the observers

public class Job implements Runnable {

  // ...
  
  @Override
  public void run() {
    System.out.println("Job " + id + " started");
    
    // ...

    System.out.println("Job " + id + " completed");
    jobRunner.jobFinished(id);
  }

}

Consider debugging this code where you’re at the end of run() method which takes you to jobFinished(). It is now even more obscure because notifyObservers() is a Java API method, not your own code. Which code gets executed next?

Unless you did a good job on commenting, it’s very hard for someone reading your code to understand that at that point odd jobs will be observed by OddJobObserver and even jobs / EvenJobObserver.

The problem gets much worse with production client / server application with 50+ observers handling different scenario.

One approach is probably to put breakpoints on every single observers and see which one is actually “interested” in our event.

I hope this highlights how important properly commenting your code is (especially for your poor colleague who has to fix your code sometime in the future).

Java and XML Injectable Properties on Spring

Here’s how you can create a property file which can be injected on Java classes as well as on bean XML configuration file.

Firstly ensure annotation config is switched on:

<context:annotation-config/>

Create your properties file, in this case I create config.properties on my classpath root (src/main/resources/config.properties on Maven-compliant project)

name=Gerry

Register this property file on your spring context using <util:properties> tag. The file will be registered as a bean class with name config

<util:properties id="config" location="classpath:/config.properties"/>

Whenever you want to inject the value into a Java class, you can use @value annotation

@Component
public class Person {

  @Value("#{config.name}")
  private String name;

  //...
}

Similarly you can do the same to xml bean config file

<bean class="my.app.Person">
  <property name="name" value="#{config.name}"/>
</bean>

Note that if you config has dots on it, you need to use the square bracket syntax or Spring will confuse the dot as property access

@Value("#{config['first.name']}")
private String name;

Multiple Environment Trick

Another trick I love is specifying multiple environment config, eg: one for prod and dev. This is common when dealing with datasource properties:

dbhost=localhost
dbhost.dev=192.168.0.20

On dev environment, I then supply -Denv=dev system property to my VM args, and do this when looking up the property:

@Value("#{systemProperties['env'] == 'dev' ? config['dbhost.dev'] : config['dbhost']}")
private String databaseHost;

Putting String List on Properties

Property files can also hold simple string list:

emails=gerry@test.com,tom@wohoo.com

When injecting this, use the split() method. Becareful with whitespaces you placed as it will be carried over

@Value(#{config['emails'].split(',')})
List<String> emails;

Multithreaded Server in Java

In Java, listening over TCP can be simply done using ServerSocket. Example below binds our program to port 5656:

ServerSocket serverSocket = new ServerSocket(5656);

To access the client’s input stream, we first need to accept the connection and obtain the Socket object. This is done through the accept method:

ServerSocket serverSocket = new ServerSocket(5656);
Socket socket = serverSocket.accept();

The program above can only accept 1 client connection. We want to able to accept infinitely many connections, hence we can use an infinite loop to keep accepting new connections:

ServerSocket serverSocket = new ServerSocket(5656);
while(true) {
  Socket socket = serverSocket.accept();
  // process client connection..
}

However once a client is connected our code has to wait until he/she disconnects before we can accept the next one. This might be ok if the work duration is very short, but if it’s very long (eg: a chatting program) then this won’t do. To solve this we can use an ExecutorService which schedules work to be done in the future (in a separate thread) and allowing the calling thread to progress. In our case we will use a fixed thread-pool executor service, which effectively limit how many active connections we can have. We will soon write the ConnectionHandler class below to process the connection.

ExecutorService executorService = Executors.newFixedThreadPool(100);
ServerSocket serverSocket = new ServerSocket(5656);
while(true) {
  Socket socket = serverSocket.accept();
  executorService.execute(new ConnectionHandler(socket));
  // go back to start of infinite loop and listen for next incoming connection
}

The ConnectionHandler class above implements Runnable and does the work of listening & writing to client. Here’s an example of what simple ConnectionHandler that just echoes messages sent by client back:

public class ConnectionHandler implements Runnable {

  private Socket socket;

  public ConnectionHandler(Socket socket) {
    this.socket = socket;
  }

  public void run() {
    BufferedReader reader = null;
    PrintWriter writer = null;
    try {
      reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
      writer = new PrintWriter(socket.getOutputStream(), true);

      // The read loop. Code only exits this loop if connection is lost / client disconnects
      while(true) {
        String line = reader.readLine();
        if(line == null) break;
        writer.println("Echo: " + line);
      }
    } catch (IOException e) {
      throw new RuntimeException(e);
    } finally {
      try {
        if(reader != null) reader.close();
        if(writer != null) writer.close();
      } catch (IOException e) {
        throw new RuntimeException(e);
      }
    }
  }

}

Keep in mind the run method will run in a separate thread, not the main thread, so make sure you synchronize reads / writes into shared resources.

Socket Timeout

Ok so far so good, we can handle multiple connection simultaneously in separate threads. But our code is still vulnerable. Recall our thread pool setting is capped at 100 which means we can only accept 100 simultaneous connection. This leave us vulnerable to denial-of-service attack, a hacker could easily open 100 connections and leave it be — denying access to other clients. One way to mitigate this is to set socket timeout:

ExecutorService executorService = Executors.newFixedThreadPool(100);
ServerSocket serverSocket = new ServerSocket(5656);
while(true) {
  Socket socket = serverSocket.accept();
  socket.setSoTimeout(3000); // inputstream's read times out if no data came after 3 seconds
  executorService.execute(new ConnectionHandler(socket));
  // go back to start of infinite loop and listen for next incoming connection
}

By setting the socket timeout, the BufferedReader.readLine() method used by ConnectionHandler above will throw SocketTimeoutException if it hasn’t had any data in 3 seconds.

public void run() {
  BufferedReader reader = null;
  PrintWriter writer = null;
  try {
    reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
    writer = new PrintWriter(socket.getOutputStream(), true);
 
    while(true) {
      // SocketTimeoutException thrown here if nothing read after 3 seconds
      String line = reader.readLine();
      if(line == null) break;
      writer.println("Echo: " + line);
    }
  } catch (SocketTimeoutException e) {
    System.out.println("Connection timed out");
  } catch (IOException e) {
    throw new RuntimeException(e);
  } finally {
    try {
      if(reader != null) reader.close();
      if(writer != null) writer.close();
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }
}