VirtualBox, Ubuntu and LAMP Stack

Came accross VirtualBox, a free & excellent virtual machine software. I decided to take it for a spin creating a Ubuntu virtual machine LAMP stack on it..

Here We Go

  1. Download and install VirtualBox
  2. Download latest Ubuntu iso installation file
  3. From VirtualBox create a new Virtual Machine. Select type: Linux and version: Ubuntu. On the next step you will be prompted with dvd drive containing the installaion disk, but instead just select the iso downloaded on step 2
  4. Go through the Ubuntu installation steps
  5. It’s also very helpful to install ssh server so you can ssh into your VM later on: sudo apt-get install openssh-server

Voila! You have ubuntu running on your Windows PC

Host and Guest

In virtualization realm, host indicates your physical PC (Windows 7 in my case), and guest is the virtual machine (Ubuntu). Most of virtual machine software documentation uses host and guest terminology heavily so make sure you’re familiar with it

Networking

This is where things get tricky. Virtual machine comes with virtual network adapters, and you have to do few configuration to setup connectivity between your virtual and physical adapters.

By default VirtualBox allows the guest machine to connect to the internet through NAT, so you can download data, browse internet etc. However if you want to run servers from the guest, it won’t be discoverable by the host or other PC in the host’s network immediately.

One approach to make them discoverable is by setting up port forwarding. You get here by going to networking section on the machine’s setting on Virtual Box

portforwarding

Note that setting port forwarding requires the port is actually free on your host machine. Hence I find it very useful to add an IP to your host’s network interface specifically for the VM so you don’t have port conflicts. In this example I added the IP 192.168.16.201 on my interface:

addip

The “AMP”

So there’s the “L – Linux” done. Now for the Apache, Mysql and Php, it can simply be done by using Ubuntu’s apt-get package manager:

  1. Open a terminal / SSH session to your Ubuntu machine
  2. Elevate into root using sudo su root
  3. apt-get install apache2
  4. apt-get install php5
  5. apt-get install mysql-server mysql-client

Few helpful notes:

  • Default doc root is /var/www
  • To start / stop apache: sudo service apache2 stopsudo service apache2 start
  • To start / stop mysql: sudo service mysql stop / sudo service mysql start
Advertisements

Spring Basic Concept: Inversion of Control and Dependency Injection

If you’re a newbie just starting to learn Spring, or even if you’ve heard a lot about Spring but not sure what it was, Martin Fowler has an excellent article about this: Inversion of Control and Dependency Injection pattern.

It was a bit old (written in 2004) but every single conceptual discussion still applies today.

If any of the following keyword doesn’t make any sense to you, then I beg you spare 20 minutes of your time to read this excellent article:

  • Inversion of Control
  • Dependency Injection
  • Service Locator

Useful Apache Configuration

Redirecting HTTPS to HTTP (and vice versa)

RewriteEngine On
RewriteCond %{HTTPS} on
RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI}

http://stackoverflow.com/questions/8371/how-do-you-redirect-https-to-http

WordPress .htaccess Rewrite Rule

This .htaccess will rewrite any path not resolving to actual file or directory. It will add “/index.php” prefix into the URL. This is required for wordpress permalink

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

Redirecting mydomain.com to http://www.mydomain.com (or the opposite)

Your user often access your site using www prefix or without it, hence you setup both URL to resolve into your webhost in your DNS. However if you don’t redirect one into the other, search engine might think it’s a completely different site (hence website statistics etc will be wrong). One approach is to do external (301) redirect from one into the other.

RewriteCond %{HTTP_HOST} ^mydomain\.com.au [NC]
RewriteRule (.*) http://www.mydomain.com.au/$1 [L,R=301]

Monitoring Apache

To enable apache monitoring, firstly make sure status module is enabled. Find following line on your httpd.conf

LoadModule status_module modules/mod_status.so

Then add following configuration section. The “Allow from” restriction will prevent arbitary IP to view this, so if your ISP provide you with static IP, put it here.

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 220.245.123.18
</Location>

You can monitor your apache server (see worker threads status etc) by going to http://mydomain.com/server-status

 

Stay Tuned!

More to come when I stumble accross them

See Also

Tomcat 7 JDBC Session Persistence

The default Tomcat session management strategy is in-memory session persisted into file when the server is shutdown gracefully. If the server dies in a cold fashion (eg: kill -9 or power outage), session data might be lost. One approach to mitigate this is to store session data into database using JDBC, aka JDBC Session Persistence.

JDBC Session Persistence can also aid load balancer failover scenario. I’d say this is an alternative to setting up (often cumbersome) TCP session replication. Note that if you have multiple cloud servers like Amazon EC2 it doesn’t come with TCP multicast feature — TCP session replication sounds like a nightmare to setup.

The Steps

  1. Ensure org.apache.catalina.session.StandardSession.ACTIVITY_CHECK or org.apache.catalina.STRICT_SERVLET_COMPLIANCE is set to true. Add line similar to following into your Tomcat’s startup.sh (if you’re on UNIX)
    export CATALINA_OPTS="-Dorg.apache.catalina.session.StandardSession.ACTIVITY_CHECK=true"

    Tomcat System Property Reference will explain what do each property means if you’re curious

  2. Create following SQL table (yes you need a database to store the session data)
    create table tomcat_sessions (
      session_id     varchar(100) not null primary key,
      valid_session  char(1) not null,
      max_inactive   int not null,
      last_access    bigint not null,
      app_name       varchar(255),
      session_data   mediumblob,
      KEY kapp_name(app_name)
    );
    
  3. Place a copy of mysql-connector-java.jar (or your DB’s JDBC driver) into $CATALINA_HOME/lib
  4. In your web app, add a META-INF/context.xml file. If you use standard maven layout you have to place it on src/main/webapp/META-INF/context.xml. You can copy the file from $CATALINA_HOME/conf/context.xml as a starting point. Then under <Context> element add following <Manager> element
    <Manager className="org.apache.catalina.session.PersistentManager"
             maxIdleBackup="10">
      <Store className="org.apache.catalina.session.JDBCStore"
             connectionURL="jdbc:mysql://localhost/mytomcat?user=root"
             driverName="com.mysql.jdbc.Driver"
             sessionAppCol="app_name"
             sessionDataCol="session_data"
             sessionIdCol="session_id"
             sessionLastAccessedCol="last_access"
             sessionMaxInactiveCol="max_inactive"
             sessionTable="tomcat_sessions"
             sessionValidCol="valid_session" />
    </Manager>
    

    Notice how the SQL column name corresponds to some of the settings above. In this configuration I used mysql database on localhost with database name “mytomcat” and username “root”. maxIdleBackup=”10″ specifies number of seconds before the in-memory session data is persisted into database.

    There are many other settings you can tweak, have a look at the Tomcat Manager Component Reference.

Fine Prints

This article is tested against Tomcat 7.0.39 but I guess it should also work with Tomcat 6. If you’ve jumped the ship from relational to MongoDB, dawsonsystems published an open source MongoDB Tomcat Session Manager on github. I haven’t got a chance to try it but it looks awesome.

Mitigating DoS Attack Using iptables

Continuing from my earlier post about iptables basics, the limit module of iptables can be used to mitigate DoS (Denial of Service) attack. Note that mitigating here means “reducing the damage”. The worst scenario is under a heavy DoS you can’t even SSH and run commands on your server. With iptables you can limit the frequency of packets — enabling you to SSH and take appropriate actions.

Following are the rules I currently use. This will only allow new incoming TCP connection on port 80 & 443 with specified frequency (see limit explanation below):

iptables -A INPUT -j ACCEPT -p tcp --dport 80 -m state --state NEW -m limit --limit 40/s --limit-burst 5 -m comment --comment 'Allow incoming HTTP'
iptables -A INPUT -j ACCEPT -p tcp --dport 443 -m state --state NEW -m limit --limit 40/s --limit-burst 5 -m comment --comment 'Allow incoming HTTPS'
  • -A INPUT: Append to the end of a chain called INPUT
  • -j ACCEPT: When rule match, accept the packet
  • -p tcp: Match only TCP protocol
  • –dport: Match given TCP port
  • -m state: Use the state module
  • –state NEW: Match only packets initiated from new connection. This rule will not match packets exchanged from an existing connections.
  • -m limit: Use the limit module
  • –limit 40/s: If more than 40 packet per second received, decrement one burst point. If no more burst point, reject the packet
  • –limit-burst 5: The initial number of burst point. A “burst” occur when the limit above is reached. On a period where limit is not reached, one burst point is regained, up to this maximum limit. If burst point is 0, subsequent burst will cause the current rule matching to fail — and iptables will try the next rules (if you setup iptables properly the packet should slip through to ‘reject all’ rule)

Add following rule to allow your program making connection to localhost (loopback interface)

iptables -A INPUT -i lo -j ACCEPT

I then append a rule to match packets exchanged from established / related connections. This is important so packet resulted from outbound connections are accepted:

iptables -A INPUT -j ACCEPT -m state --state RELATED,ESTABLISHED -m limit --limit 100/s --limit-burst 50

And finally, reject all packets not accepted by above rules. Be careful before you do this, make sure you’ve added rules to allow SSH (port 22) so you don’t lock yourself out.

iptables -A INPUT -j REJECT

Testing And Continual Adjustment

Finding the correct number for limit and burst could be hard, but what I find useful is to perform continuous monitoring and adjustment. Keep in mind your goal here is to ensure maximum capacity of the server is utilised while protecting it against DoS. One approach I like is to use the iptables LOG target. Assuming I add following rules:

iptables -A INPUT -j ACCEPT -p tcp --dport 80 -m state --state NEW -m limit --limit 40/s --limit-burst 5
iptables -A INPUT -j LOG -p tcp --dport 80 -m state --state NEW --log-prefix 'TCP 80 Burst Exhausted'

If the first rule did not match (eg: because of burst is exhausted), the LOG rule after it will match and print into your syslog (typically /var/log/messages) with prefix “TCP 80 Burst Exhausted”. LOG target will not accept / reject the packet — after logging, iptables will continue checking the subsequent ruless.

The neat thing here is you can grep ‘TCP 80 Burst Exhausted’ /var/log/messages to detect when was last time suspected DoS attack occured. If the suspected attack is determined to be false alarm, your limit and burse setting is too strict, and you should gradually increase it.

Saving and Restoring

Updating rules at specific order can be very tedious, you have to count the terminal screen lines, insert the new rule, delete the old one and so on. There’s one trick you can do, that is to leverage iptables-restore command. Everytime you save the rule using service iptables save /etc/sysconfig/iptables file is updated with your rule specification. You can update this file and restore it using iptables-restore. Beware! A syntax error will cause your rule to be skipped. It’s best to test your command first by adding it to the bottom, and use this method to re-order the rules.

AWS EC2: UNIX User Management and SSH with Password Authentication on Amazon Linux AMI

Once you’ve created your Amazon Linux EC2 instance, you will get a private key (generated during the server setup process) and a default UNIX user called ec2-user. By default password authentication is disabled (because it’s plain text password transferred over the internet). You can login using a ssh client from your PC using this command:

ssh -i /path/to/mykeypair.pem ec2-user@11.22.33.44

If you’re on windows and you use Putty it might be slightly more tricky:

  1. First you need PuttyGen tool
  2. Once you’ve downloaded and installed it, open it and select File -> Load private key. Find your keypair file
  3. PuttyGen will display success message if your keypair is valid. Then select Save private key
  4. On your putty connection parameter, go to Connection -> SSH -> Auth and use the above saved file for Private key file for authentication

Generating Encrypted Password

Standard UNIX useradd and usermod commands can be used to manage users on Amazon Linux, however you first need to know how to generate encrypted password. Supposed I want to encrypt my password string “holasenior”, run following commands:

[ec2-user@11.22.33.44]# python
Python 2.4.3 (#1, Jan  9 2013, 06:47:03)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import crypt; print

>>> crypt.crypt('holasenior','mysalt123')
'myOZ9FACMq7sA'
>>>

myOZ9FACMq7sA is your encrypted password. mysalt123 is an encryption salt to defent against dictionary attack.

Adding New User

To add new user “ironman” with password “holasenior” encrypted with salt “mysalt123”:

sudo useradd -p myOZ9FACMq7sA ironman

Changing Password of Existing User

To change password of existing user “ec2-user” to “holasenior” encrypted with salt “mysalt123”:

sudo usermod -p myOZ9FACMq7sA ec2-user

SSH with Password Authentication

WARNING: Using plain-text password authentication for SSH is dangerous, your password will be visible over the internet.

To enable password authentication, edit /etc/sshd_config file and find following line and change it to yes

PasswordAuthentication no

Posting JSON to Spring MVC Controller

Spring MVC can be setup to automatically bind incoming JSON string into a Java object. Firstly, ensure you have jackson-mapper-asl included on the classpath:

<dependency>
  <groupId>org.codehaus.jackson</groupId>
  <artifactId>jackson-mapper-asl</artifactId>
  <version>1.9.13</version>
</dependency>

Assuming the JSON string we want to bind represent a person object like this:

{
  name: "Gerry",
  age: 20,
  city: "Sydney"
}

Which will be bound into following Java class:

public class Person {
  private String name;
  private int age;
  private String city;
  // getters & setters ...
}

Declare the Spring MVC controller handler method like below. The handler method is mapped into path “addPerson” with method POST.

@RequestMapping(value = "/addPerson", method = RequestMethod.POST, headers = {"Content-type=application/json"})
public String addPerson(@RequestBody Person person) {
  logger.debug(person.toString());
  return "person";
}

Below is an example jQuery based javascript handler that posts into the above Spring MVC handler.

$.ajax({
  type: "POST",
  url: "addPerson",
  data: JSON.stringify({ name: "Gerry", age: 20, city: "Sydney" }),
  contentType: 'application/json',
  success: function(html) {
    alert(html);
  }
});