Testing TCP Firewall With Netcat

Simplest way to test if a port is open on firewall is through telnet, but problem with that is what if you don’t have anything listening? What if you’re preparing infrastructure for a new app-server deployment — telnet won’t work unless there’s something listening on the server.

With netcat you can create a simple listener that echoes whatever character passed to it. Netcat should be available on standard UNIX system via nc command.

If you want to test if port 1234 has been allowed through on firewall:

  1. On the server, run nc -l -p 1234. Netcat will wait incoming connection to port 1234.
  2. On the client, simply do telnet 11.22.33.44 1234 (assuming the server ip is 11.22.33.44)

If the server is on Windows, there’s even a Windows version of Netcat available.

Windows 7 Unable to Import Trusted Root Certification Authorities

Windows has its own trusted root CA certificates (which is also used by Google Chrome browsers). If your company is running your own internal CA, an additional CA certificate has to be added to it.

I encountered a very strange problem where everytime I import it via certmgr.msc the dialog says The import was successful but the certificate was never imported.

Solution 1: Use Administrator Elevated Privilege

  1. Click on windows icon -> Search for cmd
  2. Right click cmd.exe -> Run as Administrator
  3. From the command prompt, run certmgr.msc
  4. Continue with the import process

Solution 2: Edit Group Policy on Domain Controller

It is common for PCs in office environment to be connected to a network domain. The default domain policy is to disallow users from adding their own trusted root CA certificate.

To get around this problem, use group policy management editor on your domain controller and find **Certificate Path Validation Settings**. Update the settings into following:

cert_import2

See this Microsoft article for more info.

Mitigating DoS Attack Using iptables

Continuing from my earlier post about iptables basics, the limit module of iptables can be used to mitigate DoS (Denial of Service) attack. Note that mitigating here means “reducing the damage”. The worst scenario is under a heavy DoS you can’t even SSH and run commands on your server. With iptables you can limit the frequency of packets — enabling you to SSH and take appropriate actions.

Following are the rules I currently use. This will only allow new incoming TCP connection on port 80 & 443 with specified frequency (see limit explanation below):

iptables -A INPUT -j ACCEPT -p tcp --dport 80 -m state --state NEW -m limit --limit 40/s --limit-burst 5 -m comment --comment 'Allow incoming HTTP'
iptables -A INPUT -j ACCEPT -p tcp --dport 443 -m state --state NEW -m limit --limit 40/s --limit-burst 5 -m comment --comment 'Allow incoming HTTPS'
  • -A INPUT: Append to the end of a chain called INPUT
  • -j ACCEPT: When rule match, accept the packet
  • -p tcp: Match only TCP protocol
  • –dport: Match given TCP port
  • -m state: Use the state module
  • –state NEW: Match only packets initiated from new connection. This rule will not match packets exchanged from an existing connections.
  • -m limit: Use the limit module
  • –limit 40/s: If more than 40 packet per second received, decrement one burst point. If no more burst point, reject the packet
  • –limit-burst 5: The initial number of burst point. A “burst” occur when the limit above is reached. On a period where limit is not reached, one burst point is regained, up to this maximum limit. If burst point is 0, subsequent burst will cause the current rule matching to fail — and iptables will try the next rules (if you setup iptables properly the packet should slip through to ‘reject all’ rule)

Add following rule to allow your program making connection to localhost (loopback interface)

iptables -A INPUT -i lo -j ACCEPT

I then append a rule to match packets exchanged from established / related connections. This is important so packet resulted from outbound connections are accepted:

iptables -A INPUT -j ACCEPT -m state --state RELATED,ESTABLISHED -m limit --limit 100/s --limit-burst 50

And finally, reject all packets not accepted by above rules. Be careful before you do this, make sure you’ve added rules to allow SSH (port 22) so you don’t lock yourself out.

iptables -A INPUT -j REJECT

Testing And Continual Adjustment

Finding the correct number for limit and burst could be hard, but what I find useful is to perform continuous monitoring and adjustment. Keep in mind your goal here is to ensure maximum capacity of the server is utilised while protecting it against DoS. One approach I like is to use the iptables LOG target. Assuming I add following rules:

iptables -A INPUT -j ACCEPT -p tcp --dport 80 -m state --state NEW -m limit --limit 40/s --limit-burst 5
iptables -A INPUT -j LOG -p tcp --dport 80 -m state --state NEW --log-prefix 'TCP 80 Burst Exhausted'

If the first rule did not match (eg: because of burst is exhausted), the LOG rule after it will match and print into your syslog (typically /var/log/messages) with prefix “TCP 80 Burst Exhausted”. LOG target will not accept / reject the packet — after logging, iptables will continue checking the subsequent ruless.

The neat thing here is you can grep ‘TCP 80 Burst Exhausted’ /var/log/messages to detect when was last time suspected DoS attack occured. If the suspected attack is determined to be false alarm, your limit and burse setting is too strict, and you should gradually increase it.

Saving and Restoring

Updating rules at specific order can be very tedious, you have to count the terminal screen lines, insert the new rule, delete the old one and so on. There’s one trick you can do, that is to leverage iptables-restore command. Everytime you save the rule using service iptables save /etc/sysconfig/iptables file is updated with your rule specification. You can update this file and restore it using iptables-restore. Beware! A syntax error will cause your rule to be skipped. It’s best to test your command first by adding it to the bottom, and use this method to re-order the rules.

Configuring Firewall Packet Filtering Using iptables

This article was tested using iptables v1.3.5 running on CentOS.

Displaying Currently Active Rule

iptables -L -v -n

-v flag turns on verbose mode, and -n causes hostname to be resolved into IP when displaying.

Adding A New Rule

iptables -A INPUT -j ACCEPT -s 50.57.11.123 -m comment --comment 'Reverse proxy'

Above rule will be added to the end of INPUT chain, and when rule matches (packing coming from ip 50.57.11.123), it will be accepted

Rejecting Packets Created From Inbound Conenctions

In the following example all packets from inbound connection are rejected, but not outbound. The only inbound packets allowed are from 72.8.190.105 and 199.241.192.0/22

Chain INPUT (policy ACCEPT 704K packets, 218M bytes)
 pkts bytes target     prot opt in     out     source               destination
  36M 4776M ACCEPT     all  --  *      *       72.8.190.105         0.0.0.0/0       /* Allow incoming from Reverse Proxy*/
4439K  577M ACCEPT     all  --  *      *       199.241.192.0/22     0.0.0.0/0       /* Allow incoming from Reverse Proxy */
  10M 2897M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0       state RELATED,ESTABLISHED /* Accept incoming packets from already established conn */
14586  878K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0       /* Reject everything else */ reject-with icmp-port-unreachable

This is achieved by checking state. If incoming packet is associated with TCP connection with RELATED / ESTABLISHED then it will be allowed.

Such rule can be added using

iptables -A INPUT -j REJECT -m state --state ESTABLISHED,RELATED -m comment --comment 'Reject everything else'

Saving Rules

Use /sbin/service iptables save to persist changes for the next time the server is rebooted.

Looking up Command for Currently Configure Rules

When you saved your iptables settings, the command used to reconstruct the rules can be looked up on /etc/sysconfig/iptables file

Windows 7 nslookup Resolves But Ping And Others Failed

As a wannabe network admin at the office, I’ve been dealing with a really tricky problem for the past few days. Long story short, I have few development servers and I want to setup CNAMEs pointing to it so it’s easy to remember.

The problem is every 15-20min the CNAME will stop resolving. If I try to ping it gives name not found. However nslookup still resolves fine.

I’ve tried many different things with no luck until I finally found the problem: we mistakenly configured multiple public DNS on our DHCP settings alongside our primary internal DNS hosted on Windows Small Business Server 2011 (SBS). As shown below the configuration after we fixed it, only 1 entry is listed pointing to our internal DNS

ipconfig

It seems if multiple DNS servers are configured, windows will pick a primary one at random / round robin. If the primary happens to be the one other than our internal DNS, it won’t be able to resolve the CNAME entries.

This setting can be changed on DHCP configuration tool on SBS as shown below

dhcpconfig

And to make sure internet-wide name resolution still works. The public DNS have to be setup on the internal DNS forwarder on SBS DNS configuration

dnsforwarder

Add to the original confusion was I kept wondering why non CNAME alias still can be resolved all this time. This turns out due to the fact Windows uses not only DNS, but a set of other services to translate hostname into IP:

  1. Client check if the name queried is of its own
  2. Client checks local hosts file (eg: %Systemroot%\System32\Drivers\etc)
  3. DNS are queried
  4. NetBIOS

(Quoted from http://support.microsoft.com/kb/172218)

Notice that if DNS fails on step 3 regular PC names still can be resolved using NetBIOS, but in the case of CNAME it wouldn’t work.

How to Add Additional IP to a Network Interface on Mac OS X

See here for similar guide on Windows.

After half an hour surfing various pages, I’ve finally figured out how to assign additional IP to a network interface on Mac OS X (thanks to this apple discussion thread: https://discussions.apple.com/thread/885005?start=0&tstart=0).

Adding another IP to your network interface is a handy way of running multiple instance of a process listening on the same port (eg: Tomcat container). This is very useful to perform High-Availability (HA) setup locally on laptop.

This guide is written against OS X Mountain Lion (10.8), but hopefully not that much different for older / newer OS X version.

  1. Open Network Preferences (System Preferences > Network). On the left list box each connection made through available network interfaces are listed. On below screenshot I have two connections configured for my wireless and bluetooth network interface respectively.
  2. On OS X you need to “duplicate a connection” to add an IP to an existing network interface. Typically you would add additional IP to your primary network interface (eg: your wireless or LAN card) so it’s reachable from other host in your network. In my macbook air, since it doesn’t have a LAN port, I used the wifi network interface. Select your primary network connection (“Wi-Fi” in my case), click on the gear icon at the bottom and select Duplicate Service… You might need to click the padlock icon first to perform administrative task.
  3. Enter a name for the new connection, eg: Tomcat NIC 2 and hit Duplicate. The new connection will be created. Although it’s a different connection, it still uses the same network interface (in my case the wifi network interface). Initially the connection might not have an automatically assigned IP.
  4. If the connection does not have an automatically assigned IP, assign it manually. From Network Preferences, select the new connection, and click Advanced… button. Go to the TCP/IP tab, select Configure IPv4 Manually, provide an IPv4 address and subnet mask. Ensure the IP isn’t already used in your network (you can check it using ping).

You can verify the virtual interface has been created by using ifconfig command. Notice in my case I have got more than 1 IP assigned to my wifi interface (en0)


$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
 options=3<RXCSUM,TXCSUM>
 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
 inet 127.0.0.1 netmask 0xff000000
 inet6 ::1 prefixlen 128
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 ether 70:56:81:c0:c1:4f
 inet6 fe80::7256:81ff:fec0:c14f%en0 prefixlen 64 scopeid 0x4
 inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255
 inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
 inet 192.168.1.102 netmask 0xffffff00 broadcast 192.168.1.255
 media: autoselect
 status: active
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
 ether 02:56:81:c0:c1:4f
 media: autoselect
 status: inactive