Archives

We had an interesting email today about one of our servers denial of service attacking addresses on the internet. Our hosting providers monitor their internet pipes for malevolent traffic patterns and our server was setting off alarms. Their email instructed us that we had two hours to reply before they turned off our internet service entirely. We weren’t super pleased.

The source port of the traffic was 27015…  a common default game server port. We looked deeper into the traffic and noticed that udp packets were being blasted off to lots of remote hosts. At first this looked like regular game server traffic but on closer inspection it was not.

Some googling turned up the reason. Apparently it is possible to hand craft a packet to all of duty 4 dedicated game servers and have them reply to the wrong source… This can be done at high velocity across lots of game servers to effectively cloak your denial of service attack.  We firewalled off the source of the phony packets quickly.

The take away here (tldr) is that game servers can be used as mirrors for anonymous denial of service attacks.  Pretty clever and bad all at the same time. Hopefully patches are created to prevent this kind of thing in the future.  Even writing something like as game server can cause security problems.  The entire call of duty server network is basically one big denial of service mirror.  Wild.

Update:
If you are having this problem yourself and wondering how to fix it, here is the iptables-fu for it!
Thanks Netije!
$IPT -A INPUT -p UDP -m length --length 42 -m recent --set --name getstatus_cod
$IPT -A INPUT -p UDP -m string --algo bm --string "getstatus" -m recent --update --seconds 1 --hitcount 20 --name getstatus_cod -j DROP

Your firewall works great. Wise will be down until Windows pulls its head out of its ass.

Quadranet is having issues with our West Coast datacenter. Engineers are working on the issue right now. Sorry!

From the horses mouth:

We are currently experiencing network-related issues. Our network engineering team is already troubleshooting and working to resolve the issue.
We should have the issue resolved very shortly. Once the support department hears back from the engineering team upon resolution, we will have more details to give. Thank you for your patience and understanding.

Update: Quadranet has fixed the issue and we’re back to full capacity. Thanks!

At 4am we are going to be preforming the memory upgrade to 16 GB.

Love was just rebooted. It looks IO destroyed. This is probably due to disk use overloading on patch day. oooops.

If you want to move off Love, use the migrate server button in your user panel.

Aqua just did the same thing and was rebooted as well. It looks like the map conversion is pretty hard on the boxes!

Update: Love is running smoothly now, but aqua is a different story. Aqua is having several problems with the hard drives. We are doing a check disk at the moment. Ill update this when it is done.

Update 2: Today is not a good day for servers. Aqua is finally up after fixing a corrupt boot sector from what we think IO disk usage just choked the box. There was a problem indigo, but that was fixed. Now there is a problem with hope. Which we dont know what is wrong right now.

When we original purchased AQUA, it only came with 12 of the 16 gigs we ordered. We’re going to power this server down at 4am on the 17th to install the extra memory. When this server goes down, you’ll need to restart your servers in hgspanel for them to come back up again!

We picked 4am to effect the least amount of people, we hope this time works the best for you!

Thanks for understanding.

We applied a ‘fix’ today to force kill Minecraft modded craftbukkit servers that were not stopping themselves properly… for some reason it caused server control to become slow and… under load… stop working.

We fixed the problem and server controls should be working just fine now. People who didn’t need to start or stop their server likely had no problems!

So, we can’t fix the bukkit problem. If you have bukkit, you must do the following to stop your server until they fix it:

1 SSH to your server
2 run this command:
killall java

Sorry dudes!

We’re having an issue with apache becoming unresponsive on our web server. I emailed someone I know because I have yet been unable to fix the problem…. it’s pretty weird and I have a fishy feeling its our VPS provider that our web hosting runs on… oh man migrating HGSPanel would suck.

But – thats what we’re gonna do if we find out thats the problem!

We’re sorry for the website being unavailable. We’re working on this random oddity.

We’re sold out again. Before we toggled the site to sold out, we sold a few too many servers… Basically, we have some boxes that are running but overloaded and lagging. We have new hardware in coming up early tomorrow and we’re going to credit anyone who brings this up to support to make up for the problem.

The worst lag is on Zod.

We’re really sorry and we plan to open up for more orders tomorrow. When we open up for new orders, we will also be able to move users who are on laggy boxes to a brand new shiny server with lots of room. If you are one of the first five for a game, you might even get your own IP with a default port!

Again, if you are lagging, please contact [email protected] for credit and a server move. We’re so sorry!

Alpha (web server and db server) just hiccuped a bit with todays billing report. Some of the thread limiting I installed had an issue when we went crazy doing the nightly processes. It’s OK now and I’ll get some waits in there to make it less destructive tomorrow.

Indigo also had some heavy activity on an NS2 (Natrual Selection 2) server which appears to have bugged out and become unavailable as well. It is now back.