Site reliability improvements

Good evening everyone.

This evening we have completed a full sync and replication system to a mirrored server for this site. That means that, at random, you will be looking at either our new server in Louisville, CO... or the old server in Pennsylvania. Should one of these servers fail, the other will instantly pick up and serve requests for all users. After this happens... when the other server comes back up it will catch back up with the online server, then begin serving requests once again. Better yet, when we need to do maintenance in the future, we can route traffic to one server, upgrade the other, reverse traffic and do the remaining server... most upgrades can be done with no to minimal server downtime. (forum software updates are the exception, they have to run everywhere)

Additionally, this set up allows us to have constant replica backups to 2 places, but it also allows us to have daily snapshot backups to two additional offsite servers. There is very little chance of another huge data loss happening under these circumstances (data in 4 locations, probably not all going bad at once).

Please note that this is still a new system in these regards, please PM me if you feel that there is something odd going on with the site/account.

Also, in recognition of the season, I am now sporting the purple color, to root on the Rockies in the quest to not be laughed at in October.

It is alive. Went to bed, woke up and it was still here. [Beer]

Awesome, sounds sophisticated. Thanks for all the hard work!

So you took care of the server eating zombies?

I see the time is now accurate as well

Thanks for all of the hard work on this.

Like Jayock said if anyone notices anything different or funky, small as it may be, please contact Jayock and let him know and give him as much detail with the issues you are experiencing so he might be able to take that intel and fix it

I HUGE round of applause for our site server admins foxtrot and jayock.

Jayock has been a tremendous new asset to the team.

OK, I want to know the technical stuff, Load Balancer or DNS round robin?

DNS RR. I have a load balancer for local stuff, for HA and performance hungry services.

This is over WAN, and a single load balancer at one server location kind of defeats the geographic redundancy IMO. Maybe someday we can have a pair of F5 balancers spread out and do it all the fancy way.

Yes, great work purple team!!

Purple power!

Are our permits valid while on the Pennsylvania server ?
Thanks for all the hard work guys !! [Beer]

Excellent. It must feel good to know the plan has been implemented!

Great work guys!

Not sure if it was site wide or what happened but an hour ago I got a permanent ban message, obviously that has changed now ?? Any thoughts ?

Yep. Coffee spilled on the server. Not really. Sorry about this guys, not really banned. Help spread the word.

More details: http://ww2.ar-15.co/forums/showthread.php?t=53152

Nice work! Lovin that the site is back up and redundant.

Thanks for all the hard work!

Glad you mentioned off-site backups, a live replica of data can just mean you have 2 copies of crap when SHTF.

I keep at least a week's worth of daily backups of a critical database, in case we need to go back to an old state.

Did we loose some data during this? I had some info in my PM box and now all my PMs are dated back to 10/11.

Not during this. But the reliability improvements were done in response to:

http://www.ar-15.co/forums/showthread.php?t=49268

You must not have been around for a while to not notice this.

Great job guys, really like the new interface and speed! Thanks for gutting it out on all the issues from the last ISP! Another techie wanna know: servers metal or virtualized?

Metal, Dedicated server. We are now no longer mirrored, but that is in the works. We do have triple redundant backups, and I have a temporary place we can move the site in the event the new server catches fire or something of the sort.

Mirroring will be put back in place shortly.

Cool, sounds like you have planned to really square away the configuration. As long as you're using Linux and MySQL and not Windoze 8-). Yeah, I'm a Linux/UNIX guy trapped in a MSFT world...

100% powered by Windows Vista!! [ROFL2][ROFL2][ROFL2][ROFL2]











Ok....
Ubuntu Server 12.04 LTS x86_64 running MySQL 5.5.28, Apache 2.2.22 and PHP 5.3.10