Veterans Day

Veterans Day. It’s become another one of those days where all the social medias light up with easy to post messages and reshares. It’s sort of like the birthday wishes people send you on that thing. They didn’t know it was your birthday, but the website told them it is so they post a quick “happy birthday” on your page and forget about it. Of course, this doesn’t apply to every one and every post; but it does for the majority. It’s why I actually call people on their birthday, or send them a more direct, personal, message, and why I don’t join in the “hey it’s X day!” crowd.

But someone I play video games with decided to write why he is not fond of today, and what it really means to him. I think it shows why people should really think about their actions on days like today (or even every other day). So I got his permission to repost it here.

I am leaving his name off of this quote. If he wants to identifiy himself, he can do so in the comments before they close (30 days from posting).

Quote (no alterations made):

How I ended up in and what I thought of Desert Storm

I joined the military like many even now… not out of some misguided sense of patriotism, but because I was extremely poor. If I was going to get money for college I needed help.

So I went to the recruiters and researched everything about the branches that had college money. I settled on the Army. Mainly because the Army was well not the Navy. Pissing my father off to no end as he was a Navy Seal when they were still called frogmen. Laos, Cambodia, etc.

I took the ASVAB Armed Services Vocational Aptitude Battery. Like a SAT, ACT, or MCATs but slightly simpler. a 6 hour test that tested your ability to do math, spacial reasoning, understand words and phrases, and a coding section. I am a member of MENSA, on some scales I have a 155 others a 165 I.Q. So this test was almost a laugh. I spent 3 of the 6 hours doodling in the margins of my paper. When caught the drill sargent took my papers and graded them finding only one wrong answer in the whole test. And I even proved the question was wrong not my answer. He pulled me out of the testing area and sent me back to the recruiter for assignment in a MOS Military Occupational Specialty.

There were only three mos’s that received the combined military and Army college funding if you enlisted. Infantry, Artillery, and Armor. I had already decided that I wanted the most metal between myself and what ever enemy I could think of. So I chose Armor. 19K Armor Crew member of a M1A1 Abrams tank.

But before my recruiter would let me sign the contract designating my wish, he had a surprise. A man named Lynch in a suit wanted to talk to me. Apparently noone had ever aced the ASVAB.

Here in the US it used to be that the NSA, CIA, and FBI had to answer a direct question of who they were working for IF phrased properly. When Mr. Lynch wouldn’t or couldn’t answer my query I declined his insistant invitation to work for his organization.

So having signed my paperwork signing a contract to do my duty. I shipped off to Fort Knox, KY. Home of the gold depository. Home to the Armor Training school. Instead of doing Basic training, then being trained as a “tanker”. They put me through One Station Unit Training (OSUT) I learned to be a soldier and a tanker at the same time.

Just 3 days before graduation from my training, my orders already made out for a stint in Germany. Everything changed… Saddam Hussain, yet another man my own government had lead into power. Decided to invade Kuwait, having failed to invade Iran for over a decade. Anyone remember the Iran/Contra Scandle?

My parents and Grandparents came to the graduation, because of the nature of what was about to happen. I had stepped into a moral quagmire of serealism. I had my graduation ceremony, visited with my family for a hour then boarded a C-130 Cargo plane to fly to Ft. Stewart, GA.
But you say that’s not Iraq…

Ft. Stewart is home to the Rapid Deployment force. Four brigades of heavy mobile infantry and armor. Each brigade has 6 battalions, each battalion has 8 companies. I arrived and processed in and drew my gear in one fell swoop. I saw Georgia for 2 hours before flying direct to Saudi Arabia. We ran off the plane with our weapons loaded not knowing if our plane would be attacked. Less than 24 hours from graduation I was at war.

The build up had began and the only troops that were there before me were the Seals and one battalion of Marines. Aug. 1990.. I turned 21 that mid Sept.

A high estimate shows the Iraqi Army capable of fielding one million men and 850,000 reservists, 5,500 tanks, 3,000 artillery pieces, 700 combat aircraft and helicopters; and held 53 divisions, 20 special-forces brigades, and several regional militias, and had a strong air defense. They were veterans, having waged war with Iran for the last decade or more. This was not a insurgent war. This was real men trained hard. But with a weak heart. Ours is a volunteer force. Theirs was predominately transcripts, All men in Iraq at the time were expected to serve a term in the military. So doctors, lawyers, and other ordinary men who didn’t want to be there but out of fear of a homicidal tyrant were forced to serve or die or have their families suffer, or both.

on Feb. 14th I was sitting on the front of my tank leaned against the turret eating and listening to a BBC news report when I saw glimmers of reflected light from high in the sky.

There above me were b-52’s escort by f-16’s, f-14’s, f-15’s, f-18’s you friggin name it it was in the air. I was in the diamond shaped neutral zone some 500 miles inland, technically already in Iraq. 2 mins later I heard a hours worth of thunder. Not sporadic like boom boom boom. I mean one steady hour long explosion. We had already started ditching our civilian gear. If it wasn’t (G.I.)General Issue it was gone. I was doing the start-up for the thermal viewer and checking my bore sights. But I didn’t need it for another month. The air war had started, but not the ground war.

Mar. 1990 Just after the air war started my Sargent the gunner for the tank came down with appendicitis. And because it was the brigade XO’s (Executive Officer: 2nd in command, Lt. Colonel)
They held an all out, every man, gunnery to see who would be his new gunner. You guessed it, me.

First thing you will learn about me and one of the reasons I have suffered from moral issues about this time in my life… When I give my word, I give you my honor. I may have battled in my head the reasons we were there. But, while I was there, I was the best at it I could be.
We spent the next month doing the same drills we had been doing for the last 8+. Only now I was in the gunner’s seat.

Ground war…

We were so prepared, you could shave with how sharp we were. Just as the sun was starting to think about rizing we crossed in mass. The dim light hiding the dust clouds of our fast approach.

An M1A1ip (improved: 10 more tons of depleted uranium and chobum armor, a beefed up 1500hp jet turbine engine.) Will do 55 mph across rough terrain and still be able to hit anything between it and the horizon. Thermal sights set to a 1 degree of tempature difference means a enemy tank glows like a star. You can even read the painted numbers on a enemy tank. T-72m’s their turrets fly through the air like a frying pan when a 120mm Sabot round APFSDS (Armor Piercing Fin Stabilized Discarding Sabot)

A depleted Uranium dart about the diameter of a quarter flying at 5 miles a sec.

This is what a shot sounds like.

TC Tank Commander: “Gunner sabot tank 2 o’clock”
Me, Gunner: “Loader Sabot three round repeat”
Loader “Sabot Three rounds”
Gunner: “Tank acquired 2 o’clock”
Loader: “Clear” meaning the 1st round was loaded and he was clear of the breach.
TC: “Fire”
Everyone: “Fire”

Squeeze the trigger and the front three road wheels come off the ground and a heart stopping thunderous boom resounds.

Hit the point where the turret meets the body of the tank… And flip flip thud 4 men dead, next tank…

55 tanks in 5 days, I don’t remember how many APC’s, or trucks. I don’t even want to do the math anymore.

I killed men who didn’t want to fight. I killed for oil. And I was good at it. I still hate myself for this. Don’t get me wrong I understand your thanks, it just feels wrong to me in more ways than I can express with words. I feel a dichotomy I am proud of my service, I am proud of those who do serve. I am sickened that anyone has to serve at all. No one comes out the same. If they do they weren’t there.

I am not dominated by my veteran status. I will always have PTSD, but I have dealt with my moral issues and may regret the lessons learned. But I would never give up the person I am now to take them back. I just wish more could be enlightened without the experience.
After I returned to the states, I had plenty of time to think about what I did and saw.

The thing about Garrison is you don’t do the same thing as you do in combat at least not all the time. I had my MOS changed twice after D.S. First I changed to a PLL clerk (Parts and Load List) Thats’s the order and return clerk for a company level motorpool. The guy responsible for making sure things get tracked, ordered, returned For the entire company’s vehicles. We had early 286 PC’s with 5 1/4 in floppy drives, and for the time a whooping 10 gig harddrive. The internet was available but mostly as a email service between computers. I was convinced there was a better way to integrate the forms I filled out ad finium could be put onto the computer and sent to everyone that needed it and would only have to be filled out once. I submitted my idea to the DARPA (Defense Advanced Research Projects Agency) That’s how I got involved in Database languages. You might think I requested this move. But I hadn’t I made a decision while under fire back in Desert Storm that kinda ended my military career before it really began. My Tank Commander on the first day of fighting went catatonic he was one minute giving orders and doing his job the next just open mouthed and unresponsive. So I 100mph taped (duct tape) him so he couldn’t touch anything when he woke up I didn’t want him to start shooting us or other tanks should he be delusional.

Well I was a Private First Class (PFC) he was a Lt. Colonel, We got through the day he was removed from the tank sent back to Saudi and was “decided to resign his commission” A heroic act to some and a dangerous thought to all my future commanders. No matter what I did after that there was always blockades in my way because they thought I was a rebel. Which I was I was a free thinking evolved human. With a sense of honor and duty that didn’t completely conform.

(if your looking for my unit on the map we were the green leaf on a red background)

24th Infantry Division
operation_desert_storm

Next time you want to say to someone “thank you for your service,” or whatever else, take a moment to consider how that person might feel about what you say. Yes, you mean well, but they could very well have had to do some tragic things during that “service” that do not sit well with them. And your kind words may not have the effect you wanted them to. So maybe talk to that person first. Get an idea of how they feel before you just blurt out the in vogue patriotism of the day. If you determine they’d be receptive to it, go for it. Otherwise, just offer your understanding (or lack thereof as the case may be).

Dysthymia

With dysthymia, you may lose interest in normal daily activities, feel hopeless, lack productivity, and have low self-esteem and an overall feeling of inadequacy. People with dysthymia are often thought of as being overly critical, constantly complaining and incapable of having fun.

While I’ve made this web site mainly technical over the past few years, the domain name is still james.roomfullofmirrors.com (jrfom.com is the short form). If I don’t write about myself at least occasionally, then the domain name means nothing. If that’s not why you read this site, then you may want to skip this post. Also, this post may ramble a bit…

A month ago I went to my doctor for a routine check-up. While there, I asked him how often is it normal for someone to feel depressed. He asked me a series of questions and diagnosed me with dysthymia (now persistent depressive disorder). I had never heard of this disorder before that diagnosis. He prescribed Escitalopram to me, along with some other suggestions, and I’ve been taking it over the past month (starting the full dose every day tomorrow). It is said the drug can take up to six weeks to really take effect, but I feel that it has already made a tremendous difference. So I am writing this post to raise awareness of this disorder, as I think it may be more prevalent than the statistics claim. Everyone has heard of stuff like bipolar disorder, but dysthymia less so.

Of all the definitions I have read for dysthymia, the quote that leads off this post, from mayoclinic.org, strikes me the most. It sums up me in two short sentences. As I described it to my doctor, I have felt, for as long as I can recall, that I have had to fight off these feelings, or try to correct these behaviors, every single day. It got to the point that I was physically exhausted and mentally drained from the struggle, and just could not go on without asking a professional’s opinion. There’s just no way that is how everyone feels all the time.

Prior to this medication, I had reached the point where I did not care about a single thing. I went through my daily motions just to keep on existing, and no amount of distractions (“toys” as I’ve called them, e.g. video games or synthesizers) helped to pull me out. For example, I’d sit at work, pushing myself to do whatever needed to be done there, and daydream about going home and working on music. But by the time I got home, laying on the couch and watching pointless television until time to go to bed was way more appealing.

Occasionally I would succeed in working on music. And I think you can hear the dysthymia in my music; particularly in my most recent releases. It doesn’t matter if the songs are in a Major key, they all have some tinge of, erm, darkness to them. Truly, that’s the style of music I enjoy the most, but have I found the reason? I don’t know the answer to that yet.

But now that I’m on the medication, it’s like a different world. I feel, for lack of a better word, normal. I wouldn’t say I’m necessarily happy, I’m not, really. That’s not what the medication is supposed to do. But I do feel more eager to do things (whether I do them or not; laziness is laziness), and have been much more productive than I have been in a long time. Before, I felt like there was a seething rage just barely contained that would unleash itself with little prompting. Now, that is gone; it’s inexplicable other to say that it’s just “gone.” Yes, I get agitated and a little angry still, but nothing like before. It has been a part of me for so long I think I almost miss it, but not really. It was the most draining part, and the part that hurt my personal relationships the most.

I’m scheduled to visit my doctor for a follow-up in three and a half weeks. Maybe the medication will have other effects by then. Or maybe he’ll ask me to try something different. I don’t know. I just know that right now, today, I feel like a corner has been turned. I feel like I can advance, and that I’m not stuck in a swamp of sadness; the luck dragon has descended and snatched me up.

If any of this post resonates with you, either personally or by reminding you of someone you know, I urge you to ask a professional about it (or urge that someone you know to do so). Living with persistent depressive disorder, if that is indeed what I have (and I do believe that to be the case), is an awful way to live.

HAProxy With Keepalived For Fail Over

There are plenty of articles on the web that detail setting up HAProxy and Keepalived to create a load balancer with failover support. So why write another one? There are a couple points that I didn’t find addressed in my research; at least, not clearly. They are:

  1. How do the load balanced IP addresses get managed?
  2. What ports need to be open in the load balancers firewalls?

Instead of simply answering those questions, and making you learn the other details elsewhere, I will answer them in a full write up of a simple scenario:

  1. We have services foo.example.com, bar.example.com, and baz.example.com
  2. foo.example.com has a “public” IP 10.0.0.1 and is provided by private IPs 192.168.1.5 and 192.168.1.6
  3. bar.example.com = 10.0.0.2 provided by 192.168.2.5 and 192.168.2.6
  4. baz.example.com = 10.0.0.3 provided by 192.168.3.5 and 192.168.3.6

We’re going to setup one instance of HAProxy to load balance all three services in Virtual Machine 1 (VM1), and a second instance in VM2 to provide a fail over for VM1 (or real hardware, whatever).

Note to Red Hat users: I have a repository available that makes building HAProxy and Keepalived RPMs trivial. You can reach it at https://github.com/jsumners/failover-lb.

HAProxy

I’m not going to spend much time on the HAProxy configuration. There are many, many, well documented options for HAProxy and your actual scenario may extend beyond this simple one. If suffices to say that the following configuration will look for any traffic coming into the currently active VM (VM1 or VM2) on the public IPs 10.0.0.1, 10.0.0.2, and 10.0.0.3 on port 80. It will then proxy said traffic to one of the private servers according to the round robin load balancing algorithm and/or a cookie denoting the destination server.

global
  log 127.0.0.1   local0 debug debug
  maxconn 4096
  chroot /var/lib/haproxy
  pidfile /var/run/haproxy.pid
  user haproxy
  group haproxy
  daemon
  #debug
  #quiet

  stats socket /var/lib/haproxy/stats mode 600 level admin

defaults
  log global
  mode    http
  option  httplog
  option  dontlognull
  option  http-server-close
  option  forwardfor except 127.0.0.0/8
  option  redispatch
  option  httpclose
  retries 3
  maxconn 3000
  timeout http-request 10s
  timeout queue 1m
  timeout connect 10s
  timeout client 1m
  timeout server 1m
  timeout http-keep-alive 10s
  timeout check 10s

listen foo-http
  bind 10.0.0.1:80
  balance roundrobin
  cookie foo-server-id insert indirect nocache preserve maxidle 1h maxlife 1h domain .foo.example.com
  # proxy to port 80 and set foo-server-id cookie to "foo1"
  server foo1-http 192.168.1.5 cookie foo1
  # proxy to port 8080 and set foo-server-id cookie to "foo2"
  server foo2-http 192.168.1.6:8080 cookie foo2
  
listen bar-http
  bind 10.0.0.2:80
  balance roundrobin
  cookie bar-server-id insert indirect nocache preserve maxidle 1h maxlife 1h domain .bar.example.com
  server bar1-http 192.168.2.5 cookie bar1
  server bar2-http 192.168.2.6 cookie bar2
  
listen baz-http
  bind 10.0.0.2:80
  balance roundrobin
  cookie baz-server-id insert indirect nocache preserve maxidle 1h maxlife 2h domain .baz.example.com
  server baz1-http 192.168.3.5:8080 cookie baz1
  server baz2-http 192.168.3.6:8080 cookie baz2

Keepalived

Keepalived will be managing the fail over duties for us. It does this by:

  1. Running a script to check if a managed service is running
  2. Managing the public IP addresses for our services via IPVS
  3. Monitoring the status of a sister Keepalived process via VRRP

For our scenario, the Keepalived configuration would be:

vrrp_script chk_haproxy {
  script "killall -0 haproxy" # verify the pid existence
  interval 2 # check every 2 seconds
  weight 2 # add 2 points of prio if OK
}

vrrp_instance VI_1 {
  interface eth0 # physical interface that is connected to the network
  state MASTER
  virtual_router_id 51
  priority 101 # 101 on master, 100 on backup
  virtual_ipaddress {
    10.0.0.1 # foo.example.com
    10.0.0.2 # bar.example.com
    10.0.0.3 # baz.example.com
  }
  track_script {
    chk_haproxy
  }
}

Note that this is the only place on the system where we declare that the IPs 10.0.0.1, 10.0.0.2, and 10.0.0.3 will be managed by the kernel. We don’t declare them in a /etc/network/interfaces (Debian), /etc/sysconfig/network-scripts/ifcfg-eth0:{1,2,3} (Red Hat), or other configuration file. When Keepalived launches, it will register the IP addresses with the kernel on the physical interface eth0.

Also note that we are configuring Keepalived to verify that HAProxy is running by checking for an HAProxy process identifier every two seconds. If it doesn’t find a PID, it won’t increase the VM1’s priority, VM2 will/should increase its priority, and VM2 will take over (at which point the IPs on VM1 will be released).

Finally, for the VRRP polling to work, we simply need to make sure VM1 and VM2 accept VRRP broadcast traffic:

$ iptables -A INPUT -p vrrp -d 224.0.0.0/8 -j ACCEPT

BuddyPress And Lighttpd

This is a short post that detailing how to fix a problem with the WordPress plugin BuddyPress and Lighttpd.

The problem in question:

Users that register for a site via BuddyPress are sent an email with an account activation link. Upon visiting said link, their account is supposed to be activated thereby verifying their email address. However, with the typical configuration for a WordPress site on Lighttpd (redirecting all 404s to “/index.php”), GET request parameters are not forwarded (as they shouldn’t be, really). Since BuddyPress relies on a GET parameter for the activation key (and it should not do so) this breaks the activation process.

Solution:

The solution to this problem is to catch requests for activations and directly process them through the index.php routing handler. To do so, add an extra bit of configuration to your Lighttpd configuration for the site:

url.rewrite-final = (
  # Enable redirects for account activations that use a GET parameter
  "^/register/activate-account(.*)$" => "/index.php/$1"
)

Note that “/register/activate-account” is local to your BuddyPress configuration. It should match the URL you have configured for said action (i.e. look at an activation email to figure out what to look for).

References:

http://buddypress.org/support/topic/activation-not-possible/
http://redmine.lighttpd.net/issues/660
http://www.crobak.org/2011/01/moving-wordpress-blog-to-lighttpd/

Oracle JDBC PermGen Memory Leak Fix

Let’s talk about the Oracle JDBC drivers. If you have had the misfortune to use them, you know that there are many ways they can taint your well written application. For me, the most annoying problem is the memory leaks they leave behind when you redeploy an application without shutting down the JVM instance. Do this enough, and the drivers end up using all of the PermGen space and necessitating a restart of the JVM (undesirable when your JVM is really a web server running multiple applications). This problem has plagued me for a couple of years, and I think I have finally figured out the solution. I wrote that solution into a library I call SimpleOraclePool.

SimpleOraclePool takes care of creating a connection pool using the Oracle UCP library with the Oracle JDBC driver providing the connection. Included in the library is a ServletContextListener, named OjdbcDriverListener, that will clean up the Oracle mess when your application is unloaded (from a Servlet container, of course).

I’ll leave out the explanation of how to use it. That is all documented in the code repository’s readme and the JavaDoc included with the library. But I would like to sign this post with a message to Oracle:

....................../´¯/) 
....................,/¯../ 
.................../..../ 
............./´¯/'...'/´¯¯`·¸ 
........../'/.../..../......./¨¯\ 
........('(...´...´.... ¯~/'...') 
.........\.................'...../ 
..........''...\.......... _.·´ 
............\..............( 
..............\.............\...